U T?hTY@sddlZddlZddlZddlmZddlmZmZmZddl Z ddl m Z m Z mZddlmZe eeee jdddZe eee jd d d Zeed d dZd,e eeeeee jdddZeeee jfdddZeeeeeee e e eeed ddZeeeeee e e eeed ddZddZd-eeeeeeeeee jee jee jfddd Zd.eeeeeeeeee jee jee jfd!d"d#Zd$d%Zeeeeeeeeeeeeeeeeed&d'd(Z d)d*Z!e"d+kre!dS)/N)Path)DictOptionalTuple) ModelProto TensorProto numpy_helper) OnnxModel) input_ids batch_sizesequence_lengthdictionary_sizereturncCsp|jjjtjtjtjfksttj j |||ftj d}|jjjtjkrRt |}n|jjjtjkrlt |}|S)a`Create input tensor based on the graph input of input_ids Args: input_ids (TensorProto): graph input of the input_ids input tensor batch_size (int): batch size sequence_length (int): sequence length dictionary_size (int): vocabulary size of dictionary Returns: np.ndarray: the input tensor created )sizedtype)type tensor_type elem_typerFLOATINT32INT64AssertionErrornprandomrandintint32float32int64)r r r r datarY/var/www/html/venv/lib/python3.8/site-packages/onnxruntime/transformers/bert_test_data.pyfake_input_ids_datas   r!) segment_idsr r rcCsl|jjjtjtjtjfksttj ||ftj d}|jjjtjkrNt |}n|jjjtjkrht |}|S)a,Create input tensor based on the graph input of segment_ids Args: segment_ids (TensorProto): graph input of the token_type_ids input tensor batch_size (int): batch size sequence_length (int): sequence length Returns: np.ndarray: the input tensor created r) rrrrrrrrrzerosrrr)r"r r rrrr fake_segment_ids_data2s    r%Zmax_sequence_lengthaverage_sequence_lengthcCsL|dkr||kstd||kr4td|||Stdd|dSdS)N)rrrr&rrr get_random_lengthMs r*r)) input_maskr r r'random_sequence_length mask_typerc Cs\|jjjtjtjtjfkst|dkrrtj |tj d}|rXt |D]}t ||||<qBnt |D] }|||<q`n|dkrtj ||ftj d}|rt |D](}t ||}t |D]} d||| f<qqn4tj ||ftj d} | |d| jdd| jdf<n |dks ttj |ddtj d}|rt |D]}t ||||<q2t |dD]r}|dkr~|||d||dnd|||<|dkr|||d||dnd|d|d|<qTnTt |D]}|||<qt |dD].}|||||<|||d|d|<q|jjjtjkrtd|dYqbXtd|dntd|dt|D]N\}\}}t ||}t tj |d|dd }| | W5QRXqnd S) zOutput input tensors of test data to a directory Args: directory (str): path of a directory inputs (Dict[str, np.ndarray]): map from input name to value zCreation of the directory z failedz#Successfully created the directory  zWarning: directory z$ existed. Files will be overwritten.Zinput_.pbwbN)ospathexistsmkdirOSErrorprint enumerateitemsr from_arrayopenjoinwriteSerializeToString)r5r6indexnamerZtensorfilerrr output_test_datas  rJ) r r test_casesr verbose random_seedr r"r+r'r,r-c Cs|dk s ttj|t|g} t|D]r} t||||}|j|i}|r`t|||||j<|r|t|||| | | ||j<|rt | dkrt d|| |q.| S)aCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases dictionary_size (int): vocabulary size of dictionary for input_ids verbose (bool): print more information or not random_seed (int): random seed input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value NrzExample inputs) rrrseedr/r!rHr%r4lenr?append)r r rKr rLrMr r"r+r'r,r- all_inputsZ _test_caseZinput_1r6rrr fake_test_datas,"        rR) r r rKrNrLr r"r+r'r,r-c Cs:d} t|||| ||||||| | } t| |kr6td| S)aCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): print more information or not input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value i'z$Failed to create test data for test.)rRrOr?) r r rKrNrLr r"r+r'r,r-r rQrrr generate_test_datas$  rScCs`|t|jkrdS|j|}||}|dkr\|||}|dk r\|jdkr\||jd}|S)NZCastr)rOinputfind_graph_inputZ get_parentZop_type) onnx_model embed_nodeZ input_indexrTZ graph_inputZ parent_noderrr get_graph_input_from_embed_node%s   rX)rVinput_ids_namesegment_ids_nameinput_mask_namerc Cs|}|dk r||}|dkr0td|d}|rX||}|dkrXtd|d}|r||}|dkrtd|d|rdnd|rdnd}t||krtd|dt||||fSt|dkrtdt||d } t| dkrv| d} t|| d}t|| d}t|| d }|dkrZ|D]} | j} d | kr:| }q:|dkrltd |||fSd}d}d}|D]>} | j} d | kr| }nd | ksd| kr| }n| }q|r|r|r|||fStddS)a Find graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_model (OnnxModel): onnx model object input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Raises: ValueError: Graph does not have input named of input_ids_name or segment_ids_name or input_mask_name ValueError: Expected graph input number does not match with specified input_ids_name, segment_ids_name and input_mask_name Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask Nz Graph does not have input named r(rzExpect the graph to have z inputs. Got r.z'Expect the graph to have 3 inputs. Got ZEmbedLayerNormalizationmaskz#Failed to find attention mask inputtokensegmentz?Fail to assign 3 inputs. You might try rename the graph inputs.)Z'get_graph_inputs_excluding_initializersrU ValueErrorrOZget_nodes_by_op_typerXrHlower) rVrYrZr[Z graph_inputsr r"r+Zexpected_inputsZ embed_nodesrWrTZinput_name_lowerrrr find_bert_inputs2sf                  rb) onnx_filerYrZr[rc Cs@t}t|d}||W5QRXt|}t||||S)aFind graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_file (str): onnx model path input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask rb)rrCZParseFromStringreadr rb)rcrYrZr[modelrIrVrrr get_bert_inputss  rgcCsJt}|jddtdd|jddtddd|jd dtd d d|jd dtd dd|jddtddd|jddtddd|jddtddd|jddtd dd|jddtddd|jddddd|jdd|jdddd d|jdd!|jd"d#d$td%d&|jd'd(ddd)d|jdd*|jd+dtd,d-d|}|S).Nz--modelTzbert onnx model path.)requiredrhelpz --output_dirFz4output test data path. Default is current directory.)rhrdefaultriz --batch_sizer(zbatch size of inputz--sequence_lengthz maximum sequence length of inputz--input_ids_namezinput name for input idsz--segment_ids_namezinput name for segment idsz--input_mask_namezinput name for attention maskz --samplesz$number of test cases to be generatedz--seedr.z random seedz --verbose store_truezprint verbose information)rhactionri)rLz--only_input_tensorsz-only save input tensors and no output tensors)only_input_tensorsz-az--average_sequence_lengthz)average sequence length excluding padding)rjrriz-rz--random_sequence_lengthz3use uniform random instead of fixed sequence length)r,z --mask_typer)z^mask type: (1: mask index, 2: raw 2D mask, 3: key lengths, cumulated lengths of query and key))argparseArgumentParser add_argumentstrint set_defaults parse_args)parserargsrrr parse_argumentss   ry)rf output_dirr r rKrNrLrYrZr[rnr'r,r-c CsBt|||| \}}}t||||||||| | | }t|D](\}}tj|dt|}t||q8| rjdSddl}d| krddgndg}|j ||d}dd| D}t|D]\}}tj|dt|}| ||}t|D]T\}}t t|||}ttj|d |d d }||W5QRXqqdS) aICreate test data for a model, and save test data to a directory. Args: model (str): path of ONNX bert model output_dir (str): output directory batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): whether print more information input_ids_name (str): graph input name of input_ids segment_ids_name (str): graph input name of segment_ids input_mask_name (str): graph input name of input_mask only_input_tensors (bool): only save input tensors, average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type(int): mask type Ztest_data_set_NrZCUDAExecutionProviderZCPUExecutionProvider) providerscSsg|] }|jqSr)rH).0outputrrr Osz-create_and_save_test_data..Zoutput_r8r9)rgrSr@r:r;rDrsrJ onnxruntimeZget_available_providersZInferenceSession get_outputsrunrrBrZasarrayrCrErF)rfrzr r rKrNrLrYrZr[rnr'r,r-r r"r+rQr1r6r5rr{sessionZ output_namesresultZ output_nameZ tensor_resultrIrrr create_and_save_test_datasB"   rcCst}|jdkr|j|_|j}|dkrPt|j}tj|j d|j d|j}|dk rpt|}|j dddnt dt |j||j |j|j|j|j|j|j|j|j|j|j|jt d|dS)NrZbatch_Z_seq_T)parentsexist_okz7Directory existed. test data files will be overwritten.z Test data is saved to directory:)ryr'r rzrrfr:r;rDparentr r=r?rZsamplesrNrLrYrZr[rnr,r-)rxrzpr;rrr mainZs8   r__main__)r))NNN)NNN)#rpr:rpathlibrtypingrrrnumpyrZonnxrrrrVr rtZndarrayr!r%r*boolr4rsrJrRrSrXrbrgryrr__name__rrrr  s    I ; 4 ^ e L'