U Mh, @sddlmZddlZddlZddlZddlmZmZmZm Z ddl Z ddl m Z ddlmZerhddlZeeZejddddddd d d d Zd dZejdddddddddddZdS)) annotationsN)OptionalTuple TYPE_CHECKINGUnion) _type_utils) _beartypez torch.TensorstrzOptional['onnx.TypeProto']zonnx.TensorProto)tensornamelocationbasepathdtype_overridereturnc Cs:ddl}|dk r tj|jjn tj|j}|dk rR||jkrR||}| }||_ | |_ |j |j|j j|_|d|d}|D]"\} } |j} | | _t| | _qtj||} tj| rt| tj| } tj| s t | t!| d}|"|j#dd$W5QRX|S)aCreate a TensorProto with external data from a PyTorch tensor. The external data is saved to os.path.join(basepath, location). Args: tensor: Tensor to be saved. name: Name of the tensor (i.e., initializer name in ONNX graph). location: Relative location of the external data file (e.g., "/tmp/initializers/weight_0" when model is "/tmp/model_name.onnx"). basepath: Base path of the external data file (e.g., "/tmp/external_data" while model must be in "/tmp"). Reference for ONNX's external data format: How to load? https://github.com/onnx/onnx/blob/5dac81ac0707bdf88f56c35c0a5e8855d3534673/onnx/external_data_helper.py#L187 How to save? https://github.com/onnx/onnx/blob/5dac81ac0707bdf88f56c35c0a5e8855d3534673/onnx/external_data_helper.py#L43 How to set ONNX fields? https://github.com/onnx/onnx/blob/5dac81ac0707bdf88f56c35c0a5e8855d3534673/onnx/external_data_helper.py#L88 rN)r offsetlengthxbT)force)%onnxjit_type_utilsZ JitScalarTypeZfrom_onnx_typeZ tensor_typeZ elem_typeZ from_dtypeZdtypetoZ TensorProtor Z onnx_typeZ data_typeZdimsextendshapeZEXTERNALZ data_locationZuntyped_storagenbytesitemsZ external_dataaddkeyr valueospathjoinexistsremovedirnamemakedirsopenwritenumpytobytes)r r r r rrZ scalar_type tensor_protoZkey_value_pairskventryZexternal_data_file_pathZexternal_data_dir_pathZ data_filer-W/var/www/html/venv/lib/python3.8/site-packages/torch/onnx/_internal/fx/serialization.py'_create_tensor_proto_with_external_datas>             r/c CsNddlm}i}||ddd&}|D]}||||<q(W5QRX|S)Nr) safe_openptcpu)Z frameworkZdevice)Z safetensorsr0keysZ get_tensorr2)Zsafetensors_filer0Ztensorsfr*r-r-r.$_convert_safetensors_to_torch_formaths   r5Fz(Tuple[Union[dict, str, io.BytesIO], ...]zonnx.ModelProtoboolNone)r model_locationinitializer_locationtorch_state_dicts onnx_modelrename_initializerrc Cs ddl}i}ddt|jjD}dd|jjD} |D]} t| trN| } nt| trl| drlt | } nzt j | dd d } Wnrt t fk r} zPd t| kst| tjrtd t| tjr| dt j | dd } n| W5d} ~ XYnX| D]\} }|r| dd} | | kr,| | n8| D]2}|| sL| |r0|} | |qdq0tj|| }dd|jjD}| |kr| ||| <t|| |||| d}|jj|qq8tt|d d}|D]}|jj|=q||tj||dS)aLoad PyTorch tensors from files and add to "onnx_model" as external initializers. Output files: ONNX model file path: ONNX initializer folder: os.path.join(basepath, initializer_location) After running this function, you can do ort_sess = onnxruntime.InferenceSession(os.path.join(basepath, model_location)) to execute the model. Arguments: basepath: Base path of the ONNX external data file (e.g., "/path/to/large_model/"). model_location: Relative location of the ONNX model file. E.g., "model.onnx" so that the model file is saved to "/model.onnx". initializer_location: Relative location of the ONNX initializer folder. E.g., "initializers" so that the initializers are saved to "/initializers/". Note: When initializers are >2GB, must be the same as `model_location`. torch_state_dicts: Dictionaries or files which contain PyTorch tensors to be saved as ONNX initializers. For non-dict arguments, `torch.load` will be used to load them from file-like objects. onnx_model: ONNX model to be saved with external initializers. If an input name matches a tensor loaded from "torch_state_dicts", the tensor will be saved as that input's external initializer. rename_initializer: Replaces "." by "_" for all ONNX initializer names. Not needed by the official torch.onnx.dynamo_export. This is a hack for supporting `FXSymbolicTracer` tracer with fake tensor mode. In short, `FXSymbolicTracer` lifts FX parameters (self.linear_weight) as inputs (`def forward(self, linear_weight)`) and therefore, `.` cannot be used. rNcSsi|]\}}|j|qSr-r ).0idxr*r-r-r. sz1save_model_with_external_data..cSsh|] }|jqSr-r=)r>inputr-r-r. sz0save_model_with_external_data..z .safetensorsr2T) map_locationmmapz+mmap can only be used with files saved withzFailed to load the checkpoint with memory-map enabled, retrying without memory-map.Consider updating the checkpoint with mmap by using torch.save() on PyTorch version >= 1.6.)rC._cSsi|]}|j|jqSr-)r type)r>r*r-r-r.r@s)reverse)r enumerategraphZ initializerrA isinstancedictr endswithr5torchload RuntimeError ValueErrorioBytesIOlogwarningseekrreplacer"rrr r/popappendsortedr3save)r r8r9r:r;r<rZinitializers_to_be_deletedZexisting_initializersZonnx_input_nameselZ state_dicter r Zonnx_input_nameZrelative_tensor_file_pathZmodel_input_typesr)r?r-r-r.save_model_with_external_datausl(               r^)N)F) __future__rrRloggingrtypingrrrrrNZ torch.onnxrrZtorch.onnx._internalrr getLogger__name__rTZbeartyper/r5r^r-r-r-r.s"    T