U Mhq  @sUddlmZddlZddlZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z mZmZmZmZmZmZmZmZmZmZmZmZmZmZddlmZddlZddlZddl m!Z"ddl#m$m%Z&ddl'm(Z(ddl)m*Z*m+Z+ddl,m-Z-dd l.m/Z/m0Z0m1Z1m2Z3erPddl4Z4ddl5Z5ddl6Z6dd l7m1Z8dd l.m9Z9n(zdd l.m9Z9Wne:k rvYnXd Z;d edZ?e@eAZBe-jCZCejDGdddZEGdddZFGdddZGGdddeGZHejIddZJeGdddeZKGdddZLGd d!d!ZMGd"d#d#ZNGd$d%d%ZOGd&d'd'ejPZQGd(d)d)ZRGd*d+d+eSZTGd,d-d-eSZUGd.d/d/eSZVe*jWdd0d1d2ZXe*jWdd0d3d4d%d5d6d7ZYdd8d9d:d;ddd%dd#d/d-dd+d7dg Z[dS)?) annotationsN) defaultdict)AnyCallableDictFinalListMappingOptionalProtocolruntime_checkableSequenceSetTuple TYPE_CHECKINGTypeVarUnion)Self fake_tensor) _beartype io_adapter)infra)decomposition_tablepatcher registration serializationr) diagnosticsz Final[int]_DEFAULT_OPSET_VERSIONz)https://github.com/pytorch/pytorch/issueszreport_dynamo_export.sariflc@s&eZdZUdZded<dZded<dS)ONNXFakeContextaVA dataclass used to store context for model export using FakeTensor. This dataclass stores the FakeTensorMode instance used to convert real tensors and model parameters into fake tensors. This :attr:`ONNXFakeContext.fake_mode` is reused internally during tracing of a :class:`torch.nn.Module` into a FX :class:`GraphModule`. zfake_tensor.FakeTensorMode fake_modeNz7Optional[Tuple[Union[str, io.BytesIO, Dict[str, Any]]]]state_dict_paths)__name__ __module__ __qualname____doc____annotations__r#r)r)O/var/www/html/venv/lib/python3.8/site-packages/torch/onnx/_internal/exporter.pyr!_s  r!c @seZdZdZddddZeddddZd d d d Zej d dddddZ ej d$dddddddddZ ej d%dddddddZ ej d&ddddddd Z ej d!dd"d#ZdS)' OnnxRegistryaRegistry for ONNX functions. The registry maintains a mapping from qualified names to symbolic functions under a fixed opset version. It supports registering custom onnx-script functions and for dispatcher to dispatch calls to the appropriate function. NonereturncCs@tt|_ddlm}t|_td|jd| |j dS)zInitializes the registryrrz7torch.onnx.dynamo_export only implements opset version ze for now. If you need to use a different opset version, please register them with register_custom_op.N) rlist _registry"onnxscript.function_libs.torch_librr _opset_versionwarningswarn _initiate_registry_from_torchlibZdefault_registry)selfrr)r)r*__init__xs  zOnnxRegistry.__init__intcCs|jS)zThe ONNX opset version the exporter should target. Defaults to the latest supported ONNX opset version: 18. The default version will increment over time as ONNX continues to evolve.)r2r6r)r)r* opset_versionszOnnxRegistry.opset_versionztorchlib_registry.Registry)torchlib_registrycCs~|D]p\}}tj|}|jD]&}tj||ddd}|||q"|jD]&}tj||ddd}|||qPqdS)zPopulates the registry with ATen functions from torchlib. Args: torchlib_registry: The torchlib registry to use for populating the registry. FZ onnx_functionZ op_full_nameZ is_custom is_complexTN) itemsrOpNameZfrom_qualified_nameZ overloads ONNXFunctionqualified_name _registercomplex)r6r;Z aten_nameZaten_overloads_funcinternal_name_instanceZ overload_funcsymbolic_functionZ complex_funcr)r)r*r5s$   z-OnnxRegistry._initiate_registry_from_torchlibzregistration.OpNamezregistration.ONNXFunction)internal_qualified_namerEr.cCs|j||dS)zRegisters a ONNXFunction to an operator. Args: internal_qualified_name: The qualified name of the operator to register: OpName. symbolic_function: The ONNXFunction to register. N)r0append)r6rFrEr)r)r*rBs zOnnxRegistry._registerNFzCUnion[('onnxscript.OnnxFunction', 'onnxscript.TracedOnnxFunction')]strz Optional[str]bool)function namespaceop_nameoverloadr=r.cCs8tjj|||d}tj||d|d}|||dS)aORegisters a custom operator: torch.ops.... Args: function: The onnx-sctip function to register. namespace: The namespace of the operator to register. op_name: The name of the operator to register. overload: The overload of the operator to register. If it's default overload, leave it to None. is_complex: Whether the function is a function that handles complex valued inputs. Raises: ValueError: If the name is not in the form of 'namespace::op'. rKrLrMTr<N)rr?from_name_partsr@rArB)r6rJrKrLrMr=rDrEr)r)r* register_opszOnnxRegistry.register_opz)Optional[List[registration.ONNXFunction]])rKrLrMr.cCstjj|||d}|j|S)anReturns a list of ONNXFunctions for the given op: torch.ops.... The list is ordered by the time of registration. The custom operators should be in the second half of the list. Args: namespace: The namespace of the operator to get. op_name: The name of the operator to get. overload: The overload of the operator to get. If it's default overload, leave it to None. Returns: A list of ONNXFunctions corresponding to the given name, or None if the name is not in the registry. rN)rr?rOr0get)r6rKrLrMrDr)r)r*get_op_functionss zOnnxRegistry.get_op_functionscCs|j|||d}|dk S)aReturns whether the given op is registered: torch.ops.... Args: namespace: The namespace of the operator to check. op_name: The name of the operator to check. overload: The overload of the operator to check. If it's default overload, leave it to None. Returns: True if the given op is registered, otherwise False. rNN)rR)r6rKrLrMZ functionsr)r)r*is_registered_ops zOnnxRegistry.is_registered_opzSet[str]cCsdd|jDS)z1Returns the set of all registered function names.cSsh|] }|qSr))rA).0Z op_name_classr)r)r* sz3OnnxRegistry._all_registered_ops..)r0keysr9r)r)r*_all_registered_opssz OnnxRegistry._all_registered_ops)NF)N)N)r$r%r&r'r7propertyr:r5rbeartyperBrPrRrSrWr)r)r)r*r+os&  r+c@steZdZUdZdZded<dZded<ded<dZded <dZd ed <e j dddddd dddd d d ddZ dS) ExportOptionsaOptions to influence the TorchDynamo ONNX exporter. Attributes: dynamic_shapes: Shape information hint for input/output tensors. When ``None``, the exporter determines the most compatible setting. When ``True``, all input shapes are considered dynamic. When ``False``, all input shapes are considered static. op_level_debug: Whether to export the model with op-level debug information diagnostic_options: The diagnostic options for the exporter. fake_context: The fake context used for symbolic tracing. onnx_registry: The ONNX registry used to register ATen operators to ONNX functions. NzOptional[bool]dynamic_shapesop_level_debugDiagnosticOptionsdiagnostic_optionsOptional[ONNXFakeContext] fake_contextzOptional[OnnxRegistry] onnx_registry)r[r\r`rar^zOptional[DiagnosticOptions]cCs(||_||_||_||_|p t|_dSN)r[r\r`rar]r^)r6r[r\r`rar^r)r)r*r79s zExportOptions.__init__) r$r%r&r'r[r(r\r`rarrYr7r)r)r)r*rZs     rZc@sreZdZUdZded<ded<ded<ded<d ed <d ed <d ed<ded<ded<ejddddddZdS)ResolvedExportOptionszConsolidates :class:`ExportOptions` with default values. All unspecified options from :class:`ExportOptions` are assigned a default value. This is an internal class and its API may be changed at any time without notice. rIr[r\r]r^r!r`r+raz%Dict[torch._ops.OpOverload, Callable]rzFtorch.onnx._internal.fx.onnxfunction_dispatcher.OnnxFunctionDispatcheronnxfunction_dispatcherFXGraphExtractor fx_tracerdiagnostics.DiagnosticContextdiagnostic_contextNz-Union[ExportOptions, 'ResolvedExportOptions']HOptional[Union[torch.nn.Module, Callable, torch_export.ExportedProgram]])optionsmodelc Csddlm}m}m}t|tr|j|_|j|_|j|_|j |_ t|t j rxt|j |j sxd}t|}tt||j||j |_ |j|_|j|_|j|_|j|_ntd}tjdddddd } | |jd |_| |jt|_t|t j r| |_ n ||_ | |j d|_ |d tj|j|_| |jt|_t|j|_dd lm} | |jd |_| |j|j|_t |D].} | !d sxt"|| sxt#d| dqxdS)Nr)rdynamo_graph_extractortorch_export_graph_extractorzP'model' of type 'ExportedProgram' is only supported with 'TorchExport' FX TracerTz Optional[T]zUnion[T, Callable[[], T]])valuefallbackr.cSs|dk r |St|r|S|Srb)callable)rorpr)r)r*resolves z/ResolvedExportOptions.__init__..resolveFztorch.onnx.dynamo_export)rd_zUnresolved option '')$torch.onnx._internal.fxrrlrm isinstancercr[r\r^r` torch_exportExportedProgramrfZ TorchExportInvalidExportOptionsError ONNXProgram _from_failurerhrardrrrrYr]Z DynamoExportZDiagnosticContexttorch __version__r+Z(create_onnx_friendly_decomposition_tableZOnnxFunctionDispatcherdir startswithhasattrAssertionError) r6rjrkrrlrmmessageernrrrdkeyr)r)r*r7gsj         zResolvedExportOptions.__init__)N)r$r%r&r'r(rrYr7r)r)r)r*rcJs rcc cs~ddlm}ddlm}|jtj |dddd}t }t |d}|| |VW5QRXW5QRXt |j |_ dS) aYEnable fake mode for the duration of the context. Internally it instantiates a :class:`torch._subclasses.fake_tensor.FakeTensorMode` context manager that converts user input and model parameters into :class:`torch._subclasses.fake_tensor.FakeTensor`. A :class:`torch._subclasses.fake_tensor.FakeTensor` is a :class:`torch.Tensor` with the ability to run PyTorch code without having to actually do computation through tensors allocated on a ``meta`` device. Because there is no actual data being allocated on the device, this API allows for exporting large models without the actual memory footprint needed for executing it. It is highly recommended to enable fake mode when exporting models that are too large to fit into memory. Returns: A :class:`ONNXFakeContext` object that must be passed to :func:`dynamo_export` through the :attr:`ExportOptions.fake_context` argument. Example:: # xdoctest: +REQUIRES(env:TORCH_DOCTEST_ONNX) >>> import torch >>> import torch.onnx >>> class MyModel(torch.nn.Module): # Dummy model ... def __init__(self) -> None: ... super().__init__() ... self.linear = torch.nn.Linear(2, 2) ... def forward(self, x): ... out = self.linear(x) ... return out >>> with torch.onnx.enable_fake_mode() as fake_context: ... my_nn_module = MyModel() ... arg1 = torch.randn(2, 2, 2) # positional input 1 >>> export_options = torch.onnx.ExportOptions(fake_context=fake_context) >>> onnx_program = torch.onnx.dynamo_export( ... my_nn_module, ... arg1, ... export_options=export_options ... ) >>> # Saving model WITHOUT initializers >>> onnx_program.save("my_model_without_initializers.onnx") >>> # Saving model WITH initializers >>> onnx_program.save("my_model_with_initializers.onnx", model_state=MyModel().state_dict()) .. warning:: This API is experimental and is *NOT* backward-compatible. rr)ShapeEnvF)Zallow_scalar_outputsZallow_dynamic_output_shape_ops)Zallow_non_fake_inputsZ shape_env)r"N)torch._subclassesrZ%torch.fx.experimental.symbolic_shapesrZFakeTensorModer|Z_guardsZdetect_fake_moderZONNXTorchPatcherr!tuplepathsr#)rrr"Zpatcher_contextr`r)r)r*enable_fake_modes 2     rc@s"eZdZdZddddddZdS) ONNXProgramSerializerzProtocol for serializing an ONNX graph into a specific format (e.g. Protobuf). Note that this is an advanced usage scenario.rzio.BufferedIOBaser, onnx_program destinationr.cCsdS)aaProtocol method that must be implemented for serialization. Args: onnx_program: Represents the in-memory exported ONNX model destination: A binary IO stream or pre-allocated buffer into which the serialized model should be written. Example: A simple serializer that writes the exported :py:obj:`onnx.ModelProto` in Protobuf format to ``destination``: :: # xdoctest: +REQUIRES(env:TORCH_DOCTEST_ONNX) >>> import io >>> import torch >>> import torch.onnx >>> class MyModel(torch.nn.Module): # Dummy model ... def __init__(self) -> None: ... super().__init__() ... self.linear = torch.nn.Linear(2, 2) ... def forward(self, x): ... out = self.linear(x) ... return out >>> class ProtobufONNXProgramSerializer: ... def serialize( ... self, onnx_program: torch.onnx.ONNXProgram, destination: io.BufferedIOBase ... ) -> None: ... destination.write(onnx_program.model_proto.SerializeToString()) >>> model = MyModel() >>> arg1 = torch.randn(2, 2, 2) # positional input 1 >>> torch.onnx.dynamo_export(model, arg1).save( ... destination="exported_model.onnx", ... serializer=ProtobufONNXProgramSerializer(), ... ) Nr))r6rrr)r)r* serializes(zONNXProgramSerializer.serializeN)r$r%r&r'rr)r)r)r*rsrc@s(eZdZdZejddddddZdS) ProtobufONNXProgramSerializerz"Serializes ONNX graph as Protobuf.rzrr,rcCs2ddl}t|j|jstd||jdS)Nrz1onnx_program.ModelProto is not an onnx.ModelProto)onnxrv model_proto ModelProto ValueErrorwriteSerializeToStringr6rrrr)r)r*r5sz'ProtobufONNXProgramSerializer.serializeN)r$r%r&r'rrYrr)r)r)r*r2src@s@eZdZUdZded<ddddZejdd d d d d ZdS)"LargeProtobufONNXProgramSerializerzSerializes ONNX graph as Protobuf. Fallback to serializing as Protobuf with external data for models larger than 2GB. z Final[str]_destination_pathrH)destination_pathcCs ||_dSrb)r)r6rr)r)r*r7Hsz+LargeProtobufONNXProgramSerializer.__init__rzrr,rcCsBddl}|jtkr(||j|jn|j|j|jddddS)zQ`destination` is ignored. The model is saved to `self._destination_path` instead.rNT)Zsave_as_external_dataZall_tensors_to_one_file)rrZByteSize_PROTOBUF_SIZE_MAX_LIMITZ save_modelrrr)r)r*rKsz,LargeProtobufONNXProgramSerializer.serializeN) r$r%r&r'r(r7rrYrr)r)r)r*r@s rc@sXeZdZUdZdZded<dZded<dZded<ej dddd dddd d d Z dS) ONNXRuntimeOptionsaAOptions to influence the execution of the ONNX model through ONNX Runtime. Attributes: session_options: ONNX Runtime session options. execution_providers: ONNX Runtime execution providers to use during model execution. execution_provider_options: ONNX Runtime execution provider options. Nz0Optional[Sequence['onnxruntime.SessionOptions']]session_optionsz:Optional[Sequence[Union[str, Tuple[str, Dict[Any, Any]]]]]execution_providersz"Optional[Sequence[Dict[Any, Any]]]execution_provider_optionsrrrcCs||_||_||_dSrbr)r6rrrr)r)r*r7ss zONNXRuntimeOptions.__init__) r$r%r&r'rr(rrrrYr7r)r)r)r*r_s    rc @sneZdZUdZded<ded<ded<ded <d ed <d ed <ded<ded<ejddddddddddddddddZdddd dd!d d d"d#d$Ze dd%d&d'Z e dd%d(d)Z e dd%d*d+Z e dd%d,d-Z ejdd.dd/d0d1d2ZejdJd dd3d4d5d6Zejd7ddd8d9d:d;dd?d@ZejdAd=dBdCdDZedEddFdGdHdIZdS)KrzaAn in-memory representation of a PyTorch model that has been exported to ONNX. Args: model_proto: The exported ONNX model as an :py:obj:`onnx.ModelProto`. input_adapter: The input adapter used to convert PyTorch inputs into ONNX inputs. output_adapter: The output adapter used to convert PyTorch outputs into ONNX outputs. diagnostic_context: Context object for the SARIF diagnostic system responsible for logging errors and metadata. fake_context: The fake context used for symbolic tracing. export_exception: The exception that occurred during export, if any. model_signature: The model signature for the exported ONNX graph. zFinal[onnx.ModelProto] _model_protozFinal[io_adapter.InputAdapter]_input_adapterzFinal[io_adapter.OutputAdapter]_output_adapterz$Final[diagnostics.DiagnosticContext]_diagnostic_contextz Final[Optional[ONNXFakeContext]] _fake_contextzFinal[Optional[Exception]]_export_exceptionz2Final[Optional[torch.export.ExportGraphSignature]]_model_signaturezOFinal[Optional[Union[torch.nn.Module, Callable, torch_export.ExportedProgram]]] _model_torchN)r`export_exceptionmodel_signature model_torchzonnx.ModelProtozio_adapter.InputAdapterzio_adapter.OutputAdapterrgr_zOptional[Exception]z+Optional[torch.export.ExportGraphSignature]ri)r input_adapteroutput_adapterrhr`rrrc Cs4||_||_||_||_||_||_||_||_dSrb)rrrrrrrr) r6rrrrhr`rrrr)r)r*r7szONNXProgram.__init__)model_with_state_dictrjrzOptional[ONNXRuntimeOptions])argsrrjkwargsr.c Ost}|p|j}|jr|t}tdt j |d}t |t jjrX|}nt |tjrl|j}nd}|j||dn |j}ddl} |j|d|i|} |pt}|jp| } | j|| d} dd t| | D} | d| W5QRSQRXdS) aRuns the ONNX model using ONNX Runtime Args: args: The positional inputs to the model. kwargs: The keyword inputs to the model. model_with_state_dict: The PyTorch model to fetch state from. Required when :func:`enable_fake_mode` is used to extract real initializers as needed by the ONNX graph. options: The options to use for running the model with ONNX Runtime. Returns: The model output as computed by ONNX Runtime zCannot run model directly from `ONNXProgram` because the model was exported using `enable_fake_mode`. The model will be serialized to disk using a temporary folder ({tmpdir_path}) to populate the model with initializers before being execution.z model.onnxN) model_staterr) providerscSs i|]\}}|j|jddqS)T)force)namenumpy)rTkvr)r)r* sz(ONNXProgram.__call__..) contextlib ExitStackrr` enter_contexttempfileTemporaryDirectoryr3r4ospathjoinrvr|nnModuleZ state_dictrwrxsaverr onnxruntimeadapt_torch_inputs_to_onnxrrZget_available_providersZInferenceSessionzip get_inputsrun)r6rrjrrstackZ tmpdir_path onnx_modelrrZ onnx_inputrZ ort_sessionZonnxruntime_inputr)r)r*__call__sD        zONNXProgram.__call__r-cCs|jdk r|j|jS)z8The exported ONNX model as an :py:obj:`onnx.ModelProto`.N)rrr9r)r)r*rs zONNXProgram.model_protocCs|jS)a-The model signature for the exported ONNX graph. This information is relevant because ONNX specification often differs from PyTorch's, resulting in a ONNX graph with input and output schema different from the actual PyTorch model implementation. By using the model signature, the users can understand the inputs and outputs differences and properly execute the model in ONNX Runtime. NOTE: Model signature is only available when the ONNX graph was exported from a :class:`torch.export.ExportedProgram` object. NOTE: Any transformation done to the model that changes the model signature must be accompanied by updates to this model signature as well through :class:`InputAdaptStep` and/or :class:`OutputAdaptStep`. Example: The following model produces different sets of inputs and outputs. The first 4 inputs are model parameters (namely conv1.weight, conv2.weight, fc1.weight, fc2.weight), and the next 2 inputs are registered buffers (namely my_buffer2, my_buffer1) and finally the last 2 inputs are user inputs (namely x and b). The first output is a buffer mutation (namely my_buffer2) and the last output is the actual model output. >>> import pprint >>> class CustomModule(torch.nn.Module): ... def __init__(self): ... super().__init__() ... self.my_parameter = torch.nn.Parameter(torch.tensor(2.0)) ... self.register_buffer("my_buffer1", torch.tensor(3.0)) ... self.register_buffer("my_buffer2", torch.tensor(4.0)) ... self.conv1 = torch.nn.Conv2d(1, 32, 3, 1, bias=False) ... self.conv2 = torch.nn.Conv2d(32, 64, 3, 1, bias=False) ... self.fc1 = torch.nn.Linear(9216, 128, bias=False) ... self.fc2 = torch.nn.Linear(128, 10, bias=False) ... def forward(self, x, b): ... tensor_x = self.conv1(x) ... tensor_x = torch.nn.functional.sigmoid(tensor_x) ... tensor_x = self.conv2(tensor_x) ... tensor_x = torch.nn.functional.sigmoid(tensor_x) ... tensor_x = torch.nn.functional.max_pool2d(tensor_x, 2) ... tensor_x = torch.flatten(tensor_x, 1) ... tensor_x = self.fc1(tensor_x) ... tensor_x = torch.nn.functional.sigmoid(tensor_x) ... tensor_x = self.fc2(tensor_x) ... output = torch.nn.functional.log_softmax(tensor_x, dim=1) ... ( ... self.my_buffer2.add_(1.0) + self.my_buffer1 ... ) # Mutate buffer through in-place addition ... return output >>> inputs = (torch.rand((64, 1, 28, 28), dtype=torch.float32), torch.randn(3)) >>> exported_program = torch.export.export(CustomModule(), args=inputs).run_decompositions({}) >>> onnx_program = torch.onnx.dynamo_export(exported_program, *inputs) >>> pprint.pprint(onnx_program.model_signature) ExportGraphSignature(input_specs=[InputSpec(kind=, arg=TensorArgument(name='p_conv1_weight'), target='conv1.weight', persistent=None), InputSpec(kind=, arg=TensorArgument(name='p_conv2_weight'), target='conv2.weight', persistent=None), InputSpec(kind=, arg=TensorArgument(name='p_fc1_weight'), target='fc1.weight', persistent=None), InputSpec(kind=, arg=TensorArgument(name='p_fc2_weight'), target='fc2.weight', persistent=None), InputSpec(kind=, arg=TensorArgument(name='b_my_buffer2'), target='my_buffer2', persistent=True), InputSpec(kind=, arg=TensorArgument(name='b_my_buffer1'), target='my_buffer1', persistent=True), InputSpec(kind=, arg=TensorArgument(name='x'), target=None, persistent=None), InputSpec(kind=, arg=TensorArgument(name='b'), target=None, persistent=None)], output_specs=[OutputSpec(kind=, arg=TensorArgument(name='add'), target='my_buffer2'), OutputSpec(kind=, arg=TensorArgument(name='_log_softmax'), target=None)]) )rr9r)r)r*rs]zONNXProgram.model_signaturecCs|jS)z2The diagnostic context associated with the export.)rr9r)r)r*rh^szONNXProgram.diagnostic_contextcCs|jS)z,The fake context associated with the export.)rr9r)r)r*r`dszONNXProgram.fake_context)rz>> import torch >>> import torch.onnx >>> from typing import Dict, Tuple >>> def func_nested_input( ... x_dict: Dict[str, torch.Tensor], ... y_tuple: Tuple[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]] ... ): ... if "a" in x_dict: ... x = x_dict["a"] ... elif "b" in x_dict: ... x = x_dict["b"] ... else: ... x = torch.randn(3) ... ... y1, (y2, y3) = y_tuple ... ... return x + y1 + y2 + y3 >>> x_dict = {"a": torch.tensor(1.)} >>> y_tuple = (torch.tensor(2.), (torch.tensor(3.), torch.tensor(4.))) >>> onnx_program = torch.onnx.dynamo_export(func_nested_input, x_dict, y_tuple) >>> print(x_dict, y_tuple) {'a': tensor(1.)} (tensor(2.), (tensor(3.), tensor(4.))) >>> print(onnx_program.adapt_torch_inputs_to_onnx(x_dict, y_tuple, model_with_state_dict=func_nested_input)) (tensor(1.), tensor(2.), tensor(3.), tensor(4.)) .. warning:: This API is experimental and is *NOT* backward-compatible. N(model_with_state_dict must be specified.rk)rrrapply)r6r model_args model_kwargsr)r)r*rjsA z&ONNXProgram.adapt_torch_inputs_to_onnxz/Sequence[Union[torch.Tensor, int, float, bool]]) model_outputsrr.cCs*|p|j}|dk std|jj||dS)aConverts the PyTorch model outputs to exported ONNX model outputs format. Due to design differences, input/output format between PyTorch model and exported ONNX model are often not the same. E.g., None is allowed for PyTorch model, but are not supported by ONNX. Nested constructs of tensors are allowed for PyTorch model, but only flattened tensors are supported by ONNX, etc. The actual adapting steps are associated with each individual export. It depends on the PyTorch model, the particular set of model_args and model_kwargs used for the export, and export options. This method replays the adapting steps recorded during export. Args: model_outputs: The PyTorch model outputs. model_with_state_dict: The PyTorch model to get extra state from. If not specified, the model used during export is used. Required when :func:`enable_fake_mode` is used to extract real initializers as needed by the ONNX graph. Returns: PyTorch model outputs in exported ONNX model outputs format. Example:: # xdoctest: +REQUIRES(env:TORCH_DOCTEST_ONNX) >>> import torch >>> import torch.onnx >>> def func_returning_tuples(x, y, z): ... x = x + y ... y = y + z ... z = x + y ... return (x, (y, z)) >>> x = torch.tensor(1.) >>> y = torch.tensor(2.) >>> z = torch.tensor(3.) >>> onnx_program = torch.onnx.dynamo_export(func_returning_tuples, x, y, z) >>> pt_output = func_returning_tuples(x, y, z) >>> print(pt_output) (tensor(3.), (tensor(5.), tensor(8.))) >>> print(onnx_program.adapt_torch_outputs_to_onnx(pt_output, model_with_state_dict=func_returning_tuples)) [tensor(3.), tensor(5.), tensor(8.)] .. warning:: This API is experimental and is *NOT* backward-compatible. Nrrk)rrrr)r6rrr)r)r*adapt_torch_outputs_to_onnxs 7 z'ONNXProgram.adapt_torch_outputs_to_onnxT)include_initializersr serializerzUnion[str, io.BufferedIOBase]rIz$Optional[Union[Dict[str, Any], str]]zOptional[ONNXProgramSerializer]r,)rrrrr.c Cs|dks|dkstd|dkr:t|tr4t|}nt}g}|r|dk rlt|ttfs`td||q|jr|jjr|jjD]$}||krqt j |r||qn(t |j jj}t|D]}|j jjd=q|r&t|tstdt j |\} } | pt } | } t| | dt||j npt|trVt|d} ||| W5QRXn@z|||Wn.tk r} ztd | W5d} ~ XYnXdS) aPSaves the in-memory ONNX model to ``destination`` using specified ``serializer``. Args: destination: The destination to save the ONNX model. It can be either a string or a file-like object. When used with ``model_state``, it must be a string with a full path to the destination. If `destination` is a string, besides saving the ONNX model into a file, model weights are also stored in separate files in the same directory as the ONNX model. E.g. for `destination="/path/model.onnx"`, the initializers are saved in "/path/" folder along with "onnx.model". include_initializers: Whether to include initializers in the ONNX graph as external data. Cannot be combined with `model_state_dict`. model_state: The state_dict of the PyTorch model containing all weights on it. It can be either a string with the path to a checkpoint or a dictionary with the actual model state. The supported file formats are the same as those supported by `torch.load` and `safetensors.safe_open`. Required when :func:`enable_fake_mode` is used but real initializers are needed on the ONNX graph. serializer: The serializer to use. If not specified, the model will be serialized as Protobuf. TNzCCannot specify both `include_initializers=False` and `model_state`.zMmodel_state must be a path to the model's state_dict or the actual state_dictrzK`destination` must be a string with a path when `model_state` is specified.wbz'destination' should be provided as a path-like string when saving a model larger than 2GB. External tensor data will be saved alongside the model on disk.)rrvrHrrdictrGrr#rrexistslenrgraph initializerrange RuntimeErrorsplitgetcwdfx_serializationZsave_model_with_external_dataropenrr)r6rrrrZ_model_state_filesrZinitializer_countrsrZdestination_filenameZonnx_model_locationfexcr)r)r*rsn         zONNXProgram.saverH)rr.cCs6|ds&d|}t|t||j|dS)aLSaves the export diagnostics as a SARIF log to the specified destination path. Args: destination: The destination to save the diagnostics SARIF log. It must have a `.sarif` extension. Raises: ValueError: If the destination path does not end with `.sarif` extension. z.sarifz0'destination' must have a .sarif extension, got N)endswithlogfatalrrhdump)r6rrr)r)r*save_diagnosticsIs   zONNXProgram.save_diagnostics Exceptionr)rrhr.cCs&ddl}t|tt||dS)aZ Creates an instance of :class:`ONNXProgram` when the export process encounters a failure. In case of a failed export, this method is used to encapsulate the exception and associated diagnostic context within an :class:`ONNXProgram` instance for easier handling and debugging. Args: export_exception: The exception raised during the export process. diagnostic_context: The context associated with diagnostics during export. Returns: An instance of :class:`ONNXProgram` representing the failed ONNX program. rN)r)rrzrr InputAdapter OutputAdapter)clsrrhrr)r)r*r{[szONNXProgram._from_failure)N)r$r%r&r'r(rrYr7rrXrrrhr`rrrr classmethodr{r)r)r)r*rzsR  "E^H<Xrzcs\eZdZdZddfdd Zejdddd d d d d Zejddd ddddZZ S)rezAbstract interface for FX graph extractor engines. This class isolates FX extraction logic from the rest of the export logic. That allows a single ONNX exporter that can leverage different FX graphs.r,r-cs"tt|_t|_dSrb)superr7rrrrrr9 __class__r)r*r7s  zFXGraphExtractor.__init__rc Union[torch.nn.Module, Callable] Sequence[Any]Mapping[str, Any]torch.fx.GraphModule)rjrkrrr.cCsdS)aHAnalyzes user ``model`` and generates a FX graph. Args: options: The export options. model: The user model. model_args: The model's positional input arguments. model_kwargs: The model's keyword input arguments. Returns: The generated FX Graph. Nr))r6rjrkrrr)r)r* generate_fxszFXGraphExtractor.generate_fxrjoriginal_model fx_modulefx_module_argscCsdS)abApplies pre-export passes to the FX graph. Pre-export passes are FX-to-FX graph transformations that make the graph more palatable for the FX-to-ONNX conversion. For example, it can be used to flatten model input/output, add explicit casts to the graph, replace/decompose operators, functionalize the graph, etc. Nr))r6rjrrrr)r)r*pre_export_passessz"FXGraphExtractor.pre_export_passes) r$r%r&r'r7abcabstractmethodrr __classcell__r)r)rr*re}s rec@s<eZdZejdddddddZdd d d Zd d ZdS)Exporterrc>Union[torch.nn.Module, Callable, torch_export.ExportedProgram]rrrjrkrrcCsN||_|jdk st||_||_||_ddlm}t|jj|j sJ| dS)Nr)fx_symbolic_graph_extractor) rjrrkrrrurrvrfZFXSymbolicTracer_assert_fake_tensor_mode)r6rjrkrrrr)r)r*r7s zExporter.__init__rzr-c Csddlm}ddlm}|jj||jtjj t |n|jj |j|j|j|j}ddlm}|j|jjd}|j||jj|jjd}|jjdk ri}|jD]\}} t| tjjs| ||<q||_||jjj} zddl m!} | "| } WnRt#k r&t$%dYn4t&k rX} zt$%d | W5d} ~ XYnXtj'j(| |jj j)|jj j*|jj|jjt+|jd d|jd W5QRW5QRW5QRSQRXW5QRXW5QRXdS) Nr)DEFAULT_EXPORT_DYNAMO_CONFIG)decomposition_skip)fx_onnx_interpreter)rh)Zfx_graph_modulerdr\) optimizerzONNXScript optimizer is not available. Skipping optimization. Please `pip install onnxscript -U` to enable post-export optimization.zONNXScript optimizer failed. Skipping optimization. PLEASE REPORT A BUG AT https://github.com/microsoft/onnxscript/issues Detail: Zgraph_signature)r`rr),Ztorch.export._tracerrurrjrhZenable_decomposition_skipsr|Z_dynamoconfigpatch dataclassesasdictrfrrkrrrZFxOnnxInterpreterrrdr\r`Z initializersr>rv _subclasses FakeTensorZto_model_protorar: onnxscriptroptimize ImportErrorr3r4rrrzrrgetattr) r6rrZ graph_modulerZfx_interpreterZonnxscript_graphZinitializers_with_real_tensorsZinitializer_namerrrrr)r)r*exportsx      zExporter.exportcCstdd|j|jf}d}t|jtjjrLtdd|j |j f}|sT|rd|j j sdt dtdd|j|jf}d}t|jtjjrtdd|j |j f}|s|r|j j rt dd S) zAAsserts that the model and its input do not contain fake tensors.cSst|tjjSrbrvr|r r xr)r)r*z3Exporter._assert_fake_tensor_mode..FcSst|tjjSrbrrr)r)r*rrzJCannot export a model with fake inputs/weights without enabling fake mode.cSst|tjot|tjj Srbrvr|ZTensorr r rr)r)r*r)s cSst|tjot|tjj Srbrrr)r)r*r0s zICannot export a model with non fake inputs/weights and enabled fake mode.N)pytreeZtree_anyrrrvrkr|rr parametersbuffersrjr`r)r6Zhas_any_fake_tensorZhas_any_fake_param_or_bufferZhas_any_non_fake_tensorsZ has_any_non_fake_param_or_bufferr)r)r*rsL  z!Exporter._assert_fake_tensor_modeN)r$r%r&rrYr7rrr)r)r)r*rsMrcs(eZdZdZdddfdd ZZS)UnsatisfiedDependencyErrorz.missing_package)rcs,d|dd|d}t|t||S)NzThe installed `z4` does not support the specified ONNX opset version z. Install a newer `z,` package or specify an older opset version.r!rr:r)r* missing_opsetls z+_assert_dependencies..missing_opsetrrr r) rar:rrZdefsZonnx_opset_versionr rvZ onnx_opsetZ all_opsetsvaluesZOpset)rr"r$rrr r)r#r*_assert_dependencies`s$   r&rzOptional[ExportOptions])rkrr.c Os|dk r$t|tr|n t||d}ntt|d}t|zt||||dWStk r}z:t}|j |d|dt }t t ||j||W5d}~XYnXdS)aQExport a torch.nn.Module to an ONNX graph. Args: model: The PyTorch model to be exported to ONNX. model_args: Positional inputs to ``model``. model_kwargs: Keyword inputs to ``model``. export_options: Options to influence the export to ONNX. Returns: An in-memory representation of the exported ONNX model. **Example 1 - Simplest export** :: class MyModel(torch.nn.Module): def __init__(self) -> None: super().__init__() self.linear = torch.nn.Linear(2, 2) def forward(self, x, bias=None): out = self.linear(x) out = out + bias return out model = MyModel() kwargs = {"bias": 3.} args = (torch.randn(2, 2, 2),) onnx_program = torch.onnx.dynamo_export( model, *args, **kwargs).save("my_simple_model.onnx") **Example 2 - Exporting with dynamic shapes** :: # The previous model can be exported with dynamic shapes export_options = torch.onnx.ExportOptions(dynamic_shapes=True) onnx_program = torch.onnx.dynamo_export( model, *args, **kwargs, export_options=export_options) onnx_program.save("my_dynamic_model.onnx") By printing input dynamic dimensions we can see the input shape is no longer (2,2,2) :: >>> print(onnx_program.model_proto.graph.input[0]) name: "arg0" type { tensor_type { elem_type: 1 shape { dim { dim_param: "arg0_dim_0" } dim { dim_param: "arg0_dim_1" } dim { dim_param: "arg0_dim_2" } } } } Nrrz@Failed to export the model to ONNX. Generating SARIF report at 'z'. SARIF is a standard format for the output of static analysis tools. SARIF logs can be loaded in VS Code SARIF viewer extension, or SARIF web viewer (https://microsoft.github.io/sarif-web-component/). Please report a bug on PyTorch Github: )rvrcrZr&rrr%_DEFAULT_FAILED_EXPORT_SARIF_LOG_PATHrhr_PYTORCH_GITHUB_ISSUES_URLrrzr{)rkrrrZresolved_export_optionsrZsarif_report_pathrr)r)r* dynamo_exports2J    r)rrrrcCs$ddlm}m}|j}|j|||j|j|jdk dj|}|j |||j|jdk dj|}| ||j|}| ||}| |||j tjjt|tjjr||||}|||}|jjt|jjt|jjt|jjt |jjt!|S)Nr)analysispasses)Zenable_dynamic_axesZallow_fake_constant)"rur*r+rhZ Decomposerr[r`rZ FunctionalizeZRemoveInputMutationZInsertTypePromotionZUnsupportedFxNodesAnalysisrdZanalyzerZlevelsERRORrvr|rrZRestoreParameterAndBufferNamesZ ModularizerfrZ append_steprZRemoveNoneInputStepZRemoveNonTensorInputStepZ+ConvertComplexToRealRepresentationInputSteprZFlattenOutputStepZ,ConvertComplexToRealRepresentationOutputStep)rjrrrr*r+rhmoduler)r)r*common_pre_export_passessZ  r.r])\ __future__rrrr iologgingrrr3 collectionsrtypingrrrrrr r r r r rrrrrtyping_extensionsrr|Z torch._opsZ torch.exportrrwZtorch.utils._pytreeutilsZ_pytreerrrZtorch.onnx._internalrrZ torch.onnx._internal.diagnosticsrrurrrrrrrr r1r;rrr r(r(r'r getLoggerr$rr] dataclassr!r+rZrccontextmanagerrrrrrrzABCrerrrrryrYr&r)r.__all__r)r)r)r*s  D        )3n I/#~0)kN