U U?hL!@sddlZddlZddlZddlmZddlmZmZddlZddl Z ddl m Z ddl m Z mZddlmZeeZdeeeeejfeeeefeeeeeeeeeeeedd d d ZdS)N)Path)OptionalUnion)SymbolicShapeInference)extract_raw_data_from_modelhas_external_data)add_pre_process_metadataF) input_modeloutput_model_pathskip_optimizationskip_onnx_shapeskip_symbolic_shape auto_mergeint_maxguess_output_rankverbosesave_as_external_dataall_tensors_to_one_fileexternal_data_locationexternal_data_size_thresholdreturnc  Ks|dkr| dd}|dk s t|dk s0tdtjdd}t|}d}|stdt|tj rj|nt |}t |||||}|s|st |d}| rtj||d| | d d n t||d}t |d }zxt}||_tjj|_t|tj r6t|rtd t|\}}|t|t||}tj||d gd}~Wn.tk rztdtt YnX|}|s"|dk rt |d}| rtj||d| | d d n t||d}t|tj rt t|d}tj||d| | d d t |d}tj!"||t |}W5QRX|dkrRt|tj rH|nt |}t#|| rztj||d| | | d dn t||dS)aShape inference and model optimization, in preparation for quantization. Args: input_model: Path to the input model file or ModelProto output_model_path: Path to the output model file skip_optimization: Skip model optimization step if true. This may result in ONNX shape inference failure for some models. skip_onnx_shape: Skip ONNX shape inference. Symbolic shape inference is most effective with transformer based models. Skipping all shape inferences may reduce the effectiveness of quantization, as a tensor with unknown shape can not be quantized. skip_symbolic_shape: Skip symbolic shape inference. Symbolic shape inference is most effective with transformer based models. Skipping all shape inferences may reduce the effectiveness of quantization, as a tensor with unknown shape can not be quantized. auto_merge: For symbolic shape inference, automatically merge symbolic dims when conflict happens. int_max: For symbolic shape inference, specify the maximum value for integer to be treated as boundless for ops like slice guess_output_rank: Guess output rank to be the same as input 0 for unknown ops verbose: Logs detailed info of inference, 0: turn off, 1: warnings, 3: detailed save_as_external_data: Saving an ONNX model to external data all_tensors_to_one_file: Saving all the external data to one file external_data_location: The file location to save the external file external_data_size_threshold: The size threshold for external data NZinput_model_pathzoutput_model_path is required.z pre.quant.)prefixz&Performing symbolic shape inference...zsymbolic_shape_inferred.onnxTF)rrsize_thresholdconvert_attributezoptimized.onnxzModelProto has external data not loaded into memory, ORT cannot create session. Please load external data before calling this function. See https://onnx.ai/onnx/repo-docs/ExternalData.html for more information.ZCPUExecutionProvider) providerszYONNX Runtime Model Optimization Failed! Consider rerun with option `--skip_optimization'.zmodel_input.onnxzonnx_shape_inferred.onnx)rrlocationrr)$popAssertionErrortempfileTemporaryDirectoryrloggerinfo isinstanceonnx ModelProtoloadrZ infer_shapesstrZ save_modelsave onnxruntimeZSessionOptionsZoptimized_model_filepathZGraphOptimizationLevelZORT_ENABLE_BASICZgraph_optimization_levelr ValueErrorrZadd_external_initializerslistZSerializeToStringZInferenceSession Exceptionerror traceback format_excZshape_inferenceZinfer_shapes_pathr )r r rrrrrrrrrrrZdeprecated_kwargsZ quant_tmp_dirZ temp_pathmodelZ loaded_modelZopt_model_pathZ sess_optionZexternal_namesZexternal_valuessessZinferred_model_pathr4Z/var/www/html/venv/lib/python3.8/site-packages/onnxruntime/quantization/shape_inference.pyquant_pre_processs+               r6) NNFFFFr FrFFNr )loggingr!r0pathlibrtypingrrr&r+Z&onnxruntime.tools.symbolic_shape_inferrZ#onnxruntime.transformers.onnx_utilsrrZ quant_utilsr getLogger__name__r#r)r'boolintr6r4r4r4r5sN