U T?h @s`ddlZddlZddlZddlZddlmZeeZddZ ddZ ddZ Gd d d Z dS) N)Conv1DcCs<|jj\}}tj||}|jjj|j_|jj|j_|S)N) weightshapetorchnnLineardataT contiguousZbias)moduleZin_sizeZout_sizelinearr Z/var/www/html/venv/lib/python3.8/site-packages/onnxruntime/transformers/quantize_helper.py_conv1d_to_linears   rcCsNtdt|jD]4}|j|}t|tr@t|}||j|<qt|qdS)zsin-place This is for Dynamic Quantization, as Conv1D is not recognized by PyTorch, convert it to nn.Linear zreplace Conv1D with LinearN)loggerdebuglistZ_modules isinstancerrconv1d_to_linear)modelnamer r r r rrs    rcCs.t|dtjdd}td|S)Nztemp.p)rsaveZ state_dictospathgetsizeremove)rsizer r r_get_size_of_pytorch_model's rc@s,eZdZeejfddZedddZdS)QuantizeHelpercCsLt|tjj|tjjh|d}tdt|tdt||S)z{ Usage: model = quantize_model(model) TODO: mix of in-place and return, but results are different )dtypez'Size of full precision Torch model(MB):z"Size of quantized Torch model(MB):) rrZ quantizationquantize_dynamicrrrinfor)rr Zquantized_modelr r rquantize_torch_model/s z#QuantizeHelper.quantize_torch_modelFcCsddlm}ddlm}||jjdddtdtj |d||||dt j j id td |td tj |ddS) Nr)Path)r!T)parentsexist_okz&Size of full precision ONNX model(MB):rZDefaultTensorType)use_external_data_formatZ extra_optionszquantized model saved to:z!Size of quantized ONNX model(MB):)pathlibr$Zonnxruntime.quantizationr!parentmkdirrr"rrronnxZ TensorProtoFLOAT)Zonnx_model_pathZquantized_model_pathr'r$r!r r rquantize_onnx_model<s   z"QuantizeHelper.quantize_onnx_modelN)F)__name__ __module__ __qualname__ staticmethodrZqint8r#r-r r r rr.s r) loggingrr+rZtransformers.modeling_utilsr getLoggerr.rrrrrr r r rs