U yh=@s~ddlZddlmZmZddlmZmZddlmZddl m Z m Z m Z m Z mZmZmZmZmZmZddlmZmZmZddlmZdd lmZdd lmZmZmZm Z m!Z!m"Z"m#Z#m$Z$eee#e%d d d Z&eeee%e%e%e%e%e%e'e'e#e%ed ddZ(eeeee%e%ffeeee%e%ffee%edddZ)eeeee"ej*e+fe"ej*e'fe%edddZ,eee"ee efeeee%eee%e$efe"ee efd ddZ-eeeeedddZ.eee'e/dddZ0e"ee efe#e"ee efeeee%ed d!d"Z1e"ee efe#e"ee efeeee%ed#d$d%Z2d)e%ee%eee%eeeffee/e#ee%e$efed& d'd(Z3dS)*N) GraphModulemap_arg)GraphNode)get_new_attr_name_with_prefix) $get_node_first_input_and_output_typegetattr_from_fqnNodeInputOrOutputTypereturn_first_non_observer_nodeget_number_of_non_param_argsget_target_type_str get_arg_indices_of_inputs_to_logget_node_input_qparamsop_type_supports_shadowingget_normalized_nth_input)NSSingleResultValuesType NSSubgraphNSNodeTargetType)get_node_type_to_io_type_map)_is_activation_post_process)DictTupleCallableListAnyUnionOptionalSet)nodegmreturncCs`d}t|dr\|}|jdkrLt|jts,tt||j}t|rLt||d}|j |j d}|S)N_node_name_to_scope call_moduler) hasattrop isinstancetargetstrAssertionErrorr rrr"name)rr fqnZnode_to_use_for_fqnmoduler-M/var/www/html/venv/lib/python3.8/site-packages/torch/ao/ns/fx/graph_passes.py_maybe_get_fqn s    r/) rr logger_clslogger_node_name_suffix ref_node_name model_nameref_nameref_node_target_type results_typeindex_within_arg index_of_argr+r!c  Cs\t|j||} t||} |||j||| ||| | | }t|| ||jd| |fi}|S)z Given a starting graph of prev_node -> node -> next_node This function creates a new logger_cls obj and adds it after node, resulting in prev_node -> node -> logger_obj -> next_node r#)rr*r setattrgraph create_node)rr r0r1r2r3r4r5r6r7r8r+Zlogger_node_nameZ target_typeZ logger_objZ logger_noder-r-r._insert_logger_after_node/s.  r<)r *node_to_instrument_inputs_to_ref_node_name+node_to_instrument_outputs_to_ref_node_namer0r3r!cst}it|}fdd}|jjD]}|jdkrT|tt||d|q*||ksf||krt ||} ||krF||\} } t |} | D]} t||| }t |t kr|j }t|||d|j || | tjjd| | d |j <qt |tjjjkrt|D]@\}}|j }t|||d|j || | tjj|| | d |j <qqq||||j <||kr||\} } t|j ||d|j || | tjjdd| d |j <q*||||j <q*t||}|S)z Takes the graph of gm, adds loggers to the output of each node in nodes_to_instrument. Returns a GraphModule with the new graph. cst|fddS)Ncs |jSNr*renvr-r.hz8add_loggers_to_model..load_arg..rarBr-r.load_arggsz&add_loggers_to_model..load_argoutputrZ _ns_logger_r7r8r+)rdict named_modulesr:nodesr%rJrrr/rtyperr*r<r NODE_INPUTvaluetorchZfxZimmutable_collectionsZimmutable_list enumerate node_copy NODE_OUTPUTr)r r=r>r0r3Z new_graphmodulesrIrr+r4Z ref_node_typeZarg_indices_to_logZ node_arg_idxZnode_argZ prev_nodearg_idxargZnew_gmr-rBr.add_loggers_to_modelVs              rY) prev_node_cnode_agm_bgraph_cscale zero_pointdtype_cast_namer!c Cs~t|jd|}t||||d|di|}t|jd|} t|| ||d| di| } |dtj||| tjfi|S)NZ _input_scale_get_attrr-Z_input_zero_point_ call_function)rr*r9r;rRquantize_per_tensorZquint8) rZr[r\r]r^r_r`Zscale_node_nameZ scale_nodeZzero_point_node_nameZzero_point_noder-r-r. _insert_quantize_per_tensor_nodesF    rd) r[node_crZgm_ar\r]node_name_prefixr0node_type_to_io_type_mapr!c Cspd} d} d} d} d} d}t||||\}}t||||\}}|tjkrP|tjksx|tjkrd|tjksx|tjkr|tjkrtj} n||kr|tjkrtj j } n|tjkr|tjkrt |||}|dk rtj } |\} }nN|tjkr|tjkrd} tj } n.td|d|d|d|dt|trt||}| r~| dk rh|dk rht||||| ||S|d| |fi|SnL| r|d| || fi|S| st| }t||||d ||fi|Snt|trXg}|D]r}t||}| r|d| |fi|}||n:| s t| }t||||d ||fi|}||q|Std t|d dS) a Given a starting graph C (derived from graph B) of ... -> prev_node_c -> node_c -> ... And a corresponding related node_a, inserts the correct dtype cast node after prev_node_c to cast into the dtype expected by node_a, resulting in: dtype_cast / ... -> prev_node_c -> node_c -> ... For example, if node_c is an int8 op and node_a is an fp32 op, this function will insert a dequant. Ntozdtype cast from  z to z needs to be implementedrb call_methodr#ztype fz is not handled)rr FP32INT8ZFP16Z FP32_OR_INT8rR dequantizeUNKNOWNnnZIdentityrrcZfloat16r) format_noder&rrrdr;r9listappendrO)r[rerZrfr\r]rgr0rhZ dtype_cast_opZdtype_cast_mod_clsZdtype_cast_methodZdtype_cast_method_dtypeZdtype_cast_scaleZdtype_cast_zero_pointnode_input_type_aZ_node_output_type_aZnode_input_type_cZ_node_output_type_cnode_a_input_qparamsZnew_dtype_cast_nameZdtype_cast_modresultsZprev_node_c_innerZnew_dtype_cast_noder-r-r._insert_dtype_cast_after_nodes             rw)r[rfr\r]r!c CsD|jdkr^t|jd|}t||j}t|r:|}t|||| |j|di|}|S|jdkr$|jdkst d|jd|jdkrt t ||d |||}t|jd|}| |j|j|fi|}|St t ||d |||}t|jd|}| |j|j|t ||d fi|}|Snt d | d |jdd S)z+ Simple copy of node_a to graph_c. ra _shadow_copy_r-rkrnriztarget  is not implementedrnrrzhandling of node z with op N)r%rr*r r'rRZ is_tensordetachr9r;r)_copy_node_from_a_to_crrq)r[rfr\r]Znode_a_copy_nameZ node_a_objZ node_a_copyZarg_copyr-r-r.r|Bsn          r|) subgraph_arfnum_non_param_args_node_ar!c Cs*g}|j}||jkr,||t||d}q |||dd}|D]}||dkr^|nd}|j|dd}|dk r|\} } n|j|j} } d} | t| kr| dkrn&| dkr|dkrn|| | |sd S| d7} q| D]@} | dkrn*| dkr|dkrn|| |sd S| d7} qqJdS) z This function returns `False` if the input subgraph cannot be copied by `_insert_copy_of_subgraph_a_after_input_node_c`. This usually means that there is a corner case logic for which copy is not yet implemented. rcSsht|tr._can_insertrTZnormalize_to_only_use_kwargsNF) end_node start_nodersrreversenormalized_argumentsargskwargslenvalues) r}rfr~rNcur_noderr[Zlocal_num_non_param_args_node_anorm_args_kwargs norm_args norm_kwargscur_idx kwarg_valr-r-r._can_insert_copy_of_subgraph_apsL           r) input_node_cinput_node_c_2r}rfr\rgr!c Cst|tr|j}nt|ts t|dj}|jg}|j}||jkr\t||d}|d|q8|d} t ||| |||} t dt |D]"} || } | } t | d| |||} q| S)z* TODO(before land): real docblock rrN) r&rr:rrr)rrrinsert)_insert_copy_of_node_a_after_input_node_cranger) rrr}rfr\rgr]Z nodes_of_arZ cur_node_aZ cur_node_cZ cur_idx_arZr-r-r.-_insert_copy_of_subgraph_a_after_input_node_cs<     r)rrr[rfr\rgr!cst|tr|jnt|ts t|dj|jdd}|dk rJ|\}}n|j|j}}g} i} fdd} d} | t|kr| dkr|} n"| dkr|dk r|} n | || } | | | d7} qv| D]L\}| dkr|| |<n*| dkr|dk r|| |<n | | |<| d7} qt | } t |}|j dkrt |}t|jtsZtt|j}t|||j || | |}|S|j d kst|j |j| | |}|SdS) a Assume that node_a from graph_a has args (input, (input2)?, arg1, ...), and kwargs {kw0: kwarg0, ...} Note: input2 is optional. If it equals to None, we assume that the op has a single non-param input. If it is specified, we assume that the op has two non-param inputs. Copies the underlying values of arg1..argn and kwarg0..kwargn into gm_b, and creates the corresponding nodes in graph_c. Note: observers are ignored, so if an arg is an observer we navigate up until we find a non-observer parent. If node_a is a call_module, points the module pointed to by node_a to gm_b. Creates the copy of node_a in graph_c, with input as the first arg, and all other args and kwargs pointing to the copies of the objects in gm_b created above. An example in pictures: graph A: ======== input -------------> node_a / / / (input_2)?----------/ / / / / weight -> weight_obs / / bias ---------------- graph C (derived from B): ========================= input_node_c --> node_a_copy / / / (input_node_c_2)? / / / / weight_copy ----/ / / bias_copy ------/ rTrNcst|tr&t|}t|}|St|tttjfr<|Sttt frjD]}t|trNt dqN|St dt ddS)Nz/handling of Node inside list is not implementedzhandling for kwarg of type rz) r&rr r|intfloatrRZdtyperrrr)rO)rXrrfr\r]rr-r. _copy_arg.s   z<_insert_copy_of_node_a_after_input_node_c.._copy_argrr#)rbrk)r&rr:rrr)rrrrrsitemsrrr%r'r(r r9r;)rrr[rfr\rgrrrnew_argsZ new_kwargsrrZnew_argZ kwarg_nameZnode_a_shadows_c_nameZnew_mod_copy_nameZmod_anode_a_shadows_cr-rr.rsp3                r) name_arfname_br\matched_subgraph_pairsr0should_log_inputsrhr!c4s2|dkrt}t}it} fdd} i} i} |D]L\} }|\}}t|j|}t|j}|| ||f| |j<|| ||f| |j<q@j j D]}|j dkr| t |jd| q|| k}|| k}|s|r|r| |\}}}}n|st| |\}}}}t|jot|}|s^tdt|dt|j|d||| |j<qt|j|||\}}t|||\}}|tjko|tjko|tjko|tjk}|stdt|dt|j|d ||| |j<q|tjkr`|tjkr`t|j||}|s`tdt|dt|j|d ||| |j<qt|j|}t|||stdt|dt|j|d ||| |j<qt|j|}t|j} |r|rt|d}!t|!tr,|!j}"t |"|d |j|||t!j"j#dd| d |"j<n|t|!t$rfdd|!D}#t%|!D]>\}$}%|#|$}"t |"|d |j|||t!j"j#|$d| d |"j<qRntdt&|!d|s|r||| |j<|j}&|rt|&d}"|r t|"trt|&d}"nt|"t$r fdd|"D}"t'|j|&|"|||jd|| }'|rd}(t|'trt |'|d|(|||t!j"j#dd|d }'|'})n^t|'t$stg}*t%|'D]8\}+},t |,|d|(|||t!j"j#|+d|d }-|*(|-q|*}'|'})d}.t|j|}|dkrt|&d}.t)|'|.|||&jd}/|/|/j<|r|/}0t|0d|)krRt|0d}0q0t|)trtt*|)j}1|0j|1_+n0t|)t$st|)D]}2t*|2j}1|0j|1_+qt |/j|d|/j|||t!j,j#dd|d |/j<|r"t |j|d|j|||t!j,j#dd| d |j<q||| |j<qt-|}3|3S)a Creates a new GraphModule consisting of the graph of C, with the meaningful nodes of A shadowing the corresponding nodes of B. For example, Graph A: a0 -> op0_fp32 -> a1 -> op1_fp32 -> a2 Graph B: b0 -> op0_int8 -> b1 -> op1_int8 -> b2 matched_node_pairs: {'op0': (op0_fp32, op0_int8), 'op1': (op1_fp32, op1_int8)} Graph C (A shadows B): / dequant0 -> op0_fp32 -> logger_a_0 / dequant_1 -> op1_fp32 -> logger_a_1 / / b0 -------------> op0_int8 -> logger_b_0 --------------> op1_int8 -> logger_b_1 In a nutshell, this function does the following for each node pair: * copies the necessary attributes and modules from gm_a to gm_b, keeping names unique * adds a dtype cast op (dequant, quant, etc) * adds a copy of node_a in gm_b's graph * adds loggers to the outputs of node_a and node_b Ncst|fddS)Ncs |jSr?r@rAenv_cr-r.rDrEz6create_a_shadows_b..load_arg..rFrGrr-r.rIsz$create_a_shadows_b..load_argrJrz$skipping shadow loggers for node_b: z, start_node_a: z , unsupportedz, unknown dtype castz, unknown input qparamsz", unhandled logic in subgraph copyZ_ns_logger_b_inp_rKcsg|]}|jqSr-r@.0rXrr-r. sz&create_a_shadows_b..ztype z is not handled yetcsg|]}t|dqS)r)rr)r\r-r.r;sZ _dtype_cast_Z_ns_logger_a_inp_rrrxZ _ns_logger_a_Z _ns_logger_b_).rrrLrMrr Z base_op_noderrr:rNr%rJrrr)rprintrTr*rr rormrlrr rr/rr&rr<rrPrQrrrSrOrwrsrgetattrr2rUr)4rrfrr\rr0rrhr]rVrIZ+start_node_b_to_matched_subgraph_a_and_nameZ)end_node_b_to_matched_subgraph_a_and_nameZ match_namematchr}Z subgraph_bZref_node_type_aZref_node_type_bZnode_bZnode_b_is_start_nodeZnode_b_is_end_noder4Zall_op_types_support_shadowingrtZnode_output_type_aZnode_input_type_bZnode_output_type_bZnode_io_types_known_a_and_brur~Z fqn_base_aZ fqn_base_bZ prev_node_brZZprev_node_c_listrWrXreZdtype_cast_noder2Z input_loggerZ new_loggersZdtype_cast_idxZdtype_cast_node_innerZdtype_cast_loggerZnode_c_second_non_param_argrrZinput_logger_modZinput_logger_innerZgm_cr-)rr\r.create_a_shadows_bms$                                      r)N)4rRZtorch.fxrrZtorch.fx.graphrrZtorch.ao.quantization.fx.utilsrutilsrr r r r r rrrrZns_typesrrrZtorch.ao.ns.fx.mappingsrZtorch.ao.quantization.observerrtypingrrrrrrrrr(r/rr<rYZTensorrrdrwr|boolrrrrr-r-r-r.s 0   ( ( R     / J 3