U yh @sddlZddlZddlZddlZddlZddlmZddlmZm Z m Z m Z m Z m Z mZmZmZddlZddlmZmZmZmZmZddlmZerddlmZddlmZd d lmZm Z d d d ddgZ!Gddde"Z#Gddde#Z$Gddde#Z%ddde&e e'e e'ddd Z(ddde&e e'e e'ddd Z)ej*GdddZ+Gddde"Z,dCdd Z-ej*Gd!d"d"e+e,d#Z.ej*Gd$d%d%Z/ej*Gd&d'd'e+Z0ee.e0fZ1dDej2e'e e&d(d)dZ3e1e e'e'ge d*fd+e ed,e ed*ed-e fe e&d.fd/d0d1Z4e d2efeeed3d4d5Z5dEd7d8Z6Gd9d:d:Z7dFe eed2fe e e&efe ee e&efeee efe e e1d;dd?d@Z9e&ee e&efeee efee e&efeee efdAdBdZ:dS)GN) defaultdict) AnyCallableDictListOptionalSetTuple TYPE_CHECKINGUnion)_get_node_type BUILTIN_TYPESSUPPORTED_NODES tree_flattentree_map)ExportedProgram)Symbol)Source)ShapeEnvStrictMinMaxConstraint ConstraintDimdims dynamic_dim*refine_dynamic_shapes_from_suggested_fixesc@s\eZdZdZeddZddZddZdd Zd d Z d d Z ddZ ddZ ddZ dS)_Dimz* Metaclass for :func:`Dim` types. cCs|dkr d}|tjdkrd}|dkr:|dkr:d|dS|dkrTd|d|dS|dkrnd|d|dSd|d|d|dS) NrrzDim('z')z', max=)z', min=, max=)sysmaxsize)namemin_max_r%M/var/www/html/venv/lib/python3.8/site-packages/torch/export/dynamic_shapes.pyreadable*s z _Dim.readablecs6ttk r$tdd|jd|fddS)NzAttempted to add z to m, where an integer was expected. (Only increasing linear operations with integer coefficients are supported.)cs|SNr%xotherr%r&?z_Dim.__add__..typeintNotImplementedError__name___deriveclsr-r%r,r&__add__8s  z _Dim.__add__cCs||Sr)r%r6r%r%r&__radd__Asz _Dim.__radd__cs6ttk r$tdd|jd|fddS)NzAttempted to subtract z from r(cs|Sr)r%r*r,r%r&r.Kr/z_Dim.__sub__..r0r6r%r,r&__sub__Ds  z _Dim.__sub__cCstd|jddS)NzAttempted to negate zN. (Only increasing linear operations with integer coefficients are supported.))r3r4r6r%r%r&__rsub__Ms z _Dim.__rsub__cs>ttk sdkr,tdd|jd|fddS)NrzAttempted to multiply z with zu, where a positive integer was expected. (Only increasing linear operations with integer coefficients are supported.)cs|Sr)r%r*r,r%r&r.Zr/z_Dim.__mul__..r0r6r%r,r&__mul__Ss z _Dim.__mul__cCs||Sr)r%r6r%r%r&__rmul__\sz _Dim.__rmul__cCsddlm}t|||jS)Nr)sympify)sympyr>strr4)r7fnr>r%r%r& _derived_name_s z_Dim._derived_namecCst||tf||dS)NrootrA) _DerivedDimrBr2)r7rAr%r%r&r5dsz _Dim._deriveN)r4 __module__ __qualname____doc__ staticmethodr'r8r9r:r;r<r=rBr5r%r%r%r&r%s    rc@s(eZdZdZeddZeddZdS) _StaticDimz Meta class for static :func:`Dim` types. This class is only for setting and checking static dim constraints, and the user should never interact with it. cCs|jSr)valueselfr%r%r&minpsz_StaticDim.mincCs|jSr)rKrMr%r%r&maxtsz_StaticDim.maxN)r4rFrGrHpropertyrOrPr%r%r%r&rJhs  rJc@s0eZdZdZeddZeddZddZdS) rEa Metaclass for derived :func:`Dim` types. Currently we only support increasing linear expressions with integer coefficients. In other words, a derived Dim can always be written in the form Ax + B, where x is a regular Dim (i.e., non-derived Dim), A and B are integers, and A is positive. (In particular, the latter ensures that x < y => Ax + B < Ay + B.) These restrictions on the form of derived Dims makes the metatheory simpler: e.g., it simplifies computing ranges for derived Dims, solving for underlying regular Dims, deciding equalities between derived Dims, and so on. The function lambda x: Ax + B is expressed by `fn`, where x is a normal Dim, `root`. The range of a derived Dim is computed by mapping `fn` over the range of its `root`. cCsVddlm}|||jj}|j}|dksNtd|jd|jd|jdt|S)NrIntegerzExpected derived min value of z9 to be >= 0. Please specify an appropriate min value for (currently ).)r?rSrArDrOAssertionErrorr4r2)rNrSZ _min_symintrDr%r%r&rOs  z_DerivedDim.minc Cshddlm}|||jj}|j}|tjdks`td|jdtjdd|jd|jd t |S) NrrRrzExpected derived max value of z to be <= z.. Please specify an appropriate max value for rTrU) r?rSrArDrPr r!rVr4r2)rNrSZ _max_symintrDr%r%r&rPs (z_DerivedDim.maxcs&ttfjfdddS)Ncs|Sr))rAr*rArNr%r&r.r/z%_DerivedDim._derive..rC)rErBr2rD)rNrAr%rWr&r5s z_DerivedDim._deriveN)r4rFrGrHrQrOrPr5r%r%r%r&rEys   rErOrP)r"rOrPcCs|dkr dn|}|dkr"tjdnt|tjd}||ksPtd|d|t|tf||d}tt t dddd|_ |S) an :func:`Dim` constructs a type analogous to a named symbolic integer with a range. It can be used to describe multiple possible values of a dynamic tensor dimension. Note that different dynamic dimensions of the same tensor, or of different tensors, can be described by the same type. Args: name (str): Human-readable name for debugging. min (Optional[int]): Minimum possible value of given symbol (inclusive) max (Optional[int]): Maximum possible value of given symbol (inclusive) Returns: A type that can be used in dynamic shape specifications for tensors. Nrrz(Cannot create Dim with inconsistent min=rrXr4__main__) r r!builtinsrOrVrr2getattrinspect getmodulestackrF)r"rOrP_min_maxdimr%r%r&rs$)namesrOrPcstfdd|DS)z4 Util to create multiple :func:`Dim` types. c3s|]}t|dVqdS)rXN)r).0r"rPrOr%r& szdims..)tuple)rOrPrbr%rdr&rsc@s*eZdZUdZeed<eed<eed<dS)_ConstraintTargetz{ This represents input tensor dimensions. Don't create this class directly; instead, use :func:`dynamic_dim`. w_tensort_idraN)r4rFrGrHr__annotations__r2r%r%r%r&rgs rgcs*eZdZdZddZdfdd ZZS)_ConstraintFactoryzO Metaclass that ensures a private constructor for :class:`_Constraint` cOst|jd|jddS)N.zO has no public constructor. Please use torch.export.dynamic_dim() to create one) TypeErrorrFrG)r7argskwargsr%r%r&__call__sz_ConstraintFactory.__call__Ncst||||||Sr))superrp)r7rhriraconstraint_rangeshared debug_name __class__r%r&_createsz_ConstraintFactory._create)NN)r4rFrGrHrprw __classcell__r%r%rur&rks rkcCst||||||Sr)) _Constraintrw)rhrirarrrsrtr%r%r&_create_constraintsrzc@seZdZUdZded<dZeeed<dZee ed<ddd Z d d Z d d Z ddZ ddZddZeddZddZdS)ryz .. warning:: Do not construct :class:`_Constraint` directly, use :func:`dynamic_dim` instead. This represents constraints on input tensor dimensions, e.g., requiring them to be fully polymorphic or within some range. rrrNrsrtrcCsbddlm}ddlm}|dkr*tjd}||jj|||d@dd}t|j |j |j ||j |j S)Nrr ValueRangesrlowerupperFvrZ warn_only)%torch.fx.experimental.symbolic_shapesrtorch.utils._sympy.value_rangesr}r r!rrrrzrhrirarsrt)rNrrrr}rrr%r%r&_clone_with_range s    z_Constraint._clone_with_rangecCs |j|dS)NrrrNrr%r%r&__ge__"sz_Constraint.__ge__cCs|j|ddS)Nrrrrr%r%r&__gt__%sz_Constraint.__gt__cCs |j|dS)NrrrNrr%r%r&__le__(sz_Constraint.__le__cCs|j|ddS)Nrrrrr%r%r&__lt__+sz_Constraint.__lt__cCs tddS)NzCannot determine truth value of _Constraint. If you are trying to combine _Constraint's with logical connectives, you can specify them separately instead.)rmrMr%r%r&__bool__.sz_Constraint.__bool__cCs|j|j|jjj|jjjdSN)rirarOrPrirarrrrrrMr%r%r&serializable_spec9s z_Constraint.serializable_specc Cst|tstdt|dddlm}||jj|jj@dd}|jdkrT|j}n |jdksn|j|jksnt |j}t |j |j |j |t|j |j |j |dS)NzPA dynamic dim can be specified equal only to another dynamic dim. Equality with z is not supported.rr{Fr)rsrt) isinstanceryrmr1rrrrrrtrVrzrhrirarg)rNr-rrrrtr%r%r&__eq__Is*   z_Constraint.__eq__)rN)r4rFrGrHrjrsrrgrtr@rrrrrrrQrrr%r%r%r&rys     ry) metaclassc@s*eZdZUdZeed<ded<eed<dS) _PhantomRoota This represents the root of a derived Dim where the root does not directly specify the shape of any input dimension, but the derived Dim does. e.g., the input shapes 2*dim and dim + 1 are related via a "phantom" dim. The fields `name`, `constraint_range`, and `val` carried by a phantom root help create a symbol for it. Any derived dims with this phantom root are backed by expressions over this symbol. r"rrrvalN)r4rFrGrHr@rjr2r%r%r%r&rfs  rc@sZeZdZUdZeeefed<eed<ded<dZ e e ed<e dd Z e d d ZdS) _DerivedConstraintz This represents a derived Dim, whose root is either a regular constraint target (which directly specifies the shape of some input dimension) or a phantom root (which does so indirectly). rDrArrrNrtcCsdSr)r%rMr%r%r&rssz_DerivedConstraint.sharedcCs|j|j|jjj|jjjdSrrrMr%r%r&rs z$_DerivedConstraint.serializable_spec)r4rFrGrHr rgrrjrrtrr@rQrsrr%r%r%r&rxs   r)tindexrtc Csddlm}m}t|tjs2||jdt||dkrJ||jd||krx||jd|dd|ddd l m }dd l m }t t|t||||dtjdd d d |dS)a% .. warning:: (This feature is DEPRECATED. See :func:`Dim` instead.) :func:`dynamic_dim` constructs a :class:`_Constraint` object that describes the dynamism of a dimension ``index`` of tensor ``t``. :class:`_Constraint` objects should be passed to ``constraints`` argument of :func:`export`. Args: t (torch.Tensor): Example input tensor that have dynamic dimension size(s) index (int): Index of dynamic dimension Returns: A :class:`_Constraint` object that describes shape dynamism. It can be passed to :func:`export` so that :func:`export` does not assume static size of specified tensor, i.e. keeping it dynamic as a symbolic size rather than specializing according to size of example tracing input. Specifically :func:`dynamic_dim` can be used to express following types of dynamism. - Size of a dimension is dynamic and unbounded:: t0 = torch.rand(2, 3) t1 = torch.rand(3, 4) # First dimension of t0 can be dynamic size rather than always being static size 2 constraints = [dynamic_dim(t0, 0)] ep = export(fn, (t0, t1), constraints=constraints) - Size of a dimension is dynamic with a lower bound:: t0 = torch.rand(10, 3) t1 = torch.rand(3, 4) # First dimension of t0 can be dynamic size with a lower bound of 5 (inclusive) # Second dimension of t1 can be dynamic size with a lower bound of 2 (exclusive) constraints = [ dynamic_dim(t0, 0) >= 5, dynamic_dim(t1, 1) > 2, ] ep = export(fn, (t0, t1), constraints=constraints) - Size of a dimension is dynamic with an upper bound:: t0 = torch.rand(10, 3) t1 = torch.rand(3, 4) # First dimension of t0 can be dynamic size with a upper bound of 16 (inclusive) # Second dimension of t1 can be dynamic size with a upper bound of 8 (exclusive) constraints = [ dynamic_dim(t0, 0) <= 16, dynamic_dim(t1, 1) < 8, ] ep = export(fn, (t0, t1), constraints=constraints) - Size of a dimension is dynamic and it is always equal to size of another dynamic dimension:: t0 = torch.rand(10, 3) t1 = torch.rand(3, 4) # Sizes of second dimension of t0 and first dimension are always equal constraints = [ dynamic_dim(t0, 1) == dynamic_dim(t1, 0), ] ep = export(fn, (t0, t1), constraints=constraints) - Mix and match all types above as long as they do not express conflicting requirements r UserError UserErrorTypez0Expected tensor as input to dynamic_dim but got rz-Cannot mark 0-dimension tensors to be dynamiczCExpected the dimension passed to dynamic_dim to be in the range [0:z ] but got z., which is out of bounds for the given tensor.r{r|r~Frrt)torch._dynamo.excrrrtorchTensorZ DYNAMIC_DIMr1rarrrr}rzweakrefrefidr r!)rrrtrrrr}r%r%r&rs8E      rr)rr)rrr) constraint get_sources shape_env source_pairsderived_equalitiesphantom_symbolsc s||j|j^}|fdd|Dt|tsj|jdk r||jj|jj}|fdd|Dnt|jts||jj|jjd}nX|jj|kr||jj}n>|j |jj t j j |jjt jjjjj|jjd}|||jj<|j}|||fdS)z Updates `source_pairs`, `derived_equalities`, and `phantom_symbols` (which become fields of `EqualityConstraint`) based on a given input `constraint`. c3s|]}|fVqdSr)r%rcZ other_sourcesourcer%r&resz&_process_equalities..Nc3s|]}|fVqdSr)r%rrr%r&resr)rrrZconstraint_dim)riraextendrrrsrDrr"Z create_symbolrrZ_dynamorZConstantSourceZfxZ experimentalZsymbolic_shapesZ DimDynamicZDYNAMICrrrAappend) rrrrrrZ other_sourcesrDrAr%rr&_process_equalitiess*        r.)functreedynamic_shapesreturncs.ddfddt|f|diS)a" Customized tree_map for mapping pytrees to dynamic_shapes. For built-in types (e.g., standard collections) this behaves exactly like tree_map. OTOH for a user-defined class C registered with pytree, we cannot assume that a C containing tensors can be mapped to a C containing dynamic shapes (i.e., C may not be a polymorphic container). In that case we use the flattened form of C instead. Thus a C(**tensors) that flattens to (**tensors) will map to (**dynamic_shapes). Args: func: function to apply to each (int, float, str, bool, None, torch.Tensor) tree: input pytree dynamic_shapes: zero or more (typically one) dynamic_shapes to match Returns: output pytree mapping func to each (int, float, str, bool, None, torch.Tensor) cSs t|tkSr))r r )rr%r%r&is_leafRsz_tree_map..is_leafcsFt|}|tkr4tt||df|diS|f|SdS)Nrr)r rrZ flatten_fn)rrtypfrrr%r&rYsz_tree_map..fr)r)rrrr%rr& _tree_map:srFcCs\t|tr|}|sXt|tjjr0t|jnt|}|dk rF|ni}|j ||j S|Sr)) rrmodulernnModuler\ signatureforwardbind arguments)rrnro_is_torch_jit_tracerr%r%r& _combine_argsks  rc@s:eZdZdZddZddZddZdd Zd d d Zd S)ShapesCollectionab Builder for dynamic_shapes. Used to assign dynamic shape specifications to tensors that appear in inputs. Example:: args = ({"x": tensor_x, "others": [tensor_y, tensor_z]}) dim = torch.export.Dim(...) dynamic_shapes = torch.export.ShapesCollection() dynamic_shapes[tensor_x] = (dim, dim + 1, 8) dynamic_shapes[tensor_y] = {0: dim * 2} # This is equivalent to the following (now auto-generated): # dynamic_shapes = {"x": (dim, dim + 1, 8), "others": [{0: dim * 2}, None]} torch.export(..., args, dynamic_shapes=dynamic_shapes) cCs i|_dSr))_shapesrMr%r%r&__init__szShapesCollection.__init__cCsjt|tjstdt|t|}||jkrX|j|}||ksftd|d|n||jt|<dS)Nz'Cannot assign shape to non-tensor type z1Shapes assigned to tensor do not match: expected z, got )rrrrVr1rr)rNrshaperi_shaper%r%r& __setitem__s   zShapesCollection.__setitem__cCs$t|}||jkr|j|SdSdSr))rr)rNrrir%r%r& __getitem__s  zShapesCollection.__getitem__cCs t|jSr))lenrrMr%r%r&__len__szShapesCollection.__len__NcsNtfdd}t|||}t||}tfddjDrJtd|S)z* Generate dynamic_shapes. cs.t|}|jkr&|j|SdSdSr))rradd)rrirNt_idsr%r& find_shapes    z3ShapesCollection.dynamic_shapes..find_shapec3s|]}|kVqdSr)r%)rcri)rr%r&resz2ShapesCollection.dynamic_shapes..zSome tensors that were assigned shapes were not found in args. Maybe such tensors were copied when passing them as args? Maybe such tensors are contained in classes that were not registered with pytree?)setrranyr ValueError)rNmrnror combined_argsrr%rr&rs  zShapesCollection.dynamic_shapes)N) r4rFrGrHrrrrrr%r%r%r&r{s r)rrnrorrcs`ddlmm|dks$t|dkr(dSttigfddifddfddfd d }t||||d }t|tst|t tfst t || }|||g}D]"}|j j} | krԈ| d|_ qԈ D]Z} td d | Dr$|| n4| ^} } | rN| D]} || | kq6n || q|S)Nrrc sddlddlm}ddlmddlm}fdd}ttrj }|j kr |j d}t |j |j |j}nF|j krt|j |||j|jddd |d }||j <n |j }ttt|j||jjddd j d } t|tr| nttrXttt||jjddd j d } nBtj d } jdkr~| jk} jtjd kr| jk} | S) Nrr{) try_solver|c sxjjjdd}|}|j|}|dk rHt|dSjddjd|d|d dS) NT)integerrzExpected shape[z] = z# of input Tensor to be of the form z, where z is an integer)rrDr4rAEqrr2ZCONSTRAINT_VIOLATION)symbolexprZsolution)rrrair?tensorrr%r& root_values  "zB_process_dynamic_shapes..to_constraint..root_valuer~Fr)r"rrrrr)r?rrZtorch.utils._sympy.solverrr}rrErDr4rgrhrirarrOrPrrrrrArrJrzrLrr r!) rarrrr}rZdim_rootZroot_constraintrDr)rr%derived_constraints_with_phantom_root phantom_rootssymbols)rarr?rrr& to_constraintsp               z._process_dynamic_shapes..to_constraintc s|jkrr|j\}}|j|ks,|j|krt|j||}t|j|j|j}jd|d|d|dn|j|jf|j<dS)NzFound different definitions z and z! for the same symbolic dimension !)r4rOrPrr' INVALID_INPUT)rar#r$Zthis_Zthat_)rrboundsrr%r&check_same_bounds(s z2_process_dynamic_shapes..check_same_boundsc sRdd}t|tr|D]z\}}t|ttfrlt|trF||||}||||}|j|q|dk rjd|d|d|dqnt|tt fr0t |D]|\}}t|ttfrt|tr||||}||||}|j|q|dk rjd|d|d|dqn|dk rNjd|ddS)NcSstt|tfd|iS)NrL)rJr@r2)rrrLr%r%r&_create_static_dim8szK_process_dynamic_shapes..update_symbols.._create_static_dimzUnexpected item #z (z) in dynamic_shape z of Tensor, try None insteadzUnexpected dynamic_shape ) rdictitemsr2rr4rrrflist enumerate)rrrrrar)rrrrrr%r&update_symbols7s@         z/_process_dynamic_shapes..update_symbolscs fdd}t|||dS)NcsBt|tjr||n&|dk r>jd|dt|ddS)NzCannot associate shape z to non-tensor type z, expected None)rrrrr1)rZ dynamic_shaperrrr%r& assoc_shapeas  zB_process_dynamic_shapes..assoc_shapes..assoc_shape)r)rrrrr%r& assoc_shapes`s z-_process_dynamic_shapes..assoc_shapes)rcss|]}t|tVqdSr))rr)rcrr%r%r&resz*_process_dynamic_shapes..)rrrrrrrrrrfrVr1valuesrDr"allrr)rrnrorrrr constraintsZ$derived_constraint_with_phantom_rootZphantom_root_nameZ dynamic_dimsprimaryZothersr-r%) rrrrrrrrrr&_process_dynamic_shapessJR)     r)rcCsXi}t|ddddD]:}|dkst|tr0q|||j<t|tr|j||jj<q|S)NcSs t|tSr))rrr*r%r%r&r.r/z'_get_dim_name_mapping..)rr)rrr2r4rErD)r name_to_dimrar%r%r&_get_dim_name_mappings  r)msgrrc s ddl}ddlddlm}m}ddlm}z|dd}Wn0t k rr}z||j d|W5d}~XYnXi|dD]}|}| d |} r| d} d \} } | d |} rt | d} | d |}rt | d} t| | | d | <q|d\} }|}t|jr6t || <q|| <qt|t}D]z\}}t|t ttjfsxtt|jr||st||<|ttt|jt|trV||jjqVD]"\}}|ks||kstqifdd}t |||S)aU For working with export's dynamic shapes suggested fixes, and/or automatic dynamic shapes. Refines the given dynamic shapes spec, given a ConstraintViolation error message and the original dynamic shapes. For most cases behavior is straightforward - i.e. for suggested fixes that specialize or refine a Dim's range, or fixes that suggest a derived relation, the new dynamic shapes spec will be updated as such. e.g. Suggested fixes: dim = Dim('dim', min=3, max=6) -> this just refines the dim's range dim = 4 -> this specializes to a constant dy = dx + 1 -> dy was specified as an independent dim, but is actually tied to dx with this relation However, suggested fixes associated with derived dims can be more complicated. For example, if a suggested fix is provided for a root dim, the new derived dim value is evaluated based on the root. e.g. dx = Dim('dx') dy = dx + 2 dynamic_shapes = {"x": (dx,), "y": (dy,)} Suggested fixes: dx = 4 # specialization will lead to dy also specializing = 6 dx = Dim('dx', max=6) # dy now has max = 8 Derived dims suggested fixes can also be used to express divisibility constraints. This involves creating new root dims that aren't tied to a particular input shape. In this case the root dims won't appear directly in the new spec, but as a root of one of the dims. e.g. Suggested fixes: _dx = Dim('_dx', max=1024) # this won't appear in the return result, but dx will dx = 4*_dx # dx is now divisible by 4, with a max value of 4096 rNr)_is_supported_equivalencezSuggested fixes:rz`Suggested fixes not found in error message given to refine_dynamic_shapes_from_suggested_fixes() z(.*) = Dim\('(.*)'.*\))NNz!.* = Dim\('.*', min\=([0-9]+).*\)z.* = Dim\('.*'.*max\=([0-9]+)\)rXz = cs6|dkst|tr|S|jkr|j}t|jrt|krNt|Stt|j}|jkrr|j}n|jkst |j}j j ||\}}|}|dkrt||}|dkr|t|}|t|<|Sn|SnPt|t r2|jjkr2|jkr|jS||jj}||j<|S|S)Nrr)rr2r4Exprr@nextiter free_symbolsr"rVZpolysZ polytoolsdivrErDrA)radummyfixrrDmodulus remainderZ_dimZderived_dim_cacherZ shape_fixesr?r%r& apply_fixess8              z?refine_dynamic_shapes_from_suggested_fixes..apply_fixes)!rer?rrrrrsplitstrip Exceptionrmatchgroupr2rr>rNumberrrrrrErrVrr@rrrrDr4r)rrrrrrZshape_fixes_msgexcrrr"r_r`Z match_minZ match_maxrrootskcrr%rr&rsX+     $)NN)N)F)NNF);rZ dataclassesr\r r collectionsrtypingrrrrrrr r r rZtorch.utils._pytreer r rrrZexported_programrr?rZ torch._guardsrZfx.experimental.symbolic_shapesrr__all__r1rrJrEr@r2rr dataclassrgrkrzryrrrrrrrrrrrrr%r%r%r&s ,    C9""  k" j  4  1 J   S