U h @sddlmZddlmZmZmZmZmZddlm Z ddl m Z ddl m Z edZGd d d eZd d d dddZd d d dddZddddddZdddddddZdddddddZdd d d d!d"d#d$Zd d d d d%d&d'Zd d d d d%d(d)Zd d d d d%d*d+Zd d d d d%d,d-Zd d d d d%d.d/Zd d dd d d0d1d2Zd d d d d%d3d4Zd d d5d d6d7d8Zd d d5d d6d9d:Zd d d d5d d;dd?Z!d d d d d%d@dAZ"d d d d d%dBdCZ#d d d d d%dDdEZ$d d d d d%dFdGZ%d d d d d%dHdIZ&d d dJdKdLZ'd d d d d%dMdNZ(d d d d d%dOdPZ)d d d d d%dQdRZ*d d dSdTdUZ+d d d dVdWdXZ,d d d dVdYdZZ-d d[d\d]d^Z.d d d d d%d_d`Z/d d d d d%dadbZ0d d d d d%dcddZ1d d d d d%dedfZ2d d d d d%dgdhZ3d d d d d%didjZ4d d d d dkdldmZ5dndd d dodpdqZ6d dnd d drdsdtZ7d dndd d dudvdwZ8d d d d dxdydzZ9d d dd d d{d|d}Z:d d d d d~ddZ;d d d!dddZd d d d d"ddZ?ddddZ@d dd d dddZAdd dd dd dddZBddZCddZDddZEddZFddZGddZHddZIddZJddZKd dddddddd d d ddZLd ddd d dddZMd d d d dddZNddZOddZPd d dddd d dddZQd d d ddd d dddZRd d d dd ddddÄZSd d d ddd d dĜddƄZTd d d ddd d dĜddȄZUd d d ddd d dĜddʄZVd d d ddd d dĜdd̄ZWd d d ddd d dĜdd΄ZXd d d ddd d dĜddЄZYd d d ddd d dĜdd҄ZZddԄZ[d d d dd dd d d՜ddׄZ\d d d d d d؜ddڄZ]dd܄Z^dd d ddߜddZ_dd dd ddddZ`d d d d dddZad dnd dddZbd dnd dddZcd dnd dddZdd d dddZedddd d dddZfd ddd d d dddZgddZhdddZid d d dddZjd d d dddZkdS() annotations)ListOptionalSequenceTupleTypeVar)ir)core)mathTcseZdZfddZZS)IncompatibleTypeErrorImplcs@||_||_d|jd|j|_tt||jdS)Nzinvalid operands of type  and )type_atype_b__repr__messagesuperr__init__)selfrr __class__J/var/www/html/venv/lib/python3.8/site-packages/triton/language/semantic.pyrsz"IncompatibleTypeErrorImpl.__init__)__name__ __module__ __qualname__r __classcell__rrrrr srintz ir.builderz tl.tensor)axisbuilderreturncCs*|dkrtd|t||tjS)Nrr rz+program_id axis must be 0, 1, or 2 but got ) ValueErrortltensorZcreate_get_program_idint32r r!rrr program_idsr)cCs*|dkrtd|t||tjS)Nr#z-num_programs axis must be 0, 1, or 2 but got )r$r%r&Zcreate_get_num_programsr'r(rrr num_programs sr*ztl.dtype)a_tyb_tyr"cCs|j}|j}|j}|j}||kr0||kr,|S|S|tjjjkrN||krJ|S|S|tjjjkrl||krh|S|Std|d|dS)Nzunexpected signedness r) int_bitwidthint_signednessr%dtypeZ SIGNEDNESSZUNSIGNED TypeError)r+r,Za_rankZb_rankZa_snZb_snrrrinteger_promote_impl+sr1bool)r+r, div_or_modr"cCs|s|rtjS|s&|r,tjS|s<|rL|rFtjStjS|s\|r|rftjS|r||r|tjStjS| r| st d|d||r|j |j krt d| d| dt ||S)Nunexpected type rzCannot use /, #, or % with x because they have different signedness;this is unlikely to result in a useful answer. Cast them to the same signedness.)Zis_fp64r%float64is_fp32float32is_fp16float16is_bf16bfloat16is_intr0r.rr1)r+r,r3rrrcomputation_type_impl;s*r>None)rr allow_ptr_ar"cCsF|rB|st|||r0||kr0t|||rBt||dSN)is_ptrr is_floating)rrr@rrrcheck_ptr_type_implcs  rDFTzTuple[tl.tensor, tl.tensor])lhsrhsr!r"c Csxt|||\}}|jj}|jj}t|||t||||rp|sp|spt|||} t|| |}t|| |}||fSrA)broadcast_impl_valuetypescalarrDrBr>cast) rErFr!Z allow_lhs_ptrZ allow_rhs_ptrZarithmetic_checkr3Z lhs_sca_tyZ rhs_sca_ty ret_sca_tyrrrbinary_op_type_checking_implos     rL)inputotherr!r"cCst|||dd\}}|jj}|jj}|r<|rtd|dkrX|| |}nt |d|j }||}t||}t |||S)Nr zonly accepts size-1 tensorz2dtype must be specified when value is not a tensorrget_) rr%r&numelvalueAssertionErrorrJr$rrgetattrnamesplat)rrr/r!Z get_value_fnrrrfulls  r)rrr!r"cCsF|jrtdt|dkr"|St|j|}t||j ||S)NzCannot splat a block tensorr) rHrrlenr%rr/r& create_splatrP)rrr!r_rrrrs  r)rM dst_shape can_reorderr!r"cCsRd}|D] }||9}q|jj|kr*tdt|jj|}t||j|||S)Nr z:reshape() cannot change total number of elements in tensor) rHrr$r%rrIr&Zcreate_reshaperP)rMrrr!rsr_rrrreshape%s  r)rMr r!r"cCsZdd|jD}||d|js4t|||dSt|jj|}t| |j ||S)NcSsg|]}t|qSrr%_constexpr_to_value).0rgrrr 0szexpand_dims..r )rr!) rinsertrHrrr%rrIr&create_expand_dimsrP)rMr r!rr_rrr expand_dims/s   r)rErFrr!r"cCsX|s tdt|jdkstt|jj|jd|jdg}t||j |j |S)Nz;current implementation of `cat` always may reorder elementsr r) rrrr%rrHrIr&Z create_catrP)rErFrr!ret_typerrrcat:s "r)abr!r"cCst|||\}}|jgk}|r6t|d|}t|d|}t|jdtjrTtd}nd}|j|g}t|jj|}t | |j |j |}|rt |dgd|d}|S)NrrFrr!) rGrrrr% constexprrrHrIr&Z create_joinrPr)rrr!Z was_rank_1two new_shaperrbrrrjoinAs     r)rr!r"cCspt|jdkstt|jddks*t|jdd}t|jj|}||j \}}t ||t ||fS)Nrrr) rrrr%rrrHrIZ create_splitrPr&)rr!rrZoutLHSZoutRHSrrrsplitZs  rz Tuple[int])rMdimsr!r"cs~tjt|krtdtdd|Dttt|krJtd|tjj fdd|D}t | j ||S)Nz5permute dims must have the same length as input shapecss|]}t|VqdSrArrdrrr jszpermute..z?permute dims must be a permutation of 0, 1, ..., n-1, but were csg|]}j|qSrrrrrrrmszpermute..) rrr$sortedlistrr%rrHrIr&Z create_transrP)rMrr!rrrrpermutegs "r)rMrr!r"c Cs|js.t|j|}t||j||S|j}t|t|kr\t d|d|||krh|St |D]F\}}|||krp|dkrpt d||d|d|d|d| qpt|jj |}t| |j||S)Nz!Cannot broadcast, rank mismatch: , r z3Cannot broadcast, the expanded size of the tensor (z ) must match the existing size (z) at non-singleton dimension : ) rHrr%rr&rrPget_block_shapesrr$ enumeraterIcreate_broadcast)rMrr!r_Z src_shapeiitemrrrbroadcast_impl_shapeqs  ,rc Csd|j}|j}|rJ|sJt|j|j}t||j| |}n|s|rt|j|j}t||j| |}n|r\|r\| }| }t |t |kr t t |t |D]8}t| |jdt|jdg|}|j}| }qn`t |t |krlt t |t |D]:}t| |jdt|jdg|}|j}| }q0t |t |kst g}t|D]p\} } || } | dkr|| nH| dks| | kr|| n(tdt| dt| dt| q||kr.t|j|} t||j|| }||kr\t|j|} t||j|| }||fS)Nrr z?Cannot make_shape_compatible: incompatible dimensions at index rr)rHrr%rrIrr&rrPrrrrrrappendr$strr) rErFr!Zlhs_tyZrhs_tyZ lhs_shapeZ rhs_shape_ ret_shaperleftrightr_rrrrGsf        rGz Optional[str]Z rounding_modecCs@|dkr dS|dkrtjjS|dkr,tjjStd|ddS)NZrtneZrtzzInvalid rounding mode: z0. Supported rounding modes are 'rtne' and 'rtz'.)r ROUNDING_MODERTNEZRTZr$rrrr_str_to_rounding_modesr)rMdst_tyr!r"cCs|j}|r"t|j|j}||kr.|S|j}|j}|sJ|rVt|||S|j}|j}||krt dt |dt |t | |j |||S)Nz!Cannot bitcast data-type of size z to data-type of size )rHrr%rrIrrBrJprimitive_bitwidthr$rr&create_bitcastrPr)rMrr!src_ty src_sca_ty dst_sca_tyZsrc_bitsZdst_bitsrrrrs  rN)rMrr!fp_downcast_roundingr"cCs|j}t|tjr|j}t|tjr*|j}|rFt|j|j}||krR|S|j}|j}t |}d}| r| r|j |j kr|dkrt j j}q|t j jkrd}n$|dk rtdt|dt||s|r|jjstd|s|r2|jddk std|jd||||dS|rF| s`| rZ|s`|r~t||j||||S|r|r|r|st t |tj!|||S| o| o|j |j k}|rt|"|j|||S| o| o|j |j k} | r@t|#|j|||S|$r|$r|j%|j%ksp|j&|j&kr|'o|( } |(r|j)|} t|*| |j)} t+|| |St|,|j||| |S|-rd|$rd|(r"|j)|} t|*| |j)} t+|| |S|'rHt|.|j|||St|/|j|||S|$r|-r|(s|'st|0|j|||St|1|j|||S|2r4|$r4|j%} | d krt|3|j|||S| d kr4t+t |tj4|t|5d tj4|S|$rd|2rdt|6|j|||S|2r|2rt|7|j|||Sdstd |d |dS)NFTz]fp_downcast_rounding should be set only for truncating fp conversions. Source scalar type is z and destination type is z4fp8e4nv data type is not supported on CUDA arch < 89Zconvert_custom_typesz0target doesn't provide conversion for this type.rd@r rz cannot cast z to )8rHrr%rrrrrIrrrCrr rrr$r is_fp8e4nvoptions allow_fp8e4nvr is_fp8e4b15Z codegen_fnsgetis_fp8r&Zcreate_fp_to_fprPrr9r7r;rJr8Zcreate_fp_truncZ create_fp_extr=r-r.r^is_boolr/rrcreate_int_castZis_standard_floatingZcreate_fp_to_siZcreate_fp_to_uiZcreate_ui_to_fpZcreate_si_to_fprBZcreate_ptr_to_intint64 get_int64Zcreate_int_to_ptrr)rMrr!rrrrZuse_custom_roundingZ truncate_fpZext_fpZ sign_extendtyrZbitwidthrrrrJs                    &rJcCsDtjj}|r@|dkrtjj}n"|dkr0tjj}ntd|d|S)Nz.ca.cgCache modifier  not supported)r CACHE_MODIFIERrnCACGr$cache_modifiercacherrr_str_to_load_cache_modifierQs  rcCshtjj}|rd|dkrtjj}nF|dkr0tjj}n4|dkrBtjj}n"|dkrTtjj}ntd|d|S)Nz.wbrz.csz.wtrr)r rrnZWBrCSZWTr$rrrr_str_to_store_cache_modifier]s    rcCsDtjj}|r@|dkrtjj}n"|dkr0tjj}ntd|d|S)NZ evict_lastZ evict_firstzEviction policy r)r ZEVICTION_POLICYZNORMALZ EVICT_LASTZ EVICT_FIRSTr$)eviction_policyevictionrrr_str_to_eviction_policyms  rcCs@d}|r<|dkrtjj}n"|dkr,tjj}ntd|d|S)NzeronanzPadding option r)r PADDING_OPTIONZPAD_ZEROPAD_NANr$)padding_optionpaddingrrr_str_to_padding_optionys  rcCshtjj}|rd|dkrtjj}nF|dkr0tjj}n4|dkrBtjj}n"|dkrTtjj}ntd|d|S)NacquirereleaseZacq_relrelaxedMemory semantic r)r Z MEM_SEMANTICZACQUIRE_RELEASEZACQUIREZRELEASEZRELAXEDr$)Z sem_optionsemrrr _str_to_sems    rcCsVtjj}|rR|dkrtjj}n4|dkr0tjj}n"|dkrBtjj}ntd|d|S)NZgpuZctasysrr)r ZMEM_SYNC_SCOPEZGPUZCTAZSYSTEMr$)Z scope_optionscoperrr _str_to_scopes   r cCs|rt|ds|g}dd|D}|D],}t|trNd|krLt|ks&ntq&t|dksdtt|tt|kstdt|SdS)N__iter__cSs"g|]}t|tjr|jn|qSrrr%rrrelemrrrrsz0_canonicalize_boundary_check..rz'Duplicate dimension in `boundary_check`r)hasattrrrrrsetr)boundary_check block_shapedimrrr_canonicalize_boundary_checks *rc Cs|dk s|dk rtd|jjj} | tjks4td| rP|tjj krPtd|jj} t || }t | |j|||||| S)NK`mask` and `other` arguments cannot be specified for loading block pointers3`tl.int1` should be rewrited in `tl.make_block_ptr`z@Padding option `nan` is not supported for integer block pointers)r$rH element_tyr%r}rr=r rrrrr&Zcreate_tensor_pointer_loadrP) ptrmaskrNrrrr is_volatiler!elt_tyrrrr_load_block_pointers rc Cs|jjs"td|jd|dkr:|dk r:td|sB|rJtd|js|rj|jrjtd|r|jrtd|jr|dk rt||j|}|dk rt||j|}|jj} | j} | t j krt j } t | | j } t|| |}|dk r t|| |}|jr0|j} t | | } n| } |dkrXt ||j|||| St ||j|j|rt|jnd|||| SdS)NUnsupported ptr type z in `tl.load`z)`other` cannot be provided without `mask`z`padding_option` or `boundary_check` argument is not supported for loading a tensor ofpointers or loading a scalar. Because the compiler does not know the boundary; please use block pointers (defined by `make_block_ptr`) insteadEMask argument cannot be block type if pointer argument is not a blockzFOther argument cannot be block type if pointer argument is not a block)rHrIrBr$rrrrrr%r}int8 pointer_type address_spacerJrr&Z create_loadrPZcreate_masked_load) rrrNrrrrrr!ptr_tyrrrrrr _load_legacysH          r#zOptional[tl.tensor]rr) rrrNrrrrrr!r"c Csbt|} t|} t|} |jrF|jjrFt||||| | | || St||||| | | || SdSrA) rrrrHrBrrrr#) rrrNrrrrrr!rrrrrrloads r$)desc_ptrrrr!r"cCs<t||dd}||j|||t|t|}t||SNF require_i64)_convert_to_ir_valuesZcreate_descriptor_loadrPrrrr%r&)r%offsetsrrrHr!rgrrrdescriptor_loads r+)r%rr!r"cCs*t||dd}t||j|j|tjSr&)r)r%r&Zcreate_descriptor_storerPvoid)r%rr*r!rrrdescriptor_storesr-c Cs|dk rtd|jj}|js2t|||}|jsDtd||jksntd|d|jd|jjj|jjkstd|jjjd|jjd|jjj}|tjkstdt ||}t |||}t | |j |j |||tjS) Nrz-Value argument must be block type or a scalarz Block shape(z) and value shape(z ) mismatchzBlock element type(z) and value element type(r)r$rHrrrrrr%r}rrJr&Zcreate_tensor_pointer_storerPr,) rvalrrrrr!rrrrr_store_block_pointers"   2   r/c Cs4|jjs"td|jd|r.td|js`|jrJtd|r`|jr`td|jrt||j|}|dk rt||j|}|jj}|j}|t j krt j }t ||j }t|||}t|||}|st ||j|j||t jS|jjstdt ||j|j|j||t jS)Nrz in `tl.store`z`boundary_check` argument is not supported for storing a tensor of pointers or storing a scalar. Because the compiler does not know the boundary; please use block pointers (defined by `make_block_ptr`) insteadzFValue argument cannot be block type if pointer argument is not a blockrz"Mask must have boolean scalar type)rHrIrBr$rrrrrr%r}rr r!rJr&Z create_storerPr,rZcreate_masked_store) rr.rrrrr!r"rrrr _store_legacy;s2       r0)rr.rrrr!r"c Cspt|}t|}|js&|jjr.td|jrX|jjrXt |||||||St |||||||SdS)N"Cannot store to a constant pointer) rrrHis_constrIr$rBrrr/r0) rr.rrrrr!rrrrrstoregsr3)rcmpr.rr r!r"c CsNt|}t|}|jjj}|jdkr,tdt| |j |j |j |||jS)N)rrz9atomic_cas only supports elements with width {16, 32, 64}) rr rHrIrrr$r%r&Zcreate_atomic_casrP)rr4r.rr r!rrrr atomic_cas}s   r6z&Tuple[tl.tensor, tl.tensor, tl.tensor])rr.ropr!r"cCsF|jjstd|j|js4|jjrr?rrryr=rArBrrr atomic_minsV      rGc Csjt|||d|\}}}t|}t|}|jj}|r>tjjntjj }t | ||j |j |j |||jS)NrT)r8rr rHrIrCr r;ZFADDZADDr%r&r:rP)rr.rrr r!rCr7rrr atomic_adds rHc CsNt|||d|\}}}t|}t|}t|tjj|j |j |j |||j S)Nand) r8rr r%r&r:r r;ANDrPrHrr.rrr r!rrr atomic_ands  rLc CsNt|||d|\}}}t|}t|}t|tjj|j |j |j |||j S)Nor) r8rr r%r&r:r r;ORrPrHrKrrr atomic_ors  rOc CsNt|||d|\}}}t|}t|}t|tjj|j |j |j |||j S)Nxor) r8rr r%r&r:r r;ZXORrPrHrKrrr atomic_xors  rQc CsNt|||d|\}}}t|}t|}t|tjj|j |j |j |||j S)NZxchg) r8rr r%r&r:r r;ZXCHGrPrHrKrrr atomic_xchgs  rRcCsH||jjks(td|jjd||}|dkr.assert_dtypes_valid.._validate_dtype..rzOnly supports z. z (r)anyrr)r/ allowed_typesZ operand_nameZsupported_typesrr[r_validate_dtype<s z9dot..assert_dtypes_valid.._validate_dtypez First operandzSecond operandzUnsupported dtype ) rrrrr=Zis_int8Zis_uint8Zallow_fp8e4b15r9r;r7r~)Z lhs_dtypeZ rhs_dtyperr]r^rrrassert_dtypes_valid*s4"   **z dot..assert_dtypes_validrz+Both inputs must be either 2D or 3D; (lhs: z vs rhs: rrzFirst input shape (z) and second input shape z= are not compatible for matmul (second index of first shape (z0) must be equal to first index of second shape (r5z0All non-batch values in both first input shape (z) and second input shape (z) must be >= 16!zonly int8 supported!r rzsmall blocks not supported!rzhout_dtype=bfloat16 is unsupported. Please use out_dtype=float32/float16 and cast with `.to(tl.bfloat16)`)!rHrrr/rrrJr%r:Zdefault_dot_input_precisionrVrrrrIr=r get_int32r'r;r$r7Zget_fp32r8r9Zget_fp16rrrPrZmax_num_imprecise_acc_defaultr&Z create_dot)rErFrWrUrXrYr!r_Zlhs_rankZrhs_rankrZ ret_scalar_tyMNBr_Z acc_handlerrrdot'sf    F 0$       "  rf) conditionrgrhr!r"cCst|tj|}|jrHt|||\}}t|||\}}t|||\}}t|||dd\}}|jsvt|||\}}|j}t||j |j |j |S)NT) rJr%r}rHrrGrLr&Z create_selectrP)rgrgrhr!rr_rrrrAs  rAcCs"|rt||}n|}t||SrA)r%rr&)rgrWrZres_tyrrr wrap_tensorsrhzSequence[tl.tensor]zTuple[tl.tensor, ...])inputsr r!r"csdkr"tfddDddjjt}|ksNtd|dfddtDtfddDstd d dD|tfd dt tDS) Nc3s$|]}t||jjgddVqdS)TrN)rrrrtr!rrrszreduction..rz&reduction axis must be < inputs rank (rcsg|]\}}|kr|qSrr)rrr)r rrrszreduction..c3s|]}|jjkVqdSrA)rHrrjrrrrsz-all reduction inputs must have the same shapecSsg|] }|jqSrrPrjrrrrsc3s(|] }t||jjVqdSrArhZ get_resultrHrIrr)ri reduce_oprrrrs) tuplerHrrrrallZ create_reduceverifyr)rir region_builder_fnr!rankr)r r!rirprrr reductions rv)rir reverser!r"csdjjt}| |kr*|ksBntd|d|d|dkrR||7}D]}|jjksVtdqV|ddD|||tfdd ttDS) Nrz scan axis z must be < inputs rank (rz(all scan inputs must have the same shapecSsg|] }|jqSrrmrjrrrrsz$associative_scan..c3s(|] }t||jjVqdSrArnroriZscan_oprrrrsz#associative_scan..)rHrrrZ create_scanrsrqr)rir rtrwr!rurkrrxrassociative_scans .ry)rMnum_binsr!r"cCsJt|jdkstd|js(tdt||j|t tj |fS)Nr z histogram only supports 1D inputz%histogram only supports integer input) rrrr/r=r%r&Zcreate_histogramrPrr')rMrzr!rrr histogramsr{)rgvaluesr"cCs@tdt|jt|kr td|jdt||j|S)Nr zAShape of input to multiple_of does not match the length of valuesztt.divisibility) rtrrr$rPset_attrr make_attr get_contextrgr|rrr multiple_ofsrcCs:t|jt|krtd|jdt||j|S)NzDShape of input to max_contiguous does not match the length of valuesz tt.contiguityrrr$rPr}r r~rrrrrmax_contiguoussrcCs:t|jt|krtd|jdt||j|S)NzCShape of input to max_constancy does not match the length of valuesz tt.constancyrrrrr max_constancysr)r!r"cCst|tjSrA)r%r&Zcreate_barrierr,rlrrr debug_barriersrzList[tl.tensor])prefixargshexr!r"cCsx|ds|r|d7}|ds4|r4|ddd}t|dkrR|dsRd|}dd|D}t||||tjS)N rrrcSsg|] }|jqSrrm)rargrrrrsz device_print..)endswithr startswithr%r&Z create_printr,)rrrr!new_argsrrr device_printsr)condmsg file_namelinenor!r"c CsP|j}|s2t|jd}t||jd|}t||j||||tj S)N)r ) rHrr%rrIr&rrPZ create_assertr,)rrr func_namerr!Zcond_tyrrr device_asserts rcCs4t|trt|}t|tjr|rZd|jkr:dksNntd|jd||jSd|jkrpdksntd|jd||jSnt|tjr|j jdkstd |j std |j tj kr|r| |j||j S|j tjkr|sd std |jSd s0td t|dS)Nllz@Block pointers only support 64 bit `shape/strides`, got a value z which is out of the rangezFBlock pointers only support 32 bit `offsets/block_shape`, got a value r z*Expected a scalar in shape/strides/offsetsz8Expected an integer scalar type in shape/strides/offsetsFzzBlock pointers only support 32 bit `offsets/block_shape`, add a `.to(tl.int32)` or use regular indexing for 64 bit supportz3Unsupported element type in shape/strides/offsets: )rrr%rrrrrbr&rr/r=rrrPZ get_int64_tyr^r'rH)r!rr(rrr_convert_elem_to_ir_value s"   * *rcs,t|drfdd|DSt|gS)Nr csg|]}t|qSr)rr r!r(rrr%sz)_convert_to_ir_values..)rr)r! list_liker(rrrr)#s r))baser!r"c s:t||}t||}t||dd}|jr8|jjr@td|jjtjkrht|t tj |jj |}t dsxgddDt ddDstd t |ds|g}d d|D}t|ttt|kstd t fd d||||fDstd ||j||||}t|t t|jjS)NFr'zMExpected `base` to be a pointer type (but not a block pointer type or others)r cSs"g|]}t|tjr|jn|qSrr r rrrr;sz"make_block_ptr..css0|](}t|to&d|ko"dknVqdS)rrN)rrr rrrr<sz!make_block_ptr..zGExpected a list of constant integers (`int32_t` range) in `block_shape`cSs"g|]}t|tjr|jn|qSrr r rrrrBszrDrLrTrXrZr]r`rcrfrqrrrurwryrzr{rrrrrrrrUrrrrrrrrrrrrrrrrrrrGrrrJrrrrrr rrr#r$r+r-r/r0r3r6r8rFrGrHrLrOrQrRrVrfrArhrvryr{rrrrrrrr)rrrrrrs     (          : w    :  , ''  `   '