U yh@sjUddlmZmZmZmZddlZddlmmZ ddlm Z gZ ee e d<ejjGdddZdS))DictListOptionalTupleN)Tensor__all__c @sLeZdZd eeeeeefeeeeedddZee ed d d Z d S)_FunctionalAdamaxMbP?g?g+?:0yE>F)paramslrbetaseps weight_decayforeachmaximize_allow_empty_param_listc Cs d|kstd|d|ks,td|d|dkrDdksXntd|dd|dkrpdksntd|dd|kstd ||||d|d|d |_||_||_tjttjtt tjffi|_ t |dkr|std d |i|_ dS) Nr zInvalid learning rate: zInvalid epsilon value: rg?z#Invalid beta parameter at index 0: z#Invalid beta parameter at index 1: zInvalid weight_decay value: )rrbeta1beta2rz%optimizer got an empty parameter listr ) ValueErrordefaultsrrtorchjitZannotaterrstrstatelen param_group) selfr rrrrrrrr![/var/www/html/venv/lib/python3.8/site-packages/torch/distributed/optim/functional_adamax.py__init__s, $z_FunctionalAdamax.__init__) gradientsc Cs|jd}g}g}g}g}g}t|t|krTtddt|ddt|d}t|jd|D]\} } | dk rh|t| O}|| || | |jkri|j| <|j| } td| d<tj | tj d | d <tj | tj d | d <|j| } || d || d || dqht Lt j ||||||jd |jd |jd|jd|jd|j|j|d W5QRXdS)Nr zEthe gradients passed in does not equal to the size of the parameters!zParams length: z. zGradients length: Fr step)Z memory_formatZexp_avgZexp_infrrrrr)rrrrrrr has_complex)rrrziprZ is_complexappendrZtensorZ zeros_likeZpreserve_formatZno_gradFZadamaxrrr) r r$r Zparams_with_gradZgradsZexp_avgsZexp_infsZ state_stepsr&paramZgradientrr!r!r"r%>sh           z_FunctionalAdamax.stepN)r r r r FFF) __name__ __module__ __qualname__rrfloatrboolr#rr%r!r!r!r"rs$  (r)typingrrrrrZtorch.optim._functionalZoptimZ _functionalr)rrr__annotations__rscriptrr!r!r!r"s