U yh @sUddlZddlZddlZddlmZmZmZmZmZm Z ddl m Z ddl Z ddl mZddlmZgZeeed<dZefeejddd Zefeeejefdd d Zeaeeefd d dZe dZe dZeeefeeefdddZeeefeeefdddZdS)N)AnyCallableDictListTupleTypeVar) ParamSpec) _log_handlers__all__default) destinationreturncCsPt|\}}td|}|tjtd}||d|_|||S)Nzc10d-za%(asctime)s %(filename)s:%(lineno)s %(levelname)s p:%(processName)s t:%(threadName)s: %(message)sF) _get_logging_handlerlogging getLoggersetLevelDEBUG Formatter setFormatter propagate addHandler)r Zlogging_handlerlog_handler_namelogger formatterrO/var/www/html/venv/lib/python3.8/site-packages/torch/distributed/c10d_logger.py_get_or_create_loggers    rcCst|}t|j}||fSN)r type__name__)r Z log_handlerrrrrr's r)r c Ostr|dp|d}||d|t|dt|tt|tt|d}|ddkrtjj }d d d |D|d <n||d|d }|S) NgroupZ process_groupz, Zpg) func_nameargsZpg_namebackendZ world_sizeZ group_sizeZ global_rankZ local_rankr#nccl.css|]}t|VqdSr)str).0vrrr @sz _get_msg_dict.. nccl_version)r!r") distZis_initializedgetZ_get_process_group_name get_backendZget_world_sizeZget_ranktorchcudar$versionjoin)r!r"kwargsr msg_dictr*rrr _get_msg_dict1s$       r4_T_P)funcr cs(ttjtjtdfdd }|S)Nr"r2r c s\z ||WStk rV}z,tjf||}||d<t|W5d}~XYnXdS)Nerror) Exceptionr4r _c10d_loggerdebug)r"r2r9r3r7rrwrapperLs   z"_exception_logger..wrapper functoolswrapsr6r"r2r5r7r>rr=r_exception_loggerKs rCcs(ttjtjtdfdd }|S)Nr8csLt}||}t|}tjf||}|d|d<t||S)Nns time_spent)timetime_nsr4rr;r<)r"r2t1Z func_returnrEr3r=rrr>Zs   z_time_logger..wrapperr?rBrr=r _time_loggerYs rI) r@rrFtypingrrrrrrtyping_extensionsrr.Ztorch.distributed distributedr+Z"torch.distributed.logging_handlersr r r&__annotations__Z_DEFAULT_DESTINATIONLoggerrHandlerrr;r4r5r6rCrIrrrr s$