U Mh@sddlZddlZddlZddlmZddlmZmZmZm Z m Z m Z m Z ddl mZmZddlmZddgZe eefZeee edffZd d Zeee eeed d d ZeedddZdeeeedddZeee eedddZdS)N) Collection)DictListOptionalSetTupleTypeUnion) IterDataPipe MapDataPipe)dill_availabletraverse traverse_dps DataPipeGraphcCsdS)NZSTUBrrrH/var/www/html/venv/lib/python3.8/site-packages/torch/utils/data/graph.py_stub_unpicklersr)scan_obj only_datapipecachereturnc st}t|}tr.ddlm}||}nd}gdd}fdd}ttf} zf| D]} | ||r\| |q\z| Wn0tj t tfk rtr| nYnXW5| D]} | d|r| dqtrddlm } | dXS) Nr)PicklercSsd}t|tr>i}|D] \}}t|tttfr|||<qnLt|ttfrvg}|D]}t|tttfrT||qTnt|tttfr|}|S)N) isinstancedictitemsr r rtuplelistappend)Z ori_statestatekvrrr getstate_hook#s  z0_list_connected_datapipes..getstate_hookcs>|kst|krtn |t|tdfSdS)Nr)idNotImplementedErrorraddr)objrZcaptured_connectionsrrr reduce_hook3s  z._list_connected_datapipes..reduce_hook)extendF)ioBytesIOpicklerr Zdillr r Zset_reduce_ex_hookZset_getstate_hookr(dump PickleErrorAttributeError TypeError) rrrfpZ dill_Picklerdr!r'Zdatapipe_classesclsZ dill_extendrr&r_list_connected_datapipess:            r4)datapipercCst}t|d|dS)a Traverse the DataPipes and their attributes to extract the DataPipe graph. This only looks into the attribute from each DataPipe that is either a DataPipe and a Python collection object such as ``list``, ``tuple``, ``set`` and ``dict``. Args: datapipe: the end DataPipe of the graph Returns: A graph represented as a nested dictionary, where keys are ids of DataPipe instances and values are tuples of DataPipe instance and the sub-graph T)rr)set_traverse_helper)r5rrrrrUs)r5rrcCs:d}|s|d7}t|t|dkr(d}t}t|||S)a6 Traverse the DataPipes and their attributes to extract the DataPipe graph. [Deprecated] When ``only_dataPipe`` is specified as ``True``, it would only look into the attribute from each DataPipe that is either a DataPipe and a Python collection object such as ``list``, ``tuple``, ``set`` and ``dict``. Note: This function is deprecated. Please use `traverse_dps` instead. Args: datapipe: the end DataPipe of the graph only_datapipe: If ``False`` (default), all attributes of each DataPipe are traversed. This argument is deprecating and will be removed after the next release. Returns: A graph represented as a nested dictionary, where keys are ids of DataPipe instances and values are tuples of DataPipe instance and the sub-graph zV`traverse` function and will be removed after 1.13. Please use `traverse_dps` instead.zM And, the behavior will be changed to the equivalent of `only_datapipe=True`.NF)warningswarn FutureWarningr6r7)r5rmsgrrrrr gs )r5rrrcCst|ttfs"tdt|dt|}||kr6iS||t|||}||ifi}|D]"}||d t |||q`|S)Nz.Expected `IterDataPipe` or `MapDataPipe`, but z is found) rr r RuntimeErrortyper"r$r4copyupdater7)r5rrZdp_idrr2itemrrrr7s   r7)N)r)r+r8collections.abcrtypingrrrrrrr Ztorch.utils.datar r Ztorch.utils._import_utilsr __all__ZDataPipeintrrboolr4rr r7rrrrs $  =