U yh!@sdZddlZddlmZmZmZmZddlmZddl m Z ddl m Z m Z ejedddZejeee eejfdd d Zejee dd d Zejedd dZdS)z NOTE: This file must be imported like ``import torch.distributed.fsdp._traversal_utils`` and not like ``from torch.distirbuted.fsdp._traversal_utils import ...`` to avoid circular imports. For brevity, we may import the file as ``traversal_utils``. N)DequeListSetTuple _get_registry) _FSDPState_get_module_fsdp_state)modulereturncCst|}|dkrdSd|kS)zA Returns if ``module`` can compose with ``fully_shard``. NTZ replicater)r registryr Y/var/www/html/venv/lib/python3.8/site-packages/torch/distributed/fsdp/_traversal_utils.py _composable#src Csg}g}t}t}t|g}|r|}||t|s@q tt|D]}||krP| |qPt |}|dk r ||kr ||| || |q ||fS)a Returns a tuple containing: 1. A list of the ``_FSDPState`` instances in the module tree rooted at ``module`` without any duplicates and following the ``module.modules()`` traversal order (which is assumed to be depth-first). 2. A corresponding list of the modules owning the states in the first list. For the wrapper code path, both returned lists are the same, each containing all ``FullyShardedDataParallel`` instances. For the composable code path, this returns a list of all composable state instances and a list of the corresponding fully sharded modules. See [Note: Fully Sharded Module]. NOTE: The traversal does not proceed into any module annotated by an incompatible API (e.g. ``replicate``). N) set collectionsdequepopleftaddrreversedlistchildren appendleftr append) r fsdp_statesZ fsdp_modulesZvisited_fsdp_statesZvisited_modulesr submoduleZ child_moduleZoptional_stater r r_get_fsdp_states_with_modules1s&      rcCst|\}}|S)z*See :func:`_get_fsdp_states_with_modules`.)r)r r_r r r_get_fsdp_statesas rcCsddt|D}|S)z Returns all ``FlatParamHandle`` s in the module tree rooted at ``module`` following the rules in :func:`_get_fsdp_state`. cSsg|]}|jdk r|jqS)N)Z_handle).0Z fsdp_stater r r ls z%_get_fsdp_handles..)r)r Zhandlesr r r_get_fsdp_handlesgsr!)__doc__rtypingrrrrZtorch.nnnnZ&torch.distributed._composable.contractrZ$torch.distributed.fsdp._common_utilsrr Moduleboolrrrr!r r r rs   0