U h'@sddlZddlZddlmZddlmZmZmZmZm Z m Z m Z ddl m Z ddlmZmZddlmZdd d d d d gZdddddddZdddddddZGdddeZdS)N)Path)AnyCallableDictListOptionalTupleUnion)Image)download_and_extract_archiveverify_str_arg) VisionDatasetZkingdomZphylumclassorderfamilyZgenuszRhttps://ml-inat-competition-datasets.s3.amazonaws.com/2017/train_val_images.tar.gzzOhttps://ml-inat-competition-datasets.s3.amazonaws.com/2018/train_val2018.tar.gzzOhttps://ml-inat-competition-datasets.s3.amazonaws.com/2019/train_val2019.tar.gzzGhttps://ml-inat-competition-datasets.s3.amazonaws.com/2021/train.tar.gzzLhttps://ml-inat-competition-datasets.s3.amazonaws.com/2021/train_mini.tar.gzzEhttps://ml-inat-competition-datasets.s3.amazonaws.com/2021/val.tar.gz)2017Z2018Z2019 2021_trainZ2021_train_miniZ 2021_validZ 7c784ea5e424efaec655bd392f87301fZ b1c6952ce38f31868cc50ea72d066cc3Z c60a6e2962c9b8ccbd458d12c8582644Z e0526d53c7f7b2e3167b2b43bb2690edZ db6ed8330e634445efc8fec83ae81442Z f6f6e0e242e3d4c9569ba56400938afcc seZdZdZdeeefeeeeefee ee e ddfdd Z dd d d Z dd d d Z eeeefdddZed ddZeeedddZe d ddZdd ddZZS) INaturalistaV`iNaturalist `_ Dataset. Args: root (str or ``pathlib.Path``): Root directory of dataset where the image files are stored. This class does not require/use annotation files. version (string, optional): Which version of the dataset to download/use. One of '2017', '2018', '2019', '2021_train', '2021_train_mini', '2021_valid'. Default: `2021_train`. target_type (string or list, optional): Type of target to use, for 2021 versions, one of: - ``full``: the full category (species) - ``kingdom``: e.g. "Animalia" - ``phylum``: e.g. "Arthropoda" - ``class``: e.g. "Insecta" - ``order``: e.g. "Coleoptera" - ``family``: e.g. "Cleridae" - ``genus``: e.g. "Trichodes" for 2017-2019 versions, one of: - ``full``: the full (numeric) category - ``super``: the super category, e.g. "Amphibians" Can also be a list to output a tuple with all specified target types. Defaults to ``full``. transform (callable, optional): A function/transform that takes in a PIL image and returns a transformed version. E.g, ``transforms.RandomCrop`` target_transform (callable, optional): A function/transform that takes in the target and transforms it. download (bool, optional): If true, downloads the dataset from the internet and puts it in root directory. If dataset is already downloaded, it is not downloaded again. rfullNF)rootversion target_type transformtarget_transformdownloadreturnc s t|dt|_tjtj||||dtj |dd|rH| | sXt dg|_ i|_g|_t|tsz|g}|jdddkrdd |D|_|nd d |D|_|g|_t|j D]8\}}ttj|j|} | D]} |j|| fqqdS) Nr)rrT)exist_okzHDataset not found or corrupted. You can use download=True to download itZ2021cSsg|]}t|ddtqS)rr)r)r CATEGORIES_2021.0tr#R/var/www/html/venv/lib/python3.8/site-packages/torchvision/datasets/inaturalist.py bsz(INaturalist.__init__..cSsg|]}t|ddqS)r)rsuper)r r r#r#r$r%es)r DATASET_URLSkeysrr&__init__ospathjoinmakedirsr_check_integrity RuntimeErrorall_categoriescategories_indexcategories_map isinstancelistr _init_2021 _init_pre2021index enumeratelistdirrappend) selfrrrrrr dir_indexdir_namefilesfname __class__r#r$r)Cs,   zINaturalist.__init__)rcCstt|j|_ddtD|_t|jD]\}}|d}t |dkrZt d|d|d|dkrt d |dd |di}t t|d d D]J\}}||j|kr|j||}nt |j|}||j||<|||<q|j |q,d S)zInitialize based on 2021 layoutcSsi|] }|iqSr#r#)r!kr#r#r$ vsz*INaturalist._init_2021.._zUnexpected category name z, wrong number of piecesrZ05dzUnexpected category id z , expecting r N)sortedr*r9rr0rr1r8splitlenr/zipr2r:)r;r<r=piecesZcat_mapcatnamecat_idr#r#r$r5ps    zINaturalist._init_2021c Csbdii|_d}tt|j}t|D] \}}||jd|<tttj|j|}|D]}|jdkrx|}|d7}n0z t |}Wn"t k rt d|YnX|t |j krt |j }|j ig||d|jdg||d|j |rt d|d|i|j |<tj|||j|<q\q&t|j D]\} } | s>t d| q>d S) z$Initialize based on 2017-2019 layoutr&rrr z!Unexpected non-numeric dir name: zDuplicate category zMissing category N)r1rGr*r9rr8r+r,rint ValueErrorr/rIr2extendr0) r;Z cat_indexZsuper_categoriesZsindexZscatZ subcategoriesZsubcatZsubcat_iZold_lenZcindexcr#r#r$r6s2      zINaturalist._init_pre2021)r7rcCs|j|\}}ttj|j|j||}g}|jD],}|dkrN| |q6| |j ||q6t |dkrxt |n|d}|j dk r| |}|jdk r||}||fS)z Args: index (int): Index Returns: tuple: (image, target) where the type of target specified by target_type. rr rN)r7r openr*r+r,rr0rr:r2rItuplerr)r;r7rNr?Zimgtargetr"r#r#r$ __getitem__s       zINaturalist.__getitem__cCs t|jS)N)rIr7r;r#r#r$__len__szINaturalist.__len__) category_type category_idrcCsn|dkr|j|S||jkr.td|dn<|j|D]\}}||kr<|Sqs, $