U h? @sUddlZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlmZmZmZmZmZmZmZmZmZmZddlmZddlZddlZddlmZddl m!Z!m"Z"dZ#dRe$ee$ej%fe&dd d d Z'dSee$ej%fe&e$d ddZ(ee$ej%fe$ee)dddZ*dTee$ej%fee$e)dddZ+dUe$e&e$dddZ,e$ee$dddZ-dVe$ee$ej%feee$ej%fee$e&ddddZ.dWee$ej%fe)ee$d!d"d#Z/dXee$ej%fe$e)ee$d$d%d&Z0dYe$ee$ej%feee$ej%fee$d'd(d)Z1ee$ej%fee$ej%fee$dd*d+d,Z2e j3e j4d-Z5ee$e&fe6d.<ee$ej%fee$ej%fee$dd*d/d0Z7e2e7d1Z8ee$eee$ej%fee$ej%fee$gdffe6d2<ej9ej9ej9d3Z:ee$ed4effe6d5<d6d6d7d8Z;ee$eee$ee$ffe6d9<ee$ej%fee$ee$ee$fd:d;d<Zd?Z=d[ee$ej%feee$ej%fe)ee$ej%fd=d@dAZ>d\e$ee$ej%feee$ej%feee$ej%fee$e)ddBdCdDZ?ee$dEdFdGZ@edHe$eAZBd]eBee$eeeBee$eBdIdJdKZCd^ee$ej%fe&ejDdLdMdNZEejFejFdOdPdQZGdS)_N) AnyCallableDictIOIterableListOptionalTupleTypeVarUnion)urlparse)tqdm)#_download_file_from_remote_location_is_remote_location_availablezpytorch/vision)urlfilename chunk_sizereturnc Cstjtjj|dtid\}t|dF}t|jd.}||}r`| || t |q8W5QRXW5QRXW5QRXdS)N User-Agentheaderswb)total) urllibrequesturlopenRequest USER_AGENTopenr lengthreadwriteupdatelen)rrrresponsefhZpbarchunkr)L/var/www/html/venv/lib/python3.8/site-packages/torchvision/datasets/utils.py _urlretrieves  r+)fpathrrc CsXtjdkrtjdd}nt}t|d }||}rF||q,W5QRX|S)N) F)Zusedforsecurityrb)sys version_infohashlibmd5r r"r$ hexdigest)r-rr4fr(r)r)r* calculate_md5#s  r7)r-r4kwargsrcKs|t|f|kSN)r7)r-r4r8r)r)r* check_md51sr:)r-r4rcCs&tj|sdS|dkrdSt||S)NFT)ospathisfiler:)r-r4r)r)r*check_integrity5s  r>r.)rmax_hopsrc Cs|}dtd}t|dD]V}tjtjj||d4}|j|ksL|jdkr`|W5QRS|j}W5QRXqtd|d|d|ddS) NHEAD)ZMethodrrz Request to z exceeded z( redirects. The last redirect points to .)rrangerrrrrRecursionError)rr?Z initial_urlr_r&r)r)r*_get_redirect_url=s rF)rrcCsBt|}td|jdkrdStd|j}|dkr8dS|dS)Nz(drive|docs)[.]google[.]comz/file/d/(?P[^/]*)id)r rematchnetlocr<group)rpartsrIr)r)r*_get_google_drive_file_idMsrM)rrootrr4max_redirect_hopsrc Cs>tj|}|stj|}ttj||}tj|ddt||rXtd|dSt rjt ||nt ||d}t |}|dk rt ||||Sz"td|d|t||Wnntjjtfk r$}zF|ddd kr|d d }td |d|t||n|W5d}~XYnXt||s:td dS)aDownload a file from a url and place it in root. Args: url (str): URL to download file from root (str): Directory to place downloaded file in filename (str, optional): Name to save the file under. If None, use the basename of the URL md5 (str, optional): MD5 checksum of the download. If None, do not check max_redirect_hops (int, optional): Maximum number of redirect hops allowed Texist_okz$Using downloaded and verified file: N)r?z Downloading  to httpszhttps:zhttp:z;Failed download. Trying https -> http instead. Downloading File not found or corrupted.)r;r< expanduserbasenamefspathjoinmakedirsr>printrrrFrMdownload_file_from_google_driver+rerrorURLErrorOSErrorreplace RuntimeError)rrNrr4rOr-file_ider)r)r* download_urlZs2         rdF)rNprefixrcsBtjfddtD}|dkr>fdd|D}|S)a List all directories at a given root Args: root (str): Path to directory whose folders need to be listed prefix (bool, optional): If true, prepends the path to each result, otherwise only returns the name of the directories found cs&g|]}tjtj|r|qSr))r;r<isdirrY.0prNr)r* szlist_dir..Tcsg|]}tj|qSr)r;r<rYrhdrjr)r*rksr;r<rVlistdir)rNre directoriesr)rjr*list_dirs  rr)rNsuffixrercsDtjfddtD}|dkr@fdd|D}|S)aList all files ending with a suffix at a given root Args: root (str): Path to directory whose folders need to be listed suffix (str or tuple): Suffix of the files to match, e.g. '.png' or ('.jpg', '.png'). It uses the Python "str.endswith" method and is passed directly prefix (bool, optional): If true, prepends the path to each result, otherwise only returns the name of the files found cs0g|](}tjtj|r|r|qSr))r;r<r=rYendswithrgrNrsr)r*rks zlist_files..Tcsg|]}tj|qSr)rlrmrjr)r*rksro)rNrsrefilesr)rur* list_filess rw)rbrNrr4cCsz ddl}Wntk r(tdYnXtj|}|s>|}ttj||}tj|ddt ||rt d|rvdndd |dS|j ||d t d t ||std dS) afDownload a Google Drive file from and place it in root. Args: file_id (str): id of file to be downloaded root (str): Directory to place downloaded file in filename (str, optional): Name to save the file under. If None, use the id of the file. md5 (str, optional): MD5 checksum of the download. If None, do not check rNz`To download files from GDrive, 'gdown' is required. You can install it with 'pip install gdown'.TrPzUsing downloaded z and verified zfile: F)rGoutputquiet user_agentrU) gdownModuleNotFoundErrorrar;r<rVrXrYrZr>r[downloadr)rbrNrr4r|r-r)r)r*r\s"     r\) from_pathto_path compressionrc Cs<t||rd|ddnd}||W5QRXdS)Nzr:rAr)tarfiler extractall)rrrtarr)r)r* _extract_tars$r).bz2.xz_ZIP_COMPRESSION_MAPc Cs8tj|d|rt|ntjd}||W5QRXdS)Nr)r)zipfileZipFiler ZIP_STOREDr)rrrzipr)r)r* _extract_zipsr).tarz.zip_ARCHIVE_EXTRACTORS)r.gzr._COMPRESSED_FILE_OPENERS)rr)rr)z.tbzz.tbz2z.tgz_FILE_TYPE_ALIASES)filercCst|j}|s td|d|d}|tkr>|ft|S|tkrP||dfS|tkrt|dkr|d}|tkr||||fS|d|fStt tt tBt tB}td|d|d dS) aDetect the archive type and/or compression of a file. Args: file (str): the filename Returns: (tuple): tuple of suffix, archive type, and compression Raises: RuntimeError: if file has no suffix or suffix is not supported zFile 'zP' has no suffixes that could be used to detect the archive type and compression.NrAz&Unknown compression or archive type: 'z'. Known suffixes are: 'z'.) pathlibPathsuffixesrarrrr%sortedset)rrrsZsuffix2Zvalid_suffixesr)r)r*_detect_file_types$     r)rrremove_finishedrc Cst|\}}}|s"td|d|dkrNtt|||dk rF|nd}t|}||d*}t|d}| | W5QRXW5QRX|rt |t|S)aDecompress a file. The compression is automatically detected from the file name. Args: from_path (str): Path to the file to be decompressed. to_path (str): Path to the decompressed file. If omitted, ``from_path`` without compression extension is used. remove_finished (bool): If ``True``, remove the file after the extraction. Returns: (str): Path to the decompressed file. z*Couldn't detect a compression from suffix rBNrxr0r) rrarrr;rXr`rr r#r"remove) rrrrs archive_typerZcompressed_file_openerZrfhZwfhr)r)r* _decompress)s$" rc stjtttjfdfdd }|dkr4tj}t\}}}|svttj |tj  |d|d}||St |}||||rt |t|S)aDExtract an archive. The archive type and a possible compression is automatically detected from the file name. If the file is compressed but not an archive the call is dispatched to :func:`decompress`. Args: from_path (str): Path to the file to be extracted. to_path (str): Path to the directory the file will be extracted to. If omitted, the directory of the file is used. remove_finished (bool): If ``True``, remove the file after the extraction. Returns: (str): Path to the directory the file was extracted to. )ret_pathrcsttrt|S|SdSr9) isinstancestrr;rX)rrr)r* path_or_stras  z$extract_archive..path_or_strNrx)r)rrr rr;r<dirnamerrrYrWr`rr) rrrrrsrrrZ extractorr)rr*extract_archiveMs     r)r download_root extract_rootrr4rrcCshtj|}|dkr|}|s(tj|}t||||tj||}td|d|t|||dS)Nz Extracting rR)r;r<rVrWrdrYr[r)rrrrr4rarchiver)r)r*download_and_extract_archive}s  r)iterablercCsdddd|DdS)N'z', 'cSsg|] }t|qSr))r)rhitemr)r)r*rksz#iterable_to_str..)rY)rr)r)r*iterable_to_strsrT)valuearg valid_values custom_msgrcCs|t|ts6|dkrd}nd}|jt||d}t||dkrB|S||krx|dk rX|}nd}|j||t|d}t||S)Nz'Expected type str, but got type {type}.z:Expected type str for argument {arg}, but got type {type}.)typerzPUnknown value '{value}' for argument {arg}. Valid values are {{{valid_values}}}.)rrr)rrformatr ValueErrorr)rrrrmsgr)r)r*verify_str_args r) file_nameslice_channelsrc Cst|d}|}|dkr(tdtd|}|sDtddd|D\}}t|}|dkr~d }| }nd }t j ||d d } W5QRX|d krdnd} | |||  ddd} t j | dd} | d|ddddf} | t jS)aDRead file in .pfm format. Might contain either 1 or 3 channels of data. Args: file_name (str): Path to the file. slice_channels (int): Number of channels to slice out of the file. Useful for reading different data formats stored in .pfm files: Optical Flows, Stereo Disparity Maps, etc. r0)PFsPfzInvalid PFM files^(\d+)\s(\d+)\s$zMalformed PFM header.css|]}t|VqdSr9)int)rhdimr)r)r* sz_read_pfm..r<>r6)dtyperr.rAr)ZaxisN)r readlinerstriprrHrI ExceptiongroupsfloatnpfromfileZreshapeZ transposeflipZastypeZfloat32) rrr6headerZ dim_matchwhscaleZendiandataZ pfm_channelsr)r)r* _read_pfms&  r)trcCsD|tjj|j|fdj|jddd|jS)Nr)r) contiguousviewtorchZuint8shapeZ element_sizerr)rr)r)r*_flip_byte_ordersBr)r)r,)N)r.)NNr.)F)F)NN)NF)NF)NNNF)NNN)r)Hbz2gzipr3lzmar;os.pathrrHr1rr urllib.errorurllib.requestrtypingrrrrrrrr r r urllib.parser numpyrrZtorch.utils.model_zoor Z_internally_replaced_utilsrrrrrrr+r7boolr:r>rFrMrdrrrwr\r ZIP_BZIP2ZIP_LZMAr__annotations__rrr rrrrrrrbytesrrZndarrayrZTensorrr)r)r)r*s0   "  8"$  '    <&,-  &  3      $