/usr/local/lib64/python3.6/site-packages/torch/cuda/__pycache__
Edit: /usr/local/lib64/python3.6/site-packages/torch/cuda/__pycache__/nccl.cpython-36.pyc (3444B)
3
Egf ã
@ sR d dl Z d dlZd dlZd dlmZmZmZ dddddgZd Z dd „ Z
d
d „ Z d d
„ Z dd„ Z
eejeej f ddœdd„Zde ddfdd„Zdd e ddfddœeej eeejeej f eeeeejj eeej ddœdd„Zd eej eddœdd„Zd eej eej ddœdd„Ze ddfeej eej eddœd d„ZdS ) é N)ÚOptionalÚSequenceÚUnionÚ
all_reduceÚreduceÚ broadcastÚ
all_gatherÚreduce_scatterc C sp t tjdƒstjdƒ dS tƒ }xJ| D ]B}|jr4dS |jƒ s@dS |jsJdS |j ƒ }||kr^dS |j
|ƒ q&W dS )NÚ_nccl_all_reducez)PyTorch is not compiled with NCCL supportFT) ÚhasattrÚtorchÚ_CÚwarningsÚwarnÚsetZ is_sparseZ
is_contiguousZis_cudaZ
get_deviceÚadd)ZtensorsZdevicesZtensorZdevice© r ú;/usr/local/lib64/python3.6/site-packages/torch/cuda/nccl.pyÚ is_available
s
r c C s0 t jjƒ } | d? }| d? d@ }| d@ }|||fS )Né é iÿÿ )r r
Z
_nccl_version)ÚverÚmajorÚminorÚpatchr r r Úversion" s
r c C s
t jjƒ S )N)r r
Z_nccl_unique_idr r r r Ú unique_id* s r c C s t jj| ||ƒS )N)r r
Z_nccl_init_rank)Z num_ranksÚuidZrankr r r Ú init_rank. s r )ÚinputsÚreturnc C s& t | tjƒ st | tjƒr"tdƒ‚d S )Nz(Inputs should be a collection of tensors)Ú
isinstanceÚ collectionsÚ Containerr ÚTensorÚ TypeError)r r r r Ú_check_sequence_type2 s r& c C s4 t | ƒ |d kr| }t |ƒ tjj| ||||ƒ d S )N)r& r r
r
)r ÚoutputsÚopÚstreamsÚcommsr r r r 7 s
)r' )r ÚoutputÚrootr( r) r' r c C s” t | ƒ |d k r6|d k r"tdƒ‚qztjdƒ || }nDt|tjƒ rft|tjj ƒrftjdƒ || }n|d krv| | n|}tj
j | |||||ƒ d S )Nzä'output' and 'outputs' can not be both specified. 'outputs' is deprecated in favor of 'output', taking in a single output tensor. The signature of reduce is: reduce(inputs, output=None, root=0, op=SUM, streams=None, comms=None).zƒnccl.reduce with an output tensor list is deprecated. Please specify a single output tensor with argument 'output' instead instead.z\nccl.reduce with an output tensor list is deprecated. Please specify a single output tensor.) r& Ú
ValueErrorr r r! r r$ r" Úabcr r
Z _nccl_reduce)r r+ r, r( r) r* r' Ú_outputr r r r A s
)r r, r c C s t | ƒ tjj| |||ƒ d S )N)r& r r
Z_nccl_broadcast)r r, r) r* r r r r ` s )r r' r c C s& t | ƒ t |ƒ tjj| |||ƒ d S )N)r& r r
Z_nccl_all_gather)r r' r) r* r r r r e s )r r' r( r c C s( t | ƒ t |ƒ tjj| ||||ƒ d S )N)r& r r
Z_nccl_reduce_scatter)r r' r( r) r* r r r r k s )r NN)NN)r" r Z
torch.cudar Útypingr r r Ú__all__ZSUMr r r r r$ r& r ÚintZcudaZStreamr r r r r r r r Ú
s* P