Name	Size	Mode	Actions
api.cpython-36.pyc	26470	0644	editdlrm
backend_registry.cpython-36.pyc	8029	0644	editdlrm
constants.cpython-36.pyc	665	0644	editdlrm
functions.cpython-36.pyc	7398	0644	editdlrm
internal.cpython-36.pyc	7948	0644	editdlrm
options.cpython-36.pyc	6943	0644	editdlrm
rref_proxy.cpython-36.pyc	1585	0644	editdlrm
server_process_global_profiler.cpython-36.pyc	7839	0644	editdlrm
__init__.cpython-36.pyc	6409	0644	editdlrm

3 EgD�@s�dd
lm
Z

ddlmZ
ddlZddlmZmZm Z m Z
e eeej fZeej d�dd�Zeeefeej ej fd �d d�Zeeeej d�d d�ZGdd�de
�ZdS)�)
� _TensorPipeRpcBackendOptionsBase�
)
� constantsN)�Dict�List�Optional�Union)�device�returnc

Cs*tj
|�
}|jd
kr&td|j�d��
�
|S)NZcudazA`set_devices` expect a list of CUDA devices, but got device type �
.)�torchr �type� ValueError)
r �r�I/usr/local/lib64/python3.6/site-packages/torch/distributed/rpc/options.py� _to_devices



r)� device_mapr c
Csni}
i}x`|D]X}||}tj
|�
tj
|�
}}||krVtd
|�d||�d|���
�
||
|<|||<qW|
S)Nz9`device_map` only supports 1-to-1 mapping, trying to map z and z to )rr r)r�full_device_mapZreverse_map�
k�
vrrr�_to_device_maps









r)�devicesr c

Cstt
t|��
S)
N)�list�mapr)
rrrr�_to_device_list&s
rc s�eZ
dZd
Zejejejddddd�ee e eee ee e ffeee eed��f
dd�Ze ee e fd��f
dd�Zee d �
d d�Z�ZS)�TensorPipeRpcBackendOptionsa' The backend options for :class:`~torch.distributed.rpc.TensorPipeAgent`, derived from :class:`~torch.distributed.rpc.RpcBackendOptions`. Args: num_worker_threads (int, optional): The number of threads in the thread-pool used by :class:`~torch.distributed.rpc.TensorPipeAgent` to execute requests (default: 16). rpc_timeout (float, optional): The default timeout, in seconds, for RPC requests (default: 60 seconds). If the RPC has not completed in this timeframe, an exception indicating so will be raised. Callers can override this timeout for individual RPCs in :meth:`~torch.distributed.rpc.rpc_sync` and :meth:`~torch.distributed.rpc.rpc_async` if necessary. init_method (str, optional): The URL to initialize the distributed store used for rendezvous. It takes any value accepted for the same argument of :meth:`~torch.distributed.init_process_group` (default: ``env://``). device_maps (Dict[str, Dict], optional): Device placement mappings from this worker to the callee. Key is the callee worker name and value the dictionary (``Dict`` of ``int``, ``str``, or ``torch.device``) that maps this worker's devices to the callee worker's devices. (default: ``None``) devices (List[int, str, or ``torch.device``], optional): all local CUDA devices used by RPC agent. By Default, it will be initialized to all local devices from its own ``device_maps`` and corresponding devices from its peers' ``device_maps``. When processing CUDA RPC requests, the agent will properly synchronize CUDA streams for all devices in this ``List``. N)�num_worker_threads�rpc_timeout�init_method�device_mapsr�_transports� _channelsc
 sN|dkrind
d�|j�D�
}|dkr*gnt
|�
} t�j|
|||||| �
dS)Nc
Ssi|]\}
}t|�
|
�qSr)
r)�.0rrrrr� Ysz8TensorPipeRpcBackendOptions.__init__..)�itemsr�super�__init__) �selfrrrrrr r!Zfull_device_mapsZfull_device_list)
� __class__rrr&Ls








z$TensorPipeRpcBackendOptions.__init__)�torc s~t|�
}t
�j}|
|krlxR|j�D]F\}}|||
kr"|||
|kr"td
|�d|�d||
|���
�
q"Wt
�j|
|�
dS)a  Set device mapping between each RPC caller and callee pair. This function can be called multiple times to incrementally add device placement configurations. Args: worker_name (str): Callee name. device_map (Dict of int, str, or torch.device): Device placement mappings from this worker to the callee. This map must be invertible. Example:: >>> # both workers >>> def add(x, y): >>> print(x) # tensor([1., 1.], device='cuda:1') >>> return x + y, (x + y).to(2) >>> >>> # on worker 0 >>> options = TensorPipeRpcBackendOptions( >>> num_worker_threads=8, >>> device_maps={"worker1": {0: 1}} >>> # maps worker0's cuda:0 to worker1's cuda:1 >>> ) >>> options.set_device_map("worker1", {1: 2}) >>> # maps worker0's cuda:1 to worker1's cuda:2 >>> >>> rpc.init_rpc( >>> "worker0", >>> rank=0, >>> world_size=2, >>> backend=rpc.BackendType.TENSORPIPE, >>> rpc_backend_options=options >>> ) >>> >>> x = torch.ones(2) >>> rets = rpc.rpc_sync("worker1", add, args=(x.to(0), 1)) >>> # The first argument will be moved to cuda:1 on worker1. When >>> # sending the return value back, it will follow the invert of >>> # the device map, and hence will be moved back to cuda:0 and >>> # cuda:1 on worker0 >>> print(rets[0]) # tensor([2., 2.], device='cuda:0') >>> print(rets[1]) # tensor([2., 2.], device='cuda:1') z=`set_device_map` only supports 1-to-1 mapping, trying to map z to z and N)rr%rr$rZ_set_device_map)r'r)rrZcurr_device_mapsrr)
r(rr�set_device_mapis,




$z*TensorPipeRpcBackendOptions.set_device_map)
rcCst|
�
|_
d
S)ab
Set local devices used by the TensorPipe RPC agent. When processing CUDA RPC requests, the TensorPipe RPC agent will properly synchronize CUDA streams for all devices in this ``List``. Args: devices (List of int, str, or torch.device): local devices used by the TensorPipe RPC agent. N)rr)r'rrrr�set_devices�s z'TensorPipeRpcBackendOptions.set_devices)�__name__� __module__�__qualname__�__doc__�rpc_contantsZDEFAULT_NUM_WORKER_THREADSZDEFAULT_RPC_TIMEOUT_SECZDEFAULT_INIT_METHOD�int�float�strrr� DeviceTyperr&r*r+� __classcell__rr)
r(rr+s 





@9r)Ztorch._C._distributed_rpcr�rr0r�typingrrrrr1r3r r4rrrrrrrr�
s
 $