Name	Size	Mode	Actions
api.cpython-36.pyc	26470	0644	editdlrm
backend_registry.cpython-36.pyc	8029	0644	editdlrm
constants.cpython-36.pyc	665	0644	editdlrm
functions.cpython-36.pyc	7398	0644	editdlrm
internal.cpython-36.pyc	7948	0644	editdlrm
options.cpython-36.pyc	6943	0644	editdlrm
rref_proxy.cpython-36.pyc	1585	0644	editdlrm
server_process_global_profiler.cpython-36.pyc	7839	0644	editdlrm
__init__.cpython-36.pyc	6409	0644	editdlrm

3 Eg4)�@s�dd
lZdd
l
Z
dd
lZdd
lZdd
lZdd
lZdd
lZddlmZ
dd
l Z dd
l jZddl mZ
ej�aejZejZGdd�de�ZGdd�d�Ze�Zdd �Zd d�Zdd �Zdd�Zdd�Zdd�Zejddddg�Zejdddg�Z d
S)�N)
�Enum)
�_get_current_rpc_agentc
@seZ
dZd
ZdZdZdZdS)�RPCExecMode�sync�asyncZ async_jit�remoteN)�__name__� __module__�__qualname__ZSYNC�ASYNCZ ASYNC_JITZREMOTE�rr�J/usr/local/lib64/python3.6/site-packages/torch/distributed/rpc/internal.pyrs



rc@steZ
dZd
Zdd�Zdd�Zedd��
Zdd �Zed d��
Z dd �Z dd�Zedd��
Zdd�Z dd�Zdd�ZdS)�_InternalRPCPicklera  This class provides serialize() and deserialize() interfaces to serialize data to be "binary string + tensor table" format So for RPC python UDF function and args, non tensor data will be serialized into regular binary string, tensor data will be put into thread local tensor tables, this serialization format is consistent with builtin operator and args using JIT pickler. This format will make tensor handling in C++ much easier, e.g. attach tensor to distributed autograd graph in C++ c

Cs$tj
j�|_|j|jtj<i|_dS)
N)�copyreg�dispatch_table�copy�_dispatch_table�_tensor_reducer�torchZTensor�_class_reducer_dict)
�selfrrr �__init__(s
z_InternalRPCPickler.__init__cCs|
|jkr||j|
<dS)
N)
r)rZ obj_class�reducerrrr �_register_reducer/s
z%_InternalRPCPickler._register_reducercCs tj
|
S)
N)�_thread_local_tensor_tables�recv_tables)�cls�tensor_indexrrr �_tensor_receiver4sz$_InternalRPCPickler._tensor_receivercCs&tj
j|
�

ttj
�
d
}tj|f
fS)N�
)r�send_tables�append�lenrr)rZtensorrrrr r9s

z#_InternalRPCPickler._tensor_reducercCstj
jj|
�
S)
N)�dist�rpc�PyRRefZ_deserialize)r�rref_fork_datarrr �_py_rref_receiver?sz%_InternalRPCPickler._py_rref_receivercCs|
j�}t
j|f
fS)
N)� _serializerr')rZpy_rrefr&rrr �_py_rref_reducerCs

z$_InternalRPCPickler._py_rref_reducercCs |j|
�
S)
N)
r))rZrrefrrr � _rref_reducerGs
z!_InternalRPCPickler._rref_reducercCstj
|
�
}tjj|�
}|S)
z� Given a serialized representation of a ScriptModule created with torch.jit.save, loads and returns the ScriptModule. )�io�BytesIOr�jit�load)rZscript_module_serialized�
f�
mrrr �_script_module_receiverJs

z+_InternalRPCPickler._script_module_receivercCs&tj
�}tjj|
|�
tj|j�f
fS)
z, Serializes a ScriptModule. )r+r,rr-�saverr1�getvalue)rZ script_moduler/rrr �_script_module_reducerTs

z*_InternalRPCPickler._script_module_reducercCs�tj
�}t|�
}|j|_|j|jtjj<|j |jtjj <t|
tj j�rT|j|j|
j<x"|jj�D]}|j||j|<q`Wttd
�r�tj}nd}gt_|j|
�

tj}|dk r�|t_nt`|j�|fS)ze Serialize non tensor data into binary string, tensor data into tensor table r N)r+r,�_picklerrrr)r#r$r%r*ZRRef� isinstancerr-ZScriptModuler4� __class__r�keys�hasattrrr �dumpr3)r�objr/�
p� class_nameZold_send_tablesZtensorsrrr � serialize\s&




 

z_InternalRPCPickler.serializecCs�tt
d
�rt
j}nd}|t
_yttj|
�
�
}|j�}Wn6tk rl
}
zt|�
d}t|�
}WYdd}~XnX|dk r~|t
_nt
`|S)zJ Deserialize binary string + tensor table to original obj rNz� Default RPC pickler does not serialize function code. Ensure that UDFs are defined on both caller and callee modules.) r9rr� _unpicklerr+r,r.�AttributeError�str)r�binary_data�tensor_tableZold_recv_tablesZ unpickler�ret�
e� except_strrrr �deserialize�s





z_InternalRPCPickler.deserializeN)rr r �__doc__rr�classmethodrrr'r)r*r1r4r>rGrrrr rs  5rc

Cs tj
|�
S)
N)�_internal_rpc_picklerr>)
r;rrr r>�s
r>cCstj
||
�S)
N)rJrG)rBrCrrr rG�s
rGc
Cs�y"t|t
�r|�
|j|j|j�
}
Wnbtk r�
}
zFd
t�j��dt|�
�dt j ���}t|tj d�
t|t|�
�}
WYdd}~XnX|
S)z� This function is exclusively called from C++. See ``torch/csrc/distributed/rpc/python_rpc_handler.cpp``. Runs a Python UDF and returns its return value. Wraps any exception in ``RemoteException`` if the function raises. zOn z: �
)
�fileN)r6r@�func�args�kwargs� ExceptionrZget_worker_info�repr� traceback� format_exc�print�sys�stderr�RemoteException�type)Z python_udf�resultrErFrrr � _run_function�s



$

rZc

Cs&t|t
�r"|j|jjd
�
jd�
�
�
dS)Nzutf-8Zunicode_escape)r6rW�exception_type�msg�encode�decode)
rYrrr �_handle_exception�s

r_cCsd
j|j
|
||d�}|S)a�
Builds the key that RPC calls are profiled with using the autograd profiler. This will be the name of the corresponding Event recorded in the profiler. Args: exec_type (RPCExecMode): Type of RPC/RRef call func_name (str): Name of function being profiled. current_worker_name (str): Name of current worker. dst_worker_name (str): Name of the destination worker. Returns: String representing profiling key z {dst_worker}))Zrpc_type� func_nameZcurrent_workerZ dst_worker)�format�value)� exec_typer`�current_worker_nameZdst_worker_name�profile_keyrrr �_build_rpc_profiling_key�s



rfcCsDtj
j�std
�
�
dj|jt|
�
||�}tj
j�}tj
j||�
|S)ar This function should be called from RPC/RRef functions to create a RecordFunction object for profiling. This function also runs the before callbacks that start the profiling, though the user is responsible for running the appropriate callbacks when the function to be profiled finishes. Args: exec_type (RPCExecMode): Type of RPC/RRef call func_name (str): Name of function being profiled. current_worker_name (str): Name of current worker. dest_worker_name (str): Name of the destination worker. Returns: An instance of `torch.autograd._RecordFunction`. z$Autograd profiler should be enabled.zrpc_{}#{}({} -> {})) rZautogradZ_profiler_enabled�AssertionErrorrarbrAZ_RecordFunctionZ_run_before_callbacks)rcr`rdZdest_worker_namere�rfrrr �_start_record_function�s



ri� PythonUDFrMrNrOrWr\r[)!�collectionsrr+�picklerU� threadingrR�enumrrZtorch.distributedZdistributedr#Ztorch._C._distributed_rpcr�localr�Picklerr5� Unpicklerr?rrrJr>rGrZr_rfri� namedtuplerjrWrrrr �
s2