Name	Size	Mode	Actions
benchmark.cpython-36.pyc	208	0644	editdlrm
cffi.cpython-36.pyc	1482	0644	editdlrm
compat.cpython-36.pyc	381	0644	editdlrm
compute.cpython-36.pyc	20263	0644	editdlrm
csv.cpython-36.pyc	382	0644	editdlrm
cuda.cpython-36.pyc	405	0644	editdlrm
dataset.cpython-36.pyc	27970	0644	editdlrm
feather.cpython-36.pyc	7739	0644	editdlrm
filesystem.cpython-36.pyc	14402	0644	editdlrm
flight.cpython-36.pyc	1353	0644	editdlrm
fs.cpython-36.pyc	10395	0644	editdlrm
hdfs.cpython-36.pyc	7055	0644	editdlrm
ipc.cpython-36.pyc	7394	0644	editdlrm
json.cpython-36.pyc	231	0644	editdlrm
jvm.cpython-36.pyc	8035	0644	editdlrm
orc.cpython-36.pyc	5140	0644	editdlrm
pandas_compat.cpython-36.pyc	27695	0644	editdlrm
parquet.cpython-36.pyc	66609	0644	editdlrm
plasma.cpython-36.pyc	4069	0644	editdlrm
serialization.cpython-36.pyc	13737	0644	editdlrm
types.cpython-36.pyc	12083	0644	editdlrm
util.cpython-36.pyc	4150	0644	editdlrm
_generated_version.cpython-36.pyc	223	0644	editdlrm
__init__.cpython-36.pyc	16476	0644	editdlrm

3 8%Eg]�@s
dd
lZdd
l
ZddlmZmZmZmZmZm Z m Z mZmZm Z mZmZmZmZmZ
dd
ljZGdd�dej�ZdZGdd�dej�ZGdd �d ej�ZGd d�dej�Zdd �Zd
d
d�dd�Zdje�
e_ dd�Z!d
d
d�dd�Z"dje�
e"_ d dd�
Z#d
d
d�dd�Z$dd�
dd�Z%d
S)!�N)�IpcWriteOptions� ReadStats� WriteStats�Message� MessageReader�RecordBatchReader�_ReadPandasMixin�MetadataVersion�read_message�read_record_batch�read_schema�read_tensor�write_tensor�get_record_batch_size�get_tensor_sizec@seZ
dZd
Zdd�ZdS)�RecordBatchStreamReaderz� Reader for the Arrow streaming binary format. Parameters ---------- source : bytes/buffer-like, pyarrow.NativeFile, or file-like Python object Either an in-memory buffer, or a readable file object. cCs|j|
�

dS)
N)
�_open)�self�source�r�7/usr/local/lib64/python3.6/site-packages/pyarrow/ipc.py�__init__,s
z RecordBatchStreamReader.__init__N)�__name__� __module__�__qualname__�__doc__rrrrrr"sraNParameters ---------- sink : str, pyarrow.NativeFile, or file-like Python object Either a file path, or a writable file object. schema : pyarrow.Schema The Arrow schema for data to be written to the file. options : pyarrow.ipc.IpcWriteOptions Options for IPC serialization. If None, default values will be used: the legacy format will not be used unless overridden by setting the environment variable ARROW_PRE_0_15_IPC_FORMAT=1, and the V5 metadata version will be used unless overridden by setting the environment variable ARROW_PRE_1_0_METADATA_VERSION=1. use_legacy_format : bool, default None Deprecated in favor of setting options. Cannot be provided with options. If None, False will be used unless this default is overridden by setting the environment variable ARROW_PRE_0_15_IPC_FORMAT=1c@s&eZ
dZd
je�
Zddd�dd�ZdS)�RecordBatchStreamWriterz0Writer for the Arrow streaming binary format {}N)�use_legacy_format�optionscCst||�}|j
|
||d
�
dS)N)
r)�_get_legacy_format_defaultr)r�sink�schemarrrrrrLs

z RecordBatchStreamWriter.__init__)rrr�format�_ipc_writer_class_docrrrrrrrGs rc@seZ
dZd
Zddd�
ZdS)�RecordBatchFileReadera�
Class for reading Arrow record batch data from the Arrow binary file format Parameters ---------- source : bytes/buffer-like, pyarrow.NativeFile, or file-like Python object Either an in-memory buffer, or a readable file object footer_offset : int, default None If the file is embedded in some larger file, this is the byte offset to the very end of the file data NcCs|j|
|d
�
dS)N)
� footer_offset)
r)rrr%rrrr^s
zRecordBatchFileReader.__init__)
N)rrrrrrrrrr$Qsr$c@s&eZ
dZd
je�
Zddd�dd�ZdS)�RecordBatchFileWriterz1Writer to create the Arrow binary file format {}N)rrcCst||�}|j
|
||d
�
dS)N)
r)rr)rr r!rrrrrrhs

zRecordBatchFileWriter.__init__)rrrr"r#rrrrrrr&bs r&cCs�|dk r|
dk rtd
�
�
n$|
r>t
|
t�s:tdjt|
�
�
�
�
|
Stj}|dkrbtt t jjdd��
�
}tt t jjdd��
�
r~tj }t||d�S)Nz8Can provide at most one of options and use_legacy_formatz expected IpcWriteOptions, got {}ZARROW_PRE_0_15_IPC_FORMAT�
0ZARROW_PRE_1_0_METADATA_VERSION)r�metadata_version)� ValueError� isinstancer� TypeErrorr"�typer ZV5�bool�int�os�environ�getZV4)rrr(rrrrms












r)rrcCst||
||d
�S)N)rr)
r)r r!rrrrr� new_stream�s


r2z7Create an Arrow columnar IPC stream writer instance {}c

Cst|�
S)
a
Create reader for Arrow streaming format. Parameters ---------- source : bytes/buffer-like, pyarrow.NativeFile, or file-like Python object Either an in-memory buffer, or a readable file object. Returns ------- reader : RecordBatchStreamReader )
r)
rrrr�open_stream�s r3cCst||
||d
�S)N)rr)
r&)r r!rrrrr�new_file�s


r4z5Create an Arrow columnar IPC file writer instance {}cCst||
d
�S)a�
Create reader for Arrow file format. Parameters ---------- source : bytes/buffer-like, pyarrow.NativeFile, or file-like Python object Either an in-memory buffer, or a readable file object. footer_offset : int, default None If the file is embedded in some larger file, this is the byte offset to the very end of the file data. Returns ------- reader : RecordBatchFileReader )
r%)
r$)rr%rrr� open_file�sr5)�nthreads�preserve_indexc
 CsFtj
j||
|d
�}tj�}tj||j��}|j|�

WdQRX|j�S)a� Serialize a pandas DataFrame into a buffer protocol compatible object. Parameters ---------- df : pandas.DataFrame nthreads : int, default None Number of threads to use for conversion to Arrow, default all CPUs. preserve_index : bool, default None The default of None will store the index as a column, except for RangeIndex which is stored as metadata only. If True, always preserve the pandas index data as a column. If False, no index information is saved and the result will have a default RangeIndex. Returns ------- buf : buffer An object compatible with the buffer protocol. )r6r7N)�paZRecordBatchZfrom_pandasZBufferOutputStreamrr!Zwrite_batch�getvalue)�dfr6r7�batchr �writerrrr�serialize_pandas�s




r=T)
�use_threadsc

 Cs4tj
|�
}tj|�
�}|j�}Wd
QRX|j|
d�
S)aR
Deserialize a buffer protocol compatible object into a pandas DataFrame. Parameters ---------- buf : buffer An object compatible with the buffer protocol. use_threads : bool, default True Whether to parallelize the conversion using multiple threads. Returns ------- df : pandas.DataFrame N)
r>)r8ZBufferReaderrZread_allZ to_pandas)�bufr>Z buffer_reader�reader�tablerrr�deserialize_pandas�s


rB)
N)&r/Zpyarrowr8Zpyarrow.librrrrrrrr r rrr rrr�libZ_RecordBatchStreamReaderrr#Z_RecordBatchStreamWriterrZ_RecordBatchFileReaderr$Z_RecordBatchFileWriterr&rr2r"rr3r4r5r=rBrrrr�s"D "