Putting an Arrow table into a Pandas series takes a long time

[mrocklin]

It takes a surprisingly long time to put a list containing a PyArrow table object into an object-dtype Pandas series

In [1]: import numpy as np, pandas as pd, pyarrow as pa

In [2]: df = pd.DataFrame({'x': np.arange(1000000)})

In [3]: %time t = pa.Table.from_pandas(df)
CPU times: user 5.36 ms, sys: 3.22 ms, total: 8.58 ms
Wall time: 7.47 ms

In [4]: %time s = pd.Series([t], dtype=object)
CPU times: user 2.7 s, sys: 114 ms, total: 2.82 s
Wall time: 2.81 s

Originally reported downstream in dask/distributed#2521 by @randerzander

[jreback]

this is essentially a duplicate of #25364

[mrocklin]

Ah, my apologies. I'm not seeing exactly how, but I'm more than happy to trust your interpretation here :) Happy to close if desired.

…

On Wed, Feb 20, 2019 at 8:51 AM Jeff Reback ***@***.***> wrote: this is essentially a duplicate of #25384 <#25384> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#25389 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASszBLB2nNd9xHiex_WmoIsa8ZkWaEEks5vPX0NgaJpZM4bFp6G> .

[jreback]

i had a typo in the issue number ; check again

[mrocklin]

Thanks. Posted there. Closing.