[Python] pyarrow.scalar doesn't accept nested pyarrow values

[asfimport]

Easiest explained in code:

import pyarrow as pa

t1 = pa.struct([('x', pa.int64())])
t2 = pa.struct([('y', t1)])

t1_py = {'x', 0}
t1_pa = pa.scalar(t1_py, type=t1)

t2_py = {'y': {'x': 0}}
t2_pa = pa.scalar(t2_py, type=t2)  # works

t2_mixed = {'y': t1_pa}
t2_pa_mix = pa.scalar(t2_mixed, type=t2)  # fails
assert t2_pa_mix == t2_pa

Is there some other way to construct the pyarrow struct value corresponding to t2_pa given t1_pa directly, without going through python first (i.e. constructing t1_py)?

Reporter: Vlad Firoiu

_{Note: This issue was originally created as ARROW-15050. Please see the migration documentation for further details.}

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
[~vladfi] thanks for the report. I think this essentially comes back to ARROW-5295 (which is a similar issue but for arrays, but both arrays and scalar functions for the python->arrow conversion use the same code path I think).

The current generic python->arrow conversion layer cannot handle our own scalars, but only Python scalars (+ some numpy and pandas ones)