JNI: Pass names of children struct columns to native Arrow IPC writer [skip ci]

[firestarman]

This PR is to add the support of building the structure of column metadata from the flattened column names according to the table schema.
Since the children column metadata is required when converting cudf tables to arrow tables.

Also updating the related unit tests.

closes #7570

Signed-off-by: Firestarman [email protected]

[codecov]

Codecov Report

Merging #7598 (16fa512) into branch-0.19 (7871e7a) will increase coverage by 0.52%.
The diff coverage is 93.22%.

❗ Current head 16fa512 differs from pull request most recent head 54395f0. Consider uploading reports for the commit 54395f0 to get more accurate results

@@               Coverage Diff               @@
##           branch-0.19    #7598      +/-   ##
===============================================
+ Coverage        81.86%   82.38%   +0.52%     
===============================================
  Files              101      101              
  Lines            16884    17350     +466     
===============================================
+ Hits             13822    14294     +472     
+ Misses            3062     3056       -6     

Impacted Files	Coverage Δ
python/cudf/cudf/core/index.py	93.34% <ø> (+0.48%)	⬆️
python/cudf/cudf/core/column/column.py	87.83% <83.33%> (+0.07%)	⬆️
python/cudf/cudf/core/column/numerical.py	94.85% <85.71%> (-0.17%)	⬇️
python/cudf/cudf/core/frame.py	89.12% <89.47%> (+0.10%)	⬆️
python/cudf/cudf/core/column/decimal.py	92.75% <90.32%> (-2.12%)	⬇️
python/cudf/cudf/core/dataframe.py	90.58% <95.00%> (+0.11%)	⬆️
python/cudf/cudf/core/series.py	91.57% <95.55%> (+0.78%)	⬆️
python/cudf/cudf/core/column/string.py	86.76% <100.00%> (+0.26%)	⬆️
python/cudf/cudf/core/column_accessor.py	95.45% <100.00%> (+0.14%)	⬆️
python/cudf/cudf/core/dtypes.py	91.13% <100.00%> (+1.40%)	⬆️
... and 55 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5fea6ad...54395f0. Read the comment docs.

[firestarman]

rerun tests

[jlowe]

It's not necessary to refactor this to the flattened name approach, but there are some resource leaks that are possible with the approach used here that are not with the flattened approach and should be fixed.

[firestarman]

It's not necessary to refactor this to the flattened name approach, but there are some resource leaks that are possible with the approach used here that are not with the flattened approach and should be fixed.

I updated to align with the flattened name approach, and it is a good suggestion, because it not only reduces the code change, but also hides some column metadata details (e.g. stub meta for list type) from Java.

[firestarman]

rerun tests

[jlowe]

Thanks for updating @firestarman, this did get a lot cleaner overall. The main thing I see missing now is that the behavior of column names for nested types isn't documented in the Java APIs anywhere. If we're going the flattened names route for all writers then this should be documented on WriterBuilder, but if the flattening logic is only going to apply to Arrow IPC then its builder should override the withColumnNames method if only to provide documentation on the expected behavior.

cc: @revans2 for visibility

[firestarman]

Thanks for updating @firestarman, this did get a lot cleaner overall. The main thing I see missing now is that the behavior of column names for nested types isn't documented in the Java APIs anywhere. If we're going the flattened names route for all writers then this should be documented on WriterBuilder, but if the flattening logic is only going to apply to Arrow IPC then its builder should override the withColumnNames method if only to provide documentation on the expected behavior.

I think it is only for Arrow IPC now, so I updated its builder to override the two withXXXXNames for the documentation.

[firestarman]

@gpucibot merge

[firestarman]

@gpucibot merge

[firestarman]

Thanks Jason, learnt a lot