Fixed empty reader panic for NDJSON type infer

[Roberto-XY]

Hi there,
I am currently in the process of learning some Rust & what better way then to contribute ;)

• Now returns an error on empty readers instead of panic in infer. In deserialize as well, to have consistent behavior.
• json::coerce_data_type fails if there are no lines to be read from reader #911 also suggested handling that as DataType::Null. I decided against that, since an empty string is not valid JSON. But null is, which also panicked—fixed that. Or do you prefer a separate PR?
• added some explicit tests for null to coerce_data_type

Fixes #911

[Roberto-XY]

Is this intended? Should it not be just DataType::Boolean?

Similarly, {"a": null}" is currently infered as DataType::Struct(vec![])and not DataType::Struct(vec![Field::new("a", DataType::Null, true)])

[jorgecarleitao]

Good observation. I think it is intended - infer_array does not pass DataType::Null to coerce_data_type since arrow arrays are nullable and thus we handle Null as part before passing to coerce_data_type. The idea is here.

With that said, imo it is not the cleanest design and maybe we should clean it up?

[codecov]

Codecov Report

Merging #974 (bf3dc2a) into main (fa34796) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #974      +/-   ##
==========================================
+ Coverage   71.50%   71.52%   +0.02%     
==========================================
  Files         356      356              
  Lines       19671    19679       +8     
==========================================
+ Hits        14065    14076      +11     
+ Misses       5606     5603       -3     

Impacted Files	Coverage Δ
src/io/json/read/infer_schema.rs	91.46% <100.00%> (+0.21%)	⬆️
src/io/ndjson/read/deserialize.rs	100.00% <100.00%> (ø)
src/io/ndjson/read/file.rs	80.00% <100.00%> (+1.42%)	⬆️
src/io/json/read/deserialize.rs	67.54% <0.00%> (+0.66%)	⬆️
src/array/equal/mod.rs	88.54% <0.00%> (+2.08%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa34796...bf3dc2a. Read the comment docs.

[jorgecarleitao]

This looks wonderful. Thanks a lot, very clean and super easy to follow.

Feel welcome to use this project to learn Rust - e.g. via incomplete PRs, PR reviews and/or challenging the design or code. :)

[jorgecarleitao]

Good observation. I think it is intended - infer_array does not pass DataType::Null to coerce_data_type since arrow arrays are nullable and thus we handle Null as part before passing to coerce_data_type. The idea is here.

With that said, imo it is not the cleanest design and maybe we should clean it up?