Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Support multiple compressed inputs in read_json #12299

Closed
vuule opened this issue Dec 3, 2022 · 1 comment · Fixed by #17161
Closed

[FEA] Support multiple compressed inputs in read_json #12299

vuule opened this issue Dec 3, 2022 · 1 comment · Fixed by #17161
Labels
cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.

Comments

@vuule
Copy link
Contributor

vuule commented Dec 3, 2022

JSON reader currently concatenates data from all input sources before (optional) decompression and parsing.
This is not correct for compressed input files, as each file is compressed separately.
#12285 disables multiple inputs for compressed data.
read_json should decompress each input file before concatenating the decompressed data.

@vuule vuule added feature request New feature or request cuIO cuIO issue labels Dec 3, 2022
@karthikeyann
Copy link
Contributor

Note that Only a single source is currently supported in read_csv.
This work could help expand the same feature to csv reader.

@GregoryKimball GregoryKimball added the libcudf Affects libcudf (C++/CUDA) code. label Apr 2, 2023
@karthikeyann karthikeyann added this to the Nested JSON reader milestone Nov 12, 2024
@rapids-bot rapids-bot bot closed this as completed in 03ac845 Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants