Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Specify recover_with_null when reading JSON files #9326

Closed
andygrove opened this issue Sep 28, 2023 · 0 comments · Fixed by #9304
Closed

[FEA] Specify recover_with_null when reading JSON files #9326

andygrove opened this issue Sep 28, 2023 · 0 comments · Fixed by #9304
Assignees
Labels
feature request New feature or request

Comments

@andygrove
Copy link
Contributor

andygrove commented Sep 28, 2023

Is your feature request related to a problem? Please describe.
cuDF recently added an option that we requested (PR), where we can specify a recovery mode of recover_with_null so that invalid lines in a newline-delimited JSON file produce NULL values rather than throwing an exception. This issue is for updating the plugin to use this new feature.

Note that an initial attempt at implementing this uncovered some edge cases that need addressing in cuDF:

Describe the solution you'd like
Specify this new option when reading JSON from file and when parsing JSON with from_json in spark-rapids-jni.

Describe alternatives you've considered
None

Additional context

@andygrove andygrove added feature request New feature or request ? - Needs Triage Need team to review and classify labels Sep 28, 2023
@andygrove andygrove self-assigned this Sep 28, 2023
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Sep 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants