Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] JsonToStructs does not fallback when columnNameOfCorruptRecord is set #10484

Closed
revans2 opened this issue Feb 23, 2024 · 1 comment
Closed

[BUG] JsonToStructs does not fallback when columnNameOfCorruptRecord is set #10484

revans2 opened this issue Feb 23, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@revans2
Copy link
Collaborator

revans2 commented Feb 23, 2024

Describe the bug
ScanJson has this in it, and ScanCSV also, but even though from_json reuses a lot of the fallback checks this appears to be one that was missed. If we see that the schema includes a column with the same name as the config columnNameOfCorruptRecord, we should fall back to the CPU, because we don't have a good way to support this yet, nor anyone asking for it.
@revans2 revans2 added bug Something isn't working ? - Needs Triage Need team to review and classify labels Feb 23, 2024
@revans2 revans2 mentioned this issue Feb 26, 2024
Open
62 tasks
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Feb 27, 2024
@revans2
Copy link
Collaborator Author

revans2 commented Mar 13, 2024

This was fixed as a part of #10542

@revans2 revans2 closed this as completed Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@revans2 @mattahrens