Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle parsing of invalid S3 content type #17417

Merged
merged 1 commit into from
May 16, 2023
Merged

Handle parsing of invalid S3 content type #17417

merged 1 commit into from
May 16, 2023

Conversation

electrum
Copy link
Member

@electrum electrum commented May 9, 2023

Description

Fixes #16546

Release notes

(x) Release notes are required, with the following suggested text:

# Hive
* Fix query failure for S3 objects with empty `Content-Type` header. ({issue}`16546`)

@cla-bot cla-bot bot added the cla-signed label May 9, 2023
@electrum electrum requested review from raunaqmorarka and hashhar May 9, 2023 22:55
raunaqmorarka
raunaqmorarka approved these changes May 10, 2023
View reviewed changes
Copy link
Member

@raunaqmorarka raunaqmorarka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering why this affected only the new native reader ? Were the old readers using Hadoop's S3 filesystem client ?

@electrum
Copy link
Member Author

The new reader checks the file size up front. I don't think the Hive readers did that, which is why we never saw this problem.

@electrum electrum merged commit baeb9d5 into trinodb:master May 16, 2023
@electrum electrum deleted the hive-s3-mediatype branch May 16, 2023 06:04
@github-actions github-actions bot added this to the 418 milestone May 16, 2023
@colebow colebow mentioned this pull request May 17, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@raunaqmorarka raunaqmorarka raunaqmorarka approved these changes

@hashhar hashhar Awaiting requested review from hashhar

Assignees
No one assigned
Labels
cla-signed
Milestone
418
Development

Successfully merging this pull request may close these issues.

Queries fail when using native text file reader on S3 objects with empty Content-Type header
2 participants
@electrum @raunaqmorarka