Wrap stream from HadoopInputFile to work around #5201 #8504
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
electrum
approved these changes
Jul 14, 2021
plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/HdfsInputFile.java
Outdated
Show resolved
Hide resolved
This removes the flakiness in Parquet writer by avoiding unwarranted closing of in-use input stream. For more details, see trinodb#5201
phd3
changed the title
phd3
changed the title
|
Unrelated #8477 |
findepi
reviewed
Jul 15, 2021
Comment on lines
+60
to
+61
| // Hack: this wrapping is required to circumvent https://github.com/trinodb/trino/issues/5201 | ||
| return new HdfsInputStream(environment.doAs(user, delegate::newStream)); |
There was a problem hiding this comment.
How does wrapping help alleviate #5201? what's the mechanics?
There was a problem hiding this comment.
@findepi does this help with mechanics? #5201 (comment)
There was a problem hiding this comment.
i.e. the "rewrapping" is not done. https://github.com/apache/iceberg/blob/master/parquet/src/main/java/org/apache/iceberg/parquet/ParquetIO.java#L93
This was referenced Jul 22, 2021
Closed
phd3
added a commit
to phd3/trino
that referenced
this pull request
Jul 27, 2021
The root cause was fixed in trinodb#8504.
findepi
pushed a commit
that referenced
this pull request
Jul 29, 2021
The root cause was fixed in #8504.
This was referenced Nov 10, 2021
liupan664021
added a commit
to liupan664021/presto
that referenced
this pull request
Nov 10, 2021
Cherry-pick of trinodb/trino#8504 Co-authored-by: Pratham Desai <[email protected]>
beinan
pushed a commit
to prestodb/presto
that referenced
this pull request
Nov 18, 2021
Cherry-pick of trinodb/trino#8504 Co-authored-by: Pratham Desai <[email protected]>
sumannewton
pushed a commit
to sumannewton/trino
that referenced
this pull request
Jan 17, 2022
The root cause was fixed in trinodb#8504.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#5201 (comment)