Use UUID instead of parent directory name in computing the file prefix #18183
Conversation
In a table with multiple partition columns, it may happen that there exists a file with the same name and the same value for the least significant partition column which would lead to having the same temporary file name prefix.
| @@ -587,7 +587,7 @@ public HiveWriter createWriter(Page partitionColumns, int position, OptionalInt | |||
| if (sortedWritingTempStagingPathEnabled) { | |||
| String stagingPath = sortedWritingTempStagingPath.replace("${USER}", session.getIdentity().getUser()); | |||
| Location tempPrefix = setSchemeToFileIfAbsent(Location.of(stagingPath)); | |||
| tempFilePath = tempPrefix.appendPath(".tmp-sort.%s.%s".formatted(path.parentDirectory().fileName(), path.fileName())); | |||
| tempFilePath = tempPrefix.appendPath(".tmp-sort.%s.%s".formatted(UUID.randomUUID().toString(), path.fileName())); | |||
There was a problem hiding this comment.
i mean to change the else branch that uses parentDirectory().appendPath ie
perhaps this one needs to be changed as well, however, since path.parentDirectory().fileName() is not well defined .... we should enhance BaseS3AndGlueMetastoreTest tests with sorted tables to guard against potential breakages cc @pajaks (can be for unpartitioned tables only)
|
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua |
|
@findepi and @findinpath I assume you are going to continue work on this PR. |
|
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua |
|
Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time. |
|
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua |
|
@findinpath ping? |
|
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua |
mosabua
added
stale-ignore
Description
In a table with multiple partition columns, it may happen that there exists a file with the same name and the same value for the least significant partition column which would lead to having the same temporary file name prefix.
Discovered while working on #18178
Additional context and related issues
Release notes
(x) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text: