HBASE-27619 Bulkload fails when trying to bulkload files with invalid names after HBASE-26707 #5014
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… names after HBASE-26707 Change-Id: I2107387522b5d964d7a0057ba3e06a2acda6488d
wchevreuil
requested review from
ankitsinghal,
petersomogyi,
taklwu and
Apache9
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
taklwu
reviewed
Feb 8, 2023
| @@ -7280,7 +7280,7 @@ public Map<byte[], List<Path>> bulkLoadHFiles(Collection<Pair<byte[], String>> f | |||
| reqTmp ? null : fs.getRegionDir().toString()); | |||
| } | |||
| Pair<Path, Path> pair = null; | |||
| if (reqTmp) { | |||
| if (reqTmp || (!StoreFileInfo.isHFile(finalPath))) { | |||
There was a problem hiding this comment.
nit: maybe remove the extra () ?
Suggested change
| if (reqTmp || (!StoreFileInfo.isHFile(finalPath))) { | |
| if (reqTmp || !StoreFileInfo.isHFile(finalPath)) { |
do you have any stacktrace and example of the non-HFile ? also , when you said has been split in the bulkload process , what would this split generate?
There was a problem hiding this comment.
do you have any stacktrace and example of the non-HFile ?
For the case of split files, BulkLoadHFilesTool splits the file into two, naming the resulting files with " .bottom/.top" suffixes here. In this case, when using SFT, we'll put these files into the store dir, then when StoreFileReader tries to load it, it will try to resolve it as a reference, giving the following errors:
- At the bulkload job side:
ERROR org.apache.hadoop.hbase.regionserver.HRegion: There was a partial failure due to IO when attempting to load CF : s3a://my-s3-bucket/my-datalake/hbase/data/default/my-table/2462397a71fa06cff0c5598434a02c5f/CF/e245ce23ae9b48ebac7713568ce3a569.top
- At RS side:
FileNotFoundException: No such file or directory: s3a://my-s3-bucket/my-datalake/hbase/data/default/my-table/top/CF/e245ce23ae9b48ebac7713568ce3a569
Note in the path of error #2 that the RS assumed the file was a reference and tried tor resolve it as such.
There's also the possibility clients pass files with invalid naming format. These would also be placed in the store dir as it is, then failing the validations performed by StoreFileReader during file loading. These would manifest such as below, on RS logs:
2023-02-08T10:27:26,447 ERROR [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=49900] regionserver.HRegion(7346): There was a partial failure due to IO when attempting to load myfam : s3a://my-s3-bucket/my-datalake/hbase/data/default/my-table/3c91abacbd1a6c41d7247bd3d5a6bb00/CF/hfile_0 java.io.IOException: path=s3a://my-s3-bucket/my-datalake/hbase/data/default/my-table/3c91abacbd1a6c41d7247bd3d5a6bb00/CF/hfile_0 doesn't look like a valid StoreFile
These two are also easily reproducible by the UT added here.
Change-Id: I02f44c6eb6f71ec87184e083c0a3c6a68fdda6cc
|
🎊 +1 overall
This message was automatically generated. |
taklwu
approved these changes
Feb 8, 2023
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
wchevreuil
added a commit
that referenced
this pull request
Feb 9, 2023
… names after HBASE-26707 (#5014) Signed-off-by: Tak Lon (Stephen) Wu <[email protected]>
wchevreuil
added a commit
that referenced
this pull request
Feb 9, 2023
… names after HBASE-26707 (#5014) Signed-off-by: Tak Lon (Stephen) Wu <[email protected]>
vinayakphegde
pushed a commit
to vinayakphegde/hbase
that referenced
this pull request
Apr 4, 2024
…d files with invalid names after HBASE-26707 (apache#5014) Signed-off-by: Tak Lon (Stephen) Wu <[email protected]> Change-Id: Ia71686a35664dcc5b8944afe54772012efaa2117
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.