Add Alluxio file system #21603
Add Alluxio file system #21603
Conversation
JiamingMai
force-pushed
the
add-alluxio-file-system
branch
from
1aecc4c to
7e84237
Compare
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
4 similar comments
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
4 similar comments
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
JiamingMai
force-pushed
the
add-alluxio-file-system
branch
from
b724c3d to
ae34ab1
Compare
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
JiamingMai
force-pushed
the
add-alluxio-file-system
branch
from
ae34ab1 to
1e86c3c
Compare
github-actions
bot
added
docs
hudi
jja725
force-pushed
the
add-alluxio-file-system
branch
2 times, most recently
from
41c4bc4 to
2367351
Compare
|
@electrum All comments resolved, I can rebase the commits if everything looks good. |
jja725
force-pushed
the
add-alluxio-file-system
branch
from
2367351 to
f2dbb56
Compare
|
@mosabua I believe the test failure(trino-google-sheets) is not related to the change. Do you mind helping me rerun the test ? |
|
No need to rerun the build ... the google sheets failure is definitely a false alarm. |
|
@mosabua yeah, it's super flaky recently |
|
Thanks @JiamingMai for a great contribution! |
|
Docs are in as well now |
| import static org.assertj.core.api.Assertions.assertThat; | ||
|
|
||
| @TestInstance(TestInstance.Lifecycle.PER_CLASS) | ||
| public abstract class AbstractTestAlluxioFileSystem |
There was a problem hiding this comment.
Why do we need an abstract class? Can we merge into TestAlluxioFileSystem?
| @Test | ||
| public void testSimplifyPath() | ||
| { | ||
| String path = "test/level0-file0"; |
| String path = "test/level0-file0"; | ||
| assertThat(path).isEqualTo(AlluxioUtils.simplifyPath(path)); | ||
| path = "a/./b/../../c/"; | ||
| assertThat("c/").isEqualTo(AlluxioUtils.simplifyPath(path)); |
There was a problem hiding this comment.
The call order is wrong. It should be
assertThat(AlluxioUtils.simplifyPath("a/./b/../../c/")).isEqualTo("c/");Same for others.
This comment was marked as resolved.
This comment was marked as resolved.
The only failing test yesterday was the google sheets one, which is unrelated |
| void testContainer() | ||
| { | ||
| assertThat(ALLUXIO_MASTER_CONTAINER.isRunning()).isTrue(); | ||
| assertThat(ALLUXIO_WORKER_CONTAINER.isRunning()).isTrue(); |
There was a problem hiding this comment.
What's the motivation to verify containers' status here?
There was a problem hiding this comment.
We can remove it in a later PR. it's mostly for verify the docker status during the test
| } | ||
|
|
||
| @Test | ||
| public void convertToLocation() |
There was a problem hiding this comment.
test prefix is missing.
Please follow https://trino.io/docs/current/develop/tests.html
- Test classes should be defined as package-private and final.
- Test method must start with test, for example testExplain()
- Test methods should be defined as package-private.
Description
Add
trino-filesystem-alluxiomodule to implement the new interfaces (TrinoFileSystem,TrinoInput,TrinoInputFile,TrinoOutputFile) and support native Alluxio.Additional context and related issues
Docs in #23269
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
(x) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: