Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Rubix caching #20102

Merged
merged 1 commit into from
Feb 13, 2024
Merged

Remove Rubix caching #20102

merged 1 commit into from
Feb 13, 2024

Conversation

wendigo
Copy link
Contributor

@wendigo wendigo commented Dec 13, 2023

Rubix was abandoned 3 years ago, it's broken in majority of production cases with frequent cluster crashes, it has number of high impact CVEs and we are planning to replace it with Alluxio caching. Keeping Rubix in its current form brings more harm than good.

Description

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## Hive connector
* Remove object storage caching support. ({issue}`20102`)

@wendigo wendigo requested review from electrum and martint December 13, 2023 20:17
@cla-bot cla-bot bot added the cla-signed label Dec 13, 2023
Copy link
Member

@mosabua mosabua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we merge the replacement Alluxio caching PR first .. but I am also fine merging this now. At minimum however we need to also remove the documentation.

@wendigo
Copy link
Contributor Author

wendigo commented Dec 14, 2023

@mosabua Rubix and Alluxio are unpatched and unmaintained dependencies that have high score CVEs. Rubix doesn't work so keeping it doesn't bring any value other than false impression that hive caching is actually supported. Yeah, I'd love to see new Alluxio land as it has a higher chance of being maintained, supported and actually working.

Copy link
Member

@mosabua mosabua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets ship it!

@github-actions github-actions bot added the hudi Hudi connector label Dec 14, 2023
@jkylling
Copy link
Contributor

Thank you!

@mosabua
Copy link
Member

mosabua commented Jan 10, 2024

Imho we should merge - provided the CI failures are false alarms.

@martint
Copy link
Member

martint commented Jan 10, 2024

We’re going to merge it after the Alluxio changes get in.

@mosabua
Copy link
Member

mosabua commented Jan 10, 2024

Sounds good @martint - once we have coverage with Alluxio caching for Delta Lake from the current PR or once we have it also for Hive?

@wendigo wendigo mentioned this pull request Feb 1, 2024
@wendigo wendigo force-pushed the serafin/drop-rubix branch from ab9db53 to 3635e6a Compare February 1, 2024 11:53
@wendigo
Copy link
Contributor Author

wendigo commented Feb 1, 2024

Rebased.

@wendigo
Copy link
Contributor Author

wendigo commented Feb 8, 2024

Rebased

Rubix was abandoned 3 years ago, it's broken in majority of production cases with frequent cluster crashes,
it has number of high impact CVEs and we are planning to replace it with Alluxio caching. Keeping Rubix in its
current form brings more harm than good.
@wendigo wendigo merged commit 2be51e4 into master Feb 13, 2024
101 of 102 checks passed
@wendigo wendigo deleted the serafin/drop-rubix branch February 13, 2024 10:47
@github-actions github-actions bot added this to the 439 milestone Feb 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

5 participants