-
Notifications
You must be signed in to change notification settings - Fork 461
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CORE] Support Spark 4.0 and Deprecate JDK 8 #8574
Comments
would you send a email to gluten dev mail list to notify all developers? |
Yes, will send a mail to dev mail list. |
@weiting-chen we may move our CI to JDK17 now but still leave the legacy jdk8 support code there, we still accept PRs for JDK8. Customer who are using jdk8 needs to run the CI from their internal env. Once we release Gluten1.4, we may remove the JDK8 support completely. |
This means we would drop Spark 3.2 and Spark 3.3 support after Gluten 1.4? |
Is it possible to build Gluten with higher JDK version only for |
Can we keep the support for Spark 3.3? |
Description
Current Spark and JDK support status are in below
Spark 3.4.4 support JDK8(8u362) or above version
Spark 3.5.4 support JDK8(8u371) or above version
Spark 4.0 can only support JDK 11 and 17
For better CI resource usage, we can consider to deprecate JDK 8 support when adding Spark 4.0 support.
The plan to support Spark 4.0 will be initialized in Gluten 1.4 and done the work in Gluten 1.5.
One other consideration is in Iceberg cannot support JDK 8 with Spark 3.4.4, we have Iceberg UT may consider to separate the UT testing for Spark 3.4.4 and Iceberg or just support JDK 11/17 for better alignment.
The text was updated successfully, but these errors were encountered: