Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TASK] Rename cache serializer to a common name #2975

Closed
jlowe opened this issue Jul 20, 2021 · 3 comments
Closed

[TASK] Rename cache serializer to a common name #2975

jlowe opened this issue Jul 20, 2021 · 3 comments
Labels
duplicate This issue or pull request already exists task Work required that improves the product but is not user facing

Comments

@jlowe
Copy link
Member

jlowe commented Jul 20, 2021

After #2905 all that remains is the Spark 3.1.1 cache serializer but it is common to all Spark versions that support a pluggable cache serializer. This class should be renamed to a common package and moved out of the Spark 3.1.1 shim project.

@jlowe jlowe added the ? - Needs Triage Need team to review and classify label Jul 20, 2021
@jlowe jlowe added the task Work required that improves the product but is not user facing label Jul 20, 2021
@Salonijain27 Salonijain27 removed the ? - Needs Triage Need team to review and classify label Jul 20, 2021
@sperlingxx
Copy link
Collaborator

I am wondering how to adapt Spark 3.0.x if we move the serializer to common package. The CachedBatchSerializer was introduced at 3.1.

@jlowe
Copy link
Member Author

jlowe commented Aug 3, 2021

The module would be common to all Spark versions >= 3.1.1 rather than in a specific shim directory as it is now. That module would not compile against Spark 3.0.x.

@Salonijain27
Copy link
Contributor

duplicate of #3314

@sameerz sameerz added the duplicate This issue or pull request already exists label Sep 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists task Work required that improves the product but is not user facing
Projects
None yet
Development

No branches or pull requests

4 participants