Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Dedup of objects in Stream Output #11430

Open
mgodwan opened this issue Dec 1, 2023 · 1 comment
Open

Support Dedup of objects in Stream Output #11430

mgodwan opened this issue Dec 1, 2023 · 1 comment
Labels
distributed framework enhancement Enhancement or improvement to existing feature or request Indexing Indexing, Bulk Indexing and anything related to indexing Performance This is for any performance related enhancements or bugs

Comments

@mgodwan
Copy link
Member

mgodwan commented Dec 1, 2023

Is your feature request related to a problem? Please describe.
Today, if we write multiple primitives/objects to our streams, we've observed that the objects get duplicated always. We can save a lot of network and serialization/deserialization CPU if we can de-dup based on the references using a.

Describe the solution you'd like
Something similar to HandleTable in ObjectOutputStream used in JDK

Describe alternatives you've considered
Write custom logic for data structures and objects but that may not cover all the use cases.

Additional context

@mgodwan mgodwan added enhancement Enhancement or improvement to existing feature or request untriaged distributed framework and removed untriaged labels Dec 1, 2023
@anasalkouz anasalkouz added the Indexing Indexing, Bulk Indexing and anything related to indexing label Jan 4, 2024
@peternied
Copy link
Member

[Triage - attendees 1 2 3 4]
@mgodwan Thanks for filing, looking forward to seeing this addressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distributed framework enhancement Enhancement or improvement to existing feature or request Indexing Indexing, Bulk Indexing and anything related to indexing Performance This is for any performance related enhancements or bugs
Development

No branches or pull requests

3 participants