Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Take in account the number of deletes in the LogMergePolicy #115

Closed
fulmicoton opened this issue Apr 8, 2017 · 2 comments
Closed

Take in account the number of deletes in the LogMergePolicy #115

fulmicoton opened this issue Apr 8, 2017 · 2 comments

Comments

@fulmicoton
Copy link
Collaborator

fulmicoton commented Apr 8, 2017

The current LogMergePolicy does not take in account deletes today. As a result, some bad behavior may appear
under the following natural workload.

  • Documents are inserted continually.
  • Documents are deleted continually. For instance as they get too old.

As merge are occuring one very large segment may appear.
It might take a very long time for segment of an equivalent size to appear, so that this segment virtually never get merged.
Yet, all or almost all of its docs might be deleted.

It would be beneficial to refine the heuristics of the log merge policy to take in account the ratio of delete documents as well.

@fulmicoton
Copy link
Collaborator Author

Assigned informally to @blitzerr

@fulmicoton
Copy link
Collaborator Author

@guilload Can you mentor @blitzerr on this ticket?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants