Added short-term caching of the first column in the channel manifest #180
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Github Issue
None
What Are We Doing Here?
An inefficiency was uncovered when the head of the "manifest" table for queues and databus subscriptions have accrued many tombstones which have not yet been compacted away. Since every poll re-reads the manifest from the beginning the entire poll is slowed down as Cassandra scans over the tombstoned records. This PR briefly caches the oldest know slab ID in the manifest plus a 1 minute buffer. Future queries can be run starting at this manifest to bypass any tombstones from older, fully read and deleted slabs.
How to Test and Verify
There is no test specifically for this condition. The most important test is regression.
Risk
This is a fairly low-risk update. Even though it is at the heart of databus and queue channels the caching should serve as an optimization without risking that any manifest data goes completely unread.
Level
Medium
Required Testing
Regression
Code Review Checklist
Tests are included. If not, make sure you leave us a line or two for the reason.
Pulled down the PR and performed verification of at least being able to
build and run.
Well documented, including updates to any necessary markdown files. When
we inevitably come back to this code it will only take hours to figure out, not
days.
Consistent/Clear/Thoughtful? We are better with this code. We also aren't
a victim of rampaging consistency, and should be using this course of action.
We don't have coding standards out yet for this project, so please make sure to address any feedback regarding STYLE so the codebase remains consistent.
PR has a valid summary, and a good description.