Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tool: add level/seqnum invariant check #366

Closed
petermattis opened this issue Oct 24, 2019 · 2 comments
Closed

tool: add level/seqnum invariant check #366

petermattis opened this issue Oct 24, 2019 · 2 comments
Assignees

Comments

@petermattis
Copy link
Collaborator

RocksDB and Pebble have an invariant that for a given key the sequence numbers in higher levels will be larger than those in lower levels. This invariant also applies to the sstables within L0 and between L0 tables and data in memtables. Violating this invariant can cause all sorts of badness. Recently, a bug in RocksDB due to an interaction between intra-L0 compactions and ingestion allowed this invariant to be violated.

We should add a command which checks this invariant for a DB. This would have to walk over all of the data in the database. On the surface it seems similar to mergingIter, and it is possible that this can be a mode which mergingIter is run in. We wouldn't want to impact the performance of mergingIter, and we'd want this check to examine all keys, including range tombstones, and never skip keys. So perhaps not mergingIter, but similar.

In addition to an offline check, we should make sure that this is structured so that we can perform this check at the end of tests.

@sumeerbhola sumeerbhola self-assigned this Nov 18, 2019
@petermattis
Copy link
Collaborator Author

With #406 merged, is there anything left to do here?

@sumeerbhola
Copy link
Collaborator

This is done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants