Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: add MVCCTimeInterval block property for range keys #82667

Merged
merged 1 commit into from
Jun 27, 2022

Conversation

erikgrinaker
Copy link
Contributor

@erikgrinaker erikgrinaker commented Jun 9, 2022

This patch adds MVCCTimeInternal block property collection and
filtering for range keys, which allows using time-bound iterators with
range keys.

Range keys will only be written once the MVCCRangeTombstones version
gate is enabled.

Resolves #82596.

Release note: None

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@erikgrinaker
Copy link
Contributor Author

@jbowens This doesn't work because of the in-mem arena right? I'll just let this sit until we land compaction and can enable durability then.

@erikgrinaker erikgrinaker force-pushed the mvcc-range-tombstones-tbi branch from 8875d24 to a842942 Compare June 9, 2022 22:22
Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, but even after compaction lands, I think b-c@{5,6,7,8} will all end up in the same range-key block. Overlapping 'stacks' of range keys flushed at the same time will be output together in a single block (and really, internally a single key in a single block). To get each range key to be in a separate block you can either write them with different bounds, or flush between writes.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @aliher1911, @itsbilal, and @nicktrav)

Copy link
Contributor Author

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, figured as much, thanks.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @aliher1911, @itsbilal, and @nicktrav)

@erikgrinaker erikgrinaker force-pushed the mvcc-range-tombstones-tbi branch from a842942 to c3d4c24 Compare June 10, 2022 07:15
@erikgrinaker erikgrinaker force-pushed the mvcc-range-tombstones-tbi branch 2 times, most recently from af09932 to 078a093 Compare June 18, 2022 09:09
@erikgrinaker
Copy link
Contributor Author

Looks like DataBlockIntervalCollector receives the encoded range key, so I'll need access to decoding functions to extract the suffix. Opened cockroachdb/pebble#1774.

@erikgrinaker erikgrinaker force-pushed the mvcc-range-tombstones-tbi branch 4 times, most recently from c40ad50 to 9335edf Compare June 26, 2022 12:05
@erikgrinaker erikgrinaker marked this pull request as ready for review June 26, 2022 12:07
@erikgrinaker erikgrinaker requested a review from a team as a code owner June 26, 2022 12:07
Copy link
Contributor Author

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is ready for review now.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @aliher1911, @itsbilal, and @nicktrav)


pkg/storage/pebble.go line 330 at r4 (raw file):

// pebbleDataBlockMVCCTimeIntervalPointCollector implements
// pebble.DataBlockIntervalCollector for point keys.
type pebbleDataBlockMVCCTimeIntervalPointCollector struct {

I split out separate implementations for point and range keys to avoid having to check the key type (point vs range) in Add(), since Pebble already checks it for us when deciding which property collector to use. We could keep a common implementation if we're not too worried about checking this twice.


pkg/storage/pebble_test.go line 634 at r3 (raw file):

	// Range keys [b-c)@5, [c-d)@7, [d-e)@9 in separate blocks in a single SST.
	//
	// TODO(erikgrinaker): Seems like these all end up in the same block anyway?

@jbowens: seems like range keys end up in the same block even though the don't overlap. Is this expected, given BlockSize=1?

Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @aliher1911, @erikgrinaker, @itsbilal, and @nicktrav)


pkg/storage/pebble.go line 330 at r4 (raw file):

Previously, erikgrinaker (Erik Grinaker) wrote…

I split out separate implementations for point and range keys to avoid having to check the key type (point vs range) in Add(), since Pebble already checks it for us when deciding which property collector to use. We could keep a common implementation if we're not too worried about checking this twice.

it's all pretty simple, so the code duplication doesn't bother me 👍


pkg/storage/pebble.go line 353 at r4 (raw file):

var (
	_ sstable.DataBlockIntervalCollector      = &pebbleDataBlockMVCCTimeIntervalRangeCollector{}
	_ sstable.SuffixReplaceableBlockCollector = (*pebbleDataBlockMVCCTimeIntervalRangeCollector)(nil)

nit: is there a reason to not use (*pebbleDataBlockMVCCTimeIntervalRangeCollector)(nil) for both of these interface assertions?


pkg/storage/pebble_test.go line 634 at r3 (raw file):

Previously, erikgrinaker (Erik Grinaker) wrote…

@jbowens: seems like range keys end up in the same block even though the don't overlap. Is this expected, given BlockSize=1?

I didn't expect it, but after thinking about it, yeah, it makes sense. Created cockroachdb/pebble#1788.

This patch adds `MVCCTimeInternal` block property collection and
filtering for range keys, which allows using time-bound iterators with
range keys.

Range keys will only be written once the `MVCCRangeTombstones` version
gate is enabled.

Release note: None
@erikgrinaker erikgrinaker force-pushed the mvcc-range-tombstones-tbi branch from 9335edf to 202b112 Compare June 27, 2022 18:45
Copy link
Contributor Author

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @aliher1911, @itsbilal, and @nicktrav)


pkg/storage/pebble.go line 353 at r4 (raw file):

Previously, jbowens (Jackson Owens) wrote…

nit: is there a reason to not use (*pebbleDataBlockMVCCTimeIntervalRangeCollector)(nil) for both of these interface assertions?

Nah, this carried over from the previous code. Fixed, thanks.

@erikgrinaker
Copy link
Contributor Author

CI failures are unrelated.

TFTR!

bors r=jbowens

@craig craig bot merged commit 9dfdf1b into cockroachdb:master Jun 27, 2022
@craig
Copy link
Contributor

craig bot commented Jun 27, 2022

Build succeeded:

@erikgrinaker erikgrinaker deleted the mvcc-range-tombstones-tbi branch June 28, 2022 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

storage: add time-bound iterator handling for range keys
3 participants