Skip to content

Commit

Permalink
storage: add RevertRange RPC
Browse files Browse the repository at this point in the history
This adds a new RPC, similar to ClearRange except with the addition of
a time-range filter, for clearing revisions of keys within the passed
span that fall within the passed time range, effectively "reverting"
those writes.

Note however that this isn't a quite a true, generalized "revert" op yet
since older revisions of some keys might have been cleared by GC in the
meantime, meaning that using this to clear the newer revisions does not
cause a scan to observe the older values (but instead that scan would
now see no value). A future change could introduce a "revert window"
setting like the GC TTL that controls when a key can be GC'ed, in this
case only allowing GC of a key when the key covering it is older than
the revert window (in addition to the key itself being older than the
existing GC TTL). This could then ensure that this RPC would always be
able to revert the range to any time in the revert window at which point
it could include a check that the requested time was valid.

However that is left to a future change. The immediate need for this RPC
is rolling back a failed partial bulk ingestion during  IMPORT INTO,
during which only new keys are added and might need to be reverted, so
there are no prior revisions to worry about.

Release note: none.
  • Loading branch information
dt committed Jul 16, 2019
1 parent d7232ed commit 69d4824
Show file tree
Hide file tree
Showing 11 changed files with 3,921 additions and 1,644 deletions.
650 changes: 639 additions & 11 deletions c-deps/libroach/protos/roachpb/api.pb.cc

Large diffs are not rendered by default.

766 changes: 670 additions & 96 deletions c-deps/libroach/protos/roachpb/api.pb.h

Large diffs are not rendered by default.

16 changes: 15 additions & 1 deletion pkg/roachpb/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,9 @@ func (*DeleteRangeRequest) Method() Method { return DeleteRange }
// Method implements the Request interface.
func (*ClearRangeRequest) Method() Method { return ClearRange }

// Method implements the Request interface.
func (*RevertRangeRequest) Method() Method { return RevertRange }

// Method implements the Request interface.
func (*ScanRequest) Method() Method { return Scan }

Expand Down Expand Up @@ -630,6 +633,12 @@ func (crr *ClearRangeRequest) ShallowCopy() Request {
return &shallowCopy
}

// ShallowCopy implements the Request interface.
func (crr *RevertRangeRequest) ShallowCopy() Request {
shallowCopy := *crr
return &shallowCopy
}

// ShallowCopy implements the Request interface.
func (sr *ScanRequest) ShallowCopy() Request {
shallowCopy := *sr
Expand Down Expand Up @@ -1027,7 +1036,12 @@ func (drr *DeleteRangeRequest) flags() int {
// Note that ClearRange commands cannot be part of a transaction as
// they clear all MVCC versions.
func (*ClearRangeRequest) flags() int { return isWrite | isRange | isAlone }
func (*ScanRequest) flags() int { return isRead | isRange | isTxn | updatesReadTSCache | needsRefresh }

// Note that RevertRange commands cannot be part of a transaction as
// they clear all MVCC versions above their target time.
func (*RevertRangeRequest) flags() int { return isWrite | isRange | isAlone }

func (*ScanRequest) flags() int { return isRead | isRange | isTxn | updatesReadTSCache | needsRefresh }
func (*ReverseScanRequest) flags() int {
return isRead | isRange | isReverse | isTxn | updatesReadTSCache | needsRefresh
}
Expand Down
3,259 changes: 1,942 additions & 1,317 deletions pkg/roachpb/api.pb.go

Large diffs are not rendered by default.

43 changes: 43 additions & 0 deletions pkg/roachpb/api.proto
Original file line number Diff line number Diff line change
Expand Up @@ -286,13 +286,54 @@ message ClearRangeRequest {
option (gogoproto.equal) = true;

RequestHeader header = 1 [(gogoproto.nullable) = false, (gogoproto.embed) = true];

// StartTime specifies a lower-bound timestamp, which, if non-zero, causes
// only keys with timestamps equal or greater to be cleared. This can have the
// effect of "reverting" the range's content to some earlier time via MVCC
// (though it is only true revert if cleared keys had not caused GC of earlier
// revisions in the meantime).
util.hlc.Timestamp start_time = 2 [(gogoproto.nullable) = false];
}

// A ClearRangeResponse is the return value from the ClearRange() method.
message ClearRangeResponse {
ResponseHeader header = 1 [(gogoproto.nullable) = false, (gogoproto.embed) = true];
}


// A RevertRangeRequest specifies a range of keys in which to clear all MVCC
// revisions more recent than some TargetTime from the underlying engine. This
// directly clears any more recent MVCC revisions, so it can have the effect of
// "reverting" what the range since by clearing more recently written keys, it
// returns the MVCC-scanned values to TargetTime.
//
// Note: if a more recent MVCC revision of a key has caused an earlier revision
// to be GC'ed it is not possible to revert to that earlier revision since it is
// gone. Reverting such a range with this mechanism would instead simply cause
// the key to appear to be gone. It is up to the caller to consider if this is
// applicable to their usage and if so, take steps to avoid it. Some future work
// could add the concept of a gc.revert_ttl which would prevent GC of a key that
// otherwise meets the GC TTL if the key "shadowing" it is younger then the
// revert TTL -- i.e. ensuring keys remain revertable via this mechanism (by not
// clearing their earlier revisions) for that duraton. When such a mechanism is
// built, this command should check that the TargetTime falls within that window
// and return an error if it does not.
message RevertRangeRequest {
option (gogoproto.equal) = true;

RequestHeader header = 1 [(gogoproto.nullable) = false, (gogoproto.embed) = true];

// TargetTime specifies a the time to which to "revert" the range by clearing any
// MVCC key with a strictly higher timestamp.
util.hlc.Timestamp target_time = 2 [(gogoproto.nullable) = false];
}

// A RevertRangeResponse is the return value from the RevertRange() method.
message RevertRangeResponse {
ResponseHeader header = 1 [(gogoproto.nullable) = false, (gogoproto.embed) = true];
}


// ScanOptions is a collection of options for a batch of scans. The options
// apply to all the scans in the batch.
//
Expand Down Expand Up @@ -1549,6 +1590,7 @@ message RequestUnion {
DeleteRequest delete = 5;
DeleteRangeRequest delete_range = 6;
ClearRangeRequest clear_range = 38;
RevertRangeRequest revert_range = 48;
ScanRequest scan = 7;
BeginTransactionRequest begin_transaction = 8;
EndTransactionRequest end_transaction = 9;
Expand Down Expand Up @@ -1600,6 +1642,7 @@ message ResponseUnion {
DeleteResponse delete = 5;
DeleteRangeResponse delete_range = 6;
ClearRangeResponse clear_range = 38;
RevertRangeResponse revert_range = 48;
ScanResponse scan = 7;
BeginTransactionResponse begin_transaction = 8;
EndTransactionResponse end_transaction = 9;
Expand Down
Loading

0 comments on commit 69d4824

Please sign in to comment.