Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[coordinator] Delete related etcd keys when aggregator placement deleted #2133

Merged
merged 6 commits into from
Feb 12, 2020

Conversation

robskillington
Copy link
Collaborator

What this PR does / why we need it:

Keys related to the aggregator were not being cleaned up after placement deletion and as such there's no way to really cleanly reset an environment after aggregator has been setup (i.e. coordinator placement deleted, aggregator placement deleted and aggregator topic deleted).

Here were the remaining keys that should be removed if they still exist after placement deletion:

$ ETCDCTL_API=3 etcdctl get --prefix '' --keys-only| grep shard
_kv/default_env/shardset/1/flush
_kv/default_env/shardset/2/flush
_kv/default_env/shardset/3/flush
_kv/default_env/shardset/4/flush
_kv/default_env/shardset/5/flush
_kv/default_env/shardset/6/flush
_kv/default_env/shardset/7/flush
_kv/default_env/shardset/8/flush
_kv/default_env/shardset/9/flush

Special notes for your reviewer:

Does this PR introduce a user-facing and/or backwards incompatible change?:

NONE

Does this PR require updating code package or user-facing documentation?:

NONE

Comment on lines 500 to 501
flushTimesMgrOpts := aggregator.NewFlushTimesManagerOptions()
electionMgrOpts := aggregator.NewElectionManagerOptions()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

meganit: do this after getting the store?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, are these guaranteed to be default?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing.

Comment on lines 513 to 515
return fmt.Errorf(
"error check flush times key exists for deleted instance: %v",
flushTimesKeyErr)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think this would work better as a multierror, so you can still delete the rest of the flush time keys if one errors? Also lets us try to delete election keys too

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, will do.

Comment on lines 530 to 531
return fmt.Errorf(
"error checking election key exists for deleted instance: %v", err)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think this would work better as a multierror, so you can still delete the rest of the election keys if one errors?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, will do.

continue
}
if elem.ShardSetID() == shardSetID {
existingShardSetIDs++
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this just break?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, will do.

Comment on lines 120 to 123
if found {
continue
}
shardSetIDs = append(shardSetIDs, value)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit; probably a bit cleaner as

if !found {
 shardSetIDs = append(shardSetIDs, value)
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing.

Copy link
Collaborator

@schallert schallert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@robskillington robskillington merged commit 5df0f49 into master Feb 12, 2020
@robskillington robskillington deleted the r/delete-aggregator-keys-on-delete-placement branch February 12, 2020 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants