Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

self-driving reflector + signal handling #218

Merged
merged 13 commits into from
Apr 8, 2020
Merged

self-driving reflector + signal handling #218

merged 13 commits into from
Apr 8, 2020

Conversation

clux
Copy link
Member

@clux clux commented Apr 6, 2020

fixes #151 and fixes #152 with tokio::signal and a futures::select call.
drops a lot of Reflector logic to defer to Informer.

Also changes the api of Informer and Reflector to have clean constructors:

  • Reflector::new(api).params(lp)
  • Informer::new(api).params(lp)

Seems to work pretty well, but may need to deal with desyncs better.
Ran into the unordered resourceVersion issue on a really old namespace.
Need to double check if that can happen with old version.

The examples that are converted herein seem to work fine. Signal handler also works.

@clux
Copy link
Member Author

clux commented Apr 6, 2020

Seems like re-using Informer is the source of the new resourceVersion issues. Not sure why this is, Informers function perfectly well on their own? EDIT: NOT TRUE. Reproduced the same problem with a configmap_informer in an old namespace. Every poll after the first causes a 410 Gone desync.

Ok, seems the list + watch from the list point is more resilient than the event stream..

From k8s docs:

When the requested watch operations fail because the historical version of that resource is not available, clients must handle the case by recognizing the status code 410 Gone, clearing their local cache, performing a list operation, and starting the watch from the resourceVersion returned by that new list operation. Most client libraries offer some form of standard tool for this logic. (In Go this is called a Reflector and is located in the k8s.io/client-go/cache package.)

So, will revert the Informer dependency I guess.

@clux
Copy link
Member Author

clux commented Apr 6, 2020

Will revert the resourceVersion parsing, even by doing that, it's not solving the problem. Informer and Reflector has to list on old namespaces #219.

@clux clux changed the base branch from informer-api to master April 7, 2020 08:04
@clux clux force-pushed the reflector-rework branch 2 times, most recently from 906cc0e to 4d42621 Compare April 7, 2020 08:36
clux added 5 commits April 7, 2020 09:45
fixes #151 and fixes #152 with tokio::signal and a futures::select call.
drops a lot of Reflector logic to defer to Informer.

Seems to work pretty well, but may need to deal with desyncs better.
Ran into the unordered resourceVersion issue on a really old namespace.
Need to double check if that can happen with old version.
was an attempt to get around #219 but didn't work
this will work in kubernetes >= 1.16 - see #219
for now, don't break the it.
@clux clux force-pushed the reflector-rework branch from 4d42621 to 5796563 Compare April 7, 2020 08:45
@clux
Copy link
Member Author

clux commented Apr 7, 2020

K. Reverted that part of it for now. Now this pr is just a fix of #151 and #152

@clux clux marked this pull request as ready for review April 7, 2020 08:55
make secret reflector a little nicer to look at
@clux clux force-pushed the reflector-rework branch from 4b5ad70 to 6699646 Compare April 7, 2020 08:56
@clux clux changed the title convert Reflector to use Informer internally + self-drive self-driving reflector + signal handling Apr 7, 2020
@clux clux merged commit c88b261 into master Apr 8, 2020
@clux clux deleted the reflector-rework branch April 8, 2020 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add signal handling for runtime objects Add implicit runner inside Reflector
1 participant