Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Volume stuck in Pending state forever after a kubectl delete #2409

Closed
slaperche-scality opened this issue Apr 15, 2020 · 0 comments
Closed
Assignees
Labels
complexity:easy Something that requires less than a day to fix topic:storage Issues related to storage

Comments

@slaperche-scality
Copy link
Contributor

Component:

operator

What happened:

When trying to delete a volume whose creation was in progress, the volume stay stuck in Pending state forever.

What was expected:

We let the volume creation proceed until the end (be it with an error or a success) and then the deletion happen.

Steps to reproduce

kubectl apply -f volume.yaml && sleep 1 && kubectl delete -f volume.yaml

Resolution proposal:

Do not exit the reconciliation loop too early, in order to let deployVolume moving the process forward.

@slaperche-scality slaperche-scality added topic:storage Issues related to storage complexity:easy Something that requires less than a day to fix labels Apr 15, 2020
@slaperche-scality slaperche-scality self-assigned this Apr 15, 2020
slaperche-scality added a commit that referenced this issue Apr 15, 2020
slaperche-scality added a commit that referenced this issue Apr 15, 2020
The way we were handling a deletion while the creation is in progress
was buggy.
We were correctly detecting that we were creating the volume and thus
were postponing the deletion until the creation was done (either with
success or not).
But we were exiting the reconciliation loop too early and never had a
chance to keep continue the volume creation, thus we were forever stuck
in a Pending state (and the deletion was posptone forever as well).

The fix is easy enough though: in such case we don't exit the
reconciliation loop immediatly, to be able to call `deployVolume` and
made some progress until we're stabilized.

Refs: #2409
Signed-off-by: Sylvain Laperche <[email protected]>
@bert-e bert-e closed this as completed in d3d2764 Apr 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
complexity:easy Something that requires less than a day to fix topic:storage Issues related to storage
Projects
None yet
Development

No branches or pull requests

1 participant