panic: Stack Overflow in revokeTreeSalted #2348
Comments
|
Can you provide more information? What accessors? What leases are associated with the token of the accessor? |
|
I dont know when it happens, but when I try to revoke some accessors ; the CLI hangs and eventually vault crashes. Additionally, it seems that vault eventually tries to GC the accessors, and that also crashes the server. I've been discussing it in Gitter if you want to have a more real-time discussion which can be summarized here: https://gitter.im/hashicorp-vault/Lobby |
|
Another oddity is that the TTL is negative for some tokens. Lease ID/Duration is also blank {
"request_id": "80979339-0fcf-77e1-9de2-493709c17762",
"lease_id": "",
"lease_duration": 0,
"renewable": false,
"data": {
"accessor": "2dfaafec-....",
"creation_time": 1486354010,
"creation_ttl": 14400,
"display_name": "token-nomad-cluster",
"explicit_max_ttl": 0,
"id": "",
"meta": null,
"num_uses": 0,
"orphan": false,
"path": "auth/token/create",
"period": 14400,
"policies": [
"root"
],
"renewable": false,
"ttl": -106554
},
"warnings": null
} |
|
Another panic log: |
|
My guess is the TTL is negative because Vault keeps trying to revoke it and can't. |
dadgar
added a commit
that referenced
this issue
Feb 7, 2017
This PR updates token revocation to use a non-recursive approach. On very large trees this avoids stack overflows. Fixes #2348
|
Any update on this issue? We recently upgraded from vault 0.6.5 -> 0.9.0 and consul 0.8.x -> 1.0.1, since upgrading we are seeing something similar, there is a panic and then vault will seal up - we go and unseal and it's fine for a some number of hours and then panic again... here is what it looks like for us: |
|
Update: I compiled a vault bin with the #2478 changes included and have been running this on the past 24 hours now on our staging cluster (where we were having the above issue every 5-6 hours or so and I am yet to hit this panic. I can't tell from the github issue or PR comments if something is holding this up? Would this be a change that will get into mainline soon? It will be a pain to keep having to patch every new release with the DFS PR. |
|
It fell off the radar. I've added to the 0.9.1 milestone for us to re-review. |
|
@jefferai thank you! I'll report back on Monday if I see any regression running the PR on top of 0.9.0 - it's looking real good so far... UPDATE: running without any panics for 3 days now - confident this PR is the fix for this panic now :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Vault panics when revoking some accessors. Seems to be in the code path of
revokeTreeSaltedVault Version: 0.6.3
The text was updated successfully, but these errors were encountered: