Fix delayed status API updates in alerting and task_manager #101778
Conversation
joshdover
added
release_note:fix
Team:Core
joshdover
changed the title
joshdover
force-pushed
the
fx-status-delayed
branch
from
4f67d14 to
07e00a7
Compare
| // Wait for status to become green | ||
| let status; | ||
| const start = Date.now(); | ||
| do { | ||
| const resp = await supertest.get('/api/status'); | ||
| status = resp.status; | ||
| // Stop polling once status stabilizes OR once 40s has passed | ||
| } while (status !== 200 && Date.now() - start < 40_000); | ||
|
|
There was a problem hiding this comment.
This kind of logic is going to be more and more necessary in some tests. We should either:
- Create a centralized helper service for this like
kibana.waitForAvailable(); OR - Change how
@kbn/testwaits for Kibana to start. Currently it waits for a log message that the the server is ready. We could either add logic for polling the API or add a log message for when Kibana actually turns to available ("green") for the first time.
There was a problem hiding this comment.
We could either add logic for polling the API or add a log message for when Kibana actually turns to available ("green") for the first time.
IIRC @pgayvallet has encountered this problem before. do you remember if there were any problems with switching to this logic?
There was a problem hiding this comment.
We could either add logic for polling the API or add a log message for when Kibana actually turns to available ("green") for the first time
This is what I tried in #92568. Got reverted because it added a very significant time to the FTR CI tests. But maybe now that this PR fixes these delays, it would be better?
There was a problem hiding this comment.
I'll explore this as a follow up and see how much it increases test time. I want to keep this PR clean enough to backport to 7.13.3
There was a problem hiding this comment.
Opened a new PR for this along with an additional 5s improvement on the initial available: #102108
|
Pinging @elastic/kibana-core (Team:Core) |
joshdover
force-pushed
the
fx-status-delayed
branch
from
b2f624b to
aeb03ad
Compare
💚 Build SucceededMetrics [docs]
History
To update your PR or re-run it, just comment with: |
joshdover
added
the
auto-backport
…#102083) Co-authored-by: Josh Dover <[email protected]>
…#102084) Co-authored-by: Josh Dover <[email protected]>
Summary
Minimal fix for #101049 (for safe backporting)
Holistic fix will be added by finishing #77965
The Alerting plugin's status Observable had two issues:
setuplifecycleThis PR fixes the alerting and task_manager status Observables to be registered during setup and ensures that there is an initial status emitted (
unavailable) while also making sure the first health check isn't delayed by 5 minutes.Release note
An issue was resolved where Kibana's status API was reporting as
503 Service Unavailabledue to a slow status check in Kibana Alerting.Checklist
Delete any items that are not applicable to this PR.
For maintainers