[Filebeat] cel input - deeply nested 'state' after HTTP error #33992
Comments
botelastic
bot
added
the
needs_team
|
Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
botelastic
bot
removed
the
needs_team
|
|
6 tasks
andrewkroh
pushed a commit
that referenced
this issue
Dec 8, 2022
Previously, when an error in evaluation or result deserialisation occurred, the entire input activation container was returned instead of a valid state. The result of this was that the state would nest on each iteration of the eval event loop. In cases where errors can happen frequently, this can result in unbounded memory use. Also improve the robustness of the want_more flag; the old code would continue with the loop if state["want_more"] was unset since any(nil) does not equal false. Relates: #33992
mergify bot
pushed a commit
that referenced
this issue
Dec 8, 2022
Previously, when an error in evaluation or result deserialisation occurred, the entire input activation container was returned instead of a valid state. The result of this was that the state would nest on each iteration of the eval event loop. In cases where errors can happen frequently, this can result in unbounded memory use. Also improve the robustness of the want_more flag; the old code would continue with the loop if state["want_more"] was unset since any(nil) does not equal false. Relates: #33992 (cherry picked from commit f08eed8)
efd6
added a commit
that referenced
this issue
Dec 8, 2022
…33997) Previously, when an error in evaluation or result deserialisation occurred, the entire input activation container was returned instead of a valid state. The result of this was that the state would nest on each iteration of the eval event loop. In cases where errors can happen frequently, this can result in unbounded memory use. Also improve the robustness of the want_more flag; the old code would continue with the loop if state["want_more"] was unset since any(nil) does not equal false. Relates: #33992 (cherry picked from commit f08eed8) Co-authored-by: Dan Kortschak <[email protected]>
chrisberkhout
pushed a commit
that referenced
this issue
Jun 1, 2023
Previously, when an error in evaluation or result deserialisation occurred, the entire input activation container was returned instead of a valid state. The result of this was that the state would nest on each iteration of the eval event loop. In cases where errors can happen frequently, this can result in unbounded memory use. Also improve the robustness of the want_more flag; the old code would continue with the loop if state["want_more"] was unset since any(nil) does not equal false. Relates: #33992
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When an HTTP error occurs something bad happens.
statevalue becomes deeply nested (e.g.state: {state: {state: {state: ...} } }). See the log message below.Version: filebeat version 8.7.0 (arm64), libbeat 8.7.0 [0c77112 built 2022-12-08 04:13:10 +0000 UTC]
Reproduction
Logs
{ "log.level": "debug", "@timestamp": "2022-12-07T23:13:42.101-0500", "log.logger": "input.cel", "log.origin": { "file.name": "cel/input.go", "file.line": 198 }, "message": "request state", "service.name": "filebeat", "id": " 155307A9CE29697", "input_source": "http://localhost:9888", "input_url": "http://localhost:9888", "cel": { "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "cursor": null, "state": { "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "url": "http://localhost:9888" }, "ecs.version": "1.6.0" } }The text was updated successfully, but these errors were encountered: