From 0dc467a9ff78fb0a9162ba17ca5e7f24dda22240 Mon Sep 17 00:00:00 2001 From: "mergify[bot]" <37929162+mergify[bot]@users.noreply.github.com> Date: Wed, 23 Oct 2024 11:57:16 -0400 Subject: [PATCH] Add known issue for 8.15.x memory increase. (#1403) (#1404) * Add known issue for 8.15.x memory increase. * Apply suggestions from code review Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> --------- Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> (cherry picked from commit 0f1c582e3dacebd1458500b2c0901746457721f7) Co-authored-by: Craig MacKenzie --- .../release-notes/release-notes-8.15.asciidoc | 92 +++++++++++++++++++ 1 file changed, 92 insertions(+) diff --git a/docs/en/ingest-management/release-notes/release-notes-8.15.asciidoc b/docs/en/ingest-management/release-notes/release-notes-8.15.asciidoc index 1dff362b4..ed6494cd0 100644 --- a/docs/en/ingest-management/release-notes/release-notes-8.15.asciidoc +++ b/docs/en/ingest-management/release-notes/release-notes-8.15.asciidoc @@ -31,6 +31,31 @@ Also see: Review important information about the {fleet} and {agent} 8.15.3 release. +[discrete] +[[known-issues-8.15.3]] +=== Known issues + +[[known-issue-issue-41355-8.15.3]] +.The memory usage of {beats} based integrations is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage. +[%collapsible] +==== + +*Details* + +In 8.15, events in the memory queue are not freed when they are acknowledged (as intended), but only when they are overwritten by later events in the queue buffer. This means for example if a configuration has a queue size of 5000, but the input data is low-volume and only 100 events are active at once, then the queue will gradually store more events until reaching 5000 in memory at once, then start replacing those with new events. + +See {beats} issue link:https://github.com/elastic/beats/issues/40705[#40705]. + +*Impact* + + +Memory usage may be higher than in previous releases depending on the throughput of {agent}. A fix is planned for 8.15.4. + +- The worst memory increase is for low-throughput configs with large queues. +- For users whose queues were already sized proportionate to their throughput, memory use is increased but only marginally. +- Affected users can mitigate the higher memory usage by lowering their queue size. + +==== + [discrete] [[security-updates-8.15.3]] === Security updates @@ -64,6 +89,31 @@ Review important information about the {fleet} and {agent} 8.15.3 release. Review important information about the {fleet} and {agent} 8.15.2 release. +[discrete] +[[known-issues-8.15.2]] +=== Known issues + +[[known-issue-issue-41355-8.15.2]] +.The memory usage of {beats} based integrations is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage. +[%collapsible] +==== + +*Details* + +In 8.15, events in the memory queue are not freed when they are acknowledged (as intended), but only when they are overwritten by later events in the queue buffer. This means for example if a configuration has a queue size of 5000, but the input data is low-volume and only 100 events are active at once, then the queue will gradually store more events until reaching 5000 in memory at once, then start replacing those with new events. + +See {beats} issue link:https://github.com/elastic/beats/issues/40705[#40705]. + +*Impact* + + +Memory usage may be higher than in previous releases depending on the throughput of {agent}. A fix is planned for 8.15.4. + +- The worst memory increase is for low-throughput configs with large queues. +- For users whose queues were already sized proportionate to their throughput, memory use is increased but only marginally. +- Affected users can mitigate the higher memory usage by lowering their queue size. + +==== + [discrete] [[enhancements-8.15.2]] === Enhancements @@ -152,6 +202,27 @@ same package versions. ==== +[[known-issue-issue-41355-8.15.1]] +.The memory usage of {beats} based integrations is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage. +[%collapsible] +==== + +*Details* + +In 8.15, events in the memory queue are not freed when they are acknowledged (as intended), but only when they are overwritten by later events in the queue buffer. This means for example if a configuration has a queue size of 5000, but the input data is low-volume and only 100 events are active at once, then the queue will gradually store more events until reaching 5000 in memory at once, then start replacing those with new events. + +See {beats} issue link:https://github.com/elastic/beats/issues/40705[#40705]. + +*Impact* + + +Memory usage may be higher than in previous releases depending on the throughput of {agent}. A fix is planned for 8.15.4. + +- The worst memory increase is for low-throughput configs with large queues. +- For users whose queues were already sized proportionate to their throughput, memory use is increased but only marginally. +- Affected users can mitigate the higher memory usage by lowering their queue size. + +==== + // end 8.15.1 relnotes // begin 8.15.0 relnotes @@ -190,6 +261,27 @@ If you're using {agent} on Windows with any integration which makes use of the A ==== +[[known-issue-issue-41355]] +.The memory usage of {beats} based integrations is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage. +[%collapsible] +==== + +*Details* + +In 8.15, events in the memory queue are not freed when they are acknowledged (as intended), but only when they are overwritten by later events in the queue buffer. This means for example if a configuration has a queue size of 5000, but the input data is low-volume and only 100 events are active at once, then the queue will gradually store more events until reaching 5000 in memory at once, then start replacing those with new events. + +See {beats} issue link:https://github.com/elastic/beats/issues/40705[#40705]. + +*Impact* + + +Memory usage may be higher than in previous releases depending on the throughput of {agent}. A fix is planned for 8.15.4. + +- The worst memory increase is for low-throughput configs with large queues. +- For users whose queues were already sized proportionate to their throughput, memory use is increased but only marginally. +- Affected users can mitigate the higher memory usage by lowering their queue size. + +==== + [discrete] [[new-features-8.15.0]] === New features