Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed a bug when flushing a range from the memory buffer #455

Merged
merged 1 commit into from
Sep 29, 2021

Conversation

jmwilliams89
Copy link
Contributor

@jmwilliams89 jmwilliams89 commented Sep 29, 2021

Description of Changes

  • When flushing from a range, the memory buffer was subtracting an additional 1 before releasing resources from the underlying semaphore. This resulted in the memory buffer not reclaiming the correct amount of space after a flush. Instead, every flush would cause it to lose 1 capacity. Eventually, the semaphore will reach its max capacity and this will block all input operators from sending to the buffer, even though the buffer is empty. Unfortunately, when this happens, there are no logs to indicate the problem. Instead, it will appear as if there is no longer data to collect and/or stanza has stopped working.
  • This behavior can be reproduced by using the google output operator with a severely reduced buffer size. Otherwise, it will takes much longer to reproduce this, since the default capacity is 1048576.

Please check that the PR fulfills these requirements

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • Add a changelog entry (for non-trivial bug fixes / features)
  • CI passes

djaglowski
djaglowski previously approved these changes Sep 29, 2021
@djaglowski
Copy link
Member

Log Files Logs / Second CPU Avg (%) CPU Avg Δ (%) Memory Avg (MB) Memory Avg Δ (MB)
1 1000 1.4827332 +0.034399033 127.20918 +1.0787964
1 5000 4.931029 -0.034623623 136.18884 +0.4888153
1 10000 9.586289 -0.13798904 145.54378 +1.6126099
1 50000 45.69032 -0.1376648 173.29593 +0.80158997
1 100000 93.863304 +2.759369 234.72885 +14.567078
10 100 2.5173852 +0.6552719 133.90678 +0.41419983
10 500 5.6552057 +0.25857687 137.62823 -2.2164612
10 1000 10.86214 +0.01717949 146.54364 -0.31855774
10 5000 51.57062 -0.050445557 179.70892 -0.3560028
10 10000 97.965485 -1.3982162 222.38483 +4.230591

@jmwilliams89 jmwilliams89 merged commit 3093caf into master Sep 29, 2021
@jmwilliams89 jmwilliams89 deleted the buffer-flush-fix branch September 29, 2021 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants