Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WPLA 2.x\\ BUG - Memory usage issues #119

Open
rflinnenbank opened this issue Dec 6, 2023 · 0 comments
Open

WPLA 2.x\\ BUG - Memory usage issues #119

rflinnenbank opened this issue Dec 6, 2023 · 0 comments
Assignees
Labels
bug Something isn't working as it should and needs to be fixed! investigation This issue does not deal with fixing an issue, but currently is about finding out the cause only.

Comments

@rflinnenbank
Copy link
Collaborator

Some users of (at least) the Gunicorn image encounter high memory usage when dealing with the AROME updating process and while requesting AROME data. This results in the update process requiring up to and at times over twelve gigabytes of memory, and the request parsing to even run out of memory even when only requesting a few megabytes.

Steps to reproduce the behavior:

  1. Use Gunicorn image based on version 2.47.031.
  2. Adjust lock-file to match that of the reporting party to duplicate exact container settings.
  3. Verify occurrence of memory usage problems

Expected behavior
When updating AROME data, the maximum memory usage should barely go above the size of the files being processed (400-500Mb for now) . The same goes for regular requests, which shouldn't generate any kind of issue under several gigabytes of data processing.

Additional context
This issue is a strange one, because even in the image we install via poetry with a .lock file in the folder. This should normally mean that there aren't that many possible differences between different installations of the image. This makes one wonder even more about the reason for such huge differences in memory usage.

@rflinnenbank rflinnenbank added bug Something isn't working as it should and needs to be fixed! investigation This issue does not deal with fixing an issue, but currently is about finding out the cause only. labels Dec 6, 2023
@rflinnenbank rflinnenbank self-assigned this Dec 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working as it should and needs to be fixed! investigation This issue does not deal with fixing an issue, but currently is about finding out the cause only.
Projects
No open projects
Status: No status
Development

No branches or pull requests

1 participant