Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix memory increase during download (leak) #45

Merged
merged 3 commits into from
Jul 9, 2024
Merged

Fix memory increase during download (leak) #45

merged 3 commits into from
Jul 9, 2024

Conversation

svenrdz
Copy link
Collaborator

@svenrdz svenrdz commented Jul 8, 2024

After any completed download on a single File, the last chunk was not released and lived in memory until all files have been downloaded.

Changes

  • Release chunks from memory (rather mark it as garbage for later collection) right after it has been written to the temporary file.
  • Refactor the httpx.AsyncClient strategy to use the same client across the lifetime of a Processor object (e.g. during the lifetime of a single esgpull download command).

svenrdz added 3 commits July 8, 2024 18:29
After any completed download on a single File, the last chunk was not
released and lived in memory until all files have been downloaded.
This fix releases any chunk from memory (rather marks it as garbage for
later collection) right after it has been written to the temporary file.
@svenrdz svenrdz merged commit e2277de into main Jul 9, 2024
1 check passed
@svenrdz svenrdz added the bug Something isn't working label Jul 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant