When using MinIO as file storage uploading release attachments causes #23393
Labels
Comments
techknowlogick
added a commit
that referenced
this issue
Mar 12, 2023
When creating attachments (issue, release, repo) the file size (being part of the multipart file header) is passed through the chain of creating an attachment to ensure the MinIO client can stream the file directly instead of having to read it to memory completely at first. Fixes #23393 Co-authored-by: KN4CK3R <[email protected]> Co-authored-by: techknowlogick <[email protected]>
GiteaBot
pushed a commit
to GiteaBot/gitea
that referenced
this issue
Mar 12, 2023
When creating attachments (issue, release, repo) the file size (being part of the multipart file header) is passed through the chain of creating an attachment to ensure the MinIO client can stream the file directly instead of having to read it to memory completely at first. Fixes go-gitea#23393 Co-authored-by: KN4CK3R <[email protected]> Co-authored-by: techknowlogick <[email protected]>
KN4CK3R
added a commit
that referenced
this issue
Mar 12, 2023
Backport #23406 by @baez90 When creating attachments (issue, release, repo) the file size (being part of the multipart file header) is passed through the chain of creating an attachment to ensure the MinIO client can stream the file directly instead of having to read it to memory completely at first. Fixes #23393 Co-authored-by: Peter <[email protected]> Co-authored-by: KN4CK3R <[email protected]> Co-authored-by: techknowlogick <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Description
When creating a release e.g. with GoReleaser I noticed a memory spike when release artifacts are uploaded.
I had to increase the memory limit to ~3GB for my small self-hosted, personal Gitea instance whereas it normally barely exceeds 300MB.
I enabled the pprof endpoint, collected a heap dump and noticed that:
putObjectMultipartStreamNoLengthis the function consuming suddently >1.2GB of RAM although I was only uploading ~50MB of artifacts.The issues is obviously because the original file size is not passed through the chain so the MinIO client doesn't have access to it and has to fall back reading everything to memory.
I played around with the code and could prepare a (quite easy) fix and already tested it on my private instance and it works.
Will create a PR tomorrow (or so).
Let me know if you need further information :)
Gitea Version
v1.18.3
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
No response
Screenshots
No response
Git Version
No response
Operating System
No response
How are you running Gitea?
Gitea Helm chart
Database
PostgreSQL
The text was updated successfully, but these errors were encountered: