-
-
Notifications
You must be signed in to change notification settings - Fork 348
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feature] support processing of (many) more media types #3090
Merged
NyaaaWhatsUpDoc
merged 19 commits into
superseriousbusiness:main
from
NyaaaWhatsUpDoc:feature/switch-to-ffmpeg-media-processing
Jul 12, 2024
Merged
[feature] support processing of (many) more media types #3090
NyaaaWhatsUpDoc
merged 19 commits into
superseriousbusiness:main
from
NyaaaWhatsUpDoc:feature/switch-to-ffmpeg-media-processing
Jul 12, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
NyaaaWhatsUpDoc
commented
Jul 9, 2024
"io" | ||
"os" | ||
|
||
"codeberg.org/gruf/go-mimetypes" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is an embedded map of mimetypes by file extension: https://codeberg.org/gruf/go-mimetypes
Beautiful 😍 |
…a manager for more flexibility
NyaaaWhatsUpDoc
force-pushed
the
feature/switch-to-ffmpeg-media-processing
branch
from
July 10, 2024 13:38
da299ba
to
a91dd7d
Compare
…on separate partition
This was referenced Jul 13, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This replaces our media processing pipeline with the previous Go code we were using, to instead use embedded WebAssembly binaries of ffmpeg and ffprobe. As a result we can now easily support a multitude of extra image and video types, and adds audio file support too. This also allows us to ensure all media we processing (minus emojis) gets its metadata stripped, whereas previously it was only select types in select situations.
This also removes media file size enforcement out of the media manager itself, and instead to be handled by callers of the various media processing functions. (while still adding a check to the media manager for whether a "limit" was reached, for nicer error output). This makes the code in the media manager itself a little neater, and as part of this change minimizes the amount to which we wrap http response bodies, to ensure we can rely on Linux's sendfile syscall for optimal performance draining incoming media to /tmp. (as part of this change, i also got rid of the unnecessary separate image / video size config flags, which weren't actually being used, and replaced them with general media max local / remote media sizes).
Performance impact is yet to be measured, but given we're not doing any expensive transcodes it hopefully won't be too big.
Todo
// ...
commentsChecklist
go fmt ./...
andgolangci-lint run
.