Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore] media pipeline improvements #3110

Conversation

NyaaaWhatsUpDoc
Copy link
Member

Description

This updates our media processing to only store attachment path details on a successful store. This makes it easier to see attachments that are stored locally. It also tweaks the wording of the placeholder attachments warning such that it works for users that say don't want to store any local media whatsoever.

In the name of storage space / efficiency, the media attachments filetype column has also been updated from being a string, to be an integer that only gets stringified for the frontend. And the ffmpeg / ffprobe instance pools are updated such that instances are only allocated when they are needed now, so startup memory usage should be a bit lower. (though note that it will not "size back" after being allocated but during periods of low usage, that would end up adding significant complexity that's not worth it for server software).

Checklist

  • I/we have read the GoToSocial contribution guidelines.
  • I/we have discussed the proposed changes already, either in an issue on the repository, or in the Matrix chat.
  • I/we have not leveraged AI to create the proposed changes.
  • I/we have performed a self-review of added code.
  • I/we have written code that is legible and maintainable by others.
  • I/we have commented the added code, particularly in hard-to-understand areas.
  • I/we have made any necessary changes to documentation.
  • I/we have added tests that cover new code.
  • I/we have run tests and they pass locally with the changes.
  • I/we have run go fmt ./... and golangci-lint run.

internal/util/ptr.go Outdated Show resolved Hide resolved
@tsmethurst
Copy link
Contributor

This seems pretty reasonable to me :) Can't say no to cheeky savings in the database.

@NyaaaWhatsUpDoc NyaaaWhatsUpDoc changed the title Chore/media pipeline improvements [chore] media pipeline improvements Jul 17, 2024
@NyaaaWhatsUpDoc NyaaaWhatsUpDoc force-pushed the chore/media-pipeline-improvements branch from 179305d to b5eaf90 Compare July 17, 2024 11:59
@NyaaaWhatsUpDoc
Copy link
Member Author

NyaaaWhatsUpDoc commented Jul 17, 2024

the tests were still failing as the new lazy initialization of the ffmpeg / ffprobe webassembly modules was causing the tests to timeout that were expecting some things to be processed within a certain timeframe. so i updated the lazy initialization to store just 1 open instance at testrig init, and all other instances it will initialize as it needs them :)

@NyaaaWhatsUpDoc NyaaaWhatsUpDoc merged commit 72ba566 into superseriousbusiness:main Jul 17, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants