Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GenAI: add ability to save JPGs sent to provider #15643

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

leccelecce
Copy link
Contributor

Proposed change

This adds a config option save_thumbnails to GenAI config, which will write JPGs out to the clips directory. It's disabled by default.

Purpose is to help debug GenAI results. I'm already seeing some interesting things like multiple blank thumbnails being sent.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New feature
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code
  • Documentation Update

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • The code has been formatted using Ruff (ruff format frigate)

Copy link

netlify bot commented Dec 22, 2024

Deploy Preview for frigate-docs ready!

Name Link
🔨 Latest commit c3fa0ce
🔍 Latest deploy log https://app.netlify.com/sites/frigate-docs/deploys/67687ddc08063c0008aaaaa5
😎 Deploy Preview https://deploy-preview-15643--frigate-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@hawkeye217
Copy link
Collaborator

Curious what the other maintainers would say, but this is more of a debug feature in my opinion and shouldn't be user-facing through the config because it has the potential to write a lot of images with no automated cleanup mechanism.

Perhaps it can be something like our object track debugging code, which can just be set to True in code.

# debug object tracking

@leccelecce
Copy link
Contributor Author

Thank you for the review. I see what you mean. Code changes are a bit of a barrier for most though.

Is there any other middle ground perhaps for enabling debugging, like env vars or a command line flag?

@leccelecce
Copy link
Contributor Author

If a key concern is leaving leftovers files, happy to look at implementing a clean-up when events are expired so they don't build up.

@NickM-27
Copy link
Collaborator

NickM-27 commented Dec 22, 2024

In my opinion this is a bit different than Frigate's internal debug because the user doesn't have much control over Frigate's internal object tracking behaviors while a user has direct control over their preferred genai provider and its prompt.

This is something that most users won't be using, especially after things are working in a satisfactory manner. This is a debug capability as opposed to an additional UI / object information feature. Personally, I think it makes most sense as:

  • the config option renamed to explicitly designate it as a debug option
  • instead of some event or time based retention, change the config option to be an integer for the number of "thumbnail stacks" to keep. So that way as a user I can specify I want to keep the last 2 sets of thumbnails that were sent so that I can investigate.

@NickM-27
Copy link
Collaborator

I'd also be fine if it wasn't cleaned up by default, but with the option designated as a debug feature it will be more clear that this is a user function and not a default frigate function

@hawkeye217
Copy link
Collaborator

I'd also be fine if it wasn't cleaned up by default, but with the option designated as a debug feature it will be more clear that this is a user function and not a default frigate function

I'm fine with that too.

I'd also prefer to leave it out of the main genai docs and only keep it in the reference config, like other options we provide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants