[Support]: Identical images are sent to GenAI using Azure OpenAI

[GiorgioUghini]

Describe the problem you are having

Not opening an issue since I'm on beta-3 and beta version users are not enabled to open issues.
I want to investigate/debug the folklowing behavior:
When a new object is detected and a GenAI description should be generated, the description always says that "The provided images are identical" and I was able to confirm that to GenAI (Azure OpenAI) I'm currently sending the same image multiple time (instead of multiple images picturing the "movement" of the subject).
How can I debug this? Do you think is still an issue or is something bad in my setting?
I have default settings, use_snapshot to deafult (so False, according to doc), and default prompt.

Version

0.15.0-beta3

What browser(s) are you using?

Microsoft Edge 131.0.2903.99

Frigate config file

Not needed for the problem, I have default genai settings, use_snapshot to deafult (so False, according to doc), and default prompt.

Relevant Frigate log output

2024-12-17 13:10:03.913868144  127.0.0.1 - - [17/Dec/2024:13:10:03 +0100] "" 400 0 "-" "-" "-"
2024-12-17 13:10:14.198369165  [2024-12-17 13:10:14] httpx                          INFO    : HTTP Request: POST https://my-resource.openai.azure.com//openai/deployments/GPT-4o/chat/completions?api-version=2024-08-01-preview "HTTP/1.1 200 OK"

Relevant go2rtc log output

Not needed for the problem

FFprobe output from your camera

Not needed for the problem

Frigate stats

No response

Install method

Docker Compose

docker-compose file or Docker CLI command

services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: always
    #image: blakeblackshear/frigate:stable
    image: ghcr.io/blakeblackshear/frigate:0.15.0-beta3 #latest
    shm_size: "364mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/giorgio/frigate/config/:/config/
      - /mnt/hddmount:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "1935:1935" # RTMP feeds
    environment:
      FRIGATE_RTSP_PASSWORD: ",,,"
      LIBVA_DRIVER_NAME: "i965"
      FRIGATE_MQTT_PASSWORD: ",,,"

Object Detector

Coral

Network connection

Wired

Camera make and model

Hikvision

Screenshots of the Frigate UI's System metrics pages

No response

Any other information that may be helpful

No response

[hawkeye217]

Multiple images are sent to your GenAI provider during the lifetime of the tracked object. These are gathered during "significant" moments of the object's lifecycle. Depending on your scene and the object's action, there is a chance some of the images may look identical to the AI.

You can always change the default prompt to try to prevent the AI from saying unwanted phrases.

[GiorgioUghini]

Yes, changing the prompt is something usefult to avoid unwanted phrases, but I liked the behavior that you're trying to implement so I would reather trying to understand if there's an undercover issue rater than workaround it and lose the feature.
It happens for everything, all images are identical, for all objects and for all scenes. That is what I'm trying to investigate.
Do you have any suggestion on something that is possibly generating this behavior?

Thank you in any case for the hard work!!!

[GiorgioUghini]

@hawkeye217 - found the bug: #15550

[hawkeye217]

Great, thanks!