[Support]: Snapshot API - Sometimes 404's, sometimes doesn't crop or annotate image - no logs?

[distinctjuggle]

Describe the problem you are having

Using the snapshot API, sometimes images aren't found. Other times, images will not respect the snapshot API rules such as cropping or annotating the object time, date, etc.

The issue is intermittent, and I can't see anything explicitly relevant in the logs which catches my eye. The response from the calling API is:

{ url: "http://127.0.0.1:5000/api/events/1731438204.647007-ix6u39/snapshot.jpg", status: 404, headers: {"server": "nginx/1.25.3", "date": "Tue, 12 Nov 2024 19:03:45 GMT", "content-type": "application/json", "content-length": "45", "connection": "keep-alive"} }

In this case, it's a 404. The data provided where the image would normally be is: {"success":false,"message":"Event not found"}

This is triggered from an mqtt reader, taking the event_id directly from the mqtt messages. I don't think anything is wrong with my program, as it works most of the time. I also don't see how anything on my end would be influencing the inconsistent cropping and annotating of the snapshots from the Frigate Snapshot API.

Anything I can do to troubleshoot?

Version

0.15.0 Beta 1

What browser(s) are you using?

Irrelevant

Frigate config file

mqtt:
  enabled: true
  host: Host_IP
  port: 1883
  topic_prefix: frigate
  client_id: frigate_server




go2rtc:


  api:
    tls_listen: :443
    tls_cert: |       
      -----BEGIN CERTIFICATE-----
      
      -----END CERTIFICATE-----
    tls_key: |        
     
      -----END RSA PRIVATE KEY-----


  streams:
    Cam1:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
      - ffmpeg:Cam1#audio=opus#audio=copy
    Cam1_sub:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=2
      - ffmpeg:Cam1_sub#audio=opus#audio=copy

    Cam2:
      - rtsp://admin:[email protected]:554/Streaming/Channels/101
      - ffmpeg:Cam2#audio=opus#audio=aac#audio=copy
      # - isapi://user:[email protected]:80/
    Cam2_sub:
      - rtsp://admin:[email protected]:554/Streaming/Channels/102
      - ffmpeg:Cam2_sub#audio=opus#audio=aac#audio=copy
    Cam3:
      # - rtsp://user:[email protected]:554/h264Preview_01_main
      # - rtsp://admin:[email protected]:554/Preview_01_main
      - "ffmpeg:http://admin:[email protected]/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=Password#video=copy#audio=copy#audio=opus"
      - rtsp://user:[email protected]:554/h264Preview_01_main
      - ffmpeg:Cam3#audio=opus#audio=copy
    Cam3_sub:
      - rtsp://admin:[email protected]:554/Preview_01_sub
      - ffmpeg:Cam3_sub#audio=opus#audio=copy
    Cam4:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
      - ffmpeg:Cam4#audio=opus#audio=copy
    Cam4_sub:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=2
      - ffmpeg:Cam4_sub#audio=opus#audio=copy
    Cam5:
      - ffmpeg:rtsp://admin:[email protected]:554/Streaming/Channels/101
      - ffmpeg:Cam5#audio=opus#audio=aac#audio=copy
      - isapi://admin:[email protected]:80/
    Cam5_sub:
      - ffmpeg:rtsp://admin:[email protected]:554/Streaming/Channels/102
      - ffmpeg:Cam5_sub#audio=opus#audio=aac#audio=copy
      - isapi://admin:[email protected]:80/



  webrtc:
    candidates:
      - Host_IP:8555
      - stun:8555



cameras:
  Cam1:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Cam1
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Cam1_sub
          roles:
            - detect
    record:
      enabled: true
    live:
      stream_name: Cam1_sub
    objects:
      track:
        - person
        - cat
        - bird
        - dog


    motion:
      mask: 0.831,0.612,0.992,0.616,0.992,0.256,0.862,0.256
  Cam3:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Cam3
          # input_args: preset-http-reolink
          roles:
            - record
            - detect

    record:
      enabled: true

    motion:
      threshold: 40
      contour_area: 10
      improve_contrast: true
    objects:
      track:
        - person
        - car
        - cat
        - bicycle
        - motorcycle
        - bird
        - dog



  Cam2:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Cam2
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Cam2_sub
          roles:
            - detect
    record:
      enabled: true
    live:
      stream_name: Cam2_sub
    objects:
      track:
        - person
        - cat
        - bird
        - dog

    motion:
      threshold: 30
      contour_area: 30
      improve_contrast: true
      mask: 0.677,0.444,0,0.401,0,0,1,0,1,1,0.663,1
  Cam4:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Cam4
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Cam4_sub
          roles:
            - detect
    record:
      enabled: true
    live:
      stream_name: Cam4_sub
    objects:
      track:
        - person
        - cat
        - bird
        - dog


    motion:
      mask: 0.284,0,0.353,0.559,0.597,0.942,0.596,1,0.001,0.996,0,0
  Cam5:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Cam5
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Cam5_sub
          roles:
            - detect
    record:
      enabled: true
    live:
      stream_name: Cam5_sub
    motion:
      enabled: false  # For this to be off, Detect must also be off
      threshold: 70
      contour_area: 40
      improve_contrast: true
      mask: 0.306,0.307,0.191,0.38,0.526,0.848,0.687,0.599
    detect:
      enabled: false
    objects:
      track:
        - person
        - cat
        - bird
        - dog

ffmpeg:
  hwaccel_args: preset-nvidia-h264
  input_args: preset-rtsp-restream
  output_args:
    record: preset-record-generic-audio-copy



detectors:
  onnx:
    type: onnx


model:
  model_type: yolonas
  width: 320
  height: 320
  input_tensor: nchw
  input_pixel_format: bgr
  path: /config/model_cache/yolo_nas_s.onnx
  labelmap_path: /labelmap/coco-80.txt


detect:
 
  # Optional: enables detection for the camera (default: True)
  enabled: true



record:
  enabled: true
  expire_interval: 60
  sync_recordings: true
  retain:
  
    days: 15
   
  alerts:
    retain:
      days: 10
    pre_capture: 3
    post_capture: 7
  detections:
    retain:
      days: 10
    pre_capture: 3
    post_capture: 7

snapshots:
  enabled: true
  timestamp: true
  bounding_box: true
  crop: true
  retain:
    # Required: Default retention days (default: shown below)
    default: 10
    # Optional: Per object retention days
    objects:
      person: 15
  quality: 100


auth:
  enabled: true
  reset_admin_password: false


version: 0.15-0

Relevant Frigate log output

frigate  | 2024-11-12 14:02:16.423400546  127.0.0.1 - - [12/Nov/2024:14:02:16 -0500] "" 400 0 "-" "-" "-"
frigate  | 2024-11-12 14:03:16.447140257  127.0.0.1 - - [12/Nov/2024:14:03:16 -0500] "" 400 0 "-" "-" "-"
frigate  | 2024-11-12 14:03:45.102141925  172.18.0.1 - - [12/Nov/2024:14:03:45 -0500] "POST /api/export/Cam3/start/1731438202/end/1731438222 HTTP/1.1" 200 96 "-" "-" "-"
frigate  | 2024-11-12 14:03:45.263518223  127.0.0.1 - - [12/Nov/2024:14:03:45 -0500] "GET /vod/Cam3/start/1731438202/end/1731438222/index.m3u8 HTTP/1.1" 200 179 "-" "Lavf/61.1.100" "-"
frigate  | 2024-11-12 14:03:45.264221203  172.18.0.1 - - [12/Nov/2024:14:03:45 -0500] "GET /api/events/1731438204.647007-ix6u39/snapshot.jpg HTTP/1.1" 404 45 "-" "-" "-"
frigate  | 2024-11-12 14:03:45.464477049  127.0.0.1 - - [12/Nov/2024:14:03:45 -0500] "GET /vod/Cam3/start/1731438202/end/1731438222/seg-1-v1-a1.ts HTTP/1.1" 206 16171384 "-" "Lavf/61.1.100" "-"
frigate  | 2024-11-12 14:03:48.273702765  172.18.0.1 - - [12/Nov/2024:14:03:48 -0500] "DELETE /api/export/Cam3_squksf HTTP/1.1" 200 57 "-" "-" "-"
frigate  | 2024-11-12 14:04:16.466818083  127.0.0.1 - - [12/Nov/2024:14:04:16 -0500] "" 400 0 "-" "-" "-"
frigate  | 2024-11-12 14:05:16.486019098  127.0.0.1 - - [12/Nov/2024:14:05:16 -0500] "" 400 0 "-" "-" "-"

Relevant go2rtc log output

N/A

FFprobe output from your camera

N/A

Frigate stats

No response

Install method

Docker Compose

docker-compose file or Docker CLI command

frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:0.15.0-beta1-tensorrt
    shm_size: "1000mb" # update for your cameras based on calculation above
    devices:
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/frigate/config/:/config
      - /srv/frigate/media:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "8971:8971"
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
      - "443:443"
      - "1984:1984"
    environment:
      #FRIGATE_RTSP_PASSWORD: "password"
      - LD_LIBRARY_PATH=/usr/local/lib/python3.9/dist-packages/tensorrt:/usr/local/cuda/lib64:/usr/local/lib/python3.9/dist-packages/nvidia/cufft/lib
    mem_swappiness: 0
    memswap_limit: 0M
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1 # number of GPUs
              capabilities: [gpu]
        limits:
          memory: 10240M

Object Detector

Other

Network connection

Wired

Camera make and model

Irrelevant

Screenshots of the Frigate UI's System metrics pages

No response

Any other information that may be helpful

No response

[NickM-27]

This is triggered from an mqtt reader, taking the event_id directly from the mqtt messages. I don't think anything is wrong with my program, as it works most of the time. I also don't see how anything on my end would be influencing the inconsistent cropping and annotating of the snapshots from the Frigate Snapshot API

be sure that you are checking the has_snapshot field

as far as not respecting the snapshot settings, this happens when the object is still in progress (detected) because you can use the api to request whatever you want (cropped / not, bounding box / not, etc.). Once the event is over the snapshot is saved to disk with the config.

The way around this is to just include the query parameters of what you want on the api call. For example ?bbox=1&crop=1

[distinctjuggle]

So there might be more going on here. I've definitely fixed the non-cropping and annotating issue from your recommendation by adding the string query parameters to the API call so that's good, thank you again. That's very nice to have.

However, the completely missing image problem still remains. I attempted to check for has_snapshot like you suggested, but I'm getting a 404 on the event itself now. I guess that'd make sense, since 20 seconds surely should've been enough time for a snapshot to normally be created for an event.

Here's some logs from my API usage:

Found value: "1731472931.360635-im26mh"
event_id: "1731472931.360635-im26mh"
Found value: "new"
Found value: 1731472931.360635
Found value: "person"

Found value: 1731472931.360635
Found value: "Cam1"
Found value: "1731472931.360635-im26mh"
event_id: "1731472931.360635-im26mh"
Found value: "end"
Found value: 1731472931.360635
Found value: "person"


Now running image upload code
r rrs is. Response { url: "http://127.0.0.1:5000/api/events/1731472931.360635-im26mh", status: 404, headers: {"server": "nginx/1.25.3", "date": "Wed, 13 Nov 2024 04:42:32 GMT", "content-type": "application/json", "content-length": "17", "connection": "keep-alive"} }
value is: None
r rrs is. Response { url: "http://127.0.0.1:5000/api/events/1731472931.360635-im26mh", status: 404, headers: {"server": "nginx/1.25.3", "date": "Wed, 13 Nov 2024 04:42:34 GMT", "content-type": "application/json", "content-length": "17", "connection": "keep-alive"} }
value is: None
r rrs is. Response { url: "http://127.0.0.1:5000/api/events/1731472931.360635-im26mh", status: 404, headers: {"server": "nginx/1.25.3", "date": "Wed, 13 Nov 2024 04:42:36 GMT", "content-type": "application/json", "content-length": "17", "connection": "keep-alive"} }
value is: None
r rrs is. Response { url: "http://127.0.0.1:5000/api/events/1731472931.360635-im26mh", status: 404, headers: {"server": "nginx/1.25.3", "date": "Wed, 13 Nov 2024 04:42:38 GMT", "content-type": "application/json", "content-length": "17", "connection": "keep-alive"} }
value is: None
r rrs is. Response { url: "http://127.0.0.1:5000/api/events/1731472931.360635-im26mh", status: 404, headers: {"server": "nginx/1.25.3", "date": "Wed, 13 Nov 2024 04:42:40 GMT", "content-type": "application/json", "content-length": "17", "connection": "keep-alive"} }
value is: None

In the first part, you can see the MQTT data reading in and information being gathered from that as a result. The event_id comes in clearly.

Later, you can see it drop into the polling loop for has_snapshot a few cycles. Because the event doesn't exist (???), it can't grab the value.

---

What would make an event stop existing? Anything I can do about that? Any way I can further investigate this? I can't identify anything in particular that would cause this problem - it's not unique to a particular camera, nor would that make much sense if it was.

Again, it's intermittent and I can't identify a pattern, at least not as of now. Frigate thinks the event doesn't exist (after publishing it to mqtt as if it does) and that's really all I can tell.

[NickM-27]

has_snapshot a few cycles. Because the event doesn't exist (???), it can't grab the value.

You're supposed to pull has_snapshot from the MQTT payload itself.

The 20 seconds is not an issue, an object can't get a snapshot until it passes certain checks like that it has moved at least once in its lifetime. If an event doesn't get a clip or a snapshot by the time it is over then it won't be saved in the DB

[distinctjuggle]

So, basically you're saying I should ignore any "new" events altogether then more or less? Only updates and some end events matter as only those events will ever contain a true for has_snapshot based on what I'm observing.

I'm a bit confused by this behavior. What is the point of an event being created and published if it might be deleted in 0.1 seconds? It's extremely strange to me that an event would delete itself after announcing its presence like that. Either it thinks it's a real event or it doesn't, no? Supposedly events are only published after the false_positive flag is set to false I thought.

I know for sure that some events are being deleted under a real object condition, so if this happens once, what's to say it won't happen twice in a row for two events within a 2-3 second period of time? If I'm ignoring those brief events which pop up and disappear instantly without an update (aka, without a has_snapshot), how can I be sure I'm not getting false negatives?

---

Since I'll have to reconsider how I'm starting my exports based around this, how do the built-in clips work? Are there any traits of their length, properties, etc? Anything behavioral about them which I can rely upon? In other words, if there's a brief object which appears for 5 seconds and then leaves the camera, how long will the resulting clip be? What if it's on camera for 2 minutes before leaving? When is the clip created and how long will it be? What factors will influence this behavior? etc.

For some reason I am under the impression that they too are constantly being updated, and if you view the same "clip" at a later point in time directly from Frigate, it might have a different length compared to its original, etc. I can't really find a documentation page on their behavior though, but I swear I've seen it before. Maybe I'm confusing it with reviews?

[NickM-27]

So, basically you're saying I should ignore any "new" events altogether then more or less? Only updates and some end events matter as only those events will ever contain a true for has_snapshot based on what I'm observing.

I am not sure what your exact scenario is but in general I recommend using zones and only firing when an objects zones change (before.entered_zones.length < after.entered_zones.length). Otherwise, you just need to use the has_snapshot or has_clip field in the MQTT payload which will tell if the event is being saved or not

I'm a bit confused by this behavior. What is the point of an event being created and published if it might be deleted in 0.1 seconds? It's extremely strange to me that an event would delete itself after announcing its presence like that. Either it thinks it's a real event or it doesn't, no? Supposedly events are only published after the false_positive flag is set to false I thought.

The /events topic is a raw topic that publishes info about every object that is detected as a true positive regardless of if a review item / snapshot / clip is saved for it. This has many uses for example I don't want to save video of every car that drives by but I do want a notification when USPS / UPS / FedEx drive by, and this topic could not be used in that scenario otherwise.

I know for sure that some events are being deleted under a real object condition, so if this happens once, what's to say it won't happen twice in a row for two events within a 2-3 second period of time? If I'm ignoring those brief events which pop up and disappear instantly without an update (aka, without a has_snapshot), how can I be sure I'm not getting false negatives?

If you are getting false negatives that would be an issue but we would need examples of mqtt payloads to understand what exactly is going on. At least every new object will also have an end message as well.

In other words, if there's a brief object which appears for 5 seconds and then leaves the camera, how long will the resulting clip be? What if it's on camera for 2 minutes before leaving? When is the clip created and how long will it be? What factors will influence this behavior? etc.

it depends on your recording retention config. In your case you are keeping 15 days of all recordings so the clip would include the full time that the object was detected (start_time to end_time in the mqtt payload)