[Bug]: Events created with `POST /api/events/<camera_name>/<label>/create` do not show up anymore #13184

danieldietsch · 2024-08-19T11:45:51Z

danieldietsch
Aug 19, 2024

Describe the problem you are having

I am using various API calls to create events and record them. They are not shown in the UI anymore (even new ones created after 0.14), and despite having record: enabled: true.

Steps to reproduce

Have Frigate 0.14 with at least one camera
Send POST /api/events/<camera_name>/<label>/create, note event id
Wait some time
Send PUT /api/events/<event_id>/end
Try to find the event.

Version

0.14.0-da913d8

Frigate config file

mqtt:
  host: ***
  port: 8883
  user: frigate
  password: ***
  tls_ca_certs: /etc/ssl/certs/ca-certificates.crt
  tls_insecure: true

go2rtc:
  streams:
    keller_01: # <- for RTSP streams
      - rtsp://***:***@***:554/live/ch0 # <- stream which supports video & aac audio

ffmpeg:
  hwaccel_args: preset-vaapi

birdseye:
  enabled: true
  mode: continuous

cameras:
  keller_01:
    enabled: true
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/keller_01 # restream via go2rtc
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
            - audio
      hwaccel_args: []
    motion: {}
    audio:
      enabled: true
audio:
  enabled: true
  listen:
    - footsteps
    - speech
    - yell
    - scream
    - fire_alarm
    - door
    - slam
    - rattle
detect:
  enabled: false
record:
  enabled: true
  retain:
    days: 3
    mode: all
  events:
    pre_capture: 15
    post_capture: 15
    retain:
      default: 365
      mode: all
snapshots:
  enabled: true
  retain:
    default: 365
version: 0.14

Relevant log output

None

Operating system

Other Linux

Install method

Docker Compose

Network connection

Wired

Camera make and model

N/A

Any other information that may be helpful

No response

NickM-27 · 2024-08-19T12:02:34Z

NickM-27
Aug 19, 2024
Collaborator Sponsor

More information will be needed, we've had multiple users of this feature confirm it is working in 0.14

0 replies

danieldietsch · 2024-08-19T17:17:47Z

danieldietsch
Aug 19, 2024
Author

I am not sure what information I can provide. The logs look good:

frigate  | 2024-08-19 10:25:23.754083620  172.19.0.1 - - [19/Aug/2024:10:25:23 +0200] "POST /api/events/keller_01/Door%20opened/create HTTP/1.1" 200 95 "-" "HomeAssistant/2024.8.2 aiohttp/3.10.3 Python/3.12" "-"
frigate  | 2024-08-19 10:25:27.602055809  172.19.0.1 - - [19/Aug/2024:10:25:27 +0200] "POST /api/events/keller_01/Motion%20detected/create HTTP/1.1" 200 95 "-" "HomeAssistant/2024.8.2 aiohttp/3.10.3 Python/3.12" "-"
frigate  | 2024-08-19 10:26:12.820487344  172.19.0.1 - - [19/Aug/2024:10:26:12 +0200] "PUT /api/events/1724055923.491842-12s1ey/end HTTP/1.1" 200 55 "-" "HomeAssistant/2024.8.2 aiohttp/3.10.3 Python/3.12" "-"
frigate  | 2024-08-19 10:26:29.241709015  172.19.0.1 - - [19/Aug/2024:10:26:29 +0200] "PUT /api/events/1724055927.365277-mfaa0t/end

If I try it manually, I get the expected 200 response.

But looking at the UI, I see nothing.

Is there anything I could do to get more information?

11 replies

NickM-27 Aug 19, 2024
Collaborator Sponsor

Not sure why it would not work, I am not able to reproduce. Running curl -X POST http://127.0.0.1:5000/api/events/back_deck_cam/presence/create -d '{"duration": 30, "include_recording": true}'; creates an alert that shows up as a gif in the live view and a review item under review

danieldietsch Aug 19, 2024
Author

Can this be related to the camera not having detect enabled?

NickM-27 Aug 19, 2024
Collaborator Sponsor

just tested and that is still working fine for me

danieldietsch Aug 19, 2024
Author

Weird. I recreated the container, and now it seems to be working for me as well.

NickM-27 Aug 19, 2024
Collaborator Sponsor

interesting, please reopen if it occurs again

danieldietsch · 2024-08-21T11:59:52Z

danieldietsch
Aug 21, 2024
Author

The issue came back. It seems like it is related to the camera being unreachable for a short time at night. After loosing the connection, no events are recorded anymore. The camera is still visible and shows live recordings, but I cannot generate events.

This is the log of the connection loss:

03:31:46.868 WRN github.com/AlexxIT/go2rtc/internal/streams/producer.go:171 > error="read tcp 172.19.0.2:57838->192.168.178.78:554: i/o timeout" url=rtsp://cam01:***:554/live/ch0
[2024-08-21 03:31:52] audio.keller_01                ERROR   : ffmpeg process is not running, restarting...
127.0.0.1 - - [21/Aug/2024:03:31:56 +0200] "" 400 0 "-" "-" "-"
[2024-08-21 03:31:57] frigate.video                  ERROR   : keller_01: Unable to read frames from ffmpeg process.
[2024-08-21 03:31:57] frigate.video                  ERROR   : keller_01: ffmpeg process is not running. exiting capture thread...
[2024-08-21 03:31:57] watchdog.keller_01             ERROR   : Ffmpeg process crashed unexpectedly for keller_01.
[2024-08-21 03:31:57] watchdog.keller_01             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2024-08-21 03:31:57] ffmpeg.keller_01.detect        ERROR   : [segment @ 0x55fa69be5000] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[2024-08-21 03:31:57] ffmpeg.keller_01.detect        ERROR   : rtsp://127.0.0.1:8554/keller_01: Connection timed out
[2024-08-21 03:31:57] ffmpeg.keller_01.detect        ERROR   :     Last message repeated 1 times
03:32:00.874 WRN [rtsp] error="streams: dial tcp 192.168.178.78:554: i/o timeout" stream=keller_01
[2024-08-21 03:32:00] frigate.video                  ERROR   : keller_01: Unable to read frames from ffmpeg process.
[2024-08-21 03:32:00] frigate.video                  ERROR   : keller_01: ffmpeg process is not running. exiting capture thread...
[2024-08-21 03:32:02] ffmpeg.keller_01.audio         ERROR   : rtsp://127.0.0.1:8554/keller_01: Connection timed out
[2024-08-21 03:32:02] audio.keller_01                INFO    : Terminating the existing ffmpeg process...
[2024-08-21 03:32:02] audio.keller_01                INFO    : Waiting for ffmpeg to exit gracefully...
03:32:05.121 WRN [rtsp] error="streams: dial tcp 192.168.178.78:554: i/o timeout" stream=keller_01
[2024-08-21 03:32:05] audio.keller_01                ERROR   : ffmpeg process is not running, restarting...
[2024-08-21 03:32:07] watchdog.keller_01             ERROR   : Ffmpeg process crashed unexpectedly for keller_01.
[2024-08-21 03:32:07] watchdog.keller_01             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2024-08-21 03:32:07] ffmpeg.keller_01.detect        ERROR   : [rtsp @ 0x55de5a4b3f80] method DESCRIBE failed: 404 Not Found
[2024-08-21 03:32:07] ffmpeg.keller_01.detect        ERROR   : rtsp://127.0.0.1:8554/keller_01: Server returned 404 Not Found
03:32:10.816 WRN [rtsp] error="streams: dial tcp 192.168.178.78:554: i/o timeout" stream=keller_01
[2024-08-21 03:32:10] frigate.video                  ERROR   : keller_01: Unable to read frames from ffmpeg process.
[2024-08-21 03:32:10] frigate.video                  ERROR   : keller_01: ffmpeg process is not running. exiting capture thread...
[2024-08-21 03:32:15] ffmpeg.keller_01.audio         ERROR   : [rtsp @ 0x561af2ba7c40] method DESCRIBE failed: 404 Not Found
[2024-08-21 03:32:15] ffmpeg.keller_01.audio         ERROR   : rtsp://127.0.0.1:8554/keller_01: Server returned 404 Not Found
[2024-08-21 03:32:15] audio.keller_01                INFO    : Terminating the existing ffmpeg process...
[2024-08-21 03:32:15] audio.keller_01                INFO    : Waiting for ffmpeg to exit gracefully...
[2024-08-21 03:32:17] watchdog.keller_01             ERROR   : Ffmpeg process crashed unexpectedly for keller_01.
[2024-08-21 03:32:17] watchdog.keller_01             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2024-08-21 03:32:17] ffmpeg.keller_01.detect        ERROR   : [rtsp @ 0x56398ddddf80] method DESCRIBE failed: 404 Not Found
[2024-08-21 03:32:17] ffmpeg.keller_01.detect        ERROR   : rtsp://127.0.0.1:8554/keller_01: Server returned 404 Not Found
03:32:18.146 WRN [rtsp] error="streams: dial tcp 192.168.178.78:554: i/o timeout" stream=keller_01
[2024-08-21 03:32:18] audio.keller_01                ERROR   : ffmpeg process is not running, restarting...
03:32:21.149 WRN [rtsp] error="streams: dial tcp 192.168.178.78:554: i/o timeout" stream=keller_01
[2024-08-21 03:32:21] frigate.video                  ERROR   : keller_01: Unable to read frames from ffmpeg process.
[2024-08-21 03:32:21] frigate.video                  ERROR   : keller_01: ffmpeg process is not running. exiting capture thread...
[2024-08-21 03:32:27] watchdog.keller_01             ERROR   : Ffmpeg process crashed unexpectedly for keller_01.
[2024-08-21 03:32:27] watchdog.keller_01             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2024-08-21 03:32:27] ffmpeg.keller_01.detect        ERROR   : [rtsp @ 0x5579f9335f80] method DESCRIBE failed: 404 Not Found
[2024-08-21 03:32:27] ffmpeg.keller_01.detect        ERROR   : rtsp://127.0.0.1:8554/keller_01: Server returned 404 Not Found
[2024-08-21 03:32:28] ffmpeg.keller_01.audio         ERROR   : [rtsp @ 0x55bcb502cc40] method DESCRIBE failed: 404 Not Found
[2024-08-21 03:32:28] ffmpeg.keller_01.audio         ERROR   : rtsp://127.0.0.1:8554/keller_01: Server returned 404 Not Found
[2024-08-21 03:32:28] audio.keller_01                INFO    : Terminating the existing ffmpeg process...
[2024-08-21 03:32:28] audio.keller_01                INFO    : Waiting for ffmpeg to exit gracefully...
03:32:30.837 WRN [rtsp] error="streams: dial tcp 192.168.178.78:554: i/o timeout" stream=keller_01
[2024-08-21 03:32:30] frigate.video                  ERROR   : keller_01: Unable to read frames from ffmpeg process.
[2024-08-21 03:32:30] frigate.video                  ERROR   : keller_01: ffmpeg process is not running. exiting capture thread...
03:32:30.910 WRN [rtsp] error="streams: dial tcp 192.168.178.78:554: connect: no route to host" stream=keller_01
[2024-08-21 03:32:30] audio.keller_01                ERROR   : ffmpeg process is not running, restarting...
[2024-08-21 03:32:37] watchdog.keller_01             ERROR   : Ffmpeg process crashed unexpectedly for keller_01.
[2024-08-21 03:32:37] watchdog.keller_01             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2024-08-21 03:32:37] ffmpeg.keller_01.detect        ERROR   : [rtsp @ 0x5568dd6c4f80] method DESCRIBE failed: 404 Not Found
[2024-08-21 03:32:37] ffmpeg.keller_01.detect        ERROR   : rtsp://127.0.0.1:8554/keller_01: Server returned 404 Not Found
03:32:40.847 WRN [rtsp] error="streams: dial tcp 192.168.178.78:554: i/o timeout" stream=keller_01
[2024-08-21 03:32:40] frigate.video                  ERROR   : keller_01: Unable to read frames from ffmpeg process.
[2024-08-21 03:32:40] frigate.video                  ERROR   : keller_01: ffmpeg process is not running. exiting capture thread...
[2024-08-21 03:32:40] ffmpeg.keller_01.audio         ERROR   : [rtsp @ 0x561ddd085c40] method DESCRIBE failed: 404 Not Found
[2024-08-21 03:32:40] ffmpeg.keller_01.audio         ERROR   : rtsp://127.0.0.1:8554/keller_01: Server returned 404 Not Found
[2024-08-21 03:32:40] audio.keller_01                INFO    : Terminating the existing ffmpeg process...
[2024-08-21 03:32:40] audio.keller_01                INFO    : Waiting for ffmpeg to exit gracefully...
03:32:43.937 WRN [rtsp] error="streams: dial tcp 192.168.178.78:554: i/o timeout" stream=keller_01
[2024-08-21 03:32:43] audio.keller_01                ERROR   : ffmpeg process is not running, restarting...
[2024-08-21 03:32:47] watchdog.keller_01             ERROR   : Ffmpeg process crashed unexpectedly for keller_01.
[2024-08-21 03:32:47] watchdog.keller_01             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2024-08-21 03:32:47] ffmpeg.keller_01.detect        ERROR   : [rtsp @ 0x55aecc727f80] method DESCRIBE failed: 404 Not Found
[2024-08-21 03:32:47] ffmpeg.keller_01.detect        ERROR   : rtsp://127.0.0.1:8554/keller_01: Server returned 404 Not Found
03:32:50.866 WRN [rtsp] error="streams: dial tcp 192.168.178.78:554: i/o timeout" stream=keller_01
[2024-08-21 03:32:50] frigate.video                  ERROR   : keller_01: Unable to read frames from ffmpeg process.
[2024-08-21 03:32:50] frigate.video                  ERROR   : keller_01: ffmpeg process is not running. exiting capture thread...
[2024-08-21 03:32:53] ffmpeg.keller_01.audio         ERROR   : [rtsp @ 0x564c8081fc40] method DESCRIBE failed: 404 Not Found
[2024-08-21 03:32:53] ffmpeg.keller_01.audio         ERROR   : rtsp://127.0.0.1:8554/keller_01: Server returned 404 Not Found
[2024-08-21 03:32:53] audio.keller_01                INFO    : Terminating the existing ffmpeg process...
[2024-08-21 03:32:53] audio.keller_01                INFO    : Waiting for ffmpeg to exit gracefully...
03:32:54.205 WRN [rtsp] error="streams: dial tcp 192.168.178.78:554: connect: no route to host" stream=keller_01
[2024-08-21 03:32:54] audio.keller_01                ERROR   : ffmpeg process is not running, restarting...
127.0.0.1 - - [21/Aug/2024:03:32:56 +0200] "" 400 0 "-" "-" "-"
[2024-08-21 03:32:57] watchdog.keller_01             ERROR   : Ffmpeg process crashed unexpectedly for keller_01.
[2024-08-21 03:32:57] watchdog.keller_01             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2024-08-21 03:32:57] ffmpeg.keller_01.detect        ERROR   : [rtsp @ 0x55bdf1d27f80] method DESCRIBE failed: 404 Not Found
[2024-08-21 03:32:57] ffmpeg.keller_01.detect        ERROR   : rtsp://127.0.0.1:8554/keller_01: Server returned 404 Not Found
[2024-08-21 03:33:04] ffmpeg.keller_01.audio         ERROR   : [rtsp @ 0x562071ecec40] method DESCRIBE failed: 404 Not Found
[2024-08-21 03:33:04] ffmpeg.keller_01.audio         ERROR   : rtsp://127.0.0.1:8554/keller_01: Server returned 404 Not Found
[2024-08-21 03:33:04] audio.keller_01                INFO    : Terminating the existing ffmpeg process...
[2024-08-21 03:33:04] audio.keller_01                INFO    : Waiting for ffmpeg to exit gracefully...

After restarting the container, I also get the following exception:

[INFO] Starting go2rtc healthcheck service...
Exception in thread review_segment_maintainer:
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
172.19.0.1 - - [21/Aug/2024:13:51:23 +0200] "PUT /api/events/1724241073.402233-f9x5cd/end HTTP/1.1" 200 55 "-" "HTTPie/3.2.2" "-"
    self.run()
  File "/opt/frigate/frigate/review/maintainer.py", line 506, in run
    self.indefinite_events[camera].pop(manual_info["event_id"])
KeyError: '1724241073.402233-f9x5cd'
/usr/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
/usr/lib/python3.9/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/keller_011724241073.462239': [Errno 2] No such file or directory: '/keller_011724241073.462239'
  warnings.warn('resource_tracker: %r: %s' % (name, e))

Before the restart, events did not work, but live view did. After the restart, events where available again.

I also noticed regular messages like these, in 1 minute intervals. They are probably unrelated, but I don't know what's causing them.

127.0.0.1 - - [21/Aug/2024:03:30:56 +0200] "" 400 0 "-" "-" "-"

1 reply

danieldietsch Aug 21, 2024
Author

@NickM-27 I cannot reopen the issue.

danieldietsch · 2024-08-25T11:54:30Z

danieldietsch
Aug 25, 2024
Author

Are there any news on this? Can you replicate what is happening or can I do anything?

8 replies

danieldietsch Sep 6, 2024
Author

@NickM-27 If you have time, perhaps you can reproduce it with the steps above. In particular, if such an unended issue persists for more than 1 day, no other API-created events will be registered.

NickM-27 Sep 7, 2024
Collaborator Sponsor

not sure why that would be the case, will have to wait until I have time to test that. Also, curious what custom event that is running for that long?

danieldietsch Sep 7, 2024
Author

It is not running for that long -- but after I send end , it seems to have no end time, and will be shown as still running.

NickM-27 Sep 7, 2024
Collaborator Sponsor

So the event stays running even though you've called the end API a short time after calling create?

danieldietsch Sep 8, 2024
Author

Exactly

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Events created with `POST /api/events/<camera_name>/<label>/create` do not show up anymore #13184

{{title}}

Replies: 4 comments 20 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

[Bug]: Events created with POST /api/events/<camera_name>/<label>/create do not show up anymore #13184

danieldietsch Aug 19, 2024

Describe the problem you are having

Steps to reproduce

Version

Frigate config file

Relevant log output

Operating system

Install method

Network connection

Camera make and model

Any other information that may be helpful

Replies: 4 comments · 20 replies

NickM-27 Aug 19, 2024 Collaborator Sponsor

danieldietsch Aug 19, 2024 Author

NickM-27 Aug 19, 2024 Collaborator Sponsor

danieldietsch Aug 19, 2024 Author

NickM-27 Aug 19, 2024 Collaborator Sponsor

danieldietsch Aug 19, 2024 Author

NickM-27 Aug 19, 2024 Collaborator Sponsor

danieldietsch Aug 21, 2024 Author

danieldietsch Aug 21, 2024 Author

danieldietsch Aug 25, 2024 Author

danieldietsch Sep 6, 2024 Author

NickM-27 Sep 7, 2024 Collaborator Sponsor

danieldietsch Sep 7, 2024 Author

NickM-27 Sep 7, 2024 Collaborator Sponsor

danieldietsch Sep 8, 2024 Author

[Bug]: Events created with `POST /api/events/<camera_name>/<label>/create` do not show up anymore #13184

danieldietsch
Aug 19, 2024

Replies: 4 comments 20 replies

NickM-27
Aug 19, 2024
Collaborator Sponsor

danieldietsch
Aug 19, 2024
Author

NickM-27 Aug 19, 2024
Collaborator Sponsor

danieldietsch Aug 19, 2024
Author

NickM-27 Aug 19, 2024
Collaborator Sponsor

danieldietsch Aug 19, 2024
Author

NickM-27 Aug 19, 2024
Collaborator Sponsor

danieldietsch
Aug 21, 2024
Author

danieldietsch Aug 21, 2024
Author

danieldietsch
Aug 25, 2024
Author

danieldietsch Sep 6, 2024
Author

NickM-27 Sep 7, 2024
Collaborator Sponsor

danieldietsch Sep 7, 2024
Author

NickM-27 Sep 7, 2024
Collaborator Sponsor

danieldietsch Sep 8, 2024
Author