[Camera Support]: FFProbe Output and Audio Detection Failure?

[leaskovski]

Describe the problem you are having

Ok, so I am trying to get audio detection working, and my inital thought is that my stream doesn't have audio in it. Looking at the output of FFProbe, should it list audio streams in there as well? If I open up the rtsp stream in VLC, it plays audio fine, so do I need to be specific in my camera config?

Version

0.13.1-34fb1c2

Frigate config file

mqtt:
  host: x.x.x
  user: x
  password: x

database:
  path: /db/frigate.db

go2rtc:
  streams:
    dog_cam: tapo://xx:[email protected]
    dog_cam_sub: rtsp://xx:[email protected]/stream2
    garden_cam: rtsp://xx:[email protected]/Streaming/Channels/101/
    garden_cam_sub: rtsp://xx:[email protected]/Streaming/Channels/102/
    drive_cam: rtsp://xx:[email protected]/Streaming/Channels/101/
    drive_cam_sub: rtsp://xx:[email protected]/Streaming/Channels/102/
    doorbell_cam:
      - rtsp://xx:[email protected]:554/h264Preview_01_main
      - ffmpeg:doorbell_cam#audio=opus
    doorbell_cam_sub:
      - rtsp://xx:[email protected]:554/h264Preview_01_sub
      - ffmpeg:doorbell_cam_sub#audio=opus    

detectors:
  coral1:
    type: edgetpu
    device: pci
  
cameras:
  dog_cam:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/dog_cam
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/dog_cam_sub
          roles:
            - detect
            - audio
    audio:
      enabled: true
    detect:
      width: 1280
      height: 720
    motion:
      mask:
        - 0,0,0,54,541,54,541,0
    onvif:
      host: x.x.x.x
      port: 2020
      user: x
      password: x
  garden_cam:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garden_cam
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/garden_cam_sub
          roles:
            - detect
    detect:
      width: 640
      height: 360
    motion:
      mask:
        - 439,0,431,67,640,176,640,0
        - 0,0,210,0,210,32,0,32
  drive_cam:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/drive_cam
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/drive_cam_sub
          roles:
            - detect
    detect:
      width: 640
      height: 360
    motion:
      mask:
        - 0,0,210,0,210,32,0,32
  doorbell_cam:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/doorbell_cam
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/doorbell_cam_sub
          roles:
            - detect
    detect:
      width: 640
      height: 480

objects:
  track:
    - person
    - dog 
    
record:
  enabled: True
  events:
    retain:
      default: 10
      mode: motion

Relevant log output

2024-02-19 14:01:34.174653978  [2024-02-19 14:01:34] audio.dog_cam                  ERROR   : ffmpeg process is not running, restarting...
2024-02-19 14:01:44.184999981  [2024-02-19 14:01:44] ffmpeg.dog_cam.audio           ERROR   : Output file #0 does not contain any stream
2024-02-19 14:01:44.185016566  [2024-02-19 14:01:44] audio.dog_cam                  INFO    : Terminating the existing ffmpeg process...
2024-02-19 14:01:44.185026015  [2024-02-19 14:01:44] audio.dog_cam                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-02-19 14:01:45.741171857  [2024-02-19 14:01:45] audio.dog_cam                  ERROR   : ffmpeg process is not running, restarting...
2024-02-19 14:01:55.750353977  [2024-02-19 14:01:55] ffmpeg.dog_cam.audio           ERROR   : Output file #0 does not contain any stream

FFprobe output from your camera

[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"15/1","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":720,"width":1280}]}},{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"15/1","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":360,"width":640}]}}]

Frigate stats

No response

Operating system

Proxmox

Install method

Docker CLI

Coral version

PCIe

Network connection

Wired

Camera make and model

Tapi C200

Any other information that may be helpful

No response

[NickM-27]

your cameras most likely are not outputting aac audio and you have not added aac audio transcoding.

Easiest solution would be to add ?video&audio to the stream, example:

        - path: rtsp://127.0.0.1:8554/dog_cam_sub?video&audio
          roles:
            - detect
            - audio

[leaskovski]

I was being stupid and thought you had highlighted the change in path against the go2rtc, when actually you were updating the rtsp path in the frigate config section... this fixed the issue! DOH!

[leaskovski]

So I thought that might be the case where my go2rtc wasn't specifying audio in its config, so I have made a change to my config based on that...

go2rtc:
  streams:
    dog_cam_sub:
      - rtsp://xxx:[email protected]/stream2
      - ffmpeg:dog_cam_sub#audio=pcma

This hasn't made any difference though. FFProbe still isn't reporting the audio stream. If I use VLC to connect to the rtsp stream (rtsp://xxx:[email protected]/stream2), I can see the video stream and the PCMA audio stream.

There must be something wrong with my go2rtc config that is blocking the audio from the stream.

[NickM-27]

Your audio is already pcm so adding a second pcma transcode isn't going to change anything. You can to transcode to aac if you want to do that.

[leaskovski]

I tired transcoding to opus as I have other cameras with that setup. That said, this is the only camera I have tried to configure audio for, so I have no idea if those other ones work.

I can't seem to get this working, so I guess I will have to forgo audio detection for the time being.