[Support]: need advice on tweaks to make PTZ autotracking better/faster #10183

guycobb2 · 2024-03-01T23:47:30Z

guycobb2
Mar 1, 2024

Describe the problem you are having

simply looking for config tweaks that could improve the performance of the ptz autotracking. if vehicles go by too fast, it can't keep up. it seems that a lot of times it won't start tracking people, more so at night. just anything i can do to make the experience better. im quite impressed with the results shown in the frigate documents..... looking to achieve something more like that.

clip_1-garage_ptz_1709334332.782342-1709334347.088556.mp4

clip_1-garage_ptz_1709303806.885861-1709303820.465465.mp4

Version

0.13.2-6476F8A

Frigate config file

mqtt:
  enabled: true
  host: 192.168.10.5
  port: 1883
  #user: xxxx
  # Optional: password
  # NOTE: MQTT password can be specified with an environment variables that must begin with 'FRIGATE_'.
  #       e.g. password: '{FRIGATE_MQTT_PASSWORD}'
  #password: xxxx

detectors:
  #ov:
  #  type: openvino
  #  device: AUTO
  #  model: 
  #    path: /openvino-model/ssdlite_mobilenet_v2.xml
#model:
#  width: 300
#  height: 300
#  input_tensor: nhwc
#  input_pixel_format: bgr
#  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

  coral:
    type: edgetpu
    device: usb




go2rtc:
  streams:
    1-garage_ptz:
    - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
    - ffmpeg:1-garage_ptz#video=mjpeg
    2-garage_door:
    - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
    - ffmpeg:2-garage_door#video=mjpeg
    3-garage:
    - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
    - ffmpeg:3-garage#video=mjpeg
    4-utility_door:
    - rtsp://admin1:[email protected]:554/stream2
    - ffmpeg:4-utility_door#video=mjpeg
    5-kitchen:
    - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
    - ffmpeg:5-kitchen#video=mjpeg
    6-guys_room:
    - rtsp://admin1:[email protected]:554/stream2
    - ffmpeg:6-guys_room#video=mjpeg
  webrtc:
    candidates:
    - 192.168.10.5:8555
    - 100.0.0.0/8:8555
    - stun:8555

cameras:
  1-garage_ptz:
    ffmpeg:
      hwaccel_args: preset-vaapi
      output_args:
        record: preset-record-generic
      inputs:
      # - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=2
        # roles:
        # - detect
      - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
        roles:
        - detect
        - record
    detect:
      max_disappeared: 50
      enabled: true
      width: 1920
      height: 1080
      fps: 10
      stationary:
        interval: 50
        threshold: 50
    objects:
      track:
      - person
      - dog
      - car
      - cat
      - umbrella
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 50
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 2073600
          # Optional: minimum width/height of the bounding box for the detected object (default: 0)
          min_ratio: 0.5
          # Optional: maximum width/height of the bounding box for the detected object (default: 24000000)
          max_ratio: 2.0
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.5
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.65
          # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
          # Checks based on the bottom center of the bounding box of the object
          #mask: 0,0,1000,0,1000,200,0,200
        car:
          threshold: 0.70
          # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
    onvif:
      # Required: host of the camera being connected to.
      host: 192.168.40.4
      # Optional: ONVIF port for device (default: shown below).
      port: 8099
      # Optional: username for login.
      # NOTE: Some devices require admin to access ONVIF.
      user: admin
      # Optional: password for login.
      password: admin
      # Optional: PTZ camera object autotracking. Keeps a moving object in
      # the center of the frame by automatically moving the PTZ camera.
      autotracking:
        # Optional: enable/disable object autotracking. (default: shown below)
        enabled: true
        # Optional: calibrate the camera on startup (default: shown below)
        # A calibration will move the PTZ in increments and measure the time it takes to move.
        # The results are used to help estimate the position of tracked objects after a camera move.
        # Frigate will update your config file automatically after a calibration with
        # a "movement_weights" entry for the camera. You should then set calibrate_on_startup to False.
        calibrate_on_startup: false
        # Optional: the mode to use for zooming in/out on objects during autotracking. (default: shown below)
        # Available options are: disabled, absolute, and relative
        #   disabled - don't zoom in/out on autotracked objects, use pan/tilt only
        #   absolute - use absolute zooming (supported by most PTZ capable cameras)
        #   relative - use relative zooming (not supported on all PTZs, but makes concurrent pan/tilt/zoom movements)
        zooming: relative
        # Optional: A value to change the behavior of zooming on autotracked objects. (default: shown below)
        # A lower value will keep more of the scene in view around a tracked object.
        # A higher value will zoom in more on a tracked object, but Frigate may lose tracking more quickly.
        # The value should be between 0.1 and 0.75
        zoom_factor: 0.35
        # Optional: list of objects to track from labelmap.txt (default: shown below)
        track:
        - person
        - dog
        - cat
        - car
        # Required: Begin automatically tracking an object when it enters any of the listed zones.
        required_zones:
        - full_screen
        # Required: Name of ONVIF preset in camera's firmware to return to when tracking is over. (default: shown below)
        return_preset: home
        # Optional: Seconds to delay before returning to preset. (default: shown below)
        timeout: 20
        # Optional: Values generated automatically by a camera calibration. Do not modify these manually. (default: shown below)
        movement_weights: 0.04, 1.0, 1.2348265647888184, 1.3114162588632237, 0.22082424317636798
    zones:
      full_screen:
        coordinates: 1915,1075,1915,5,5,5,5,1075
  3-garage:
    ffmpeg:
      hwaccel_args: preset-vaapi
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
      - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
        roles:
        - detect
      - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
        roles:
      #  - detect
        - record
    detect:
      max_disappeared: 50
      enabled: true
      width: 704
      height: 480
      fps: 5
      stationary:
        interval: 50
        threshold: 50
    objects:
      track:
      - person
      - dog
      - car
      - cat
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 100
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 337920
          # Optional: minimum width/height of the bounding box for the detected object (default: 0)
          min_ratio: 0.5
          # Optional: maximum width/height of the bounding box for the detected object (default: 24000000)
          max_ratio: 2.0
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.6
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.80
          # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
          # Checks based on the bottom center of the bounding box of the object
          #mask: 0,0,1000,0,1000,200,0,200
  2-garage_door:
    ffmpeg:
      hwaccel_args: preset-vaapi
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
      - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
        roles:
        - detect
      - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
        roles:
      #  - detect
        - record
    detect:
      max_disappeared: 50
      enabled: true
      width: 704
      height: 480
      fps: 5
      stationary:
        interval: 50
        threshold: 50
    objects:
      track:
      - person
      - dog
      - umbrella
      - car
      - cat
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 100
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 337920
          # Optional: minimum width/height of the bounding box for the detected object (default: 0)
          min_ratio: 0.5
          # Optional: maximum width/height of the bounding box for the detected object (default: 24000000)
          max_ratio: 2.0
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.6
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.80
          # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
          # Checks based on the bottom center of the bounding box of the object
          #mask: 0,0,1000,0,1000,200,0,200
  4-utility_door:
    ffmpeg:
      hwaccel_args: preset-vaapi
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
      - path: rtsp://admin1:[email protected]:554/stream2
        roles:
        - detect
      - path: rtsp://admin1:[email protected]:554/stream1
        roles:
      #  - detect
        - record
    detect:
      max_disappeared: 50
      enabled: true
      width: 640
      height: 360
      fps: 5
      stationary:
        interval: 50
        threshold: 50
    objects:
      track:
      - person
      - dog
      - car
      - cat
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 50
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 230400
          # Optional: minimum width/height of the bounding box for the detected object (default: 0)
          min_ratio: 0.5
          # Optional: maximum width/height of the bounding box for the detected object (default: 24000000)
          max_ratio: 2.0
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.6
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.80
          # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
          # Checks based on the bottom center of the bounding box of the object
          #mask: 0,0,1000,0,1000,200,0,200
  5-kitchen:
    ffmpeg:
      hwaccel_args: preset-vaapi
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
      - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
        roles:
        - detect
      - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
        roles:
      #  - detect
        - record
    detect:
      max_disappeared: 50
      enabled: true
      width: 704
      height: 480
      fps: 5
      stationary:
        interval: 50
        threshold: 50
    #motion:
      #mask:
    objects:
      track:
      - person
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 100
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 337920
          # Optional: minimum width/height of the bounding box for the detected object (default: 0)
          min_ratio: 0.5
          # Optional: maximum width/height of the bounding box for the detected object (default: 24000000)
          max_ratio: 2.0
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.6
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.80
          # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
          # Checks based on the bottom center of the bounding box of the object
          mask:
  6-guys_room:
    ffmpeg:
      hwaccel_args: preset-vaapi
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
      - path: rtsp://admin1:[email protected]:554/stream2
        roles:
        - detect
      - path: rtsp://admin1:[email protected]:554/stream1
        roles:
      #  - detect
        - record
    detect:
      max_disappeared: 50
      enabled: true
      width: 640
      height: 360
      fps: 5
      stationary:
        interval: 50
        threshold: 50
    objects:
      track:
      - person
      filters:
        person:
          # Optional: minimum width*height of the bounding box for the detected object (default: 0)
          min_area: 50
          # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
          max_area: 230400
          # Optional: minimum width/height of the bounding box for the detected object (default: 0)
          min_ratio: 0.5
          # Optional: maximum width/height of the bounding box for the detected object (default: 24000000)
          max_ratio: 2.0
          # Optional: minimum score for the object to initiate tracking (default: shown below)
          min_score: 0.6
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
          threshold: 0.80
          # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
          # Checks based on the bottom center of the bounding box of the object
          #mask: 0,0,1000,0,1000,200,0,200

record:
  enabled: true
  retain:
    days: 14
    mode: all
  events:
    retain:
      default: 14
      mode: active_objects

snapshots:
  enabled: true
  timestamp: false
  bounding_box: false
  crop: false
  retain:
    default: 7

rtmp:
  enabled: false

birdseye:
  # Optional: Enable birdseye view (default: shown below)
  enabled: true
  # Optional: Restream birdseye via RTSP (default: shown below)
  # NOTE: Enabling this will set birdseye to run 24/7 which may increase CPU usage somewhat.
  restream: true
  # Optional: Width of the output resolution (default: shown below)
  width: 1280
  # Optional: Height of the output resolution (default: shown below)
  height: 720
  # Optional: Encoding quality of the mpeg1 feed (default: shown below)
  # 1 is the highest quality, and 31 is the lowest. Lower quality feeds utilize less CPU resources.
  quality: 8
  # Optional: Mode of the view. Available options are: objects, motion, and continuous
  #   objects - cameras are included if they have had a tracked object within the last 30 seconds
  #   motion - cameras are included if motion was detected in the last 30 seconds
  #   continuous - all cameras are included always
  mode: objects

Relevant log output

2024-03-01 17:17:19.885564622  [INFO] Preparing Frigate...
2024-03-01 17:17:19.891799651  [INFO] Starting Frigate...
2024-03-01 17:17:20.956924005  [2024-03-01 17:17:20] frigate.app                    INFO    : Starting Frigate (0.13.2-6476f8a)
2024-03-01 17:17:21.003575713  [2024-03-01 17:17:21] peewee_migrate.logs            INFO    : Starting migrations
2024-03-01 17:17:21.007432956  [2024-03-01 17:17:21] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-03-01 17:17:21.105528325  [2024-03-01 17:17:21] frigate.app                    INFO    : Recording process started: 586
2024-03-01 17:17:21.106669899  [2024-03-01 17:17:21] frigate.app                    INFO    : go2rtc process pid: 89
2024-03-01 17:17:21.121979079  [2024-03-01 17:17:21] detector.coral                 INFO    : Starting detection process: 596
2024-03-01 17:17:23.728603977  [2024-03-01 17:17:21] frigate.app                    INFO    : Output process started: 598
2024-03-01 17:17:23.728706886  [2024-03-01 17:17:21] frigate.app                    INFO    : Camera processor started for 1-garage_ptz: 628
2024-03-01 17:17:23.730627178  [2024-03-01 17:17:21] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2024-03-01 17:17:23.730659666  [2024-03-01 17:17:21] frigate.app                    INFO    : Camera processor started for 3-garage: 629
2024-03-01 17:17:23.730743031  [2024-03-01 17:17:23] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2024-03-01 17:17:23.730761223  [2024-03-01 17:17:21] frigate.app                    INFO    : Camera processor started for 2-garage_door: 632
2024-03-01 17:17:23.730780215  [2024-03-01 17:17:21] frigate.app                    INFO    : Camera processor started for 4-utility_door: 634
2024-03-01 17:17:23.730794263  [2024-03-01 17:17:21] frigate.app                    INFO    : Camera processor started for 5-kitchen: 636
2024-03-01 17:17:23.730809108  [2024-03-01 17:17:21] frigate.app                    INFO    : Camera processor started for 6-guys_room: 638
2024-03-01 17:17:23.730825560  [2024-03-01 17:17:21] frigate.app                    INFO    : Capture process started for 1-garage_ptz: 640
2024-03-01 17:17:23.730840376  [2024-03-01 17:17:21] frigate.app                    INFO    : Capture process started for 3-garage: 643
2024-03-01 17:17:23.730855435  [2024-03-01 17:17:21] frigate.app                    INFO    : Capture process started for 2-garage_door: 647
2024-03-01 17:17:23.730871094  [2024-03-01 17:17:21] frigate.app                    INFO    : Capture process started for 4-utility_door: 651
2024-03-01 17:17:23.730885624  [2024-03-01 17:17:21] frigate.app                    INFO    : Capture process started for 5-kitchen: 658
2024-03-01 17:17:23.730902665  [2024-03-01 17:17:21] frigate.app                    INFO    : Capture process started for 6-guys_room: 665

FFprobe output from your camera

[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"0/0","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":0,"width":0},{"avg_frame_rate":"0/0","bit_rate":"64000","codec_long_name":"PCMA-law/G.711A-law"}]}}]

Frigate stats

{"cameras":{"1-garage_ptz":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":10.2,"capture_pid":640,"detection_enabled":1,"detection_fps":0.1,"ffmpeg_pid":644,"pid":628,"process_fps":10.2,"skipped_fps":0.0},"2-garage_door":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.0,"capture_pid":647,"detection_enabled":1,"detection_fps":0.1,"ffmpeg_pid":653,"pid":632,"process_fps":5.0,"skipped_fps":0.0},"3-garage":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.0,"capture_pid":643,"detection_enabled":1,"detection_fps":0.1,"ffmpeg_pid":648,"pid":629,"process_fps":5.0,"skipped_fps":0.0},"4-utility_door":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.0,"capture_pid":651,"detection_enabled":1,"detection_fps":0.2,"ffmpeg_pid":660,"pid":634,"process_fps":5.0,"skipped_fps":0.0},"5-kitchen":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.0,"capture_pid":658,"detection_enabled":1,"detection_fps":0.1,"ffmpeg_pid":667,"pid":636,"process_fps":5.0,"skipped_fps":0.0},"6-guys_room":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.1,"capture_pid":665,"detection_enabled":1,"detection_fps":1.1,"ffmpeg_pid":674,"pid":638,"process_fps":5.0,"skipped_fps":0.0}},"cpu_usages":{"1":{"cmdline":"/package/admin/s6/command/s6-svscan -d4 -- /run/service","cpu":"0.0","cpu_average":"0","mem":"0.0"},"15":{"cmdline":"s6-supervise s6-linux-init-shutdownd","cpu":"0.0","cpu_average":"0","mem":"0.0"},"16":{"cmdline":"/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B","cpu":"0.0","cpu_average":"0","mem":"0.0"},"24":{"cmdline":"s6-supervise s6rc-oneshot-runner","cpu":"0.0","cpu_average":"0","mem":"0.0"},"25":{"cmdline":"s6-supervise s6rc-fdholder","cpu":"0.0","cpu_average":"0","mem":"0.0"},"26":{"cmdline":"s6-supervise frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"27":{"cmdline":"s6-supervise frigate-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"28":{"cmdline":"s6-supervise go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"29":{"cmdline":"s6-supervise go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"30":{"cmdline":"s6-supervise go2rtc-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"31":{"cmdline":"s6-supervise nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"32":{"cmdline":"s6-supervise nginx-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"41":{"cmdline":"/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules","cpu":"0.0","cpu_average":"0","mem":"0.0"},"42":{"cmdline":"/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --","cpu":"0.0","cpu_average":"0","mem":"0.0"},"80":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"81":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"82":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"89":{"cmdline":"/usr/local/go2rtc/bin/go2rtc -config=/dev/shm/go2rtc.yaml","cpu":"0.0","cpu_average":"0","mem":"0.0"},"97":{"cmdline":"bash ./run.user go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"98":{"cmdline":"python3 -u -m frigate","cpu":"1.7","cpu_average":"3","mem":"1.9"},"113":{"cmdline":"nginx: master process nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"145":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"146":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.1"},"147":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"148":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"155":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"177":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"213":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"251":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"287":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"325":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"362":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"397":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"435":{"cmdline":"nginx: cache manager process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"584":{"cmdline":"frigate.logger       ","cpu":"0.0","cpu_average":"0","mem":"0.3"},"586":{"cmdline":"frigate.recording_manager","cpu":"3.6","cpu_average":"3","mem":"0.5"},"595":{"cmdline":"/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(61)","cpu":"0.1","cpu_average":"0","mem":"0.0"},"596":{"cmdline":"frigate.detector.coral","cpu":"0.3","cpu_average":"2","mem":"0.4"},"598":{"cmdline":"frigate.output       ","cpu":"1.1","cpu_average":"5","mem":"0.4"},"603":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1920x1080 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.2"},"604":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 704x480 -i pipe: -f mpegts -s 1056x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.1"},"605":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 704x480 -i pipe: -f mpegts -s 1056x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.1"},"606":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 640x360 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.1"},"607":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 704x480 -i pipe: -f mpegts -s 1056x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.1"},"608":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 640x360 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.1"},"609":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.2","cpu_average":"1","mem":"0.2"},"628":{"cmdline":"frigate.process:1-garage_ptz","cpu":"1.3","cpu_average":"2","mem":"0.6"},"629":{"cmdline":"frigate.process:3-garage","cpu":"0.6","cpu_average":"0","mem":"0.5"},"632":{"cmdline":"frigate.process:2-garage_door","cpu":"0.7","cpu_average":"0","mem":"0.5"},"634":{"cmdline":"frigate.process:4-utility_door","cpu":"0.8","cpu_average":"0","mem":"0.5"},"636":{"cmdline":"frigate.process:5-kitchen","cpu":"0.6","cpu_average":"3","mem":"0.5"},"638":{"cmdline":"frigate.process:6-guys_room","cpu":"1.3","cpu_average":"1","mem":"0.5"},"640":{"cmdline":"frigate.capture:1-garage_ptz","cpu":"3.1","cpu_average":"3","mem":"0.5"},"643":{"cmdline":"frigate.capture:3-garage","cpu":"0.3","cpu_average":"0","mem":"0.5"},"644":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://*:*@192.168.40.4:554/cam/realmonitor?channel=1&subtype=0 -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/1-garage_ptz@%Y%m%d%H%M%S%z.mp4 -r 10 -vf fps=10,scale_vaapi=w=1920:h=1080:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"5.1","cpu_average":"5","mem":"0.2"},"647":{"cmdline":"frigate.capture:2-garage_door","cpu":"0.3","cpu_average":"0","mem":"0.5"},"648":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://*:*@192.168.40.3:554/cam/realmonitor?channel=1&subtype=1 -r 5 -vf fps=5,scale_vaapi=w=704:h=480:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"1.2","cpu_average":"1","mem":"0.2"},"651":{"cmdline":"frigate.capture:4-utility_door","cpu":"0.2","cpu_average":"0","mem":"0.5"},"653":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://*:*@192.168.40.5:554/cam/realmonitor?channel=1&subtype=1 -r 5 -vf fps=5,scale_vaapi=w=704:h=480:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"1.1","cpu_average":"1","mem":"0.2"},"658":{"cmdline":"frigate.capture:5-kitchen","cpu":"0.3","cpu_average":"0","mem":"0.5"},"659":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://*:*@192.168.40.3:554/cam/realmonitor?channel=1&subtype=0 -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy /tmp/cache/3-garage@%Y%m%d%H%M%S%z.mp4","cpu":"0.6","cpu_average":"0","mem":"0.1"},"660":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://*:*@192.168.40.6:554/stream2 -r 5 -vf fps=5,scale_vaapi=w=640:h=360:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"0.7","cpu_average":"0","mem":"0.2"},"665":{"cmdline":"frigate.capture:6-guys_room","cpu":"0.3","cpu_average":"0","mem":"0.5"},"667":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://*:*@192.168.40.2:554/cam/realmonitor?channel=1&subtype=1 -r 5 -vf fps=5,scale_vaapi=w=704:h=480:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"1.2","cpu_average":"1","mem":"0.2"},"668":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://*:*@192.168.40.5:554/cam/realmonitor?channel=1&subtype=0 -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/2-garage_door@%Y%m%d%H%M%S%z.mp4","cpu":"1.1","cpu_average":"1","mem":"0.1"},"673":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://*:*@192.168.40.6:554/stream1 -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/4-utility_door@%Y%m%d%H%M%S%z.mp4","cpu":"0.8","cpu_average":"0","mem":"0.1"},"674":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://*:*@192.168.40.7:554/stream2 -r 5 -vf fps=5,scale_vaapi=w=640:h=360:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"0.6","cpu_average":"0","mem":"0.2"},"677":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://*:*@192.168.40.2:554/cam/realmonitor?channel=1&subtype=0 -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/5-kitchen@%Y%m%d%H%M%S%z.mp4","cpu":"1.0","cpu_average":"0","mem":"0.1"},"680":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://*:*@192.168.40.7:554/stream1 -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/6-guys_room@%Y%m%d%H%M%S%z.mp4","cpu":"0.8","cpu_average":"0","mem":"0.1"},"10959":{"cmdline":"sh","cpu":"0.0","cpu_average":"0","mem":"0.0"},"12940":{"cmdline":"sleep 30s","cpu":"0.0","cpu_average":"0","mem":"0.0"}},"detection_fps":1.7,"detectors":{"coral":{"detection_start":0.0,"inference_speed":7.59,"pid":596}},"gpu_usages":{"intel-vaapi":{"gpu":"2.03%","mem":"-%"}},"processes":{"go2rtc":{"pid":89},"logger":{"pid":584},"recording":{"pid":586}},"service":{"last_updated":1709336287,"latest_version":"0.13.2","storage":{"/dev/shm":{"free":3073.5,"mount_type":"tmpfs","total":3084.0,"used":10.5},"/media/frigate/clips":{"free":16620523.6,"mount_type":"fuse.shfs","total":102582371.5,"used":85961847.9},"/media/frigate/recordings":{"free":16620523.6,"mount_type":"fuse.shfs","total":102582371.5,"used":85961847.9},"/tmp/cache":{"free":934.9,"mount_type":"tmpfs","total":953.7,"used":18.8}},"temperatures":{},"uptime":1246,"version":"0.13.2-6476f8a"}}

Operating system

UNRAID

Install method

Docker CLI

Coral version

USB

Network connection

Wired

Camera make and model

EmpireTech PTZ1A2M-25X

Any other information that may be helpful

No response

NickM-27 · 2024-03-02T00:00:36Z

NickM-27
Mar 2, 2024
Collaborator Sponsor

CC @hawkeye217

0 replies

hawkeye217 · 2024-03-02T01:00:26Z

hawkeye217
Mar 2, 2024
Collaborator

Glad you're able to try out autotracking! There's a few things at play here.

Zooming is still an experimental feature, especially relative zooming. Camera calibration helps Frigate estimate an object's position based on its velocity before the camera moves. In Frigate 0.13, this calibration does not take measurements for relative zooming, only panning and tilting. So for objects that require zooming, your zoom_factor and your initial zoom of the camera preset will have a dramatic effect on this estimate. For the car in your first example video to be tracked, you should reduce the amount of time it takes for your camera to zoom in - either through a tighter initial zoom on the camera's preset, a lower zoom_factor, or both.
This is why a full-frame zone isn't recommended, as the docs say. Frigate can get a better estimate of an object's velocity after it has entered the frame and before it enters the required zone for autotracking.
In your second example with the dog, I have a feeling Frigate just lost tracking of the object probably due to the fact that dogs are not always detected very well by the default COCO model. A better trained Frigate+ would probably do very well for you. Another maintainer has had great results with his dog being detected in various lighting conditions with his trained Frigate+ model.

1 reply

guycobb2 Mar 2, 2024
Author

Good info to know. I’ll see if I can do anything to speed up the zooming speed (don’t think I can) or will drop the zoom factor.

How can I have the entire frame be monitored without a full frame zone? If it comes in view of the camera and can positively identify it, I want it tracked.

I’ve considered trying to learn more about frigate+ as it seems very reasonably priced and I’d love to add the ability to identify FedEx trucks, certain people or whatever else it may bring to the table.

What about detector framerate and resolution? I was running at 720x480 and 5 fps but tried upping to 1080p with 10fps to hopefully better identify things further away. What’s the best approach here?

hawkeye217 · 2024-03-02T02:51:35Z

hawkeye217
Mar 2, 2024
Collaborator

Good info to know. I’ll see if I can do anything to speed up the zooming speed (don’t think I can) or will drop the zoom factor.

The zooming motor speed is likely fixed, but speeding it up would definitely help if it's possible. You just want to minimize the amount of zooming that has to occur.

How can I have the entire frame be monitored without a full frame zone? If it comes in view of the camera and can positively identify it, I want it tracked.

Any autotracking algorithm is a balancing act between solid tracking of important objects and trying to catch every moving object. Frigate estimates the velocity of objects entering your frame and moves the camera based on those velocity estimates. If your zone is full-frame, there is very little velocity data for Frigate to use in a movement estimation. The more data Frigate has before your object enters your required_zone, the more accurate the tracking will be.

Personally, I've configured my required_zones to autotrack objects that enter into important areas around my property - areas that would actually be significant to security and any "gap" areas where my other cameras don't give great coverage of.

I’ve considered trying to learn more about frigate+ as it seems very reasonably priced and I’d love to add the ability to identify FedEx trucks, certain people or whatever else it may bring to the table.

While I'm just a contributor to Frigate, I have heard many great reports from other users about their experience with Frigate+. It's only improving as the months progress.

What about detector framerate and resolution? I was running at 720x480 and 5 fps but tried upping to 1080p with 10fps to hopefully better identify things further away. What’s the best approach here?

After a lot of testing and reports from other users during the beta period, 5fps should be sufficient for everyone. Higher rates only increase resource usage and, depending on your server configuration, may lead to unwanted dropped frames. A higher detect resolution would theoretically be beneficial for smaller objects in the frame.

Frigate's autotracking was designed to support low-end PTZs that don't have native autotracking features and be a supplement to "spotter" stationary cameras rather than a "catch all" for anything that moves in the frame. In an ideal world, an autotracking camera would be as responsive as the human eye and brain to be able to track every moving object in a frame. For the kind of tracking you're looking for, you may want to consider a high-end PTZ with much more responsive motors and firmware tracking algorithm. It will always outperform Frigate's way of autotracking with ONVIF.

1 reply

guycobb2 Mar 2, 2024
Author

settings did have PTZ speed options, but I had already turned it up to the max speed.

I created 16 equal zones across the full frame..... good idea or bad? I also dropped the frame rate for detection back down to 5fps.

the PTZ I'm using isn't a crazy $1k+ unit, but it's not a complete turd box either. It does have onboard AI auto tracking and whilst it does a pretty good job, it's limited to just people and vehicles. I like the idea of Frigate doing the tracking if possible since it can recognize quite a few more items. but, ultimately if it doesn't work well enough I can fall back to the onboard tracking.....which is what i've been using.

hawkeye217 · 2024-03-02T03:06:31Z

hawkeye217
Mar 2, 2024
Collaborator

settings did have PTZ speed options, but I had already turned it up to the max speed.

I created 16 equal zones across the full frame..... good idea or bad? I also dropped the frame rate for detection back down to 5fps.

If all 16 zones are part of your required_zones for autotracking, it's no different than a full-frame zone.

I should mention that an object that enters a required_zone will continue to be autotracked after it leaves a required_zone. Use required zones for the more important areas of your scene (driveways, entryways, sidewalks, etc).

the PTZ I'm using isn't a crazy $1k+ unit, but it's not a complete turd box either. It does have onboard AI auto tracking and whilst it does a pretty good job, it's limited to just people and vehicles. I like the idea of Frigate doing the tracking if possible since it can recognize quite a few more items. but, ultimately if it doesn't work well enough I can fall back to the onboard tracking.....which is what i've been using.

I noticed you're running an EmpireTech/Dahua PTZ1A2M-25X. The autotracking algorithm in that is always going to outperform what Frigate can do with ONVIF.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Support]: need advice on tweaks to make PTZ autotracking better/faster #10183

{{title}}

Replies: 4 comments 2 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

[Support]: need advice on tweaks to make PTZ autotracking better/faster #10183

guycobb2 Mar 1, 2024

Describe the problem you are having

Version

Frigate config file

Relevant log output

FFprobe output from your camera

Frigate stats

Operating system

Install method

Coral version

Network connection

Camera make and model

Any other information that may be helpful

Replies: 4 comments · 2 replies

NickM-27 Mar 2, 2024 Collaborator Sponsor

hawkeye217 Mar 2, 2024 Collaborator

guycobb2 Mar 2, 2024 Author

hawkeye217 Mar 2, 2024 Collaborator

guycobb2 Mar 2, 2024 Author

hawkeye217 Mar 2, 2024 Collaborator

guycobb2
Mar 1, 2024

Replies: 4 comments 2 replies

NickM-27
Mar 2, 2024
Collaborator Sponsor

hawkeye217
Mar 2, 2024
Collaborator

guycobb2 Mar 2, 2024
Author

hawkeye217
Mar 2, 2024
Collaborator

guycobb2 Mar 2, 2024
Author

hawkeye217
Mar 2, 2024
Collaborator