Crashing every few hours: CPU goes to 100%, network to 0%, freezes

[DSew20]

I have been running Frigate without issue for several months. Then a week or so ago started getting issues where it just stopped working. When I reboot the machine it works fine for a few hours, then the CPU hits 100% and the system freezes.

System:

• Frigate v0.13.2 on Docker - all at the latest versions and updated
• 4 cameras of varying types
• Coral edge tpu
• Hardware Proxmox on a Intel NUC

What I have tried

• Rebooting - Frigate VM and Proxmox server
• Rebuilding the docker container
• Updating the docker container (to 0.13.2)

There are no errors in the log at the point the system jumps to 100% and freezes.

What do I need - some pointers re where to look and how to troubleshoot? As above, was running great. Nothing was changed in the setup and it just started having issue. After 3 days of trying to find the reason run out of ideas.

Config:

`
mqtt:
  enabled: true
  host: 192.168.yy.xx
  user: aaabbb
  password: aaabbb
  port: 1883

database:
  path: /media/database/frigate.db

ffmpeg:
  hwaccel_args: preset-vaapi

detectors:
  coral:
    type: edgetpu
    device: usb

#Global Object Settings
objects:
  track:
    - person
    - umbrella
  filters:
    person:
      min_area: 5000
      max_area: 100000


go2rtc:
  streams:
    garage:
      - rtsp://xxx.xxx.xxx.xxx:554/11
    garage_sub:
      - rtsp://xxx.xxx.xxx.xxx:554/12
    front-door:
      - rtsp://aaaaa:[email protected]:554/mpeg/720p/media.amp
    hik_front:
      - rtsp://aaaaa:[email protected]:554/Streaming/Channels/101
    hik_front_sub:
      - rtsp://aaaaa:[email protected]:554/Streaming/Channels/102
    hik_rear:
      - rtsp://aaaaa:[email protected]:554/Streaming/Channels/101
    hik_rear_sub:
      - rtsp://aaaaa:[email protected]:554/Streaming/Channels/102
      
cameras:
  #######################################################################################################
  # FRONT DOOR
  #######################################################################################################
  front-door:
    ffmpeg:
      inputs:
        - path: 'rtsp://aaaaa:[email protected]:554/mpeg/720p/media.amp'
          roles:
            - record
            - detect
    detect:
      width: 640   #1280
      height: 360  #720
      fps: 5       #30
    objects:
      track:
        - person
        - umbrella
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 21
    record:
      enabled: true
      retain:
        days: 2
      events:
        retain:
          default: 21
  
  #######################################################################################################
  # HIK VISION REAR GARDEN
  #######################################################################################################
  rear:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/hik_rear
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/hik_rear_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
    detect:

      width: 640
      height: 480
      fps: 6
    snapshots:
      enabled: True
      timestamp: false
      bounding_box: True
      retain:
        default: 21
    record:
      enabled: True
      retain:
        days: 3
      events:
        retain:
          default: 21

  #######################################################################################################
  # HIK VISION FRONT
  #######################################################################################################
  HikVision_front:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/hik_front
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/hik_front_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
    detect:
      width: 640
      height: 380
      fps: 6
    objects:
      track:
        - person
        - car
        - dog
        - cat
        - fox
        - bicycle
        - umbrella
    snapshots:
      enabled: True
      timestamp: false
      bounding_box: True
      retain:
        default: 21
    record:
      enabled: True
      retain:
        days: 7
      events:
        retain:
          default: 21
    motion:
      mask:
        - 0,0,614,0,597,142,385,106,379,149,280,162,244,148,0,163
    zones:
      driveway:
        coordinates: 0,380,589,380,619,285,464,301,389,185,282,162,0,189,47,252,0,290
        #objects:
        #  - car
  #######################################################################################################
  # GARAGE
  #######################################################################################################
  garage:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/garage_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
    detect:
      width: 640
      height: 352
      fps: 5
    objects:
      track:
        - person
        - car
        - bicycle
        - umbrella
      filters:
        car:
          threshold: 0.75
          
    snapshots:
      enabled: True
      timestamp: false
      bounding_box: True
      retain:
        default: 21
    record:
      enabled: True
      retain:
        days: 5
      events:
        retain:
          default: 21
    motion:
      mask:
        - 318,199,266,352,0,352,0,276
        - 190,183,104,222,99,159,144,125 
    zones:
      driveway:
        coordinates: 265,79,270,117,151,122,50,171,75,89,203,57
        objects:
          - car
          - person
      front_garden:
        coordinates: 325,239,364,352,603,352,511,190,520,86,516,0,300,59
        objects:
          - person
          - umbrella
      garage_front:
        coordinates: 81,279,292,205,264,112,160,128,58,176
        objects:
          - person

`

[ipkryss]

I have the exact same problem, running frigate as lxc in proxmox and with a coral TPU. All of a sudden I notice that the VM/LXC is running at 100% CPU and 100% swap and I have to reboot it to get it running again.

[hawkeye217]

This is likely the same issue with the Proxmox kernel that multiple users have seen recently.