[Support]: /dev/shm keeps filling up despite increasing

[bluesk1d]

Describe the problem you are having

After updating to 15 beta 1 and starting the container, I noted the /dev/shm warning indicating I would need at least 167MB. I updated to 256 but it eventually filled up and crashed. I updated again to 512MB and it lasted a few days longer but still crashed trying to write to /dev/shm/logs/nginx/current.

Am I missing an additional config change?

Version

0.15.0-33825f6

What browser(s) are you using?

No response

Frigate config file

semantic_search:
  enabled: True
  reindex: False
model:
  path: plus://28b2d....
motion:
  threshold: 30
  contour_area: 10
mqtt:
  host: 192.168.1.6
  user: mqtt
  password: xxxxxx
ffmpeg:
  hwaccel_args: preset-vaapi
go2rtc:
  streams:
    porch_east:
      - rtsp://192.168.1.1:7447/
    doorbell:
      - rtsp://192.168.1.1:7447/
    backyard:
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.30.116:554/H264/ch1/main/av_stream
    garage_exterior:
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.30.169:554/H264/ch1/main/av_stream
    garage_interior:
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.30.143:554/stream1
detectors:
  coral:
    type: edgetpu
    device: usb
cameras:
  porch_east:
    review:
      alerts:
        required_zones:
          - Porch
          - Frontyard
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/porch_east
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record
    detect:
      fps: 5
      width: 1280
      height: 720
    motion:
      mask:
        - 0.177,0,0.18,0.04,0,0.039,0,0
        - 0.515,0.326,0.534,0.19,0.646,0.117,0.649,0.074,0.548,0,0.422,0,0.417,0.049,0.386,0.082,0.237,0.077,0.231,0.026,0.19,0.02,0.233,0.285,0.319,0.449,0.337,0.589,0.334,0.778,0.31,0.884,0.289,0.613,0,0.76,0,0,1,0,1,1,0.719,1,0.609,0.724,0.625,0.483,0.669,0.424,0.656,0.392
    zones:
      Porch:
        coordinates: 349,720,247,477,0,544,0,720
      West sideyard:
        coordinates: 1167,720,893,559,345,537,390,720
      Frontyard:
        coordinates: 740,339,891,560,341,539,314,372
      Sidewalk:
        coordinates: 750,340,789,297,890,290,886,249,686,252,627,282,363,298,294,276,321,373
      Street:
        coordinates: 564,0,709,0,853,66,889,240,275,269,250,107,585,72,633,55
    record:
      enabled: true
      retain:
        days: 0
  doorbell:
    audio:
      enabled: true
      min_volume: 200
      listen:
        - scream
        - speech
        - yell
        - gunshot
    review:
      alerts:
        required_zones:
          - Porch
          - Frontyard
          - Driveway
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/doorbell
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record
            - audio
    detect:
      fps: 5
      width: 1200
      height: 900
    motion:
      mask:
        - 1,0.447,0.88,0.459,0.878,0.441,0.834,0.448,0.688,0.427,0.689,0.455,0.651,0.541,0.623,0.547,0.603,0,1,0
        - 0,0.045,0.175,0.049,0.182,0,0,0
    zones:
      Porch:
        coordinates: 446,810,458,789,926,627,1171,706,1121,900,404,900,410,813
      Frontyard:
        coordinates: 1171,694,1200,585,1067,556,954,623
      Driveway:
        coordinates: 971,616,1057,553,854,507,748,525,747,658,854,613
      Sidewalk:
        coordinates: 957,528,1200,583,1200,556,947,507
      Street:
        coordinates: 761,442,935,462,1057,475,1179,496,1160,547,990,513,749,470
    record:
      enabled: true
      retain:
        days: 0
  backyard:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/backyard
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record
    detect:
      fps: 5
      width: 1920
      height: 1080
    motion:
      mask:
        - 0.11,0.272,0.106,0.09,0.211,0.087,0.212,0,0,0,0,0.42
        - 0.334,0.378,0.454,0.439,0.489,0.534,0.42,0.651,0.326,0.535,0.268,0.465,0.243,0.407,0.186,0.276,0.251,0.135,0.26,0.236,0.291,0.317
        - 0.49,0.08,0.49,0.047,0.579,0.015,0.593,0.049,0.599,0.083,0.594,0.112,0.633,0.127,0.64,0.154,0.594,0.188,0.588,0.144,0.5,0.104
    zones:
      Deck:
        coordinates: 1792,508,1816,373,1527,248,1217,479,1516,614,1321,960,1538,1080,1724,711
      Patio:
        coordinates: 
          1216,476,1210,540,1283,578,1216,644,1433,765,1317,973,1517,1080,1046,1080,946,1011,970,978,944,954,1056,854,944,778,968,754,883,686,951,635,486,337,940,140,1473,295
      Backyard_perimeter:
        coordinates: 
          0.24,0.29,0.506,0.1,0.768,0.256,0.797,0.23,0.946,0.347,0.969,0.083,0.851,0,0.481,0,0.221,0.162,0,0.425,0,1,0.542,1,0.481,0.914,0.546,0.801,0.488,0.723,0.509,0.706,0.46,0.629,0.488,0.589
        inertia: 3
  garage_interior:
    detect:
      width: 1280
      height: 720
      fps: 5
    record:
      enabled: true
      retain:
        days: 1
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage_interior
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record
    motion:
      mask:
        - 0.317,0.042,0.002,0.042,0,0,0.314,0
    zones:
      Garage:
        coordinates: 1,0.655,0.612,0.358,0.177,0.516,0,0.592,0,1,1,1
        loitering_time: 0
        inertia: 3
      Driveway:
        coordinates: 0.803,0.271,0.991,0.44,0.961,0.624,0.613,0.358
        loitering_time: 0
      East_sideyard:
        coordinates: 0.803,0.27,0.735,0.228,0.615,0.252,0.612,0.355
        loitering_time: 0
  garage_exterior:
    review:
      alerts:
        required_zones:
          - Porch
          - Frontyard
          - Driveway
          - East sideyard
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage_exterior
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record
    detect:
      width: 2560
      height: 720
      fps: 5
    motion:
      mask: 
        0,0.858,0.143,0.51,0.121,0.423,0.13,0.37,0.213,0.393,0.238,0.285,0.225,0.057,0.338,0.002,0.679,0,0.717,0.095,0.68,0.109,0.657,0.231,0.628,0.413,0.639,0.563,0.694,0.591,0.834,0.508,0.866,0.445,0.902,0.544,0.92,0.465,0.956,0.559,0.977,0.439,0.999,0.498,1,0,0,0
    zones:
      Porch:
        coordinates: 
          2078,720,1763,515,1686,644,1544,720,1403,510,1644,448,1849,523,2067,404,2233,329,2279,495,2303,468,2351,486,2432,462,2491,468,2318,720
      Frontyard:
        coordinates: 1709,422,1716,253,1490,199,1231,180,1405,507
      East sideyard:
        coordinates: 314,441,148,468,0,531,0,720,339,720,571,427,709,195,571,218
      Driveway:
        coordinates: 1531,720,1364,437,1221,180,947,180,695,200,556,425,348,720
      Sidewalk:
        coordinates: 
          375,210,650,158,877,131,1196,121,1418,137,1730,187,1714,234,1503,193,1236,178,934,180,764,191,627,203,483,225,369,241,192,276,166,254
      Street:
        coordinates: 
          1395,0,1692,51,1635,0,1716,0,1870,96,1786,196,1227,123,806,139,496,190,152,257,137,230,323,148,583,63,823,0
objects:
  track:
    - person
    - cat
    - dog
    - package
  filters:
    car:
      min_score: .85
      threshold: .95
    dog:
      min_score: .8
      threshold: .94
    cat:
      min_score: .65
      threshold: .90
    package:
      min_score: .65
      threshold: .9
    person:
      min_score: .80
      threshold: .96
record:
  enabled: true
  retain:
    days: 14
    mode: all
  alerts:
    retain:
      days: 7
  detections:
    retain:
      days: 7
review:
  alerts:
    labels:
      - person
snapshots:
  enabled: true
version: 0.15-0

Relevant Frigate log output

s6-log: warning: unable to write to /dev/shm/logs/nginx/current: No space left on device
s6-log: warning: unable to write to /dev/shm/logs/nginx/current: No space left on device
s6-log: warning: unable to write to /dev/shm/logs/nginx/current: No space left on device
s6-log: warning: unable to write to /dev/shm/logs/nginx/current: No space left on device
s6-log: warning: unable to write to /dev/shm/logs/nginx/current: No space left on device

Relevant go2rtc log output

na

FFprobe output from your camera

na

Frigate stats

No response

Install method

Docker CLI

docker-compose file or Docker CLI command

--shm-size=512m

Object Detector

Coral

Network connection

Wired

Camera make and model

NA

Screenshots of the Frigate UI's System metrics pages

No response

Any other information that may be helpful

No response

[ivanjx]

i usually use 1gb on v15

[effoc]

Filled 1g today so expanded now to 2Gb have the same problem

[gashalot]

I can also confirm that this is a regression with 0.15. I've got 8 4k cameras running, and the system routinely fills 1G of SHM. On 0.14, 768Mb of SHM was adequate.

I logged into the container on the last crash and checked usage of SHM. None of the log files take up a material amount of space, but the cameras themselves do. Here is an example listing of the disk usage for a single camera at the time of crash. All have similar consumption patterns.

The only fix here is a hard restart of frigate every few hours, which is entirely untenable in a production situation.

300K	deck
7.3M	deck1731702143.819199
7.3M	deck1731702144.017641
7.3M	deck1731702144.217096
7.3M	deck1731702144.415887
7.3M	deck1731702144.620421
7.3M	deck1731702144.817344
7.3M	deck1731702145.010544
7.3M	deck1731702145.21396
7.3M	deck1731702145.414621
7.3M	deck1731702145.614321
7.3M	deck1731702145.816226
7.3M	deck1731702146.014961
7.3M	deck1731702146.214983
7.3M	deck1731702146.41527
7.3M	deck1731702146.612055
7.3M	deck1731702146.815829
0	deck1731702147.008172

[NickM-27]

This is fixed in the next version