"Uncaught RangeError: offset is out of bounds" Error

[maxi1134]

Checklist:

• [ x] I updated to the latest version available
• [x ] I cleared the cache of my browser

Release with the issue:
2.0.0
Last working release (if known):
the one before (Unsure which number )
Browser and Operating System:
Chrome, Ipad Fullkiosk and the HA app on a android table

Description of problem:

My video feeds don't load (White screen) and this appears in the logs since update to latest version

Additional information:

[dermotduffy]

Have not encountered that before. Can you provide your card configuration, and describe what you're doing when those errors arrive? If you have a stacktrace or anything else on on the Javascript console (F12 in Chrome) that would be really helpful.

[dermotduffy]

Also when you visit the Frigate server web UI, do the 'Live' and 'Debug' video streams work okay, or do you see a similar error?

[Mincka]

I also get that one randomly on a specific device (Surface Pro / Windows 10 / Edge Latest).
It is mixed with thousands of repeating Uncaught RuntimeError: memory access out of bounds.

In the HA log, I get thousands of Script error and my logs are flooded.

2021-11-14 08:27:27 ERROR (MainThread) [frontend.js.latest.202111090] http://homeassistant.local:8123/hacsfiles/frigate-hass-card/frigate-hass-card.js?hacstag=394082552200:557:39882 Uncaught RangeError: offset is out of bounds
2021-11-14 08:27:27 ERROR (MainThread) [frontend.js.latest.202111090] :0:0 Script error.
2021-11-14 08:27:27 ERROR (MainThread) [frontend.js.latest.202111090] http://homeassistant.local:8123/hacsfiles/frigate-hass-card/frigate-hass-card.js?hacstag=394082552200:557:39882 Uncaught RangeError: offset is out of bounds
2021-11-14 08:27:27 ERROR (MainThread) [frontend.js.latest.202111090] :0:0 Script error.
2021-11-14 08:27:27 ERROR (MainThread) [frontend.js.latest.202111090] :0:0 Script error.
2021-11-14 08:27:27 ERROR (MainThread) [frontend.js.latest.202111090] :0:0 Script error.
2021-11-14 08:27:27 ERROR (MainThread) [frontend.js.latest.202111090] :0:0 Script error.
2021-11-14 08:27:28 ERROR (MainThread) [frontend.js.latest.202111090] :0:0 Script error.
2021-11-14 08:27:28 ERROR (MainThread) [frontend.js.latest.202111090] :0:0 Script error.

In the console of the browser:

       Uncaught RuntimeError: memory access out of bounds
    at 00025b02:0x5b8a
    at 00025b02:0x56d9
    at 00025b02:0x3f4
    at e.t.initializeWasmDecoder (frigate-hass-card.js?hacstag=394082552200:557)
    at e.t.bufferWrite (frigate-hass-card.js?hacstag=394082552200:557)
    at e.write (frigate-hass-card.js?hacstag=394082552200:557)
    at e.t.write (frigate-hass-card.js?hacstag=394082552200:557)
    at A.e.packetComplete (frigate-hass-card.js?hacstag=394082552200:557)
    at A.e.parsePacket (frigate-hass-card.js?hacstag=394082552200:557)
    at A.e.write (frigate-hass-card.js?hacstag=394082552200:557)
$func52 @ 00025b02:0x5b8a
$_malloc @ 00025b02:0x56d9
$_mpeg1_decoder_create @ 00025b02:0x3f4
t.initializeWasmDecoder @ frigate-hass-card.js?hacstag=394082552200:557
t.bufferWrite @ frigate-hass-card.js?hacstag=394082552200:557
e.write @ frigate-hass-card.js?hacstag=394082552200:557
t.write @ frigate-hass-card.js?hacstag=394082552200:557
e.packetComplete @ frigate-hass-card.js?hacstag=394082552200:557
e.parsePacket @ frigate-hass-card.js?hacstag=394082552200:557
e.write @ frigate-hass-card.js?hacstag=394082552200:557
e.onMessage @ frigate-hass-card.js?hacstag=394082552200:557
00025b02:0x5b8a

The cards stay white and don't recover. The only way to fix it is to reload the page. I have an extension to reload the page every 30 minutes my kiosk view (because it does not refresh automatically if I make changes to the dashboard on another device). Sometimes (maybe every 10 reloads), the cards can't load and these errors loop. I am not able to reproduce it easily. :(

[Mincka]

Related errors upstream:
cycjimmy/jsmpeg-player#33
cycjimmy/jsmpeg-player#27
phoboslab/jsmpeg#268

[Mincka]

I don't have the stack trace for RangeError: offset is out of bounds".
From what I saw in the related issues upstream, it happens when the page is indeed reloaded multiple times and it may be related to improper garbage collection. So in my case, I just stopped the auto-reload extension. It was more useful when I add authentication errors in the previous version.

@maxi1134 : Do you know if you get this error on a browser which has been open for a long time or on a computer with low memory?

[maxi1134]

@Mincka I do leave the browser open for hours on end on the video view!

I also get white screens and no loading after a few reloads of the page.

32GB of ram here

[blakeblackshear]

I have seen this when using very high resolutions.

[maxi1134]

@blakeblackshear I am using 2 and 4k cameras! ( 5 and 8MP for the biggest ones ).

That might be the issue?

[Mincka]

Not my case. Over 4 cards, 3 streams are low resolution (640x360), another is 1920x1080.
Having multiple cards may lead to a similar result: high memory usage.

[dermotduffy]

I cannot reproduce this with any of my cameras -- but I suspect this is just because they are lower resolution / lower bitrate than the cameras you are experimenting with.

Right now both the Frigate card and the Frigate web UI use a 4MB JSMPEG videoBufferSize. If I reduce that value, I can reproduce errors that look like the ones @Mincka reports (although I need to reduce it substantially). This may suggest that increasing it could help.

If anyone can readily reproduce this and is comfortable doing their own build, try setting a much higher value here and lets see if it has any effect?

The upstream issue also suggest that the ffmpeg parameters used for the encoding on the Frigate end could also be something to investigate/fiddle with.

[Mincka]

Tested with videoBufferSize: 1024 * 1024 * 40 and it happened again, but only 1 time for about 60 reloads (don't say I didn't try lol).
I am sure it was my build that was loaded when it crashed since I used a custom version number.
I would say it's better and it looks like to load a little bit faster. Since it's hard to reproduce, I will check if it happens again in the coming days.

It would be great if @maxi1134 could also test it.

Update your frigate-hass-card.js from config/www/community/frigate-hass-card with the code below:
frigate-hass-card.js.txt

Delete the frigate-hass-card.gz file. Ctrl+F5 and/or Shift+Ctrl+R to reload.

Edit: Anyhow, would it be possible to catch these exceptions from the card and trigger a reset to prevent an unexpected flooding?

[dermotduffy]

I added a way to specify custom JSMPEG parameters in the card YAML (as specifying a giant video buffer size for most users is probably unnecessarily inefficient). In the next build, you can do something like:

live:
  jsmpeg:
    options:
      videoBufferSize: 41943040

... which is equivalent to the custom build @Mincka did.

@Mincka also mentioned a potential garbage collection problem in the upstream JSMPEG library -- I think they're on to something. If I set a very low videoBufferSize I get the same console spam. If I subsequently set a new videoBufferSize (without reloading) I still continue to get the console spam. I don't believe this is the card failing to do its garbage collection, as the card intentionally destroys the JSMPEG player at the right moments. I discovered that if I add disableWebAssembly: true I no longer get this strange behavior, so my theory is the memory issue is related to how JSMPEG uses WASM.

[maxi1134]

Tested with videoBufferSize: 1024 * 1024 * 40 and it happened again, but only 1 time for about 60 reloads (don't say I didn't try lol). I am sure it was my build that was loaded when it crashed since I used a custom version number. I would say it's better and it looks like to load a little bit faster. Since it's hard to reproduce, I will check if it happens again in the coming days.

It would be great if @maxi1134 could also test it.

Update your frigate-hass-card.js from config/www/community/frigate-hass-card with the code below: frigate-hass-card.js.txt

Delete the frigate-hass-card.gz file. Ctrl+F5 and/or Shift+Ctrl+R to reload.

Edit: Anyhow, would it be possible to catch these exceptions from the card and trigger a reset to prevent an unexpected flooding?

I replaced my file and the errors continue!

Update: It worked for a little while on Firefox but after a few hours of streaming the error is back in the console.

[Mincka]

It also still happen in my case. I try a new build with videoBufferSize: 1024 * 1024 * 400 just to see if it happens again.
The YAML additions are very welcome and I will try to use them after this last quick and dirty attempt.

[Mincka]

It's quite sad but the error happened again the first time I reloaded the page with the latest build (with 419430400 as buffer size). In the end, I am not sure this setting is responsible (at least alone) for this error. I will try with disableWebAssembly as soon as possible.

If the exception bubbles up to the console / HA, I guess you could catch it and reset the card @dermotduffy? Of course it would be best to find the root cause, but I think it could be helpful to prevent an unexpected log flooding from the card. when the dependencies fail for some reason.

[dermotduffy]

It seems like it would be simple, but because the errors are not really related to the card all, and not in any of the call paths I control (they are buried deep inside the JSMPEG libraries in async code that gets triggered from Websocket traffic), there isn't even a straightforward way for me to notice or catch these errors. I'll see if I can hack something in, but it won't be pretty...

I'm not sure reset the card is right, as there's no guarantee it'll work after the reset either and it could easily generate a reset-loop.

[Mincka]

OK I understand. :( I am now running the latest version with only disableWebAssembly as true. I will let you know if it prevents it.

[Mincka]

No errors today. With web assembly disabled, the stream seams to take a little bit longer but that's OK for an initial load on a kiosk. It could be a fix for me. I will check back in a few days. Thanks!

[Mincka]

Still no error on the Surface Pro used as kiosk while using only this:

jsmpeg:
  options:
    disableWebAssembly: true

I consider this workaround good enough for me. Thank you. Maybe something to add to the Troubleshooting section?

[maxi1134]

Is the DisableWebAssemble option available in the latest HACS version?

If not, how can I install it!?

Thanks!

[dermotduffy]

@maxi1134 It's not, you'll either need to do your own custom build or wait for the next beta version (probably within the next week).

[maxi1134]

I got a new, yet similar error today!

2021-11-25 15:20:16 ERROR (MainThread) [frontend.js.latest.202111090] http://192.168.0.15:8123/hacsfiles/frigate-hass-card/frigate-hass-card.js?hacstag=394082552200:557:66175 Uncaught RangeError: WebAssembly.Memory.grow(): Unable to grow instance memory

[maxi1134]

I added

jsmpeg:
options:
disableWebAssembly: true

and it seems to work!

[dermotduffy]

Just because the configuration was not rejected, doesn't mean that option was honored! Unless you did a fresh build yourself, it should not have any effect.

FYI: I'm planning a new build this weekend, just trying to squish a few more bugs first ...

[maxi1134]

I did a fresh build! ( Only took me 4 hours to figure out how 😅 )

[maxi1134]

It seems to work after a reboot!

But after a while, my logs get filled with this:

[dermotduffy]

@maxi1134 Can you see if you can get me the stacktrace for one of those in your Javascript console (see example from earlier in this thread)? I'm not sure there's much I can do, but I can try to mute the log spam at least...

[dermotduffy]

I had another look at this, and even wrapping the calls to prevent the exception spam is an exercise in futility -- when it gets into this situation, it seems like the exceptions can stem from a variety of different sync & async methods -- wrapping one only just causes the exceptions to pop out somewhere else. So I am going to close this bug, as I'm struggling to identify a sensible action to take that doesn't just depend on the upstream bugs being resolved.

For those in this situation, if the disableWebAssembly or videoBufferSize tricks don't work for you, and/or if lowering your resolution/bitrate aren't options or don't help, using a different live provider in the card (e.g. WebRTC) may be the necessary direction.

Of course, if anyone else has bright ideas of things that can be done in the card, I'd love to hear it.