RPI4 Hardware Acceleration Fix #14

Artiume · 2020-01-01T20:50:24Z

If you are new to Docker or this application our issue tracker is ONLY used for reporting bugs or requesting features. Please use our discord server for general support.

Expected Behavior

HWA should work.

Current Behavior

HWA fails to work.

Steps to Reproduce

I wrote this guide for HWA. In order to get HWA working, I had to make a few hacks and would like to pass on what I learned to the lsio team.

version: "3.7"

services:
  jellyfin:
    image: linuxserver/jellyfin
    devices:
      - /dev/vchiq:/dev/vchiq ##HWA Chip
    container_name: jellyfin
    network_mode: host
    environment:
      PUID: 0
      PGID: 0
      TZ: ${TZ}
      UMASK_SET: "022"
    volumes:
      - /data/brick/jellyfin:/config
      - /media:/media
      - /dev/shm:/config/data/transcoding-temp/transcodes
      - /opt/vc/lib:/opt/vc/lib   ## OpenMax Libraries
      - /etc/ld.so.conf.d/00-vmcs.conf:/etc/ld.so.conf.d/00-vmcs.conf ## Path needed for libraries
    restart: always

I also needed to install the library libomxil-bellagio-dev

This installed libomxil-bellagio0, libomxil-bellagio-bin and libomxil-bellagio-dev. I'm not sure which were essential but -dev installed all 3. I also had to run the container as root, but that might have been due to the Path hacks or permissions else where, I'm not sure.

Environment

OS: Raspbian
CPU architecture: x86_64/arm32/arm64 Rpi4
How docker service was installed:

Command used to create docker container (run/create/compose/screenshot)

Docker logs

The text was updated successfully, but these errors were encountered:

thelamer · 2020-01-03T22:16:01Z

I am down for embedding some stuff into the aarch64 image to support openmax , what kind of performance difference did you see and what codecs are currently supported ?
Last time I played with it, it was basically audio only and very negligible.

Artiume · 2020-01-03T22:51:20Z

I can encode x264 using HWA. I still have to decode x265 using software (until it can be patched to include it). The HEVC decoder will require a bump in video ram but the x264 encoder seems to work fine with the default 78Mb. I'm not sure what audio accelerations are available.

x265 10bit 4.2 Mbps > x264 8bit 7.8 Mbps. No audio conversion, I hover around 320% to 380% CPU usage but no stuttering, I can seek to a new position in the video in ~5 seconds. If I turn off HWA. Video will stutter every couple seconds and isn't watchable.

Here are the ffmpeg logs, not sure what's up with libx264, the speed is all sorts out of whack lol.

  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_omx))
  Stream #0:1 -> #0:1 (copy)

  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (h264_omx), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 7430 kb/s, 23.98 fps, 90k tbn, 23.98 tbc (default)
    Metadata:
      encoder         : Lavc58.54.100 h264_omx
    Stream #0:1: Audio: aac (LC), 48000 Hz, 5.1, fltp (default)
frame=    9 fps=0.0 q=-0.0 size=N/A time=00:00:00.83 bitrate=N/A speed=1.61x    
frame=   24 fps= 23 q=-0.0 size=N/A time=00:00:01.23 bitrate=N/A speed=1.18x    
frame=   36 fps= 23 q=-0.0 size=N/A time=00:00:01.74 bitrate=N/A speed=1.11x    
frame=   47 fps= 22 q=-0.0 size=N/A time=00:00:02.28 bitrate=N/A speed=1.07x    
frame=   63 fps= 24 q=-0.0 size=N/A time=00:00:02.94 bitrate=N/A speed=1.11x    
[hls @ 0x2d7c5e0] Opening '/config/data/transcoding-temp/transcodes/transcodes/71eab540ce83b7b33dd0dc6d9c54b6541.ts' for writing
frame=   77 fps= 24 q=-0.0 size=N/A time=00:00:03.45 bitrate=N/A speed= 1.1x    
frame=   87 fps= 24 q=-0.0 size=N/A time=00:00:03.96 bitrate=N/A speed=1.09x    
frame=   94 fps= 23 q=-0.0 size=N/A time=00:00:04.16 bitrate=N/A speed=0.999x    
frame=  104 fps= 22 q=-0.0 size=N/A time=00:00:04.64 bitrate=N/A speed=0.991x    
frame=  113 fps= 22 q=-0.0 size=N/A time=00:00:04.90 bitrate=N/A speed=0.939x    
frame=  122 fps= 21 q=-0.0 size=N/A time=00:00:05.44 bitrate=N/A speed=0.949x    
frame=  132 fps= 21 q=-0.0 size=N/A time=00:00:05.75 bitrate=N/A speed=0.924x

  Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (copy)

  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (libx264), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 23.98 fps, 90k tbn, 23.98 tbc (default)
    Metadata:
      encoder         : Lavc58.54.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 7430000/0/0 buffer size: 14861000 vbv_delay: -1
    Stream #0:1: Audio: aac (LC), 48000 Hz, 5.1, fltp (default)
frame=    7 fps=0.0 q=0.0 size=N/A time=00:06:38.10 bitrate=N/A speed= 769x    
frame=   17 fps= 15 q=0.0 size=N/A time=00:06:38.61 bitrate=N/A speed= 348x    
frame=   25 fps= 15 q=28.0 size=N/A time=00:06:38.95 bitrate=N/A speed= 235x    
frame=   32 fps= 15 q=25.0 size=N/A time=00:06:39.29 bitrate=N/A speed= 181x    
frame=   40 fps= 14 q=25.0 size=N/A time=00:06:39.46 bitrate=N/A speed= 144x    
frame=   49 fps= 15 q=28.0 size=N/A time=00:06:39.97 bitrate=N/A speed= 121x    
frame=   56 fps= 14 q=28.0 size=N/A time=00:06:40.17 bitrate=N/A speed= 103x    
frame=   64 fps= 14 q=28.0 size=N/A time=00:06:40.51 bitrate=N/A speed=89.7x    
frame=   72 fps= 14 q=28.0 size=N/A time=00:06:41.00 bitrate=N/A speed=  80x    
frame=   79 fps= 14 q=28.0 size=N/A time=00:06:41.17 bitrate=N/A speed=72.3x    
frame=   85 fps= 14 q=28.0 size=N/A time=00:06:41.42 bitrate=N/A speed=  66x    
frame=   94 fps= 14 q=28.0 size=N/A time=00:06:41.76 bitrate=N/A speed=60.7x    
frame=  102 fps= 14 q=28.0 size=N/A time=00:06:42.11 bitrate=N/A speed=55.8x    
frame=  110 fps= 14 q=28.0 size=N/A time=00:06:42.45 bitrate=N/A speed=52.1x    
[hls @ 0x2b168e0] Opening '/config/data/transcoding-temp/transcodes/transcodes/transcodes/7d7890a8bd5d7efc8303d30e0c49776b134.ts' for writing
frame=  117 fps= 14 q=28.0 size=N/A time=00:06:42.79 bitrate=N/A speed=48.2x    
frame=  124 fps= 14 q=28.0 size=N/A time=00:06:42.96 bitrate=N/A speed=45.1x    
frame=  130 fps= 14 q=28.0 size=N/A time=00:06:43.22 bitrate=N/A speed=42.7x

thelamer · 2020-01-03T22:53:56Z

cool stuff.

What is the contents of /dev/vchiq on your system, we should be able to automate the permissions issue if I know what is in there.

Artiume · 2020-01-03T22:56:49Z

pi@raspberrypi:~ $ ls /dev/vchiq
/dev/vchiq

The file is text based, would you like it posted?

thelamer · 2020-01-03T23:05:05Z

no , just weird that it is a single file like that , we use the following find command with other video devices to pull their information and add the abc user in this container to their group to be able to directly open them.

find /dev/dri /dev/dvb -type c -print

Does this actually print the device when run ?

find /dev/vchiq -type c -print

Artiume · 2020-01-03T23:06:22Z

nope, it returned nothing.

thelamer · 2020-01-03T23:09:33Z

who is it owned by ? ls -l

Artiume · 2020-01-03T23:10:33Z

crw-rw---- 1 root video 243, 0 Jan 1 21:16 vchiq

thelamer · 2020-01-04T00:22:34Z

Can you try this out (change your puid and pgid to what you use local)

version: "3.7"

services:
  jellyfin:
    image: lsiodev/jellyfin:v10.4.3-pkg-5749f6c2-dev-6d4ba8d69ee572a9c934a39852967ce5d844eaae
    devices:
      - /dev/vchiq:/dev/vchiq ##HWA Chip
    container_name: jellyfin
    network_mode: host
    environment:
      PUID: 1000
      PGID: 1000
      TZ: ${TZ}
      UMASK_SET: "022"
    volumes:
      - /data/brick/jellyfin:/config
      - /media:/media
      - /dev/shm:/config/data/transcoding-temp/transcodes
      - /opt/vc/lib:/opt/vc/lib   ## OpenMax Libraries
      - /etc/ld.so.conf.d/00-vmcs.conf:/etc/ld.so.conf.d/00-vmcs.conf ## Path needed for libraries
    restart: always

This should fix you having to run as root and needing to install openmax bins, but I cannot embed the Rpi firmware stuff in the images so that will always be a bind mount.
Let me know, and don't forget to chown stuff back from a root user to test or use alternative locations for testing.

Artiume · 2020-01-04T02:57:08Z

No dice. the package didnt install properly. it couldn't find the latest and had to fallback to .9.3.4.

1000:1000, no update - ffmpeg exited 255, no useful logs

Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_omx))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[Parsed_subtitles_1 @ 0x2bf6440] Shaper: FriBidi 0.19.7 (SIMPLE) HarfBuzz-ng 1.7.2 (COMPLEX)
[Parsed_subtitles_1 @ 0x2bf6440] Using font provider fontconfig

root

Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_omx))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[Parsed_subtitles_1 @ 0x2dfe5a0] Shaper: FriBidi 0.19.7 (SIMPLE) HarfBuzz-ng 1.7.2 (COMPLEX)
[Parsed_subtitles_1 @ 0x2dfe5a0] Using font provider fontconfig
[h264_omx @ 0x2e4e1e0] Using OMX.broadcom.video_encode
[h264_omx @ 0x2e4e1e0] OMX_GetHandle(OMX.broadcom.video_encode) failed: 80001005
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x2e1df30] Qavg: 65421.898
[aac @ 0x2e1df30] 2 frames left in the queue on closing
Conversion failed!

update logs

debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/arm-linux-gnueabihf/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/arm-linux-gnueabihf/perl5/5.26 /usr/share/perl5 /usr/lib/arm-linux-gnueabihf/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/arm-linux-gnueabihf/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, <> line 1.)
debconf: falling back to frontend: Teletype
Selecting previously unselected package libomxil-bellagio-dev. 
Preparing to unpack .../libomxil-bellagio-dev_0.9.3-4_armhf.deb ...

FFMpeg exited with code 1 ffmpeg failing due to the encoder failing.

OMX_GetHandle(OMX.broadcom.video_encode) failed: 80001005 Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

So this is where I'm at, I'll look into it tomorrow as to what I'm missing.

Artiume · 2020-01-04T15:15:41Z

So while running as root, adding abc to the video group fixed it. let me start from scratch and see if that's the issue
Edit: I think i was playing an x264 file

thelamer · 2020-01-04T22:35:03Z

try this tag:
lsiodev/jellyfin:v10.4.3-pkg-5749f6c2-dev-635df2acc3ce03024f255cd07e2c6ff2dfcc2653

I was using a file guard for the device ownership logic should have been -e, this should automate the video group add like we do on x86.

Artiume · 2020-01-04T23:11:20Z

so something funky is going on with the permissions. With both patches, I can't get HWA working at all while with the normal linuxserver/jellyfin, installing the 3 packages makes it work. Also, we're missing some permissions for abc. When I run it as abc, ffmpeg will exit with code 255 and won't have accessed the openmax module. Root will exit ffmpeg with code 1 for the accelerator failing.

pi@raspberrypi:/dev $ sudo docker exec -it jellyfinpi apt-get install libomxil-bellagio-dev -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libomxil-bellagio-bin libomxil-bellagio0
Suggested packages:
  libomxil-bellagio-doc libomxil-bellagio0-components-base
The following NEW packages will be installed:
  libomxil-bellagio-bin libomxil-bellagio-dev libomxil-bellagio0
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 131 kB of archives.
After this operation, 684 kB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports bionic/universe armhf libomxil-bellagio-bin armhf 0.9.3-4 [10.9 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports bionic/universe armhf libomxil-bellagio0 armhf 0.9.3-4 [27.9 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports bionic/universe armhf libomxil-bellagio-dev armhf 0.9.3-4 [91.9 kB]
Fetched 131 kB in 0s (325 kB/s)
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 3.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/arm-linux-gnueabihf/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/arm-linux-gnueabihf/perl5/5.26 /usr/share/perl5 /usr/lib/arm-linux-gnueabihf/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/arm-linux-gnueabihf/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, <> line 3.)
debconf: falling back to frontend: Teletype
Selecting previously unselected package libomxil-bellagio-bin.
(Reading database ... 9177 files and directories currently installed.)
Preparing to unpack .../libomxil-bellagio-bin_0.9.3-4_armhf.deb ...
Unpacking libomxil-bellagio-bin (0.9.3-4) ...
Selecting previously unselected package libomxil-bellagio0.
Preparing to unpack .../libomxil-bellagio0_0.9.3-4_armhf.deb ...
Unpacking libomxil-bellagio0 (0.9.3-4) ...
Selecting previously unselected package libomxil-bellagio-dev.
Preparing to unpack .../libomxil-bellagio-dev_0.9.3-4_armhf.deb ...
Unpacking libomxil-bellagio-dev (0.9.3-4) ...
Setting up libomxil-bellagio-bin (0.9.3-4) ...
Setting up libomxil-bellagio0 (0.9.3-4) ...
Setting up libomxil-bellagio-dev (0.9.3-4) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...

thelamer · 2020-01-05T01:44:33Z

I gotta check, because what you are saying is not possible those libs are specifcially installed :
https://github.com/linuxserver/docker-jellyfin/blob/arm-hardware/Dockerfile.armhf#L30-L31

$ docker run --rm --entrypoint '/bin/sh' lsiodev/jellyfin:arm32v7-v10.4.3-pkg-5749f6c2-dev-635df2acc3ce03024f255cd07e2c6ff2dfcc2653 -c 'apt list -qq --installed | sed "s#/.*now ##g" | cut -d" " -f1 | grep -i bellagio' 

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libomxil-bellagio-bin0.9.3-4
libomxil-bellagio00.9.3-4

You are swapping the tag and using the lsiodev endpoint right ?

Artiume · 2020-01-05T12:15:32Z

Yeah, the install log was based on the normal image, I was providing it in case it could provide some info. Any idea why I can't use HWA as root on your dev images?

thelamer · 2020-01-06T17:15:10Z

I have not forgotten about this, but it is very difficult to make something optimal without local hardware to test on.
A couple more commands I guess:

To see the perms and ensure abc is a member of the video group :

cat /etc/groups |grep abc
ls -l /dev/vchiq

You are mounting in a custom ldconf file but unless you run ldconfig at some point in the container those bins will not be registered .

Artiume · 2020-01-07T11:53:21Z

dev image

pi@raspberrypi:~ $ sudo docker exec -it jellyfinpi cat /etc/group |grep abc
users:x:100:abc
abc:x:1000:

pi@raspberrypi:~ $ sudo docker exec -it jellyfinpi ls -l /dev/vchiq
crw-rw---- 1 root video 243, 0 Jan  7 06:32 /dev/vchiq

Confirmed that ldconfig was the issue. After running that command, I could use HWA as root.

thelamer · 2020-01-07T17:48:33Z

So the only thing left is to determine how I can test for that vchiq device effectively -f and -e are false, can you run this script on your host? :

#! /bin/bash
DEVICE=/dev/vchiq
for type in -b -c -d -e -f -G -h -g -k -L -O -p -r -S -s -u -w -x; do
  if test ${type} "$DEVICE"; then
    echo "$DEVICE exists using ${type}"
  fi
done

This will let me determine the test operator to use to find the existence of this device and add abc to it's group.
Also we should be able to remove the need to mount /etc/ld.so.conf.d/00-vmcs.conf:/etc/ld.so.conf.d/00-vmcs.conf I can just test for the existence of /opt/vc/lib and load libs from it.

Artiume · 2020-01-10T01:59:55Z

pi@raspberrypi:~/bin $ ./test
/dev/vchiq exists using -c
/dev/vchiq exists using -e
/dev/vchiq exists using -r
/dev/vchiq exists using -w

thelamer · 2020-01-10T02:19:52Z

Thank you sir, will have something to test in the next two hours

thelamer · 2020-01-10T03:40:53Z

@Artiume please destroy/stop your existing container and run just this compose with your PUID and GUID swapped out for what has access to your /media stuff:

version: "3.7"
services:
  jellyfin:
    image: lsiodev/jellyfin:v10.4.3-pkg-5749f6c2-dev-0b4c6d34c0d10a8bb12e967b2a7ea23ee433b4bf
    devices:
      - /dev/vchiq:/dev/vchiq
    network_mode: host
    environment:
      PUID: 1000
      PGID: 1000
    volumes:
      - /media:/media
      - /opt/vc/lib:/opt/vc/lib

This is an ephemeral test, and does not mount up anything that is not needed.

Artiume · 2020-01-10T03:55:23Z

success :)

for my own learning, what all did you do? I'm also not quite sure what the test operators were for /dev/vchiq

thelamer · 2020-01-10T03:56:51Z

All the logic is here:
https://github.com/linuxserver/docker-jellyfin/blob/arm-hardware/root/etc/cont-init.d/40-gid-video
We already had stuff to add abc to video just needed the lib loading.

I'll PR this into the main image now that you have confirmed it works.

Artiume mentioned this issue Jan 1, 2020

Update to HW Accel to show how to pass devices and update Rpi4 Information jellyfin-archive/jellyfin-docs#183

Merged

thelamer closed this as completed Jan 10, 2020

thelamer mentioned this issue Jan 10, 2020

Arm hardware OpenMAX docs and logic #15

Merged

Artiume mentioned this issue Mar 3, 2020

Fix Docker packages jellyfin/jellyfin#2499

Merged

whiteowl3 mentioned this issue Jun 17, 2021

HowTo: OMX HWA in docker on RPI4 Ubuntu Server 64 bit Host jellyfin/jellyfin#6195

Closed

whiteowl3 mentioned this issue Jun 17, 2021

Jellyfin OMX HWA in docker on RPI4 Ubuntu Server 64 bit Host #114

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RPI4 Hardware Acceleration Fix #14

RPI4 Hardware Acceleration Fix #14

Artiume commented Jan 1, 2020 •

edited

Loading

thelamer commented Jan 3, 2020

Artiume commented Jan 3, 2020

thelamer commented Jan 3, 2020

Artiume commented Jan 3, 2020

thelamer commented Jan 3, 2020 •

edited

Loading

Artiume commented Jan 3, 2020

thelamer commented Jan 3, 2020

Artiume commented Jan 3, 2020

thelamer commented Jan 4, 2020

Artiume commented Jan 4, 2020 •

edited

Loading

Artiume commented Jan 4, 2020 •

edited

Loading

thelamer commented Jan 4, 2020

Artiume commented Jan 4, 2020

thelamer commented Jan 5, 2020

Artiume commented Jan 5, 2020

thelamer commented Jan 6, 2020

Artiume commented Jan 7, 2020

thelamer commented Jan 7, 2020

Artiume commented Jan 10, 2020

thelamer commented Jan 10, 2020

thelamer commented Jan 10, 2020

Artiume commented Jan 10, 2020

thelamer commented Jan 10, 2020

RPI4 Hardware Acceleration Fix #14

RPI4 Hardware Acceleration Fix #14

Comments

Artiume commented Jan 1, 2020 • edited Loading

Expected Behavior

Current Behavior

Steps to Reproduce

Environment

Command used to create docker container (run/create/compose/screenshot)

Docker logs

thelamer commented Jan 3, 2020

Artiume commented Jan 3, 2020

thelamer commented Jan 3, 2020

Artiume commented Jan 3, 2020

thelamer commented Jan 3, 2020 • edited Loading

Artiume commented Jan 3, 2020

thelamer commented Jan 3, 2020

Artiume commented Jan 3, 2020

thelamer commented Jan 4, 2020

Artiume commented Jan 4, 2020 • edited Loading

Artiume commented Jan 4, 2020 • edited Loading

thelamer commented Jan 4, 2020

Artiume commented Jan 4, 2020

thelamer commented Jan 5, 2020

Artiume commented Jan 5, 2020

thelamer commented Jan 6, 2020

Artiume commented Jan 7, 2020

thelamer commented Jan 7, 2020

Artiume commented Jan 10, 2020

thelamer commented Jan 10, 2020

thelamer commented Jan 10, 2020

Artiume commented Jan 10, 2020

thelamer commented Jan 10, 2020

Artiume commented Jan 1, 2020 •

edited

Loading

thelamer commented Jan 3, 2020 •

edited

Loading

Artiume commented Jan 4, 2020 •

edited

Loading

Artiume commented Jan 4, 2020 •

edited

Loading