Update Rockchip presets

[MarcA711]

Hey,

In #768 I requested the implementation of presets for FFmpeg with Rockchip HW acceleration. These presets need to be updated, because the FFmpeg fork that most users used at this time was deprecated in favor of a new fork.
Now, most Rockchip users use this fork from nyanmisaka. Here is the documentation on how to use the fork and here is a static version of this fork that I compiled.

In short, these presets allow transcoding and resizing a video using hardware:

// transcode to h264
ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -i input.mp4 -vf scale_rkrga=w={XXX}:h={XXX}:force_original_aspect_ratio=0 -c:v h264_rkmpp -g 50 -bf 0 -profile:v high -level:v 4.1 output.mp4
// transcode to hevc
ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -i input.mp4 -vf scale_rkrga=w={XXX}:h={XXX}:force_original_aspect_ratio=0 -c:v hevc_rkmpp -g 50 -bf 0 -profile:v main -level:v 5.1 output.mp4

I would really appreciate if you could update the presets and I can help testing.

[nyanmisaka]

go2rtc/internal/ffmpeg/ffmpeg_test.go

Line 163 in 5fa31fe

require.Equal(t, `ffmpeg -hide_banner -fflags nobuffer -flags low_delay -i http://example.com -c:v h264_rkmpp_encoder -g 50 -bf 0 -profile:v high -level:v 4.1 -an -vf "transpose=1,transpose=1" -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}`, args.String())

FYI there's also a HW accelerated transpose/crop filter vpp_rkrga. The usage is similar to Intel QSV's vpp_qsv filter.

[MarcA711]

Hey @AlexxIT , I just wanted to ask if the presets will be updated soon? It would be very nice to see them in the next version of Frigate. However, don't feel pressured. If you don't have time right now, that's fine too!
In any case, thanks for your help.

[AlexxIT]

Hi. It's a big task for me. I need to replicate the entire environment so I don't have to make edits blindly.
The task is in the plans, but I'm doing other tasks in the meanwhile
https://github.com/AlexxIT/go2rtc/issues/assigned/AlexxIT

[MarcA711]

Thank you for the clarification! Let me know, if I can help you somehow.

[bugui2007]

Hi guys,
I'm using frigate 14 beta2 and I noticed that the predefined functions like #rotate=90 and #audio=xxx were not working on my rockchip 3588 (it's an Orange PI 5 plus).

Is there any relationship between audio transcoding and this issue here?

Thanks in advance,
Thank you so much.
PS: Congratulations on this great project. Go2RTC is the perfect knife for streaming video!

[MarcA711]

AFAIK, these options shouldn't cause any issues if you don't use #hardware.

[AlexxIT]

I don't have problems with transcoding. But have problems with scale on my Rockchip.
Maybe because not best linux from LubanCat developers...

Maybe someone can help me with downloading latest version. Because I don't have Baidu account.
I have Thinkcore Pi TP-0W / LubanCat Zero W / RK3566.

Baidu Cloud Link: https://pan.baidu.com/s/19t8AZV9SYTdjn2uObBiSGA Extraction code: hslu
3-Linux > rk356x > Ubuntu > Universal > 22.04 > 20240320 > rk3566 > ubuntu22.04-lite

cat@lubancat:~$ ./ffmpeg -hide_banner -hwaccel rkmpp -hwaccel_output_format drm_prime -re -i bbb5.mp4 -c:v h264_rkmpp -g 50 -bf 0 -profile:v high -level:v 4.1 -an -vf "vpp_rkrga=640:-1" -y out.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb5.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.16.100
  Duration: 00:00:05.02, start: 0.000000, bitrate: 2606 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 2481 kb/s, 24 fps, 24 tbr, 24k tbn (default)
    Metadata:
      handler_name    : (C) 2007 Google Inc. v08.13.2007.
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      handler_name    : (C) 2007 Google Inc. v08.13.2007.
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_rkmpp) -> h264 (h264_rkmpp))
Press [q] to stop, [?] for help
librga fail to get driver version! Compatibility mode will be enabled.

65 im2d_rga_impl rga_version_below_minimun_range_user_driver(322): The driver may be compatible, but it is best to update the driver to version 1.2.4. You can try to update the SDK or update the <SDK>/kernel/drivers/video/rockchip/rga3 directory individually. current version: librga 1.10.0, driver .
rga_api version 1.10.0_[8]
[Parsed_vpp_rkrga_0 @ 0x558edbb130] RGA async blit returned invalid fence_fd: 0
[vf#0:0 @ 0x558ee1c650] Error while filtering: Generic error in an external library
Failed to inject frame into filter network: Generic error in an external library
Error while filtering: Generic error in an external library
[out#0/mp4 @ 0x558edcafa0] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=N/A bitrate=N/A speed=N/A    
Conversion failed!

[MarcA711]

Sorry, I can't help with downloading the newer image.

Maybe sudo helps, if your issue is due to insufficient permission. But it seems your issue is related to outdated drivers.

However, I can confirm that the above command works.

[AlexxIT]

LubanCat also build ffmpeg for their boards. Maybe something useful there:
https://github.com/LubanCat/buildroot/tree/rk356x/package/ffmpeg

[MarcA711]

This is pretty old and based on the original FFmpeg fork from rockchip, that was discontinued due to license issues. I don't think it works on up to date systems.

Currently, most users use this fork from nyanmisaka: https://github.com/nyanmisaka/ffmpeg-rockchip
It works best in my opinion and is actively meintained. So go2rtc should implement presets for this fork.

[nyanmisaka]

You can try commenting out these lines in FFmpeg to avoid the check for a valid value for the kernel returned fence_fd. It might work, but ultimately you're better off updating your RGA kernel drivers to the latest and enabling Multi-RGA and Async-RGA in defconfig, which will help performance a lot.

https://github.com/nyanmisaka/ffmpeg-rockchip/blob/9efe5bcff082d6538702d01c9b7126f40da27361/libavfilter/rkrga_common.c#L1109-L1114

[AlexxIT]

Thanks. Building custom FFmpeg and updating RGA kernel drivers are too complicated things for me :)

I can run some docker container if there are pre-made ones with the right contents.
Or I'll wait for someone to help me download the latest Linux from Baidu for my test board.

[AlexxIT]

I have tested latest Ubuntu from LubanCat developers. Same error.
Also I can't run docker on this board. Don't know why.
Just a waste of money...

[AlexxIT]

@MarcA711 what about Frigate's docker container? If I can run it on this board, then theoretically it should have correct version of library?

[MarcA711]

I don't think it will work. Your problem is outdated kernel drivers and docker containers use the same kernel as the host.

Is there a lot that has to be done to update the presets? Maybe I am able to implement the presets if you point me to the right files. It should be similar to nvidia for example. However, I don't know Go.

Alternatively, maybe I could give you access to a docker container on my board using https://tmate.io/. Do you need docker for your tests or do you just want to run ffmpeg commands?

[AlexxIT]

go2rtc doesn't just use presets. It rewrites the full ffmpeg command from firmware to hardware version. By modifying inputs and filters. Nvidia not best example. VAAPI has better API.

https://github.com/AlexxIT/go2rtc/blob/master/internal/ffmpeg/hardware/hardware.go

[MarcA711]

Thank you, I will have a look. Maybe I find out how to do it.

And for your board, you can maybe ask here for an image: https://github.com/Joshua-Riek/ubuntu-rockchip
He already added support for other lubancat boards and it seems to be now big deal: Joshua-Riek/ubuntu-rockchip#327. With an up to date base os, this will be a powerful board I guess.

[AlexxIT]

Thanks. I think RockChip support is really important in go2rtc. I bought this board specifically for that purpose. It was pretty cheap. I didn't think there would be so many problems with it.

[MarcA711]

Yes, I think the rockchip platform is important. Rockchip is perhaps the most important platform after Raspberry Pi.

I was able to setup a dev environment on my board and hope to be able to implement the presets myself.

[AlexxIT]

It's not necessary to run go2rtc or dev on board. You can just fix code on any PC, run tests, and check output string after parseArgs on board's console. But change input and output to files.

go2rtc/internal/ffmpeg/ffmpeg_test.go

Lines 195 to 205 in ba34855

	func TestParseArgsHwRKMPP(t *testing.T) {
	// [HTTP-MJPEG] video will be transcoded to H264
	args := parseArgs("http://example.com#video=h264#hardware=rkmpp")
	require.Equal(t, `ffmpeg -hide_banner -fflags nobuffer -flags low_delay -i http://example.com -c:v h264_rkmpp_encoder -g 50 -bf 0 -profile:v high -level:v 4.1 -an -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}`, args.String())
	args = parseArgs("http://example.com#video=h264#rotate=180#hardware=rkmpp")
	require.Equal(t, `ffmpeg -hide_banner -fflags nobuffer -flags low_delay -i http://example.com -c:v h264_rkmpp_encoder -g 50 -bf 0 -profile:v high -level:v 4.1 -an -vf "transpose=1,transpose=1" -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}`, args.String())
	args = parseArgs("http://example.com#video=h264#height=320#hardware=rkmpp")
	require.Equal(t, `ffmpeg -hide_banner -fflags nobuffer -flags low_delay -i http://example.com -c:v h264_rkmpp_encoder -g 50 -bf 0 -profile:v high -level:v 4.1 -height 320 -an -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}`, args.String())
	}

[MarcA711]

This helps a lot. Thank you!