-
Notifications
You must be signed in to change notification settings - Fork 17
io modes
As of commit https://github.com/InES-HPMM/linux-l4t/commit/75f0f45e02cffb6e02b7e6fcf1f2757a9b0cafa2
The v4l2src GStreamer plugin offers different methods for pushing buffers through a pipeline. A user can choose the mode by setting the io-mode property. The variants are (using Gstreamer 1.8.0):
- 1 - Read/Write: Copy buffers when passing between elements
- 2 - MMAP: Memory mapping, allocated by Kernel
- 3 - Userptr: User Memory, allocated by user space application
- 4 - Dmabuf: Use Buffers of a Hardware DMA
- 5 - Dmabuf Import: ? (new in Gst 1.8.0)
To improve execution speed of a Video Processing pipeline, it is desired to reduce the copying of buffers to a minimum, since this is an expensive CPU operation. The most difficult operation in this regard is passing buffers to a hardware-accelerated block (e.g. scaler) and getting them back into the pipeline. Presentation [2] shows that when using MMAP, there is a copy operation required when passing the buffers and one more copy when getting them back. Userptr is an improvement, since a copy is only required for getting the buffers back. Using Dmabuf is described as the optimal solution, since both passing and getting buffers back can be done without the need to copy.
The current state of L4T R24.1 and the drivers allows the following scenarios:
(Default is GSt 1.2.4. But in situations with (*) GSt 1.8.0 was used.)
| R/W | MMAP | Userptr | Dmabuf | |
|---|---|---|---|---|
| (GSt 1.2.4) | (GSt 1.2.4) | (GSt 1.8.0) | (GSt 1.8.0) | |
| xvimagesink | 30fps 100%CPU | 20fps 70%CPU | (*) 30fps 30%CPU | (*) 20 or 30fps 100%CPU some missed buffers |
| videoconvert & nvoverlaysink | Not supported | 6 fps 180%CPU | (*) 30fps 50%CPU needs additional Patch, see below | (*) 6fps 180%CPU |
| nvvidconv & nvoverlaysink (with NVMM) | 26 fps 80%CPU | 30 fps 80%CPU | Error | Error |
| nvvidconv(NVMM) & nvvidconv & xvimagesink | 18fps 85%CPU | 20fps 80%CPU | Error | Error |
(*) In order to use these modes, this patch needs to be applied to GStreamer Plugins Good (1.8.0): changes to GStreamer
Pipelines:
- Userptr xvimagesink
gst-launch-1.0 v4l2src io-mode=3 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=3840, height=2160, framerate=30/1, format=UYVY' ! xvimagesink- MMAP hardware-accelerated
gst-launch-1.0 v4l2src io-mode=2 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=3840, height=2160, framerate=30/1, format=UYVY' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=3840, height=2160, framerate=30/1, format=I420' ! nvoverlaysink sync=false- MMAP hardware scale workaround & xvimagesink
gst-launch-1.0 v4l2src io-mode=2 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=3840, height=2160, framerate=30/1, format=UYVY' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=3840, height=2160, framerate=30/1, format=I420' ! nvvidconv ! 'video/x-raw, width=3840, height=2160, framerate=30/1, format=I420' ! xvimagesink sync=falseIn order to use the videoconvert element with the Userptr mode on the TX1, the memory alignment needs to be adapted to 64 Bytes. The following patch can be applied to GStreamer-Plugins-Base (1.8.0):
--- gstvideofilter_ORIG.c 2016-06-09 08:03:54.403918724 +0000
+++ ./gst-plugins-base-1.8.0/gst-libs/gst/video/gstvideofilter.c 2016-06-09 07:47:24.284560434 +0000
@@ -82,7 +82,8 @@
if (gst_query_get_n_allocation_pools (query) == 0) {
GstStructure *structure;
GstAllocator *allocator = NULL;
- GstAllocationParams params = { 0, 15, 0, 0, };
+// GstAllocationParams params = { 0, 15, 0, 0, };
+ GstAllocationParams params = { 0, 63, 0, 0, };
if (gst_query_get_n_allocation_params (query) > 0)
gst_query_parse_nth_allocation_param (query, 0, &allocator, ¶ms);With this patch, we can use videoconvert as follows and get 30 FPS at 2160p resolution:
# videoconvert -> xvimagesink
gst-launch-1.0 v4l2src io-mode=3 ! 'video/x-raw, width=3840, height=2160, framerate=30/1, format=UYVY' ! videoconvert ! 'video/x-raw, width=3840, height=2160, framerate=30/1, format=I420' ! xvimagesink sync=false -vvv
# videoconvert -> nvoverlaysink
gst-launch-1.0 v4l2src io-mode=3 ! 'video/x-raw, width=3840, height=2160, framerate=30/1, format=UYVY' ! videoconvert ! 'video/x-raw, width=3840, height=2160, framerate=30/1, format=I420' ! nvoverlaysink sync=false -vvv