better x264 tuning for type of content

[totaam]

Related to #1107, #1232 and #800.

We (almost) already know if the content we are encoding using the video encoders is really real video or just fast moving.
This is what we use to enable b-frames (#800), we should also use this to tune x264 better.

[totaam]

• r13020: use the simplest motion search with non-video content (DIA - diagonal)
• r13021 + r13022: make it possible to use different x264 "tune" option

For details on tune, see: Explanation of x264 tune.

ie:

XPRA_B_FRAMES=0 \
XPRA_FORCE_CSC_MODE=YUV420P \
XPRA_X264_PROFILE=high10 \
XPRA_X264_TUNE=film \
    xpra --bind-tcp=0.0.0.0:10000 start --speed=100 -d x264

Will use YUV420P csc mode, high10 profile, film tune with ultrafast preset (speed).
In this case, using "film" without b-frames is not particularly useful!

The main question here is to figure out what is the best setting to use, both for video and non-video content.
There are going to be many quality vs cpu vs bandwidth vs latency trade offs to choose from, so maybe using x264 directly from the command line would be quicker.

[totaam]

As of r13051, we are also more likely to use colour subsampling for video regions, and less likely to use it for other areas (the rest of the window).

[totaam]

As of r13052, we will use "film" tuning (intended for high-bitrate/high-quality movie content. Lower deblocking is used here) for video regions, and "zerolatency" (optimization for fast encoding and low latency streaming) for everything else.

Here's the "-d x264" output for a video region:

using default profile=high444
x264 context=0x7fabf40497c0,    BGRX  300x300  preset=faster, profile=high444, tune=film
 me=HEX, me_range=16, mv_range=-1, opencl=False, b-frames=1, max delayed frames=1
 vfr-input=0, lookahead=0, sync-lookahead=-1, mb-tree=1, bframe-adaptive=FAST
 threads=auto, sliced-threads=True
x264 encode 1 delayed frames after 0
x264 encode frame 0 as  IDR slice with 7 nals, total   15737 bytes, keyframe=True , delayed=1
x264 encode frame 1 as    P slice with 4 nals, total   12211 bytes, keyframe=False, delayed=1
x264 encode frame 2 as    B slice with 4 nals, total    7849 bytes, keyframe=False, delayed=1
x264 encode frame 3 as    P slice with 4 nals, total   12390 bytes, keyframe=False, delayed=1
x264 encode frame 4 as    B slice with 4 nals, total    7695 bytes, keyframe=False, delayed=1

And for non-video (no b-frames):

using default profile=high444
x264 context=0x7efc480151c0,    BGRX  498x316  quality=99, speed=74, source=unknown
 preset=faster, profile=high444, tune=zerolatency
 me=DIA, me_range=16, mv_range=-1, opencl=False, b-frames=0, max delayed frames=0
 vfr-input=0, lookahead=-1, sync-lookahead=0, mb-tree=0, bframe-adaptive=FAST
 threads=auto, sliced-threads=True
x264 encode frame     0 as  IDR slice with 7 nals, total  124041 bytes, keyframe=True , delayed=0
x264 encode frame     1 as    P slice with 4 nals, total     154 bytes, keyframe=False, delayed=0
x264 encode frame     2 as    P slice with 4 nals, total     278 bytes, keyframe=False, delayed=0
x264 encode frame     3 as    P slice with 4 nals, total     109 bytes, keyframe=False, delayed=0

This should be tested together with #800 as there is a lot of overlap.

[totaam]

r13056 allows us to detect the whole window as a video region, so that we will correctly use colour subsampling and more aggressive compression settings for full screen video (or anything full screen with high fps, like glxgears).

If this makes us wrongly detect the whole window as video area, we may need to tweak the video region detection code #967 (#410 - see also #1135, #1060)

r13058 also exposes the tune option via xpra info:

$ xpra info | grep tune
window.2.encoder.tune=film

[totaam]

Ready for testing, including b-frames from #800.

We want to:

• verify that when we have a video region, we use "film" tuning and enable b-frames, non-video areas that still use h264 should still be using "zerolatency" (those should be rare)
• ideally, measure the bandwidth savings and cpu costs of those new settings - do we need to adjust the speed (or even quality) setting to compensate?
• verify that this doesn't negatively impact av-sync (synchronize sound with video frames #835 av-sync finer tuning #1164)

Note: if we end up with a blurry window and delayed paints because the whole window is mistakenly identified as a video region then this needs to go in #967, not here.

[totaam]

2016-07-25 18:32:10: maxmylyn commented

---

Using a Fedora 23 trunk r13086 built from source server and client:

• When watching a YouTube video, or something with VLC, I verified that the film tuning is being used. However, typing quickly into a text box (like Trac tickets...) does trigger an h264 region, also using film, not zerolatency.

• AV Sync does not appear to be affected. It's still very good - under 20ms off from what I can tell.

Note: if we end up with a blurry window and delayed paints because the whole window is mistakenly identified as a video region then this needs to go in #967, not here.

I do see this, will leave a comment in #967.

---

ideally, measure the bandwidth savings and cpu costs of those new settings - do we need to adjust the speed (or even quality) setting to compensate?

I'll talk about this with Smo and see what he thinks. Last I heard he was still setting up his testing environment. We also got a new hire and she should be able to do this once she's all set up.

[totaam]

However, typing quickly into a text box (like Trac tickets...) does trigger an h264 region, also using film, not zerolatency.

Anything that is mistakenly detected as a video region will end up using film.
You must have had a video region detected at that point, you can verify with xpra info | grep region.
So unless this is not the case, it does not belong here (x264 tuning) but in #967 (video region detection).

---

Last I heard he was still setting up his testing environment.

Note: you will need to run one or more sets of tests with extra environment variables to compare with and without b-frames and tuning.

[totaam]

r13436 further lowers the quality of "true" video (more than 10 frames per second, and at least 20 frames have been processed using the given video region)
As of r13439, we also only enable "b-frames" and the "video" encoder tuning for "true" video.
See also #1232: scrolling should now be detected and compressed better. There is now more information in

xpra info | grep subregion
xpra info | grep -e "window\.[0-9]*\.encoder.*"

[totaam]

See #1232 comment:16 which now exposes the "video-mode" flag for "true" video. (>10fps, etc..)

As of r13775, we can switch to and from "true" video on the fly, see #800#comment:25

[totaam]

2016-10-28 19:59:33: maxmylyn commented

---

Alright, so I think it's time to make a decision on this one.

It looks like we (afarr and myself) aren't gonna get someone who can write the tests, and aren't going to be given the time to write them ourselves.

So, @antoine - what would you like to do with this ticket? The x264 tuning works well in my experience, and I haven't had any issues with it. We can hand it to Smo, in the hopes that he will have time to include it in his tests? Probably not the best idea as he's already busy.

[totaam]

Closing and hoping for the best :/