Add an option to jitter the shadow dithering pattern every frame

[Calinou]

• Related to Implement temporal antialiasing (TAA) in the Vulkan renderer #3401.

Describe the project you are working on

The Godot editor 🙂

Describe the problem or limitation you are having in your project

Shadow mapping in Godot 4.0 exhibits a noticeable dithering pattern at lower resolutions, especially when a light's Shadow Blur property is increased above its default value or when PCSS shadows are used (when a light's Size is set above 0).

This can be alleviated by increasing the shadow filter quality, but it has a significant cost.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Add an option to jitter the shadow dithering pattern every frame.

We can also exploit the fact that monitors do not have instant response times, and use this to let noise smooth itself out even when TAA is disabled. This is best done on high refresh rate monitors with high enough frame rates (80 FPS or more).

In addition, jittering the shadow pattern also avoids the feeling of having a pattern that "follows" the camera, which is distracting in motion.

Both examples below have the exact same performance. However, the bottom examples look markedly smoother:

Without shadow jitter

Click to view at full size. Images have been scaled by a factor of 4× with nearest-neighbor filtering.

Standard DirectionalLight shadows	PCSS OmniLight shadows

With shadow jitter (accumulation over 3 frames simulated)

This is not what you get when taking a screenshot, but the non-instant response time of a monitor will effectively get you something like this on your screen in motion.

Standard DirectionalLight shadows	PCSS OmniLight shadows

In the future, this jitter option could be extended to various post-processing effects. SSAO, SSR and SSIL's perceptual quality should all benefit from jittering. Depth of field already has a jitter option available.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

To prepare for the introduction of temporal antialiasing, I recommend adding an enum project setting rendering/shadows/use_jitter:

• Auto (default). Jitter the shadow pattern when TAA is enabled, but don't jitter it when TAA is disabled.
• Enabled. Always jitter the shadow pattern, even when TAA is disabled. This can be used on high refresh rate monitors to smooth out shadow rendering without requiring the use of TAA, as long as a high enough framerate is maintained.
• Disabled. Never jitter the shadow pattern, even when TAA is enabled. This can be used when you want to use TAA, but optimal image stability is desired.

I have a WIP implementation of this feature here: https://github.com/Calinou/godot/tree/shadow-filter-jitter
The randomness distribution could be improved – I'm just offsetting the pattern by a fixed amount every frame right now.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, as shadow rendering is performed in core.

Is there a reason why this should be core and not an add-on in the asset library?

See above.

[Mickeon]

I like it. Feels very old school, being a way to take advantage of an external limitation to save on processing power.

Speaking of which, even at 60 FPS, in your average environment, the jitter is likely going to be barely noticeable to the human eye.

Is there any commercial game out there that uses a similar technique to what's being described?

[Calinou]

Is there any commercial game out there that uses a similar technique to what's being described?

Most AAA games that use TAA nowadays use jittering to improve screen-space effect quality. I'm not aware of this being done without TAA though, but it's certainly worth trying.

Some old games/consoles make use of temporal dithering to hide color banding (back when most games rendered with 16 bpp colors). You can see this in action in a lot of Nintendo 64 games, especially when approaching alpha-blended surfaces.

[mrjustaguy]

Do note that FXAA slightly helps with the shadow quality too, it's quite possible that FXAA+Jitter would result in an even more significant improvement visually for fairly cheap. MSAA ofc doesn't do this.

[Saul2022]

Maybe this article can help on fickering https://www.alexandre-pestana.com/shadows-using-dithering-and-temporal-supersampling/

[Lasuch69]

It would be awesome to include a shader uniform with current direction of jitter to use for dithering and other use cases.

With jitter (custom solution not optimal for real-world scenario):

Without jitter:

[Calinou]

It would be awesome to include a shader uniform with current direction of jitter to use for dithering and other use cases.

You can pass Engine.get_frames_drawn() % 16 as an int uniform and use it in your custom dithering jitter.

[mrjustaguy]

This is only accurate for TAA, FSR2 calculates it's own based on resolution..

[Calinou]

This is only accurate for TAA, FSR2 calculates it's own based on resolution..

A method to return the number of frames required to fully accumulate the current TAA method could be added, but I think you'll still have to modulate it against Engine.get_frames_drawn() and pass it as a uniform manually. Adding a new global variable for shaders has a cost even if you don't use the feature, so it needs to be considered carefully. (To avoid breaking shaders when TAA is disabled, it would still need to be present, even if it always returns 0.)