Restore and fix post-process shadow blurring in 3D

[Calinou]

When uncommented, this function resulted in the dithering pattern being less noticeable. However, it also introduced jaggies in distant shadows, which are especially jarring in motion. Its performance cost was fairly low, but it wasn't worth the tradeoff.

The filter was quite effective for blurry shadows that are up close, so maybe it could be resurrected in the future based on shadow texel density relative to the camera.

In the future, temporal antialiasing will be added to optionally hide the dithering pattern by changing it every frame (letting the accumulation buffer do its job of averaging it all). On high refresh rate monitors, the shadow filtering pattern can be jittered to let the monitor's response time perform this averaging (without relying on TAA).

Preview

To see the negative impact of the filter on distant shadows, look at the top of the first screenshot, and compare it with the Filtering uncommented screenshot.

Current shadow rendering

Filtering uncommented

[clayjohn]

I think we can actually fix this by using dFdxFine and dFdyFine and clamp the result so it doesn't exceed 1.0
I tested the below code and it removes the artifacts for me from the previous implementation on Intel and AMD. It would be great if you can test on NVidia. Maybe we can fix this instead of removing it!

	interp_shadow -= dFdxFine(interp_shadow) * (float(fc2.x & 1) - 0.5);
	interp_shadow -= dFdyFine(interp_shadow) * (float(fc2.y & 1) - 0.5);
	interp_shadow = min(interp_shadow, 1.0);

You'll also need to change the call to blur_shadow in the directional light code to shadow=blur_shadow()

Edit: Here is what the effect looks like when working properly

Before

After

Before

After

[Calinou]

Thanks for the suggestion! I added you as a co-author of the commit 🙂

I've applied your suggestion (and pushed a new commit overriding the old one), but the blurring still seems to look bad. It almost looks like shadow maps are rendered at half resoution in the viewport, which results in distant shadows looking pixelated.

Can you test the project below on your GPU?

Testing project: test_shadow_blur_pass.zip

1024×600

Before	After (this PR)

2560×1440

Before	After (this PR)

[clayjohn]

What is happening is the blurring applies only within screen aligned 2x2 quads. So in distant areas (where the penumbra is less than 2 pixels wide) it is creating aliasing as the shadow is much higher frequency than 2x2.

[Calinou]

What is happening is the blurring applies only within screen aligned 2x2 quads. So in distant areas (where the penumbra is less than 2 pixels wide) it is creating aliasing as the shadow is much higher frequency than 2x2.

Could we detect the penumbra's length or use a distance-based metric to avoid this?

[clayjohn]

What is happening is the blurring applies only within screen aligned 2x2 quads. So in distant areas (where the penumbra is less than 2 pixels wide) it is creating aliasing as the shadow is much higher frequency than 2x2.

Could we detect the penumbra's length or use a distance-based metric to avoid this?

Yep! We could probably reuse ddx and ddy (based on vertex position) which are already calculated. I'm not sure what the performance implication would be though.

[mrjustaguy]

I've got a dumb question... is this post proc done in screen space or what?
I mean if it is, wouldn't it maybe be worth seeing if Gaussian Blur has good results?

[clayjohn]

I've got a dumb question... is this post proc done in screen space or what? I mean if it is, wouldn't it maybe be worth seeing if Gaussian Blur has good results?

It's not actually done as a post process. It's done at render time. And it only works within each 2x2 fragment quad, so guassian blur isn't an option

[Calinou]

Note that godotengine/godot-proposals#7178 may be worth considering as an alternative, although it aims to resolve a more specific issue (aliasing due to shadow texel density being too high at certain spots of the image).

[mrjustaguy]

Well the Post-processing also reduces the graininess a little, which that wouldn't.

Though what was suggested in the proposal kinda hit a dead end, as the simple angle based blur increase causes it's own issues, and I have no ideas how exactly I'd go about creating my own Anisotropic Filtering shader to use for Shadows or how to Apply Hardware Filtering to it properly (I know Anisotropic Filtering works just fine without mipmaps, I tested, but unsure about how it'd respond to eh, 4 maps in 1 texture even if I figured out how to apply it...)