Tighter Shadow Caster Culling causes some object shadows to not render for a Frame

[mrjustaguy]

Tested versions

Reproducable in 4.3 dev 5, or any build prior that has #84745

System information

Godot v4.3.dev (0728468bd) - Windows 10.0.22621 - Vulkan (Forward+) - dedicated NVIDIA GeForce GTX 1050 Ti (NVIDIA; 31.0.15.5176) - Intel(R) Core(TM) i3-10105F CPU @ 3.70GHz (8 Threads)

Issue description

There seems to be an edge case where Tighter Shadow Caster Culling is inaccurately culling some shadow casters away when the lights initially move, but the next frame when they stop moving the missing casters get rendered again.

The issue doesn't seem to come from the fact they're Skinned Meshes, as replacing them with a box of the same AABBs also suffers from this problem.

This happens with both Omni Lights and Spot Lights (replace OmniLight3D with a Spot Light and rotate it -90 degrees to light the floor, it'll still flicker the shadow casters)

Steps to reproduce

Open MRP, and run it, press Home a few times to rotate the lights an increment at a time until some dummies (Female 26 and Light 1 for me) start displaying no shadows for a frame after rotation. Use End to rotate in the opposite direction, to allow for retriggering.

Disable Tighter Shadow Caster Culling and repeat, observe no issues.

Minimal reproduction project (MRP)

Shadows.zip

[lawnjelly]

mrjustguy_tighterculling_bug.zip

Here's a simpler MRP. Will examine in more detail today.

The flicker for one frame is likely due to some logic the culling does to juggle between partial update (tighter culling) and full update (regular culling). My best guess so far is it disappears on the partial cull and reappears on the regular cull. But debugging should reveal what is happening.

[lawnjelly]

Pinning it down. The logic between partial update and regular seems fine, it is giving rise to the flicker, but the reason is that the cull on the partial update is incorrectly culling the box.

Am running the numbers now to find out why this is the case.

UPDATE:
I think I'm getting there. It's been tough to debug.

It seems to be a float precision problem at the exact point the light is close to the frustum plane (possibly frustum edge, the 3 point plane equation is subject to error when close to colinear), the normal of the extra culling plane created between the light and the points of the view frustum is subject to considerable floating point error, and can lead to a large change in the estimate of the distance between the cube and the culling plane, resulting in it being incorrectly culled.

I suspect the answer may be to disable these extra culling planes if the light is in this situation, or perform some bodging to work around it. If this is correct, it is not surprising it hasn't been reported earlier as it would seem to only occur in this rare configuration.

UPDATE:
Can confirm the problem is occurring when the light to frustum edge is near colinear. This explains the dodgy normal.

[mrjustaguy]

I can confirm it seems to be a precision issue, increasing the cull margin reduces the flickering, until you get to a large enough margin for it to go away completely (from 0.1m to 1m)