Add stencil support to spatial materials #80710

apples · 2023-08-17T10:48:14Z

Adds stencil buffer support to spatial materials.

Linked Issues

Supersedes #78542

Implements the proposed design from godotengine/godot-proposals#7174

Currently integrates #73527

Testing with the sample project https://github.com/apples/godot-stencil-demo

Differences from the design

Added compare operator ALWAYS. The ALWAYS operator is needed for some effects.
Added outline and xray presets ahead of schedule because it was easy and made testing easier.
- The way that next_pass materials are handled is a bit strange right now, but I tried to make it as non-destructive as possible.

Currently implemented

Added stencil_mode to shaders, which works very similarly to render_mode.
- read - enables stencil comparisons.
- write - enables stencil writes on depth pass.
- write_depth_fail - enables stencil writes on depth fail.
- compare_(never|less|equal|less_or_equal|greater|not_equal|greater_or_equal|always) - sets comparison operator.
- (integer) - sets the reference value.
BaseMaterial3D stencil properties.
- stencil_mode - choose between disabled, custom, or presets.
- stencil_flags - set read/write/write_depth_fail flags.
- stencil_compare - set stencil comparison operator.
- stencil_reference - set stencil reference value.
- stencil_effect_color - used by outline and xray presets.
- stencil_outline_thickness - used by outline preset.
BaseMaterial3D stencil presets.
- STENCIL_MODE_OUTLINE - adds a next pass which uses the grow property to create an outline.
- STENCIL_MODE_XRAY - adds a next pass which uses depth_test_disabled to draw a silhouette of the object behind other geometry.

Supported Renderers

Forward+
Mobile
Compatibility

Depth Prepass

Stencil effects work well when rendered with a second opaque pass immediately following the current opaque pass. However, with depth prepass enabled, the depth information from any stencil materials is not available for screen-space effects. For instance, with SSAO enabled, opaque stencil materials have strong visual artifacts. Due to this, there might not be a good way to support opaque stencil effects when depth prepass is enabled.

GLES3

More work is needed for GLES3 support. The depth buffer attachment for the scene FBO must be changed to a depth-stencil attachment. Making this change without breaking existing projects will be difficult.

See f2ec38d for an experimental implementation, and find concerns in this comment: Add stencil support to spatial materials #80710 (comment).

Sorting Problems

Currently the biggest annoyance with sorting, is that next_pass materials aren't necessarily rendered immediately after their parent. Users have to rely entirely on render_priority to get correct sorting.

Individual stencil effects which have material passes in both the opaque and the alpha passes will have more difficulty being correctly sorted. This might make effects such as portals and impossible geometry more challenging to implement.

scene/resources/material.h

servers/rendering/renderer_rd/forward_clustered/render_forward_clustered.cpp

servers/rendering/renderer_rd/forward_clustered/scene_shader_forward_clustered.h

servers/rendering/renderer_rd/forward_mobile/scene_shader_forward_mobile.h

servers/rendering/shader_compiler.cpp

scene/resources/material.cpp

QbieShay · 2023-08-17T14:22:20Z

Woahh awesome @apples ! Thank you so much! I am busy busy until next week but I'll review it then!

servers/rendering/renderer_rd/forward_clustered/render_forward_clustered.cpp

scene/resources/material.cpp

Calinou

Tested locally (rebased on top of master ff5c884), it mostly works as expected. MSAA, TAA and FXAA correctly smooth out stencil rendering as expected too.

In the demo project, the Wind Waker Light appears much darker when using the Mobile rendering method though. I assume this is due to the use of a RGB10A2 color buffer instead of RGBA16, but maybe this is due to the 2× color multiplier that has to be applied? It doesn't affect the x-ray outline on the second screenshot further below.

Forward+	Mobile

Forward+	Mobile

I also noticed the stencil modes don't feature syntax highlighting and autocompletion in the shader editor, unlike the render modes:

Shader syntax validation also doesn't work if the stencil type has anything written after write, even if it makes no sense:

Here, I don't get a shader compilation error but I logically should.

I also suggest changing the default outline color in the material preset to be opaque black (or opaque white), so that you can see it immediately after setting it up. Right now, the default color is fully transparent.

PS: Using a Material Overlay property, it's possible to have an object that has both an outline and x-ray at the same time 🙂

Also, if you're curious about how a stencil outline compares with an inverted hull outline. Inverted hull on the left, stencil outline on the right:

Calinou · 2023-08-18T09:41:52Z

Is it possible to create stencil shadows that don't appear through walls with this PR? Logically, it seems feasible, but I've tried to modify the Wind Waker light shader to no avail.

apples · 2023-08-18T18:54:54Z

Is it possible to create stencil shadows that don't appear through walls with this PR?

@Calinou My intuition says that stencil shadows would require the write operations INCREMENT/DECREMENT to be exposed, but there might be some clever way to implement them without those operators.

Right now, since the write operator isn't exposed, it's just set to REPLACE internally.

the Wind Waker Light appears much darker when using the Mobile rendering method

I noticed this too and haven't had a chance to debug it. It's something to do with how the raw color values are being mixed (multiply), it seems to be clamping the values to 1.0 at some point. The xray/outline effects in my demo just use standard alpha blending with regular colors, so the problem doesn't affect them.

EDIT: If we want to expose those write operations for stencil shadows, we'll likely need to expose the compare/write masks as well. This can't really be done in the material properties for bloat reasons, but we could expose them in the shader stencil_mode without any trouble. (edit again: this is assuming it's necessary at all, which it might not be)

QbieShay · 2023-08-22T08:29:00Z

Tested with my project, I didn't attempt windwaker lights because I have completely forgot how the setup for that is supposed to work.

I tested my "particles passing through hole in the ground" scene and also the uotline and xray mode.

My own particles worked great! I am glad that this syntax seems indeed to be an update from the full stencil one:

Outlines&Xray: works great! The default configuration however seems to use stencil reference 0 and not work, it started working as soon as i changed it to something that isn't 0 (maybe that has to do with the mask like you were saying @apples ?)

QbieShay · 2023-08-22T08:32:19Z

RE: stencil shadows

I have not seen a huge demand amongst users for this. Since it seems to add some complexity to the API, i'd leave it for a future PR in case there's need for it. I'd specifically wait to see if anyone needs those features also for other reasons.

The internet seems to think stencil shadows are a bit outdated ad not "worth it" with current programmable pipelines?
(based on this, https://www.reddit.com/r/opengl/comments/4mjkv1/comment/d3w8m28/?utm_source=share&utm_medium=web2x&context=3 )

QbieShay

Tested on my test project for stencil, looks good!

I've noticed this PR also adds depth test operations, which I'm assuming they're needed for the stencil write if depth fail.

While exposing the whole thing just for this may be overkill, I see no harm in adding it for the users, I think they'll find ways to use it (plus i think the requested windwaker thing isn't possible without it).

Thank you Apples! My comments on default configurations can be tweaked in a later PR (and doesn't need to be you since changing defaults is something good for a first contributor as well ^^ )

Calinou · 2023-08-22T15:28:04Z

The internet seems to think stencil shadows are a bit outdated ad not "worth it" with current programmable pipelines?

This is true, but sometimes art direction prevails above all else 🙂

QbieShay · 2023-08-22T15:52:40Z

Oh. Didn't know they looked different! I really know nothing of them

Lielay9 · 2023-08-22T17:01:36Z

RE: stencil shadows

I have not seen a huge demand amongst users for this. Since it seems to add some complexity to the API, i'd leave it for a future PR in case there's need for it. I'd specifically wait to see if anyone needs those features also for other reasons.

Wasn't the original proposal #3373 made explicitly because of stencil shadows? I can attest to that at the very least one of the 40 thumbs ups was for those. If they won't be supported the proposal should probably be reopened. 👀

The internet seems to think stencil shadows are a bit outdated ad not "worth it" with current programmable pipelines? (based on this, https://www.reddit.com/r/opengl/comments/4mjkv1/comment/d3w8m28/?utm_source=share&utm_medium=web2x&context=3 )

I wouldn't put much weight to a 7 year old comment with 5 likes. As Calinou alluded to, stencil shadows have a distinct look and properties that are better fit for stylized games. I've commonly encountered them in platforming games, where the environment uses shadow maps and characters have down pointing stencil shadows. I could've swore some of the mario games worked like this but I can't recall which.

QbieShay · 2023-08-22T17:11:21Z

@Lielay9 could you test this PR, and see if it supports them? I don't understand how they work, so I can't quite test them myself.

You can go on the "checks" tab and at the right there is an "artifacts" menu drop down - you'll find a build for this PR there.

Wasn't the original proposal godotengine/godot-proposals#3373 made explicitly because of stencil shadows? I can attest to that at the very least one of the 40 thumbs ups was for those. If they won't be supported the proposal should probably be reopened. 👀

The proposal was indeed for stencil shadows specifically, but they cannot be done without stencil at all. It should be reassessed with this API.

apples · 2023-08-22T21:53:13Z

@QbieShay

The default configuration however seems to use stencil reference 0 and not work, it started working as soon as i changed it to something that isn't 0 (maybe that has to do with the mask like you were saying @apples ?)

Nothing to do with masks, it's just that the buffer is initialized to all zeros every frame, so choosing 0 as a default reference value is kind of useless. It could easily be changed to 1 as the default.

I've noticed this PR also adds depth test operations, which I'm assuming they're needed for the stencil write if depth fail.

Some stencil effects (Wind Waker lights in particular) will need the depth test operator exposed. Since #73527 seems like it's on track to be merged, I included it in this PR for testing purposes. I'm hoping that PR gets merged first so I can just rebase this one on top of it.

apples · 2023-08-22T21:53:30Z

Regarding stencil shadows:

The internet seems to think stencil shadows are a bit outdated ad not "worth it" with current programmable pipelines?

Modern shadows are typically based on shadow maps, which can be a bit fuzzy/pixelated since they are stored as textures. Stencil shadows, in comparison, use geometry-based stencil volumes instead of textures, so they have infinitely crisp edges. Some users will desire this for artistic reasons, typically because they're trying to follow the style of retro games.

If they [stencil shadows] won't be supported the proposal should probably be reopened.

I don't think that'll be necessary. This PR is a good foundation, and will be very easy to expand upon in future PRs.

Additionally, stencil shadows might require some other changes not directly related to stencils, such as exposing depth clamping, which is needed for infinite shadow volumes. Also, finding ways to generate the infinite shadow volumes in the first place and avoid culling them is outside the scope of this PR.

Rest assured: I am one of those people who wants stencil shadows. It is not something forgotten with this PR. It just might take multiple PRs to fully support them.

QbieShay · 2023-08-23T07:36:38Z

Some stencil effects (Wind Waker lights in particular) will need the depth test operator exposed. Since #73527 seems like it's on track to be merged, I included it in this PR for testing purposes. I'm hoping that PR gets merged first so I can just rebase this one on top of it.

@clayjohn ?

Lielay9 · 2023-08-23T22:55:40Z

2023-08-24.01-26-39.mp4

Well, I guess this is best you can do now. Works if volumes don't overlap; a simple torus, for example, will break it.

Calinou My intuition says that stencil shadows would require the write operations INCREMENT/DECREMENT to be exposed, but there might be some clever way to implement them without those operators.

These are indeed necessary for shadows. Besides that, the only thing missing would be culling the shadows correctly which is outside the scope of this PR. Thereafter, stencil shadows are viable, if not exactly painless to use.

Extra note: Some primitive shapes seem to have small holes/gaps and therefore didn't play too nicely with the code generating the shadow meshes.

Calinou · 2023-08-23T23:39:19Z

Extra note: Some primitive shapes seem to have small holes/gaps and therefore didn't play too nicely with the code generating the shadow meshes.

Could you post your code somewhere? I think the current primitive meshes are fine as they are, although some of them have hard seams as you'd typically expect (e.g. boxes aren't smooth-shaded).

AThousandShips

Style looks good otherwise

AThousandShips · 2023-12-17T16:02:39Z

scene/resources/material.cpp

@@ -2492,6 +2594,171 @@ BaseMaterial3D::EmissionOperator BaseMaterial3D::get_emission_operator() const {
 	return emission_op;
 }

+void BaseMaterial3D::_prepare_stencil_effect() {
+	const Ref<Material> current_next_pass = get_next_pass();


Suggested change

const Ref<Material> current_next_pass = get_next_pass();

Ref<Material> current_next_pass = get_next_pass();

While const indicates intention here it doesn't really do anything except prevent reassignment of the variable itself, any operation is still allowed (and it is treated as a non-const pointer)

AThousandShips · 2023-12-17T16:02:52Z

scene/resources/material.cpp

+}
+
+Ref<BaseMaterial3D> BaseMaterial3D::_get_stencil_next_pass() const {
+	const Ref<Material> current_next_pass = get_next_pass();


Suggested change

const Ref<Material> current_next_pass = get_next_pass();

Ref<Material> current_next_pass = get_next_pass();

AThousandShips · 2023-12-17T16:05:45Z

servers/rendering/renderer_rd/forward_clustered/scene_shader_forward_clustered.cpp

 		depth_stencil_state.enable_depth_write = depth_draw != DEPTH_DRAW_DISABLED ? true : false;
+
+		RD::CompareOperator depth_function_rd_table[DEPTH_FUNCTION_MAX] = {


Suggested change

RD::CompareOperator depth_function_rd_table[DEPTH_FUNCTION_MAX] = {

static const RD::CompareOperator depth_function_rd_table[DEPTH_FUNCTION_MAX] = {

Here as well (even though it's not done in the other cases it's cleaner IMO)

AThousandShips · 2023-12-17T16:05:56Z

servers/rendering/renderer_rd/forward_clustered/scene_shader_forward_clustered.cpp

+
+	depth_stencil_state.enable_stencil = stencil_enabled;
+	if (stencil_enabled) {
+		RD::CompareOperator stencil_compare_rd_table[STENCIL_COMPARE_MAX] = {


Suggested change

RD::CompareOperator stencil_compare_rd_table[STENCIL_COMPARE_MAX] = {

static const RD::CompareOperator stencil_compare_rd_table[STENCIL_COMPARE_MAX] = {

AThousandShips · 2023-12-17T16:06:11Z

servers/rendering/renderer_rd/forward_mobile/scene_shader_forward_mobile.cpp

 		depth_stencil_state.enable_depth_write = depth_draw != DEPTH_DRAW_DISABLED ? true : false;
+
+		RD::CompareOperator depth_function_rd_table[DEPTH_FUNCTION_MAX] = {


Suggested change

RD::CompareOperator depth_function_rd_table[DEPTH_FUNCTION_MAX] = {

static const RD::CompareOperator depth_function_rd_table[DEPTH_FUNCTION_MAX] = {

AThousandShips · 2023-12-17T16:06:18Z

servers/rendering/renderer_rd/forward_mobile/scene_shader_forward_mobile.cpp

+
+	depth_stencil_state.enable_stencil = stencil_enabled;
+	if (stencil_enabled) {
+		RD::CompareOperator stencil_compare_rd_table[STENCIL_COMPARE_MAX] = {


Suggested change

RD::CompareOperator stencil_compare_rd_table[STENCIL_COMPARE_MAX] = {

static const RD::CompareOperator stencil_compare_rd_table[STENCIL_COMPARE_MAX] = {

Alia5 · 2023-12-18T21:25:32Z

Really hoping this will be added in 4.3
Rebased this to master, dunno if I broke the FSR2 implementation in the process.
Maybe someone could rebase this for people to play in case this doesn't get merged soon-ish which I'd suspect based on #80710 (comment)

Nevertheless stencil support truly is an awesome and much needed feature

EnlightenedOne · 2023-12-20T01:30:58Z

I share @Alia5 sentiment on this one, as-is it does allow for the most simple case to be delivered. It certainly lowers the bar for engagement people can stencil their third person character behind opaque and transparent geometry and draw them through it with various effects.

Here is a custom dither (with a bit of alpha blending when not discarded for a softer look):

I noticed that the xray colour drawn in the editor can go stale (as shown in the picture) changing the stencil mode flushes things and gets further colour changes applied immediately this might be some Godot UI quirk instead of an issue with the feature but just in case it is not I thought I would flag it.

carsonclarke570 · 2024-05-13T15:02:32Z

Would it be possible to at least rebase these changes on to master for people (me included lol) to try these changes out with the most recent master changes? I'd do it myself but I was struggling to resolve some of the conflicts properly.

QbieShay · 2024-05-13T18:56:36Z

We usually don't ask contributors to rebase constantly unless we're close to merging their contribution, in en effort to respect people's time. However it's been a while so maybe there's merit in rebasing.

carsonclarke570 · 2024-05-13T19:02:32Z

We usually don't ask contributors to rebase constantly unless we're close to merging their contribution, in en effort to respect people's time. However it's been a while so maybe there's merit in rebasing.

Apologies, I don't mean to be disrespectful and do not mean to apply pressure on the authors. I wasn't sure who or where to contact after I was having trouble getting any help with doing this rebase in the Discord or the Rocketchat and it had been a while since there was an update so I figured there was no harm in asking here.

QbieShay · 2024-05-13T19:16:47Z

No worries no harm done :) You can try joining the https://chat.godotengine.org/channel/rendering channel in the dev chat if you struggle with rebase, folks there can help

RadiantUwU · 2024-06-25T01:05:42Z

Any updates? This has been not updated for so long, this branch is literally nearly 8000 commits behind master.

apples · 2024-07-20T00:04:34Z

I found some free time, so I decided to do the rebase since it seems like some people need it.

It was a pretty clean rebase. I just had to flip the depth functions due to the recent reverse Z changes.

I'm not sure how well this interacts with the new motion buffer, I haven't had a chance to test it yet.

I also updated the demo project (https://github.com/apples/godot-stencil-demo).

Updated stencil features to match new design Fixed stencil opaque pass Added STENCIL_RW_WRITE_ALWAYS Added docs for BaseMaterial3D stencil operations Added mobile renderer support Renamed stencil rw modes Implemented property hiding for stencil fields Updated docs Apply suggestions from code review Co-authored-by: A Thousand Ships <[email protected]> Update scene/resources/material.cpp Co-authored-by: RedMser <[email protected]> Update servers/rendering/renderer_rd/forward_clustered/render_forward_clustered.cpp Co-authored-by: A Thousand Ships <[email protected]> Added missing cases Fixed stencil_mode autocompletion Fixed syntax highlighting of stencil_mode Added stencil support to VisualShader Added stencil support for GLES3 Changed stencil property default values in BaseMaterial3D Revert "Added stencil support for GLES3" This reverts commit c6f581b. Removed mask_uses_ref project setting Changed depth functions to better match PR godotengine#73527 Apply suggestions from code review Co-authored-by: A Thousand Ships <[email protected]> Changed set_stencil_effect_color to pass by const ref Fixed errors

AThousandShips

Just some nit-picks and improvements for when you have the time

AThousandShips · 2024-05-04T16:30:35Z

scene/resources/material.cpp

+
+	Ref<BaseMaterial3D> stencil_next_pass;
+
+	if (!current_next_pass.is_valid() || !current_next_pass->has_meta("_stencil_owned")) {


Suggested change

if (!current_next_pass.is_valid() || !current_next_pass->has_meta("_stencil_owned")) {

if (current_next_pass.is_null() || !current_next_pass->has_meta("_stencil_owned")) {

AThousandShips · 2024-07-20T08:20:17Z

editor/plugins/text_shader_editor.cpp

+				for (int j = 0; j < render_modes.size(); j++) {
+					const ShaderLanguage::ModeInfo &mode_info = render_modes[j];
+
+					if (!mode_info.options.is_empty()) {
+						for (int k = 0; k < mode_info.options.size(); k++) {
+							built_ins.push_back(String(mode_info.name) + "_" + String(mode_info.options[k]));


Suggested change

for (int j = 0; j < render_modes.size(); j++) {

const ShaderLanguage::ModeInfo &mode_info = render_modes[j];

if (!mode_info.options.is_empty()) {

for (int k = 0; k < mode_info.options.size(); k++) {

built_ins.push_back(String(mode_info.name) + "_" + String(mode_info.options[k]));

for (const ShaderLanguage::ModeInfo &mode_info : render_modes) {

if (!mode_info.options.is_empty()) {

for (const StringName &option : mode_info.options) {

built_ins.push_back(String(mode_info.name) + "_" + String(option));

Use for-each syntax when applicable

AThousandShips · 2024-07-20T08:20:23Z

editor/plugins/text_shader_editor.cpp

-				if (!mode_info.options.is_empty()) {
-					for (int k = 0; k < mode_info.options.size(); k++) {
-						built_ins.push_back(String(mode_info.name) + "_" + String(mode_info.options[k]));
+				for (int j = 0; j < stencil_modes.size(); j++) {


AThousandShips · 2024-07-20T08:20:31Z

editor/plugins/text_shader_editor.cpp

+		{
+			const Vector<ShaderLanguage::ModeInfo> &shader_modes = ShaderTypes::get_singleton()->get_modes(RenderingServer::ShaderMode(shader->get_mode()));
+
+			for (int i = 0; i < shader_modes.size(); i++) {


AThousandShips · 2024-07-20T08:20:43Z

editor/plugins/text_shader_editor.cpp


-		for (int i = 0; i < modes.size(); i++) {
-			const ShaderLanguage::ModeInfo &mode_info = modes[i];
+			for (int i = 0; i < stencil_modes.size(); i++) {


AThousandShips · 2024-07-20T08:21:30Z

scene/resources/visual_shader.cpp

+		}
+
+		// Add flags afterward.
+		for (int i = 0; i < flag_names.size(); i++) {


AThousandShips · 2024-07-20T08:22:17Z

servers/rendering/shader_language.cpp

+		for (int i = 0; i < RenderingServer::SHADER_MAX; i++) {
+			const Vector<ModeInfo> modes = p_is_stencil ? ShaderTypes::get_singleton()->get_stencil_modes(RenderingServer::ShaderMode(i)) : ShaderTypes::get_singleton()->get_modes(RenderingServer::ShaderMode(i));
+
+			for (int j = 0; j < modes.size(); j++) {


AThousandShips · 2024-07-20T08:22:30Z

servers/rendering/shader_language.cpp

+			}
+		}
+	} else {
+		for (int i = 0; i < p_modes.size(); i++) {


AThousandShips · 2024-07-20T08:22:42Z

servers/rendering/shader_language.cpp

+				for (int i = 0; i < RenderingServer::SHADER_MAX; i++) {
+					const Vector<ModeInfo> modes = ShaderTypes::get_singleton()->get_stencil_modes(RenderingServer::ShaderMode(i));
+
+					for (int j = 0; j < modes.size(); j++) {


AThousandShips · 2024-07-20T08:22:53Z

servers/rendering/shader_language.cpp

+					}
+				}
+			} else {
+				for (int i = 0; i < p_info.stencil_modes.size(); i++) {


RadiantUwU · 2024-08-15T19:24:17Z

Is this gonna make it into 4.4?

clayjohn · 2024-08-15T20:22:01Z

Is this gonna make it into 4.4?

Not sure yet. The status right now is that this is blocked until we add the ability to control the order of rendering commands (i.e. add a Rendering Compositor). No one has volunteered to work on that yet, which leaves this PR blocked.

We are discussing if there is a way to merge this before adding a compositor, but that discussion is ongoing

RadiantUwU · 2024-08-17T17:33:14Z

Will it implement things such as custom writing to stencil buffers and reading from them for colors? Just a function to be usable if stencil mode is used would be enough.
This would allow things such as custom screen space refraction to potentially work better.

EnlightenedOne · 2024-11-06T03:35:10Z

doc/classes/BaseMaterial3D.xml

@@ -151,6 +151,10 @@
 		<member name="depth_draw_mode" type="int" setter="set_depth_draw_mode" getter="get_depth_draw_mode" enum="BaseMaterial3D.DepthDrawMode" default="0">
 			Determines when depth rendering takes place. See [enum DepthDrawMode]. See also [member transparency].
 		</member>
+		<member name="depth_function" type="int" setter="set_depth_function" getter="get_depth_function" enum="BaseMaterial3D.DepthFunction" default="0">


I believe the default should be "5" (or the enum should be reordered)?

EnlightenedOne · 2024-11-06T16:49:04Z

Hi @apples,

First off thank you for this tremendous feature. I had a play with it off main and I think one thing missing from it is a means to render to the stencil buffer "only" in the opaque pass, this much like a shadow mesh for transparent geometry would allow transparent geometry to be stenciled more easily for the most simple of use cases:

I am going to take a look at options for this off my branch, hope it is useful feedback.

I also noticed a bug the default for a new or existing VisualShader is a Depth Mode of "Less Or Equal"/0 not "Greater Or Equal"/5 if you import this into an existing scene it trashes the rendering. It is possible other parts of Godot make the same assumption, I think you need to reorder the enum so that the first item is the safe default value. On my fork I am experimenting with modifying the VisualShader to cope with a non 0 default value... it worked but for safety I reordered the list.

Thanks,
EO

EnlightenedOne · 2024-11-07T15:52:04Z

Found 2 other bugs, in generated shader code no space is included between depth draw and depth function modes:
render_mode blend_mix, depth_draw_opaque,depth_function_greater_or_equal, cull_back, diffuse_burley, specular_schlick_ggx;

If you have XRay mode on an object then hide it behind a StandardMaterial3D it will draw your XRay coloured object just fine. Add a Rim to the StandardMaterial3D that is in the way and it stops working. I suspect it is a bug rather than a draw order issue will dig on it soon.

apples · 2024-11-07T16:17:55Z

@EnlightenedOne Thanks for testing! If I can find some spare time, I'll see if I can get those issues fixed.

If you have XRay mode on an object then hide it behind a StandardMaterial3D it will draw your XRay coloured object just fine. Add a Rim to the StandardMaterial3D that is in the way and it stops working. I suspect it is a bug rather than a draw order issue will dig on it soon.

I suspect this is an opaque vs transparency pass issue, but I'll need to look into exactly how rim is implemented.

EnlightenedOne · 2024-11-07T20:26:46Z

scene/resources/material.cpp

@@ -727,6 +727,35 @@ void BaseMaterial3D::_update_shader() {
 			break; // Internal value, skip.
 	}

+	switch (depth_function) {
+		case DEPTH_FUNCTION_LESS_OR_EQUAL:
+			code += ",depth_function_less_or_equal";


Here is where the missing space comes from other material shader generators include the space after the comma

EnlightenedOne · 2024-11-08T09:06:38Z

@EnlightenedOne Thanks for testing! If I can find some spare time, I'll see if I can get those issues fixed.

If you have XRay mode on an object then hide it behind a StandardMaterial3D it will draw your XRay coloured object just fine. Add a Rim to the StandardMaterial3D that is in the way and it stops working. I suspect it is a bug rather than a draw order issue will dig on it soon.

I suspect this is an opaque vs transparency pass issue, but I'll need to look into exactly how rim is implemented.

Issue was user error. The issue comes from the sort order of the opaque materials "render_list[RENDER_LIST_OPAQUE].sort_by_key();". I didn't realise you had to give your player a priority greater than that of anything you want them to be drawn in front of. If they are of equal priority you get material id/age of material based sorting. So if all of your scene has a priority of 0 your player material should be priority 1+ and your xray material 0 or -1 to get the desired effect. Equally any opaque thing that should obscure the player needs to have a priority of 2 in the above example.

EnlightenedOne · 2024-11-18T00:05:03Z

As a pre-compositor way to stencil behind transparent objects I landed on adding an optional render pass/RenderListType for post alpha stencil materials (flag as optionally drawn late) so that depth/stencil changes from the alpha pass can influence the XRay's next pass material. You have to set a stencil write on all your transparent objects that you want to overwrite the opaque stencil. Motion blur wont interact with the stuff drawn based on stencils after the transparent pass. To use this mode you can set the alpha to 0 and just write to the stencil buffer in the transparent layer.

My naive approach was to implement depth/stencil only writes in the opaque pass and sort them to happen after all other opaque objects and before stencil overlay materials. I was confident the motion buffer could be massaged to work with this in theory but I hit trouble as RD::PipelineColorBlendState::Attachment with the write rgba all set to false caused black pixels to get drawn in the opaque pass in Vulkan Forward+. I adopted RenderDoc to help me diagnose issues after that bump.

apples requested review from a team as code owners August 17, 2023 10:48

AThousandShips reviewed Aug 17, 2023

View reviewed changes

apples marked this pull request as draft August 17, 2023 11:01

AThousandShips added enhancement topic:rendering topic:shaders labels Aug 17, 2023

AThousandShips added this to the 4.x milestone Aug 17, 2023

RedMser reviewed Aug 17, 2023

View reviewed changes

scene/resources/material.cpp Outdated Show resolved Hide resolved

AThousandShips reviewed Aug 17, 2023

View reviewed changes

servers/rendering/renderer_rd/forward_clustered/render_forward_clustered.cpp Outdated Show resolved Hide resolved

AThousandShips reviewed Aug 17, 2023

View reviewed changes

scene/resources/material.cpp Show resolved Hide resolved

Calinou reviewed Aug 18, 2023

View reviewed changes

apples force-pushed the 7174-apples-stencil branch from 0a1e4e9 to be2f6cf Compare August 20, 2023 04:56

QbieShay approved these changes Aug 22, 2023

View reviewed changes

This comment was marked as off-topic.

Sign in to view

YuriSizov changed the title ~~Stencil support in spatial materials~~ Add stencil support to spatial materials Dec 16, 2023

AThousandShips approved these changes Dec 17, 2023

View reviewed changes

jbromberg mentioned this pull request Dec 17, 2023

Godot 4 Ultimate Flexible Outline and X-Ray Shader alfredbaudisch/GodotShaderCollection#3

Open

akien-mga mentioned this pull request May 6, 2024

Fix CPUParticles3D using angle incorrectly when ROTATE_Y is set. #89595

Merged

adamscott added the topic:vfx label Jul 1, 2024

Lielay9 mentioned this pull request Jul 2, 2024

Round primitive meshes contain gaps. #93865

Closed

apples force-pushed the 7174-apples-stencil branch from f8040f5 to 6d98df0 Compare July 20, 2024 00:00

apples added 2 commits July 19, 2024 23:21

Add depth function for spatial materials

47ff5a9

apples force-pushed the 7174-apples-stencil branch from 6d98df0 to 1c1b7a8 Compare July 20, 2024 04:29

AThousandShips reviewed Jul 20, 2024

View reviewed changes

Glorax mentioned this pull request Aug 21, 2024

Implement a Rendering Compositor godotengine/godot-proposals#7916

Open

EnlightenedOne reviewed Nov 6, 2024

View reviewed changes

EnlightenedOne reviewed Nov 7, 2024

View reviewed changes

	const Ref<Material> current_next_pass = get_next_pass();
	Ref<Material> current_next_pass = get_next_pass();

		depth_stencil_state.enable_depth_write = depth_draw != DEPTH_DRAW_DISABLED ? true : false;

		RD::CompareOperator depth_function_rd_table[DEPTH_FUNCTION_MAX] = {

	RD::CompareOperator depth_function_rd_table[DEPTH_FUNCTION_MAX] = {
	static const RD::CompareOperator depth_function_rd_table[DEPTH_FUNCTION_MAX] = {

	RD::CompareOperator stencil_compare_rd_table[STENCIL_COMPARE_MAX] = {
	static const RD::CompareOperator stencil_compare_rd_table[STENCIL_COMPARE_MAX] = {


		Ref<BaseMaterial3D> stencil_next_pass;

		if (!current_next_pass.is_valid() \|\| !current_next_pass->has_meta("_stencil_owned")) {

	if (!current_next_pass.is_valid() \|\| !current_next_pass->has_meta("_stencil_owned")) {
	if (current_next_pass.is_null() \|\| !current_next_pass->has_meta("_stencil_owned")) {

Add stencil support to spatial materials #80710

Are you sure you want to change the base?

Add stencil support to spatial materials #80710

Conversation

apples commented Aug 17, 2023 • edited Loading

Linked Issues

Differences from the design

Currently implemented

Supported Renderers

Depth Prepass

GLES3

Sorting Problems

QbieShay commented Aug 17, 2023

Calinou left a comment • edited Loading

Choose a reason for hiding this comment

Calinou commented Aug 18, 2023

apples commented Aug 18, 2023 • edited Loading

QbieShay commented Aug 22, 2023

QbieShay commented Aug 22, 2023

QbieShay left a comment

Choose a reason for hiding this comment

Calinou commented Aug 22, 2023

QbieShay commented Aug 22, 2023

Lielay9 commented Aug 22, 2023 • edited Loading

QbieShay commented Aug 22, 2023 • edited Loading

apples commented Aug 22, 2023

apples commented Aug 22, 2023

QbieShay commented Aug 23, 2023

Lielay9 commented Aug 23, 2023 • edited Loading

Calinou commented Aug 23, 2023 • edited Loading

This comment was marked as off-topic.

AThousandShips left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Alia5 commented Dec 18, 2023 • edited Loading

EnlightenedOne commented Dec 20, 2023 • edited Loading

carsonclarke570 commented May 13, 2024

QbieShay commented May 13, 2024

carsonclarke570 commented May 13, 2024

QbieShay commented May 13, 2024

RadiantUwU commented Jun 25, 2024

apples commented Jul 20, 2024

AThousandShips left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RadiantUwU commented Aug 15, 2024

clayjohn commented Aug 15, 2024

RadiantUwU commented Aug 17, 2024

EnlightenedOne Nov 6, 2024 • edited Loading

Choose a reason for hiding this comment

EnlightenedOne commented Nov 6, 2024 • edited Loading

EnlightenedOne commented Nov 7, 2024

apples commented Nov 7, 2024

Choose a reason for hiding this comment

EnlightenedOne commented Nov 8, 2024 • edited Loading

EnlightenedOne commented Nov 18, 2024 • edited Loading

apples commented Aug 17, 2023 •

edited

Loading

Calinou left a comment •

edited

Loading

apples commented Aug 18, 2023 •

edited

Loading

Lielay9 commented Aug 22, 2023 •

edited

Loading

QbieShay commented Aug 22, 2023 •

edited

Loading

Lielay9 commented Aug 23, 2023 •

edited

Loading

Calinou commented Aug 23, 2023 •

edited

Loading

Alia5 commented Dec 18, 2023 •

edited

Loading

EnlightenedOne commented Dec 20, 2023 •

edited

Loading

EnlightenedOne Nov 6, 2024 •

edited

Loading

EnlightenedOne commented Nov 6, 2024 •

edited

Loading

EnlightenedOne commented Nov 8, 2024 •

edited

Loading

EnlightenedOne commented Nov 18, 2024 •

edited

Loading