How to handle additive transparency

[andybak]

The lack of support for additive blending has been discussed on the main glTF repo:

KhronosGroup/glTF#1189

and in the broader context of blending modes here:

KhronosGroup/glTF#1302

Unreal's glTF export plugins and their web viewer use an extension called EPIC_blend_modes but I can't find any documentation on it beyond the implementation here: https://github.com/ue4plugins/GLTFWebViewer

So we have two potential extensions - one documented with no implementation and the other with a single undocumented implementation.

I'm also wondering if regular PBR emissive could somehow be used to partially fake additive blending.

I'm surprised at the lack of interest in additive blending considering it's a fairly standard feature in most realtime and non-realtime engines and renderers and is a common technique to implement light beams, sparks, flames etc.

It is also useful as for avoiding issues with sorting of transparent objects. Additive blending is inherently order independent. This is one of the reasons that Open Brush uses is so extensively (z-sorting of brush strokes is especially problematic)

[andybak]

/cc @hybridherbst @donmccurdy

Just keeping a braindump here so I don't clutter up the issue tracker on UnityGLTF

I'm tempted to attempt to implement KHR_blend as documented as an import/export plugin for UnityGLTF and use the new Open Brush export implementation as a test case. I'm not sure how to move KhronosGroup/glTF#1302 forward or whether it really matters. I'm more of a "rough consensus and running code" kind of guy and the concept of "unofficial but widely supported extensions" seems to be the way things are going with glTF.

[hybridherbst]

Wasn't aware of this proposal, thanks for the ping!

@pfcDorn can you take a look? I think with the new plugin infrastructure it totally makes sense to support this in UnityGLTF, and if someone doesn't want it they just turn the plugin off.

@andybak we may want to rename the extension to EXT_blend if there is no plan at Khronos to ratify KHR_blend.

(Regarding implementations, Needle Engine does indeed support additive blending in glTF as part of a bigger NEEDLE_* extension but not in a standardization-happy format.)

[andybak]

@andybak we may want to rename the extension to EXT_blend

Yep. If we're going back to the drawing board I would consider some other changes. Is the current proposal too specific to how glsl/hlsl define blending ops? Is it too flexible and therefore complex?

How would an importer for something like Blender work? I'm no Blender expert and I struggled to define an additive material at all using a Cycles material node graph.

I'd be happy with an extension that just handled additive alone. I think the "additive vs alpha/multiplicative vs stencil" distinction accounts for 99% of use cases I see in the real world and additive is the only one of the three that is currently absent from glTF.

[andybak]

I checked Shader Graph and it supports four transparent blending modes. They aren't documented but they are the same four that URP supports in the lit shader:

https://docs.unity3d.com/Packages/[email protected]/manual/lit-shader.html

1. Alpha
2. Premultiply
3. Additive
4. Multiply

For an opaque shader you additionally get alpha clipping which is arguably a 5th mode. I'm still personally only really interested in additive but I wonder if adding Multiply and Premultiply would be a good baseline set (given that alpha and alpha clipping are already supported)

[donmccurdy]

In hindsight, I agree that my KHR_blend proposal was too heavily influenced by OpenGL and WebGL APIs. I don't feel strongly one way or the other about whether it was too flexible or complex. If full flexibility were preferred, the WebGPU API for blending might be good inspiration:

https://www.w3.org/TR/webgpu/#blend-state

Also, better to have string enums than integer GL constants.

[andybak]

In an ideal world this would simply be another potential value for alphaMode property: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#alpha-coverage

If we want to ease the path towards standardization Maybe the extension should simply be an "extended alphaMode" - one that follows the same syntax and is used in preference for any client that supports the extension.

In which case I'd be tempted to name the extension something like EXT_advanced_alphaMode

[hybridherbst]

If it's just another allowed value on that then some existing materials wouldn't translate over, for example "mesh has alpha cutout at 0.5 and is additively blended".

[andybak]

If it's just another allowed value on that then some existing materials wouldn't translate over, for example "mesh has alpha cutout at 0.5 and is additively blended".

My first thought is that setting any values on EXT_advanced_alphaMode would indicate that the "legacy" alphaMode, and alphaCutoff should be ignored.

Is this consistent with how other extensions interact with existing properties?

[donmccurdy]

I tend to view this as a two-part problem:

1. alphaMode = "BLEND" and alphaMode = "MASK" should not be mutually exclusive. This could be fixed with a new enum value like alphaMode = "BLEND_AND_MASK" for backward compatibility. Perhaps not required for this extension. Other enum values could be added someday, like alphaMode = "HASHED"
2. New blend operations should added by some additional property, like alphaBlendOperation: "add", which has a default value equivalent to the current alpha blending.

[andybak]

The existing alphaMode supports:

• OPAQUE
• MASK
• BLEND

EPIC_blend_modes has the following valid values:

• Opaque
• Masked
• Translucent
• Additive
• Modulate
• Alpha Composite

https://docs.unrealengine.com/5.1/en-US/how-the-gltf-exporter-handles-unreal-engine-content/#blendmodesupport

and their behavior is described here: https://docs.unrealengine.com/5.1/en-US/material-blend-modes-in-unreal-engine/

As mentioned previously Unity Shader Graph has:

• Opaque
• Opaque / alpha clipping
• Transparent / Alpha
• Transparent / Additive Blending
• Transparent / Multiplicative Blending
• Transparent / Premultiplied Alpha

This matches the Unreal list perfectly from my brief reading. (I listed them in the same order above so you can see the matches)

three.js is a bit different and closer to @donmccurdy 's suggestion in that alpha clipping and other blending modes are not mutually exclusive. I don't know unreal very well but this is doable in Unity. Alpha clipping would have to be done manually in non-opaque shaders.

three.js doesn't have premultiplied blend but it does have subtractive blend. I'm personally in favour of leaving both out of this extension and going with the common subset.

I'm still not clear how Blender handles this. It used to have additive as a simple setting but it was removed. I also suspect Cycles and Eevee might behave differently - I only looked at Cycles.

I know Blender isn't the only non-realtime 3d app to consider but it's the one I have handy so I'm using it as a proxy for others.

[andybak]

This seems to be the best Blender equivalent: https://developer.blender.org/docs/images/Eevee2.81_transparent_bsdf2.png

The emission strength needs to probably be higher than 1.0 to match typical the look of typical glsl additive.

EDIT fix link because Blender website doesn't update url when viewing an image. Grumble...

[andybak]

Thinking about how interactions with the existing alphaMode would work:

Opaque
Mask
Blend

The "common subset" discussed above reduces to:

Opaque
Opaque / alpha clipping    
Transparent / Alpha
Transparent / Additive Blending
Transparent / Multiplicative Blending

Three of these map perfectly to existing alphaModes:

Opaque (Opaque)
Opaque / alpha clipping (Mask)
Transparent / Alpha (Blend)

Which leaves:

Transparent / Additive Blending
Transparent / Multiplicative Blending

Simply adding a new alphaMode property that has possible values of add/multiply would give us these two.

@donmccurdy said:

If it's just another allowed value on that then some existing materials wouldn't translate over, for example "mesh has alpha cutout at 0.5 and is additively blended".

Not sure what "existing materials" you're referring to? I'm assuming we'd doing "SrcAlpha One" in GL terms so the existing alpha is taking into account. You can't specify a hard cutoff but you can do alpha-masking. Is "alpha clip plus additive" used much?

I just did a survey of Open Brush shaders (the ones used in brushes which is the reason we want additive blending) and it's actually fairly complex.

On the whole we're doing Blend One One but there's a few cases of Blend SrcAlpha One - I need to take a look to see how much difference this makes (we can probably modify the textures on export to remove the distinction between these two cases).

However we're also using an even mix of BlendOp Add, Min with the default (BlendOp Add, Add) and just for fun the mobile variants of two shaders use BlendOp Max, Min which I presume is a performance optimisation...

[aaronfranke]

Since there is a discussion of allowing "alpha clip plus additive" and/or alphaMode = "BLEND_AND_MASK", perhaps the extension should have a boolean for "also use mask" or "always use clip", not sure what to name it. This would allow you to combine any other blend mode with clip/mask (but we'd note that this boolean shouldn't be set for OPAQUE or MASK to avoid redundant representations of those two existing states).

However, can somebody clarify: If using a hypothetical "BLEND_AND_MASK" mode, is the cutoff value for "everything below this is fully transparent and everything above is blended" or is it "everything below this is blended and everything above is fully opaque"?

Which leaves:

What about HASH, SUBTRACT, MAX, MIN? If we do as I suggest above and include a mask/clip boolean, then it might make sense to have this extension be comprehensive (even if most implementations don't support some of them), otherwise to do HASH with a mask/clip you'd need two extensions to avoid overlap (this ext for the bool, plus another ext).

[hybridherbst]

BLEND_AND_MASK : I believe it would be "everything below this is fully transparent and everything above is blended" in line with how Unity and Unreal handle "transparent + masked" surfaces. From a shader perspective it would probably be "everything below is discarded and everything above is blended".

I think the question here starts to become "should we handle explicit cases, or should we just do blend modes". I believe the explicit cases are more useful in the context of glTF.

[andybak]

OK. I sense a tension between a "maximalist" and "minimalist" spec here. I was aiming towards the latter because it felt like both:

1. The common cases are overwhelming more useful and widely supported
2. It's easier to get implementers on board with a small and obviously practical extension.

I'm a bit dubious about alpha-hashing as it's not an out of the box mode in most of the platforms I looked at. And doesn't it potentially need another parameter? It also raises the question of what specific dither patterns to support. I know the term "hashing" implies a very specific algorithm but the underlying "fake opacity via selective discard" can be implemented in many ways.

@aaronfranke are you mostly in favour of it because it's well specified in Godot?

I guess I was arguing for a spec that had the common subset of modes. They seem to be both the most useful, easy to implement and widely supported in existing shader frameworks. I know implementers don't have to implement the whole extension but partial implementations are a pain point for users - and I fear some implementers might be dissuaded from supporting an extension if they can't easily support all of it.

[aaronfranke]

@andybak Hash is in Godot, plus @donmccurdy mentioned it here. I don't have any other reason, I have not used it personally (nor have I used most of these, as I'm not an expert in alpha blending modes). We can remove it if desired, leaving it to another extension, especially if it needs another parameter. @donmccurdy Can you weigh in on this?

[andybak]

So - the minimalist spec would be just add and multiply (the common subset that seems to be universally supported) plus a bool for "always clip".

I've already mentioned my doubts about hash. I don't recall seeing subtract used much "in the wild". Premultiply I'm unsure about.

[hybridherbst]

I don't have a very strong opinion but would be favourable towards including "alpha hashing" as mode. My reasoning would be that it's a way for asset authors to handle multi-layer transparencies that doesn't have a equivalent in glTF currently, and it isn't really possible to "guess" that an asset should use alpha hashing in a general case.

One curious case I've seen in the wild is that when models are opaque and use an opacity value and an alpha cutoff value, then e.g. Apple's QuickLook will interpret that as alpha hashing.

[aaronfranke]

@hybridherbst To keep support for this mode, I'd like help collecting information on which apps have support for it (and what those apps call it).

Also, do we need to add a new parameter for it? Godot has alpha_hash_scale. Or, if this is just a scale, does it make sense to bake this somehow to avoid the extra parameter, or is it important to have the parameter explicitly?

[andybak]

collecting information on which apps have support for it

A lot depends on what you mean by "support". With Unity I looked at what Shader Graph supported. Unity does support code-based shaders but it's currently a rather fluid situation (the old render pipeline is deprecated, writing code shaders in URP and HDRP is poorly documented if you want to integrate with fog/lighting etc, the new unified pipeline is not yet available).

I don't know enough about other ecosystems but my data above was based on what seemed to be supported in the obvious "out of the box" PBR shaders or graphical authoring system. With full hlsl/glsl access I presume anything is possible but it might be more work for implementers.

We probably need to try a toy implementation across a few platforms or canvas some experts on those platforms.

(also - some platforms offer support for low-level shader writing but with big caveats. i.e. a lot of Unity integration SDKs only support shader graph shaders. You can't build for Apple VisionOS if you use code shaders etc)