Provide a super simple way for engines to add support for spec gloss

[pjcozzi]

See

• https://github.com/KhronosGroup/glTF/tree/master/extensions/Khronos/KHR_materials_pbrSpecularGlossiness/examples
• https://github.com/KhronosGroup/glTF/tree/master/extensions/Khronos/KHR_materials_pbrSpecularGlossiness#best-practices

[emackey]

For what it's worth, the "lossless" conversion goes the other way. Spec/Gloss is more flexible than Metal/Rough, and there is a direct, simple conversion from metal/rough to spec/gloss, but not the other way around. See KhronosGroup/glTF#696 (comment) and some OSGjs sample code.

Basically the metallic map is used to lerp the baseColor, and the roughness channel is inverted to form glossiness.

// Pseudocode conversion from (baseColor, metallic, roughness) to (diffuse, specular, glossiness)

diffuse.rgb = baseColor.rgb * (1.0 - metallic);
specular.rgb = mix(vec3(0.04), baseColor.rgb, metallic);
glossiness = 1.0 - roughness;

As such, there's no lossless "reverse" conversion, since diffuse and specular could be unrelated colors, there's no way to combine them into a sensible baseColor.

Also note the vec3(0.04) hard-coded dielectric specularity in there. The spec/gloss extension gives the artist control over dielectric specularity, which is, I'm told, important to some.

[pjcozzi]

@emackey I thought this stage would provide a super simple way for engines to add support for spec gloss. What do you suggest? Engines should just natively support both? Or engines could natively support spec gloss and then convert metal rough to it?

[emackey]

Yes, I believe the accepted wisdom is that for engines that want to support both, the best practice is to "natively" support PBR in the form of spec/gloss, and have a simple transformation in the fragment shader to get there from metal/rough when needed. You can see this is what OSGjs is doing.

I forgot to check what glTF-WebGL-PBR actually does about this issue, so I just looked at it now. Here's the relevant code there. Looks like the transform from metal/rough to spec/gloss is already hard-coded in the frag shader. We just need a way to make that transform conditional, so that the spec/gloss maps can drive those values directly, without using that hard-coded 0.04 in there.

[pjcozzi]

Gotcha, thanks!

[bghgary]

@emackey
The diffuseColor and specularColor in the glTF-WebGL-PBR shader are inputs to the BRDF equations and are not the same as the diffuse and specular in the spec-gloss extension. Converting from the spec-gloss extension inputs to the BRDF inputs is documented near the end of this section. The conversion from metal-rough to spec-gloss should be something similar to this conversion function. The factor/texture combo also needs to be taken into account during the conversion.

@pjcozzi

provide a super simple way for engines to add support for spec gloss

Perhaps the title should be edited to avoid confusion.

[pjcozzi]

provide a super simple way for engines to add support for spec gloss

Perhaps the title should be edited to avoid confusion.

Updated.

[emackey]

Started thinking about this again recently. The links broke during the great extension folder re-org, here are updated links:

The diffuseColor and specularColor in the glTF-WebGL-PBR shader are inputs to the BRDF equations and are not the same as the diffuse and specular in the spec-gloss extension. Converting from the spec-gloss extension inputs to the BRDF inputs is documented near the end of this section. The conversion from metal-rough to spec-gloss should be something similar to this conversion function. The factor/texture combo also needs to be taken into account during the conversion.

Looking at the formulas at the bottom of that section, I see alpha is (1 - glossiness) squared, which is the same as the roughness-squared that glTF-WebGL-PBR uses. The other formula is this:

Cdiff = diffuse.rgb * (1 - max(specular.r, specular.g, specular.b))

Here it looks like we're making sure that the diffuse channel goes dark for "metals" (or anytime there's high specular, since this is not metal/rough workflow we're talking about). But I'm confused, shouldn't a standard spec/gloss workflow already have this calculation baked into the supplied diffuse map?

For example, in WaterBottle's spec/gloss assets, one can clearly see a particular yellowish color of the bottle. Some pixels have this color in the diffuse map, with black in the spec map. Other pixels have this color in the spec map with black in the diffuse map. This seems very akin to the metal/rough workflow where one base color has some metal areas and some non-metal areas. But why would I take the spec/gloss version's diffuse map and re-multiply it by (1 - max(specular.r, specular.g, specular.b))? It looks like that multiplication is already baked into the map.

[bghgary]

I really need to write up how I came up with the math. It's in my backlog. The gist of it is to solve for the workflow inputs given the two sets of equations for metallic-roughness and specular-glossiness. I will try to spend some time doing this.

But I'm confused, shouldn't a standard spec/gloss workflow already have this calculation baked into the supplied diffuse map?

All the implementations of spec/gloss that I've seen does this to ensure energy conservation. As far as I know, it is not baked into the diffuse map.

WaterBottle's spec/gloss assets

The original textures for the water bottle was authored with metal/rough. The spec/gloss textures were created using the exact formula in the conversion function in the link you mentioned.

[emackey]

The original textures for the water bottle was authored with metal/rough. The spec/gloss textures were created using the exact formula in the conversion function in the link you mentioned.

Oh I think I see. The conversion from metal/rough to spec/gloss explicitly divides out the "oneMinusMaxSpecular" part, presumably expecting the runtime to multiply that back in later, as per the Cdiff formula above.

[emackey]

@lilleyse I think I understand what's needed in the shader. If you or someone can get a branch of gltf-pipeline going with the mechanical hookup of specGloss, to get those textures/factors loaded into uniforms and the corresponding metal/rough textures/factors properly ignored for such models, I think the shader changes will be fairly simple from there. Up to you if/when you want to prioritize this.

https://github.com/AnalyticalGraphicsInc/gltf-pipeline/blob/25177db38c6c7874969acc02da339176d72b2607/lib/processPbrMetallicRoughness.js#L533-L536

In the above snippet, roughness is simply 1.0 - glossiness, and alpha is still the square of that. Glossiness comes from the alpha channel of the spec/gloss texture map, multiplied by the last component of the spec/gloss factor. f0 becomes identical to specularColor, which no longer uses mix and is now just the raw inputs from the specular texturemap's RGB channels multiplied by the specular factor. The only gotcha is the diffuseColor above, which is the diffuse texture map multiplied by the diffuse factor, multiplied by (1.0 - max(specularColor.r, specularColor.g, specularColor.b)).

With these changes (applied only for spec/gloss models), roughness and metalness no longer appear anywhere in the shader, and the spec/gloss shader hookup would be complete.

[lilleyse]

Moved to CesiumGS/cesium#6921, gltf-pipeline is no longer generating PBR shaders.