[PBR extension] Additional maps (normal maps, displacement, ...?)

[mlimper]

We need to check if / how bump maps / normal maps could be integrated.

[cedricpinson]

Yeah it's clearly missing, why not adding a normal and bump channel ? I guess we only need to decide if we add extra information like normal map coordinate (DX/GL) or if the coordinates must be fixed.

[mlimper]

I guess we only need to decide if we add extra information like normal map coordinate (DX/GL) or if the coordinates must be fixed.

Agreed - maybe also information about the space of the normals... could make sense to allow for object space and tangent space?

[cedricpinson]

yeah maybe, I dont see a lot of usage or request for this on our plateform, but it's maybe safer to add this information

[erich666]

As usual, I'd say "do as three.js does", and they have these - normal maps seem like a minimum since they've been in use well over a decade in interactive apps. Heck, I get requests to add normal maps and specular maps to Mineways, of all things - people do use these. Firms such as Simplygon use these in order to make content work on the Hololens, for example, see this video, they use a normal map to capture fine detail instead of a zillion polys.

Three.js goes wild in this area: normal/bump, displacement (vertex shifts), roughness, metalness, alpha, and the IBL (environment map). I do hope you're planning on IBL, that's a must (@cedricpinson I suspect will agree, as Sketchfab uses these a lot). Lighting doesn't get covered in the PBR spec, and unfortunately lighting matters to materials. Again, I'd just steal the three.js model, maybe lopping off some of the more obscure textures for now.

yeah maybe, I dont see a lot of usage or request for this on our plateform, but it's maybe safer to add this information

I agree, though have limited knowledge in this area. By the "WW3JS do?" principle, I don't see it there, so let's not include it here.

[cedricpinson]

Ok here some stats on sketchfab:

PBR workflow usage:
80% metalness
20% specular

Channels usage:
normal/bump 100%
emit 30%
f0 30% only used on metalness workflow
opacity 15%
ao 60%
cavity 12%
roughness 76%
glossiness 24%

[mlimper]

I agree, though have limited knowledge in this area. By the "WW3JS do?" principle, I don't see it there, so let's not include it here.

Agreed. Converters can also generate tangent-space versions of input maps in object space, (although that means they will have to generate tangents and binormals on the way). So, let's stick to tangent-space normal maps for now.

Looking at the listed maps (here), I am currently wondering if we really opacity. In our first proposal, alpha was part of the diffuseColor (glossiness workflow) or baseColor (metalness workflow). For artists, it might be cleaner to have alpha as a separate texture, but for delivery and rendering (which glTF should be all about), it might be better to pack alpha together with base / diffuse color into a single RGBA texture?

The other maps that are mentioned, 'normals', 'emissive' and 'AO', should probably be added, as it seems artists frequently use them (especially AO).

I am wondering a bit about 'normals' vs. 'bump': could we just support 'normals' and make tools generate a normal map for a given bump map? The "WW3JSD" approach suggests to support bump maps (https://threejs.org/examples/webgl_materials_bumpmap.html), but it would be nice to facilitate implementation of the PBR spec, given that normal maps / bump maps could be efficiently reduced to a "normal maps only" representation in a conversion tool (similar to object space / tangent space normal maps that would be reduced to a "tangent space only" representation).

[mlimper]

@FslNopper You mentioned f0_metal and f0_nonMetal (#696 (comment)). Can we put both into a single texture and simply use the "metalness" information to decide how those values should be interpreted?

[GunpowderGuy]

Bump maps are necessary , because they can be used for lighrning or displacement , so a normal map only representation is not posible

[FslNopper]

The f0_metal and f0_nonMetal is an idea to define both the metal and glossy model parameters in one single PBR material. But now I think this would make things too complicated. Now, I just recommend to have two separate PBR materials - both as an extension

[FslNopper]

I agree, we should have bump maps as well. But I think we should not put them into PBR directly.
From my point of view, bump and also like normal maps can also be used for the Phong model.
It would be nice. if the glTF next material model is somehow like Lego bricks:
Phong parameters
Metal PBR parameters
Glossy PBR parameters
Geometry parameters like bump, normal and displacement parameters

So, e.g. phong with geometry would be possible as well.

[mlimper]

Bump maps are necessary , because they can be used for lighrning or displacement , so a normal map only representation is not posible

I am aware of that! My point is just "if they are mostly used for lighting, to have something very similar to a normal map (compare the threejs example), we may actually cover 95% of the use cases without requiring to support bump maps"

[mlimper]

Geometry parameters like bump, normal and displacement parameters

So, e.g. phong with geometry would be possible as well.

That is correct, and I believe that we have to make a tradeoff here:
If we put everything into tiny, separate extensions, there is a danger of fragmentation, where an asset that uses a bunch of extensions might not be rendered consistently across several platforms as they all only support a very specific subset of extensions (for example, they would just support the Metal PBR model and not support the Glossy PBR extension). On the other hand, it is true that the idea of having a PBR model is actually separate from the idea to use a normal map or bump map. BRDF theory describes geometry on a micro-scale, normal maps and bump maps on a meso-scale, and, as you already pointed out, they can be used with other shading models (such as Blinn-Phong) as well.

The same does actually apply to AO. This is also not a core concept related to PBR / BRDF theory.

We already pointed out that PBR support is basically worthless if your renderer does not support normal maps. Therefore, if we now create a separate extension for normal maps / bump maps, it would be interesting to see if we could make the PBR extension require that the normal map / bump map extension is supported as well. This way, you would never get PBR without guaranteed support for normal / bump maps, but you would always be able to use normal maps / bump maps without PBR, if you have any use for that.

@pjcozzi Would that be aligned with the basic concepts of glTF?

[FslNopper]

The "geometry" parameters are optional for Phong and required for PBR. Maybe we should follow this approach.
There will be quite a lot of non-PBR applications out there e.g. embedded and/or automotive. So, it would be great to have normal maps - even AO - as an extension for non PBR.

[mlimper]

Regarding F0, quoting the current spec draft:

Metal-Roughness:

The base color has two different interpretations depending on the value of metallic, which is defined as 0 for dielectrics and 1 for metals.
For metallic = 1, base is the specific measured reflectance value (F0).
For metallic = 0, base represents the reflected diffuse color of the material.
In this model it is not possible to specify a reflectance value for non-metals, where a reflectance value of 4% (0.04) is often used.

Specular-Glossiness:

The specular value defines specular reflectance at normal incidence (F0).

We do not employ an explicit F0 texture right now.

@jeffdr @cedricpinson Would that work for you?

[FslNopper]

I think the original idea was, from where F0 is gathered:

If it is the metallic workflow F0 is "sampled" 0.04 for dielectrics and sampled from the base color map for metallic.

Furthermore, for the Glossy workflow, F0 would be be sampled from the specular map.

This would allow both models as one PBR, but there would be a (complicated) conversion overhead in the rendering engine.

So, I agree not to have an explicit F0 texture and keep it like it is.

[mlimper]

If it is the metallic workflow F0 is "sampled" 0.04 for dielectrics and sampled from the base color map for metallic.

Yes, that was the idea - since the typical deviation of F0 among dielectrics seems to marginal (according to the allegorithmic PBR guide, values typically vary between 0.02 and 0.05, also compare the Unity blog post)

[cedricpinson]

@mlimper Typically if the user provide this data we will want to use it, but it will be out of the spec. So it's probably better to describe extra optionals channels in the specs and recommend how to use it. For this case if we find F0 we use it, else we use 4%
I think it's more flexible to say that you can have optionals texture, but not sure how it will be break the logic of schema / extension.

Hope it makes sense

[jeffdr]

Yes in metalness "mode" you can usually get by without an F0 texture; that just sorta provides an extra degree of control for dielectric materials which is often not needed.

[mlimper]

Well, usually we will try to keep the spec really limited to the parts that are absolutely necessary, to keep the burden on a spec-conformant implementation as low as possible.

In this specific case, it might be OK to have an F0 texture for the dielectric materials as an optional component, since it seems that - even if an implementation would ignore that one - the rendered results wouldn't look too different.

Just to be clear - that optional texture would only be used to read the F0 values for the non-metallic parts, correct? For the metallic parts, we would still use the base color map.

[cedricpinson]

Yes the values in F0 texture are only used for non metallic parts

[tparisi]

Has anyone checked these additional proposed channels against Unity and Unreal specs?

[sbtron]

@tparisi everything in glTF is a subset of what Unity and Unreal support. Both engines have additional maps but we stayed away from the complex ones that are hard (if not impossible) to do in webGL. These could always be added in future updates.

[pjcozzi]

Updated in #826

Also see #948