[PBR extension] Parameter Set

[mlimper]

Initially, we have proposed two parameter sets: "Specular-Glossiness" (short: "glossiness") and "Metallic-Roughness" (short: "roughness"). One the one hand, this allows for more freedom, as assets may be given in one or the other format. On the other hand, it might be better to have only one parameter set, for increased simplicity, ease of implementation and portability.

If we agree to use only one parameter set, we will have to decide which one that should be. The roughness model might be popular and easy to use with several existing systems. The glossiness model allows for a bit more freedom, as the proposed specular component has three channels.

[pjcozzi]

I don't have the PBR experience to say which to go with, but I agree with only having one model in the extension; if it is needed, another extension could do another model. This is inline with the spirit of glTF, which is to keep the client as simple as possible and not provide too many different ways to do the "same" thing.

[lexaknyazev]

if it is needed, another extension could do another model

In that case, all PBR materials extensions should agree on light object model and probably keep lights in yet another extension.

Also, since material.extensions can contain many objects, there should be some restrictions on co-existence of two PBR extensions in one material.

[mlimper]

Yes, lighting is another topic that we need to discuss at some point. Will this be part of the extension, or yet another extension?

Adding @vorg @jeffdr @selim-bekkar-sb @tparisi to the conversation.

@ alll: What is your opinion about the number of parameter sets we should support? What would you vote for:

• glossiness model
• roughness model
• both

Let's keep in mind that glTF is a format for efficient asset delivery, not for editing. Furthermore, consider that, on the one hand, we certainly want to have an expressive model, but, on the other hand, we are still fine when we can serve 95% of the use cases (not necessarily 100%).

Please add any one you like to this voting - and thanks in advance for your votes :-)

[jeffdr]

I would vote 'glossiness', but I don't have a strong opinion. Not much need to support both IMO since one is pretty easily derived from the other for conversions.

[vorg]

I would go for Metallic-Roughness with potentially changing Roughness to Smoothness = 1 - Rougness (as per Unity and ThreeJS)

[mlimper]

Ok, it seems a lot of frameworks / implementations support both models, which could be a reason to go for both

[tsturm]

@mlimper - That is right, a lot of frameworks support both material workflows including Marmoset Toolbag, Sketchfab, Unity, Substance Designer etc.

[cedricpinson]

yeah a lot of sketchfab users uses metal-roughness workflow, but some specular-glossiness. I dont have a strong opinion on that but both are widely supported in software so it's maybe better to have both in the spec.

[cedricpinson]

How is handled F0 texture ? do you plan to support it ?

[tsturm]

Yes, I think it makes totally sense to support F0 textures in the metal-roughness workflow to change the
constant F0 value for dielectrics.

[pjcozzi]

glTF is designed for runtime; often there may be many workflows that generate the same runtime data so I would only support the two different approaches if one can not faithfully be converted to the other, and they are both widely used; otherwise, supporting both adds spec and client complexity that glTF aims to avoid.

[RemiArnaud]

Yes gltf is for runtime, but I would be careful with your statement above. gltf intent is not to dictate how runtime works or unify to one single run-time implementation.
However I do agree that client complexity should be avoided, but not by imposing a rendering model. gltf is a common language and a mechanism for run-times to get what they need, and only what they need, through a parametrised offline or online pipeline conversion process.

Going back to PBR, there is no clear winner and several rendering approaches should be available with Gltf. It won't make clients more complex, as each client should get only the representation it needs.

[mlimper]

It won't make clients more complex, as each client should get only the representation it needs.

I am not sure if that should be the way to go, as we will want clients to support either the full extension or nothing. Otherwise, we could end up with fragmentation, where each client / writer combination supports only one model, which would lead to glTF PBR assets not being exchangeable across implementation.

To judge whether we should support both, I guess it makes sense to have a simple example implementation first, so one could clearly see what would happen in a client that aims to support both.

[cedricpinson]

I have an old example here https://cedricpinson.github.io/osgjs-website/examples/pbr/ and the code to use the different workflow metalness-roughness / specular-glossiness is here https://github.com/cedricpinson/osgjs/blob/master/examples/pbr/shaders/pbrReferenceFragment.glsl#L248-L255
It could be simplified

[mlimper]

Thanks, very cool! Looks very straightforward.

[tparisi]

@mlimper I agree 100% we should not be working on client-specific PBR here... defeats the purpose of glTF and could lead to fragmentation.

[RemiArnaud]

@tparisi @mlimper

It's never been a gltf goal that run-times must be able to load all gltf content. It's nice that they can, but the main goal is to make it easy for runtime a to use the language and not to unify and limit usage.

It's more important for gltf to satisfy the needs for direct consumption of 3D content, meaning ability to represent what is needed and only what is needed by a given run-time, than to see that it is possible to write a universal player.

[emackey]

only what is needed by a given run-time, than to see that it is possible to write a universal player.

I'm perhaps biased coming from Cesium, but certainly I think Cesium wants to be able to ingest as many glTFs as possible. I would imagine ThreeJS has the same goal.

If we end up in a state where the typical glTF model is highly tuned for a particular target runtime, and can't be loaded by a generic ThreeJS glTF loader or a generic Cesium glTF loader, then glTF is not going to lay claim to being the "JPEG of 3D" as was mentioned at the Khronos BoF this year.

There are already 3D formats that target specific runtimes. There are already formats like OBJ that can deliver simple geometry to almost any 3D application, so long as you don't need PBR shading on such objects. I hope I'm not alone in expecting glTF to fill a long-standing gap in standards, delivering models with high-quality shading and animations to a range of generic 3D model loaders on multiple platforms.

[mlimper]

I hope I'm not alone in expecting glTF to fill a long-standing gap in standards, delivering models with high-quality shading and animations to a range of generic 3D model loaders on multiple platforms.

Agree to 100%!

glTF will be the "JPEG for 3D" - of maybe the "PDF for 3D" (although PDF has 3D... but that's a different topic ;- )... saying, it will be a standard 3D delivery format, where assets should be exchangeable across different rendering platforms and look as similar as possible.

[emackey]

So to bring this back on topic, let's consider the question of which PBR parameter set to use. I'm not an expert in these parameter sets, but in light of the above discussion, it seems clear that splitting them into two separate PBR extensions would introduce too much fragmentation. This fragmentation becomes a show-stopper issue if core GLSL is replaced by core PBR as proposed in #733. The remaining options then, as best I understand them, are:

• Option A, a single PBR extension supports two different parameter sets. glTF writer toolchains may choose to output only one of the two, but all generic glTF readers must be able to load and render both.
• Option B, the PBR experts here select one of the two parameter sets as the official one. glTF readers can then focus on a single parameter set, a great advantage for readers which may be on thin clients or in resource-constrained environments. However, glTF writers must be able to somehow adapt models that have been authored with alternate parameters, to fit the selected format.

Is there a simple conversion from one parameter set to the other, without loss of visual fidelity? If there's a straightforward conversion, then it makes sense to take option B: include that conversion in the glTF writers, making the job of the readers that much simpler, which is a key goal.

But if there's no deterministic conversion that preserves the artist's intent for the model (that is, if converting one to the other causes loss of perceived quality), then unfortunately we must consider asking readers to support both parameter sets. We would run the risk that some readers may choose to only implement one of the two, causing fragmentation.

[jeffdr]

It sounds like, given the goals of this format, option B is preferable. If that's the case, then I would feel fairly strongly that the metalness parameterization should not be used there (use specular/gloss instead). The reason being that it's fairly easy to convert metalness/roughness to specular/gloss, but the reverse is not always possible. The specular setup is the most general purpose; so if you're going with only one in the official specification, that should be it.

[significant-bit]

From a toolsmith's perspective:
If I have a Blender --> Unreal pipeline, and both are using the same parameters / material model, the asset delivery format should support that. Converting to other parameters on save then reverse converting them on load doesn't help me at all. Would this use case be better served with custom shaders instead of using a "standard PBR" extension?

Insisting glTF should support only one seems like insisting images should be either color or grayscale. The JPEG of 2D does both. QED

[jeffdr]

The idea, I suppose, is that you wouldn't be converting them on load - everyone writing apps conforming to the glTF spec would just use the specular model for their shaders, which would always match the content.

That's a problem for people who might have their own reasons to use metalness shaders at runtime (though off the top of my head I can't think of any), and so I'll have to defer to the group to decide whether supporting only one workflow is wise for glTF. But if it is going to be only one, it should be "specular".

[emackey]

Thanks @jeffdr for the conversion clarification.

@significant-bit I hadn't considered the case where a runtime actually wants the metal/rough parameters that are hard (impossible?) to convert to in a lossless way. Does the Unreal pipeline really require metal/roughness parameters, or enable some rendering with those parameters that can't be accomplished with spec/gloss? What would this pipeline do with glTF models that did use the spec/gloss parameters?

In general, it sounds to me like the primary PBR transmission format should be spec/gloss, with glTF authoring toolchains strongly encouraged to convert any metal/rough to spec/gloss parameters on the way through. I would imagine that most rendering engines, certainly the WebGL-based ones, would find the spec/gloss model more similar to classic rendering and have an easier time supporting that (remembering that we're trying to limit complexity of the loaders, since they may be on mobile devices etc).

If there are real, concrete advantages for some engines to rendering directly with metal/rough, then perhaps a separate metal/rough PBR extension is called for after all. This would enable glTF to target these runtimes directly, but again, would lead to fragmentation. In this case I would think that #733 would promote specifically the spec/gloss PBR to core glTF, and leave metal/rough as just an extension. The extension may not gain much traction unless it is trivial for the runtime to convert on-the-fly the metal/rough textures to spec/gloss. From WebGL's point of view, it's always better to preprocess what you can during authoring, rather than make the client do it at runtime.

What do folks think of this interpretation?

[vorg]

Hi guys, I've been following following the spec evolution since the begging and always was on the metallic side so i feel like it's the last chance to sum up what we are giving up here if we go with specular:

• I did a survey some time ago to check who is using what and metallic / metalness was in majority
  
  Notes:

  • Unity supports both models, specular and metallic - not shown in the table

  Update:

  • BabylonJS is using specular/glossines (named reflectivity/microsurface)
  • Substance that is often used to create PBR texture supports both too)
  • Sketchfab is using both

• As @jeffdr mentioned before, metallic workflow has memory benefits as it need just one channel with 0 or 1. All color (albedo + specular) are in the baseColor texture. Specular need both textures to be RGB.

• Yes, specular workflow avoids artifacts at the bounduaries between metal and dielectric, and allows bigger flexiblity but simplicity of mettalic makes it harder to create unrealistic materials

So as much as personally I prefer metalness as I think is more elegant conceptually (color + roughness + mask) vs specular workflow (diffuse colors + specular colors + inverted roughness) even if i'm giving up flexibility. Saying that I'll adapt as I want to build my tools around glTF not the other way around.

[emackey]

Interesting point that the metallic workflow saves two channels of texture data, that could make a compelling argument to support both.

Earlier in this thread, @cedricpinson posted this sample code that shows the choice of metal/rough vs spec/gloss as being just an extra 4 lines of fragment shader code inside a compiler directive. Is that really all it takes to support both? Not counting glTF parse code of course. So adding a couple lines of GLSL and using metallic workflow frees up two channels of texture memory?

[stephomi]

I would say I'm in favor of 2 workflows (in one single extension).
But if I'd have to choose for one workflow, I'd go for the metalness.

Metalness workflow saves 1 texture channel (and most of the time 2 channels since F0 channel is seldomly used).

Specular workflow takes more memory and allow for more freedom but is more prone to error (which is more prone to error energy conservation).
But it some cases it has better filtering (for example here you find similar comparison in the awesome allego pbr guide that everyone should read ) so rendering wise it's sometimes better.

would find the spec/gloss model more similar to classic rendering and have an easier time supporting that

It's similar only by the name, if you translate naively old classic rendering textures to a pbr specular workflow, you'll end up with a model way too bright.

Also about glossiness/roughness, as @vorg pasted in his survey, they are completely independant from the metalness/specular workflow.
UE4 uses metalness and roughness while other engine uses metal+glossiness.
Maybe the 2 channels should be present regardless of the workflow too (although mutally exclusive).

[FslNopper]

From my side, I prefer the metallic/roughness approach and should be standard to be glTF compliant. The specular/glossiness could/should be done as an extension.

I suggest to store the following PBR values. I do not go into the ambient, normal and other texture maps, as I think the approach is clear:

baseColor
metallic
roughness
f0_metal
f0_nonMetal

Furthermore, each of the above values can either be:

value (scalar, vector and so on)
or
texture

Regarding the texture, there are several access / sample methods:

Channels:
r, g, b, a
rgb, rgba
luminance

[mlimper]

Looks like Metallic-Roughness is really the most popular approach, and that it would allow us to cover the very most use cases directly:
#699 (comment)

My main point is that specular/glossiness is not an energy-conserving system in any real way, which to my mind disqualifies it from being called a PBR-based model.

And that's another good point.

The specular/glossiness could/should be done as an extension.

Agreed!

So, if no one will strongly protest within the next few weeks, we will consider this issue as closed and proceed by limiting ourselves, for now, to the metallic-roughness parameter set. We will figure out the exact set of maps for this model (#699). Specular/Glossiness could be added later on, as an(other) extension.

[jeffdr]

I can see the appeal to sticking with one or the other instead of supporting both, and I can buy that a majority of artists are using the Metalness workflow, so going with that simple approach makes some sense. You're shutting out some existing content from automatic conversion by doing this, but maybe that doesn't matter.

My main point is that specular/glossiness is not an energy-conserving system in any real way, which to my mind disqualifies it from being called a PBR-based model.

This is false, as I've stated before in this thread. The spec/gloss model does energy conservation just fine, and algebraically in almost the exact same way metalness does (the diffuse contribution is simply masked by the inverse of the reflectivity in the shader, and no special treatment by the artist is required). A PBR specular/gloss shader is sometimes confused in this way with "pre-PBR" shaders that also used spec and gloss maps without energy conservation.

[cedricpinson]

Agree with @jeffdr it's clearly easy to support both, not need to discard one.

[mlimper]

Alright, thanks for the quick feedback.

As we initially proposed to support both, we are, of course, also fine with that ;-)
I'd just like to make a decision soon, so that we can close this one.

@pjcozzi @erich666
Any thoughts?

[emackey]

+1 for both.

[pjcozzi]

Whatever folks decide is fine with me; doing one is clean, but doing both doesn't sound like an implementation burden.

I believe @sbtron has a demo showing the two shading models, and the difference was not significant. @sbtron can you comment and share a link?

[sbtron]

@bghgary has an example showing conversion between the two workflows: https://bghgary.github.io/glTF/convert-between-workflows/
Heres some example assets created with both workflows - https://sbtron.github.io/BabylonJS-glTFLoader/

@bghgary can comment on this more?

[bghgary]

@bghgary can comment on this more?

I agree with having both for PBR as an extension. However, when considering putting PBR into the core spec (#733), we should pick a workflow that is the min-bar to maximize adoption, which in this case is metallic-roughness. Specular-glossiness workflow can be an extension.

There are low-resource devices out there that will only support metallic-roughness since they have a hard requirement on the number of channels passed to the shaders to limit resource usage. These devices will not be able to support specular-glossiness without implementing a slow conversion on load.

Consider these scenarios:

1. Runtime implementation supports both. Anything we chose is fine.
2. Runtime implementation supports specular-glossiness only. Runtimes for low-resource devices can be modified to support metallic-roughness. It is relatively cheap to add support for metallic-roughness without introducing more resources.
3. Runtime implementation supports metallic-roughness only. Runtimes for low-resource devices cannot be easily modified to support specular-glossiness since it cannot handle the extra channels. The only other option I see is to convert to metallic-roughness on load. This will result in lossy conversion and longer load times if spec supports both and specular-glossiness is used, or the spec supports specular-glossiness only.

If only metallic-roughness is part of the core spec, we can require all runtimes to implement the core spec (metallic-roughness) and get consistent adoption. For scenarios that require specular-glossiness, which may not be implemented by all runtimes, they can use a specular-glossiness extension.

[pjcozzi]

Just so everyone is on the same page: we do intend to make PBR core spec ASAP.

@bghgary's proposal of metallic-roughness in core + specular-glossiness extension (that would come out at the same time) sounds pragmatic. I expect the extension would become widely supported.

I don't have the domain knowledge to contribute meaningfully here, but I'd like to ask that we aim to converge on this by next Monday, Jan 9 (just the core/extension decision, smaller spec details can converge later) so the Khronos folks can get all the spec work lined up.

@mlimper @jeffdr @erich666 @cedricpinson @emackey @FslNopper @vorg and anyone else?

[jeffdr]

I'm onboard with metallic/roughness + spec/gloss extension. The virtue of its simplicity probably outweighs the drawbacks.

[mlimper]

Fine with that!

[pjcozzi]

Given everyone's feedback above, we are going with @bghgary's proposal of metallic-roughness in core + specular-glossiness extension (that would come out at the same time) sounds pragmatic.

@mlimper let's finalize all the work in your current extension then we will cut-and-paste-ish part of it to core spec. OK?

[mlimper]

Yes, let's do it like that.

[xelatihy]

@pjcozzi @mlimper One more opinion on this.

While I realize that this is close, there is a significant problem with using metallic-roughness in that many materials cannot be represented this way, while the can be represented using specular-glossiness.

For example all models saved as OBJ cannot be converted to PBRT with metallic-roughness, so I am not sure that adoption is better if we have very little models as input.

I agree with @bghgary about the low performance devices, but the question is how many of those devices will around in a couple of years when glTF will become (I hope) mainstream. Also consider that to render PBR materials accurately there are many more resources needed for shadowing computation compare to material shading.

Furthermore, although this might not be the of interests here, most academic literature as shown that diffuse-specular cover a significant space of useful materials, while metallic-roughness is only halpful for editing at the price of subsetting the representable materials substantially.

For example, many materials fitted from real measurement cannot be projected onto metallic-roughness without loosing some look.

I hope this decision is not finalized yet.

[xelatihy]

I would pur both parameter set and have the asset author choose which one. This has the benefit of @bghgary for metallic-roughness, as well as can represent most materials using spcular-glossiness, without subsetting the materials types supported.

[emackey]

@xelatihy point of clarification. It's been explained (to me, among other people) a few times in the PBR discussions in this repo that the PBR version of albedo/spec/gloss is not at all the same shading model as a traditional diffuse/ambient/spec/gloss. To support things like classic OBJ models, we would use the KHR_materials_common extension, with the traditional shading model, not PBR's spec/gloss extension.

[emackey]

Perhaps this warrants a non-normative note in the spec, as I imagine many folks encountering PBR would initially scratch their heads wondering why spec/gloss was relegated to an extension.

[FslNopper]

@xelatihy Low power (in any meaning) devices will probably always be around. In automotive, there will be still a lot of such devices. Also, some Internet of Things devices will also have displays and they will be for sure low power devices.

[ghost]

I agree with all people pushing to support both metal + spec workflow by default. All big player like UE / Unity / Sketchfab / Marmoset / Substance support both. Pushing by default in one direction, is not the best way imo to build a good agnostic specs.

[pjcozzi]

We are collecting requirements for next-gen PBR in glTF, please chime in to #1442.