Validation layer complains about unprovided input which is unused due to specialization constants

[DataBeaver]

Describe the Issue
The attached SPIR-V module declares a vertex shader input "tangent" which is used for normal mapping. This feature is controlled by the specialization constant "use_normal_map", which defaults to false. Even when not enabled, the validation layer gives this message:

Validation Error: [ UNASSIGNED-CoreValidation-Shader-InputNotProduced ] Object 0: handle = 0x100000000010, type = VK_OBJECT_TYPE_SHADER_MODULE; | MessageID = 0x23e43bb7 | Vertex shader consumes input at location 3 but not provided

phong.zip

Environment:

• OS: Debian unstable
• GPU: GeForce GTX 980Ti
• Options enabled (synchronization, best practices, etc.): Debian defaults

Additional context
The SPIR-V module is compiled with my custom shader compiler. The shaders it produces pass spirv-val and work in an OpenGL application, but compiler bugs are not impossible.

My Vulkan rendering code is also a work in progress so I haven't yet been able to test if the shader works in Vulkan despite the validation error.

@sfricke-samsung

[sfricke-samsung]

What value are you passing into the specialized constants where you are getting the unwanted validation error warning?

[DataBeaver]

Input to my shader class:

(gdb) print spec_values
$18 = std::map with 9 elements = {["use_diffuse_map"] = 0, ["use_emission"] = 0, ["use_emission_map"] = 0, ["use_normal_map"] = 0, 
  ["use_reflectivity"] = 0, ["use_reflectivity_map"] = 0, ["use_shininess_map"] = 0, ["use_specular"] = 0, 
  ["use_specular_map"] = 0}

Mapped to Vulkan structures:

(gdb) print *spec_info
$21 = {mapEntryCount = 9, pMapEntries = 0x555555924050, dataSize = 36, pData = 0x5555559240e0}
(gdb) print *spec_info->pMapEntries@9
$22 = {{constantID = 815633763, offset = 0, size = 4}, {constantID = 1389555148, offset = 4, size = 4}, {constantID = 652242916, 
    offset = 8, size = 4}, {constantID = 1191522301, offset = 12, size = 4}, {constantID = 1633307910, offset = 16, size = 4}, {
    constantID = 2119707162, offset = 20, size = 4}, {constantID = 1140758303, offset = 24, size = 4}, {constantID = 1418341623, 
    offset = 28, size = 4}, {constantID = 1119793582, offset = 32, size = 4}}
(gdb) x/9dd spec_info->pData
0x5555559240e0: 0       0       0       0
0x5555559240f0: 0       0       0       0
0x555555924100: 0

[ncesario-lunarg]

At first glance, this sounds similar to #3477. i.e., if "dead code" is triggering the error--and that "dead code" can be identified via a constant/specialization constant, then #3499 should resolve it.

[ncesario-lunarg]

@sfricke-samsung if you think the "eliminate dead branches" spirv-opt pass will not help here, I'll go ahead and assign this to you per offline discussion.

[DataBeaver]

I cloned the repository, merged #3499, and built my own copy of the validation layers. However I'm still getting validation errors. I've also improved my own code to detect which bindings are actually used based on the control flow graph and specialization constants and not include unused ones in descriptor set layouts. Due to this change there's more errors than before.

Validation Error: [ VUID-VkGraphicsPipelineCreateInfo-layout-00756 ] Object 0: handle = 0x55966134d0d8, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x45717876 | Shader uses descriptor slot 0.11 (expected VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER, VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER_DYNAMIC, VK_DESCRIPTOR_TYPE_INLINE_UNIFORM_BLOCK_EXT) but not declared in pipeline layout The Vulkan spec states: layout must be consistent with all shaders specified in pStages (https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#VUID-VkGraphicsPipelineCreateInfo-layout-00756)

This message repeats for descriptor slot 0.42. These errors correspond to ShadowMap and EnvMap uniform blocks.

Validation Error: [ VUID-VkGraphicsPipelineCreateInfo-layout-00756 ] Object 0: handle = 0x55966134d0d8, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x45717876 | Shader uses descriptor slot 0.70 (expected VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER) but not declared in pipeline layout The Vulkan spec states: layout must be consistent with all shaders specified in pStages (https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#VUID-VkGraphicsPipelineCreateInfo-layout-00756)

This message repeats with descriptor slots 0.16, 0.66, 0.73, 0.17, 0.12, 0.27 and 0.58. These correspond to various textures.

Validation Error: [ UNASSIGNED-CoreValidation-Shader-InputNotProduced ] Object 0: handle = 0x980f360000000011, type = VK_OBJECT_TYPE_SHADER_MODULE; | MessageID = 0x23e43bb7 | Vertex shader consumes input at location 3 but not provided

This is the same as in my original report

Validation Error: [ VUID-vkCmdDrawIndexed-None-02699 ] Object 0: handle = 0x980b0000000002e, type = VK_OBJECT_TYPE_DESCRIPTOR_SET; | MessageID = 0xa44449d4 | VkDescriptorSet 0x980b0000000002e[] encountered the following validation error at vkCmdDrawIndexed time: Attempting to validate DrawState for binding #11 which is an invalid binding for this descriptor set. The Vulkan spec states: Descriptors in each bound descriptor set, specified via vkCmdBindDescriptorSets, must be valid if they are statically used by the VkPipeline bound to the pipeline bind point used by this command (https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#VUID-vkCmdDrawIndexed-None-02699)

This repeats on every draw command using the shader.

I checked from /proc//maps that the program is actually using my custom version of the validation layer rather than the system version. It's still possible I messed something up in the compilation process.

Just in case it might be useful, here's a RenderDoc capture using the shader: vulkan_capture.zip
The render output is a bit glitchy because it's drawing directy on the swapchain image without a depth buffer. However that should be completely unrelated to these validation errors.

[ncesario-lunarg]

Thanks for the capture!

One thing to note with #3499 is that spirv-val will run more shaders than before. That said, I don't think that in and of itself should be causing the additional errors you're seeing.

[DataBeaver]

No, the additional validation errors are due to my own code changes. At the time of originally reporting the issue my code created descriptor set layouts including all resources declared by the shader, so the validation layer's view of what the shader uses and what the descriptor set layout provides matched. But now my code detects which resources are actually used by the shader based on specialization constant values and ignores unused resources. The validation layer still thinks them to be used though and gives errors because they're not included in the descriptor set layouts.

My understanding is that since the shader does not statically use those resources, I do not have to declare them in the descriptor set layouts[1]. So those validation errors seem to be false positives as well and probably share the same root cause of lacking dead code elimination. Why #3499 did not fix that I do not know.

[1]There's a possibility this makes pipeline switching more expensive because there will be more different layouts used by the program, but that's a different topic.

[ncesario-lunarg]

Why #3499 did not fix that I do not know.

That's a good question. I'm hoping to work on this a bit more today and will also try to run your capture as well.

[sfricke-samsung]

So finally got time to look at this... a few things:

I do agree that spirv-opt should be smart enough when %use_normal_map is set to false that it could eliminate the %56 block

---

I am just trying to do this with spirv-opt without the Validation Layers (as we can figure out where the mis-match is later)... but this is not a valid Vulkan SPIR-V module

VUID-StandaloneSpirv-OriginLowerLeft-04653
The OriginLowerLeft execution mode must not be used; fragment entry points must declare OriginUpperLeft

You need to make sure it this correct (glslangValidator has a --invert-y flag that will flip the values for you so you can keep your GLSL the same)

---

Even if I manually change it to OriginUpperLeft I am getting errors because %world_obj_matrix is a UniformConstant which means it is being used with a STORAGE_IMAGE or SAMPLER/SAMPLED_IMAGE so not sure what is happening here as its just a matrix of floats

VUID-StandaloneSpirv-UniformConstant-04655
Any variable in the UniformConstant storage class must be typed as either OpTypeImage, OpTypeSampler, OpTypeSampledImage, OpTypeAccelerationStructureKHR, or an array of one of these types

---

My next goal is to make a reduced SPIR-V module with the same high level goal of testing a OpSpecConstantFalse variable set to false can optimize the input from being used

[sfricke-samsung]

I also want to note that as an app, you can do what ever you want to your SPIR-V prior to sending it to through the validation layers (and the driver in general), thinking more about it, I don't see this actually being a "validation layer bug" and more of an issue that your GLSL-to-SPIR-V workflow doesn't have a way to eliminate the unwanted branches (but more importantly, your workflow should remove the OpVariable so the validation layers don't see it in the first place)

My bet is spirv-opt is not doing it because it is an Interface variable (its in OpEntryPoint)

[DataBeaver]

Sorry, I must have accidentally compiled the module in OpenGL mode. Can you try with this instead: phong.zip

This gets a pass from spirv-val --target-env vulkan1.2 phong.spv. It should be the same as the one used in the RenderDoc capture.

[DataBeaver]

I also want to note that as an app, you can do what ever you want to your SPIR-V prior to sending it to through the validation layers (and the driver in general), thinking more about it, I don't see this actually being a "validation layer bug" and more of an issue that your GLSL-to-SPIR-V workflow doesn't have a way to eliminate the unwanted branches (but more importantly, your workflow should remove the OpVariable so the validation layers don't see it in the first place)

My bet is spirv-opt is not doing it because it is an Interface variable (its in OpEntryPoint)

My compiler is capable of eliminating those variables with a dead code elimination pass, however it means I will have to apply specialization constants before compiling the code into SPIR-V. I already do this for GLSL shaders in OpenGL, since those don't support specialization constants in the first place. It'll be some extra work if I want to ship binary shaders since I'll have to add parsing for either SPIR-V or some custom format, but that's doable too.

You are right that strictly speaking Vulkan's static use rules say that it's determined solely by the presence of instructions, not whether those instructions can get executed. Is this not an intended use for specialization constants then? If I can't use them to control shader features like this, what can I use them for?

[DataBeaver]

This is getting increasingly tangential, but upon thinking on it further I realized I won't actually have to convert SPIR-V back to an AST in order to apply specialization. Since I can already process the CFG enough to recognize which variables are used, it should be fairly straightforward to emit modified SPIR-V where the dead parts have been removed. So I suppose that's the way to go then.

[sfricke-samsung]

Is this not an intended use for specialization constants then? If I can't use them to control shader features like this, what can I use them for?

No you have a point... I think the issue is how spec constant are suppose to be used for things such as branch dead code elimination such in the case you want... let me poke around some other people who might have another point of view on this matter

[sfricke-samsung]

@DataBeaver so finally got back to looking at spec constants and when asked inside the Vulkan WG I was pointed to this issue KhronosGroup/Vulkan-Docs#1671 on how for things under "static use rules" rules can't assume that compilers will do dead code elimination

[DataBeaver]

Yeah, I kinda figured that. And as I thought, it was a fairly simple affair to take an already compiled SPIR-V module and strip out any CFG blocks which can be determined to never be visited due to specialization constants. I'm even stripping out the declarations of now-unused variables, though no further than that (unused types and functions are left in). This keeps the validation layer happy while avoiding the need to pre-generate all possible combinations of spec constant values. I guess my use case is too complex for Vulkan's spec constant API.

[spencer-lunarg]

closing, explained above about spec constants