Implement texture mipmap generation via compute pipelines #5757
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Direct3D 11 doesn't like reading from "image2D"-like types with complicated formats like rgba8.
We want compatibility with 10_0 feature levels, which use Shader Model 4 (cs_4_0). Shader Model 4 only supports UAV resources of type `RWByteAddressBuffer` or `RWStructuredBuffer` (i.e. structured buffers).
The overhead required to make this work on older Direct3D versions is quite bad compared to using the framebuffer method or driver implementation.
Metal mostly support this (needs testing on multiple devices for confirmation), and D3D11 also supports this according to documentations (cs_5_0 profile only).
It's incorrect for texture dimensions that are not a multiple of the threadgroup size. Using texture width is more accurate.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Preface
Since #5508, custom mipmap generation was implemented to reduce the overhead in the case of uploading small textures to an atlas.
The implementation revolved around binding one level of a texture as a sampler and another level as a render target (FBO), and using
textureto downsample the larger texture and store the result in the smaller texture.This was pretty straightforward for OpenGL and Metal, but D3D11 and Vulkan weren't very supportive. Both of them don't seem to like the idea of binding the same texture as both an input and output of the shader.
That was worked around by allocating another texture to bind as the input for the shader, but that required preparing the texture content for each level, which means switching between encoders on Metal (and Veldrid's Vulkan implementation doesn't handle that well?).
Compute-based mipmap generation
After exploring through compute shaders for the past week, I've came up with a simple implementation that avoids creating extra resources as much as possible.
The concept of it boils down to a compute shader accepting an input texture, a linear sampler, and a uniform buffer supplying the uploaded region. The compute shader downsamples from the texture according to the specified uploaded region and the current invocation ID, and stores the result in the output texture.
For Veldrid, this is implemented by splitting each texture into texture views in
VeldridTextureResources, bind one view for sampling and another for output, and supply a re-usable uniform buffer with region data.For Legacy OpenGL, this is implemented similarly, but texture views aren't required as each texture resembles a sampler by itself, so the properties can be modified accordingly for each dispatch call without creating extra resources.
Compute shaders is shown to be supported by 80% of systems according to this game, but the rendering-based path still exists as a fall back for older hardware since it still behaves 10x better than driver implementation. It can be further improved by using texture views and caching VBOs, but I'm leaving it as-is in this PR.
Better rectangle merging implementation
After writing a test scene for mipmap generation, it turns out there was a high overhead coming from the rectangle merging logic added in #5508.
The loop works well for a few overlapping regions, but can get very complex when queuing a high number of uploads at different regions.
The overhead can get as high as ~500ms, which is 100x higher than the overhead of mipmap generation itself.
After discussing this on discord with @smoogipoo, I've came up with a different implementation that follows the nature of texture atlases and produces as less rectangles as possible to simplify the mipmap generation process.
The implementation accepts all uploaded regions, and produces one or two rectangles, one covering all uploaded regions that are horizontally after the top-left uploaded region, and another rectangle covering all uploaded regions that are horizontally before the top-left uploaded region.
(each highlighted box represents a region that's worked on by the mipmap generation logic)
On master, mipmaps would be generated for every single uploaded data (including the paddings around the texture) as can be seen above, this resulted in too many vertices and unnecessary fills.
With the new implementation, all uploaded regions are formed into two rectangles to dispatch as many threads as possible without overlapping / unnecessary work.
The new implementation is also ~75x faster than the original logic, which should reduce stutters in osu! when starting gameplay, as there are a lot of textures getting added to the atlas at once.
As an aside, I have noticed both the computing and rendering implementations missing certain textures when testing with osu!, I'll leave this PR blocked from merge for now until I have the time to investigate it or someone else can look into it.
Requires testing on all configurations below:
GL_ARB_compute_shader)GL_ARB_compute_shaderandGL_ARB_texture_view)GL_ARB_compute_shader)GL_ARB_compute_shaderandGL_ARB_texture_view)GL_ARB_compute_shader)GL_ARB_compute_shader)GL_ARB_compute_shaderandGL_ARB_texture_view)Optional configurations: