Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to prefiltered environment maps #1420

Merged
Merged
Changes from 1 commit
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
3996bf5
Update .gitignore with MaterialXView's CMake-generated files
ApoorvaJ Jun 28, 2023
dd47637
Update .gitignore with compilation artifacts
ApoorvaJ Jun 28, 2023
3013728
Add some XCode files to the .gitignore
ApoorvaJ Jun 29, 2023
266385c
Extend the image class to allow uploading custom mipmaps
ApoorvaJ Jun 30, 2023
8ae3d94
Implemented GPU rendering to the different mips
ApoorvaJ Jul 3, 2023
76d314d
Add an environment pre-convolution shader
ApoorvaJ Jul 4, 2023
fb312d3
Implement per-mip logic for the convolution shader
ApoorvaJ Jul 4, 2023
fb55bd1
Inverse equirectangular lookup
ApoorvaJ Jul 5, 2023
687da38
WIP convolution shader
ApoorvaJ Jul 5, 2023
c6493bf
Finish the convolution shader
ApoorvaJ Jul 5, 2023
fa7d436
Increase sample count
ApoorvaJ Jul 5, 2023
fb08a15
Fix the broken FIS sampling, since it relies on the environment map b…
ApoorvaJ Jul 5, 2023
96f2520
Change convolved env texture format from RGBA8Unorm to RGB16Float
ApoorvaJ Jul 6, 2023
ee5090b
Fix GLSL broadcasting error
ApoorvaJ Jul 6, 2023
5b793a1
WIP implement OpenGL path
ApoorvaJ Jul 6, 2023
afc608e
Removed some unused code
ApoorvaJ Jul 6, 2023
55d9822
Undo the addition of a "useCustomMipMaps" bool on Images
ApoorvaJ Jul 6, 2023
9d7dedc
Bind uniforms to the opengl impl
ApoorvaJ Jul 6, 2023
1cc0004
Fix OpenGL uniform upload
ApoorvaJ Jul 6, 2023
26bbc3d
Delete temp code
ApoorvaJ Jul 7, 2023
5cd4526
Add back direct lighting UI
ApoorvaJ Jul 7, 2023
aaeb75c
Implement lazy-generation of the environment preconvolution
ApoorvaJ Jul 7, 2023
dd307fe
Call the lazy convolver in the OpenGL path
ApoorvaJ Jul 7, 2023
4ad2868
Hoist metal shader generation outside the mip loop
ApoorvaJ Jul 11, 2023
9b66cae
Undo changes to gitignore
ApoorvaJ Jul 19, 2023
87d9bdb
Clean up OpenGL code
ApoorvaJ Jul 19, 2023
6ff67a9
Remove unused comment
ApoorvaJ Jul 19, 2023
e57f920
Revert unncessary change
ApoorvaJ Jul 19, 2023
23d1871
Revert unused change in header file
ApoorvaJ Jul 19, 2023
5439181
Remove custom GGX function in favor of built-in one
ApoorvaJ Sep 18, 2023
14ab9e9
Lazy-init convolution on OpenGL, and don't rebuild the map afterwards
ApoorvaJ Sep 18, 2023
d40e456
Reorder constructor initialization to get rid of warnings
ApoorvaJ Sep 19, 2023
e71ea99
Add a separate param for whether a texture is used as a render target
ApoorvaJ Sep 19, 2023
e006bb0
Attempt to fix the VSCode warning 'unreferenced formal parameter'
ApoorvaJ Sep 19, 2023
7e8bd86
Merge branch 'main' into fix-split-sum-ibl
jstone-lucasfilm Dec 15, 2023
e9bf596
Merge branch 'main' into fix-split-sum-ibl
jstone-lucasfilm Dec 16, 2023
bfd659d
Merge branch 'main' into fix-split-sum-ibl
jstone-lucasfilm Dec 23, 2023
6de22f3
Restore original mx_environment_radiance
jstone-lucasfilm Dec 23, 2023
31c606d
Minor naming update
jstone-lucasfilm Dec 23, 2023
12ef1d8
Minor formatting updates
jstone-lucasfilm Dec 23, 2023
86c89ce
Minor naming updates
jstone-lucasfilm Dec 23, 2023
5735998
Additional naming updates
jstone-lucasfilm Dec 23, 2023
983d05f
Harmonize on prefilter over preconvolve
jstone-lucasfilm Dec 23, 2023
f0e5eef
Additional terminology updates
jstone-lucasfilm Dec 24, 2023
41b7d07
Add virtual method
jstone-lucasfilm Dec 24, 2023
e5738d0
Clarify shader variable names
jstone-lucasfilm Dec 24, 2023
7ff97b0
Harmonize GGX importance sampling
jstone-lucasfilm Dec 24, 2023
207b4eb
Simplify shader math
jstone-lucasfilm Dec 24, 2023
3f3f7aa
Clarify shader math
jstone-lucasfilm Dec 26, 2023
8b74333
Harmonize on VNDF sampling
jstone-lucasfilm Dec 26, 2023
9613f48
Standardize shader math
jstone-lucasfilm Dec 26, 2023
eacae2e
Update orthonormal basis logic
jstone-lucasfilm Dec 26, 2023
31f35e7
Use filtered environment samples
jstone-lucasfilm Dec 26, 2023
0711edd
Refine documentation for new functions
jstone-lucasfilm Dec 26, 2023
2ac301c
Minor clarifications
jstone-lucasfilm Dec 26, 2023
7ece761
Reduce sample count
jstone-lucasfilm Dec 26, 2023
ac4a526
Update documentation
jstone-lucasfilm Dec 26, 2023
0763963
Clarify function names
jstone-lucasfilm Dec 26, 2023
1bfd917
Update documentation
jstone-lucasfilm Dec 26, 2023
1348ece
Minor shader reorganization
jstone-lucasfilm Dec 29, 2023
0dd8059
Fix off-by-one error
jstone-lucasfilm Dec 29, 2023
f2b0e37
Add default arguments
jstone-lucasfilm Dec 29, 2023
cf9e332
Remove unused initializer
jstone-lucasfilm Dec 29, 2023
00c47a6
Restore original getMaxMipCount
jstone-lucasfilm Dec 29, 2023
d82c08b
Increase sample count for accuracy
jstone-lucasfilm Dec 29, 2023
ab08644
Fix warning message
jstone-lucasfilm Dec 29, 2023
6c77f06
Clarify comment
jstone-lucasfilm Dec 29, 2023
2b8605f
Remove extra newline
jstone-lucasfilm Dec 29, 2023
5145328
Add missing initializer
jstone-lucasfilm Dec 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Undo the addition of a "useCustomMipMaps" bool on Images
Remove function call in OpenGL
ApoorvaJ committed Sep 18, 2023
commit 55d9822926ca920fe9a1f7e3157427c928b987ff
19 changes: 2 additions & 17 deletions source/MaterialXRender/Image.cpp
Original file line number Diff line number Diff line change
@@ -89,12 +89,11 @@ UnsignedIntPair getMaxDimensions(const vector<ImagePtr>& imageVec)
// Image methods
//

Image::Image(unsigned int width, unsigned int height, unsigned int channelCount, BaseType baseType, bool useCustomMipMaps) :
Image::Image(unsigned int width, unsigned int height, unsigned int channelCount, BaseType baseType) :
_width(width),
_height(height),
_channelCount(channelCount),
_baseType(baseType),
_useCustomMipMaps(useCustomMipMaps),
_resourceBuffer(nullptr),
_resourceBufferDeallocator(nullptr),
_resourceId(0)
@@ -506,21 +505,7 @@ void Image::writeTable(const FilePath& filePath, unsigned int channel)
void Image::createResourceBuffer()
{
releaseResourceBuffer();
size_t numTexels = 0;
if (_useCustomMipMaps)
{
int w = _width;
int h = _height;
while (w > 0 && h > 0) {
numTexels += w * h;
w /= 2;
h /= 2;
}
}
else
{
numTexels = _width * _height;
}
size_t numTexels = _width * _height;
_resourceBuffer = malloc(numTexels * _channelCount * getBaseStride());
_resourceBufferDeallocator = nullptr;
}
13 changes: 3 additions & 10 deletions source/MaterialXRender/Image.h
Original file line number Diff line number Diff line change
@@ -59,11 +59,10 @@ class MX_RENDER_API Image
unsigned int width,
unsigned int height,
unsigned int channelCount,
BaseType baseType = BaseType::UINT8,
bool useCustomMipMaps = false
BaseType baseType = BaseType::UINT8
)
{
return ImagePtr(new Image(width, height, channelCount, baseType, useCustomMipMaps));
return ImagePtr(new Image(width, height, channelCount, baseType));
}

~Image();
@@ -95,11 +94,6 @@ class MX_RENDER_API Image
return _baseType;
}

bool getUseCustomMipMaps() const
{
return _useCustomMipMaps;
}

/// Return the stride of our base type in bytes.
unsigned int getBaseStride() const;

@@ -218,14 +212,13 @@ class MX_RENDER_API Image
/// @}

protected:
Image(unsigned int width, unsigned int height, unsigned int channelCount, BaseType baseType, bool useCustomMipMaps);
Image(unsigned int width, unsigned int height, unsigned int channelCount, BaseType baseType);

protected:
unsigned int _width;
unsigned int _height;
unsigned int _channelCount;
BaseType _baseType;
bool _useCustomMipMaps;

void* _resourceBuffer;
ImageBufferDeallocator _resourceBufferDeallocator;
79 changes: 26 additions & 53 deletions source/MaterialXRenderMsl/MetalTextureHandler.mm
Original file line number Diff line number Diff line change
@@ -148,13 +148,10 @@
texDesc.width = image->getWidth();
texDesc.height = image->getHeight();
texDesc.mipmapLevelCount = generateMipMaps ? image->getMaxMipCount() : 1;
texDesc.usage = MTLTextureUsageShaderRead |
// For now, we set generate mip maps flag off,
// when we want to use the texture as render target
((!generateMipMaps || image->getUseCustomMipMaps()) ? MTLTextureUsageRenderTarget : 0);
texDesc.usage = MTLTextureUsageShaderRead | MTLTextureUsageRenderTarget;
texDesc.resourceOptions = MTLResourceStorageModePrivate;
texDesc.pixelFormat = pixelFormat;
if(generateMipMaps && !image->getUseCustomMipMaps())
if(generateMipMaps)
{
if(image->getChannelCount() == 1)
{
@@ -193,40 +190,30 @@

NSUInteger channelCount = image->getChannelCount();

int numBytesPerTexel = channelCount * getTextureBaseTypeSize(image->getBaseType());
int numUploadMips = image->getUseCustomMipMaps() ? image->getMaxMipCount() : 1;
size_t numTexelsAcrossMips = 0;
if (image->getUseCustomMipMaps())
{
int w = image->getWidth();
int h = image->getHeight();
while (w > 0 && h > 0) {
numTexelsAcrossMips += w * h;
w /= 2;
h /= 2;
}
}
else
{
numTexelsAcrossMips = image->getWidth() * image->getHeight();
}
NSUInteger sourceBytesPerRow =
image->getWidth() *
channelCount *
getTextureBaseTypeSize(image->getBaseType());
NSUInteger sourceBytesPerImage =
sourceBytesPerRow *
image->getHeight();

std::vector<float> rearrangedDataF;
std::vector<unsigned char> rearrangedDataC;
void* imageData = image->getResourceBuffer();


if ((pixelFormat == MTLPixelFormatRGBA32Float || pixelFormat == MTLPixelFormatRGBA8Unorm) && channelCount == 3)
{
bool isFloat = pixelFormat == MTLPixelFormatRGBA32Float;

numBytesPerTexel = numBytesPerTexel / 3 * 4;
sourceBytesPerRow = sourceBytesPerRow / 3 * 4;
sourceBytesPerImage = sourceBytesPerImage / 3 * 4;

size_t srcIdx = 0;

if(isFloat)
{
rearrangedDataF.resize(numTexelsAcrossMips * numBytesPerTexel / sizeof(float));
rearrangedDataF.resize(sourceBytesPerImage / sizeof(float));
for(size_t dstIdx = 0; dstIdx < rearrangedDataF.size(); ++dstIdx)
{
if((dstIdx & 0x3) == 3)
@@ -242,7 +229,7 @@
}
else
{
rearrangedDataC.resize(numTexelsAcrossMips * numBytesPerTexel);
rearrangedDataC.resize(sourceBytesPerImage);
for(size_t dstIdx = 0; dstIdx < rearrangedDataC.size(); ++dstIdx)
{
if((dstIdx & 0x3) == 3)
@@ -263,36 +250,22 @@
id<MTLBuffer> buffer = nil;
if(imageData)
{
buffer = [_device
newBufferWithBytes:imageData
length:(numTexelsAcrossMips * numBytesPerTexel)
options:MTLStorageModeShared];
int width = image->getWidth();
int height = image->getHeight();
int sourceOffset = 0;
for (int mipLevel = 0; mipLevel < numUploadMips; mipLevel++)
{
NSUInteger sourceBytesPerRow = width * numBytesPerTexel;
NSUInteger sourceBytesPerImage = height * sourceBytesPerRow;
[blitCmdEncoder
copyFromBuffer:buffer
sourceOffset:sourceOffset
sourceBytesPerRow:sourceBytesPerRow
sourceBytesPerImage:sourceBytesPerImage
sourceSize:MTLSizeMake(width, height, 1)
toTexture:texture
destinationSlice:0
destinationLevel:mipLevel
destinationOrigin:MTLOriginMake(0, 0, 0)];
sourceOffset += width * height * numBytesPerTexel;
width = width / 2;
height = height / 2;
}
buffer = [_device newBufferWithBytes:imageData
length:sourceBytesPerImage
options:MTLStorageModeShared];
[blitCmdEncoder copyFromBuffer:buffer sourceOffset:0
sourceBytesPerRow:sourceBytesPerRow
sourceBytesPerImage:sourceBytesPerImage
sourceSize:MTLSizeMake(image->getWidth(), image->getHeight(), 1)
toTexture:texture
destinationSlice:0
destinationLevel:0
destinationOrigin:MTLOriginMake(0, 0, 0)];
}

if(generateMipMaps && !image->getUseCustomMipMaps() && image->getMaxMipCount() > 1)
if(generateMipMaps && image->getMaxMipCount() > 1)
[blitCmdEncoder generateMipmapsForTexture:texture];

[blitCmdEncoder endEncoding];

[cmdBuffer commit];
3 changes: 0 additions & 3 deletions source/MaterialXRenderMsl/MslPipelineStateObject.mm
Original file line number Diff line number Diff line change
@@ -608,9 +608,6 @@ int GetStrideOfMetalType(MTLDataType type)
samplingProperties.vaddressMode = ImageSamplingProperties::AddressMode::CLAMP;
samplingProperties.filterType = ImageSamplingProperties::FilterType::LINEAR;

// This `bindImage` call is actually uploading the image on the first call,
// when the image doesn't exist on the GPU. We need to upload custom mips
// during this stage.
static_cast<MaterialX::MetalTextureHandler*>
(imageHandler.get())->bindImage(env.second, samplingProperties);
bindTexture(renderCmdEncoder, (unsigned int)arg.index, env.second, imageHandler);
2 changes: 1 addition & 1 deletion source/MaterialXView/RenderPipelineGL.cpp
Original file line number Diff line number Diff line change
@@ -136,7 +136,7 @@ mx::ImagePtr GLRenderPipeline::convolveEnvironment()
int h = srcTex->getHeight();

// Create texture to hold the convolved environment.
mx::ImagePtr outTex = mx::Image::create(w, h, 3, mx::Image::BaseType::HALF, true);
mx::ImagePtr outTex = mx::Image::create(w, h, 3, mx::Image::BaseType::HALF);
glImageHandler->createRenderResources(outTex, true); // TODO: Is this needed?


4 changes: 2 additions & 2 deletions source/MaterialXView/RenderPipelineMetal.mm
Original file line number Diff line number Diff line change
@@ -166,8 +166,8 @@
int w = srcTex->getWidth();
int h = srcTex->getHeight();

mx::ImagePtr outTex = mx::Image::create(w, h, 3, mx::Image::BaseType::HALF, true);
mtlImageHandler->createRenderResources(outTex, true); // TODO: Is this needed?
mx::ImagePtr outTex = mx::Image::create(w, h, 3, mx::Image::BaseType::HALF);
mtlImageHandler->createRenderResources(outTex, true);
id<MTLTexture> metalTex = mtlImageHandler->getAssociatedMetalTexture(outTex);