Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add GL3Plus support for macOS #209

Closed
wants to merge 1 commit into from
Closed

add GL3Plus support for macOS #209

wants to merge 1 commit into from

Conversation

srmainwaring
Copy link
Contributor

This PR is to port GL3Plus from Ogre2.1 to Ogre2.2 for macOS.

This is a work in progress. A number of the tutorials and samples are running with some rendering errors on the more advanced techniques. I have a basic knowledge of OpenGL but am not an expert and would appreciate a review by the Ogre Team to help with the port.

I've reworked my initial changes to follow the fall back code implemented for GL_ARB_texture_storage extensions in the now deprecated OgreGL3PlusTexture class (although for the card I am using with macOS Big Sur 11.4 this extension is actually supported even though GL4.2 is not).

My main concern is the implementation of GL3PlusTextureGpu::copyTo where I am not sure if I am using staging textures correctly (or whether this is the recommended approach).

Questions / To do

  1. Commits need to be squashed (and rebased if necessary)

  2. Window resizing is not correctly handled in the Cocoa window (but it is not crashing)

  3. GL3PlusTextureGpu::createInternalResourcesImpl

    • check compressed texture fallback
    • check use of PixelFormatGpuUtils::getSizeBytes
    • check converting from depthOrSlices to depth and slices
    • see if legacy code can be compacted (combine treatment of compressed/uncompressed textures)

  4. GL3PlusTextureGpu::copyTo

    • check correct usage of staging texture
    • understand how to handle sub images in fallback code (i.e. src.X != 0, etc)

  5. GL3PlusTextureGpu::copyViaFramebuffer

    • understand why this is not used to implement fall back case in GL3PlusTextureGpu::copyTo
    • understand why the case !srcIsFboAble && dstIsFboAble is not handled

Samples / Tutorials

The following tutorials are running with warnings about various missing vertex shader attributes, but otherwise appear to be running correctly.

  • Sample_Tutorial01_Initialization
  • Sample_Tutorial02_VariableFramerate
  • Sample_Tutorial03_DeterministicLoop

Other samples:

  1. Sample_StereoRendering

    • The last two rows of cubes may appear / disappear depending on the camera position

  2. Sample_DynamicGeometry

    • Appears ok

  3. Sample_PbsMaterials

    • Textures are not correctly rendered on the floor / cubes

  4. Sample_ShadowMapDebugging

    • The debug windows are rendering red rather than black

  5. Sample_Refractions

    • Takes a very long time to compile shaders

  6. Sample_Decals

    • Exits with error: Fragment Program 100000005PixelShader_ps failed to compile

@darksylinc
Copy link
Member

darksylinc commented Jul 4, 2021 •
edited
Loading

Hi!

When I said you could implement your own gl3LegacyTexStorage1D (which btw could also handle compressed formats too!) to simplify the code I actually meant this:

glTexStorage1D = gl3LegacyTexStorage1D;

// This is possible because we use gl3w where the GL function is actually a macro:
#define glTexStorage1D gl3wProcs.gl.TexStorage1D

// where gl3wProcs.gl.TexStorage1D is a C function pointer loaded dynamically at runtime:

typedef void (APIENTRYP PFNGLTEXSTORAGE1DPROC) (GLenum target, GLsizei levels, GLenum internalformat, GLsizei width);
union GL3WProcs {
	GL3WglProc ptr[772];
        struct {
        ...
        PFNGLTEXSTORAGE1DPROC TexStorage1D;
        ...
        } gl;
}

// These pointers get filled when Ogre calls gl3wInit()
// After this call, you can override the function pointer with your emulated version

This way Ogre would be completely unaware that glTexStorage is being emulated and existing codepaths could run unmodified.

In theory we can do the same with glFramebufferParameteri by stub-ing them out with functions that do nothing, but it may be better to avoid calling them explicitly since Ogre should be aware of drivers not implementing this functionality.

  • GL3PlusTextureGpu::copyTo

    • check correct usage of staging texture
    • understand how to handle sub images in fallback code (i.e. src.X != 0, etc)

Staging Texture stuff: Staging Textures are actually CPU -> GPU, whereas copyTo is GPU -> GPU.

In order to use a StagingTexture fallback you would need to do GPU -> CPU (AsyncTicket) then CPU -> GPU (StagingTexture). This is what Ogre 2.1 did and was super slow, but it works.

Given that we're being forced to throw async transfers into the trash here, you can actually use Image2::convertFromTexture or Image2::copyContentsToMemory which handles all edge cases of AsyncTicket synchronously; then use Image2::uploadTo for synchronous upload (which handles all the edge cases of StagingTexture).

  • GL3PlusTextureGpu::copyViaFramebuffer

    • understand why this is not used to implement fall back case in GL3PlusTextureGpu::copyTo
    • understand why the case !srcIsFboAble && dstIsFboAble is not handled

Framebuffer copies need src and dst to be both FBO. The assumption was that caller code would never call copyViaFramebuffer if this prerequisite wasn't met.

We never had to deal much with this problem because fortunately glCopyImageSubData ended up being widely supported (even if the GPU is super old, a driver update is all what was needed to provide GL_ARB_copy_image; and if the driver is no longer updated then you're on Windows, where D3D11 driver is a much better target as that driver's GL implementation is super unstable anyway).

@darksylinc
Copy link
Member

darksylinc commented Jul 4, 2021 •
edited
Loading

One more detail: We implemented HW mipmap generation of textures loading from disk as:

  1. Create temporary texture that is RenderTarget
  2. Generate mipmaps
  3. copyTo from tmp to final texture; this way the final texture does not have an implicit RenderTarget flag

If we implement copyTo fallback as GPU -> CPU -> GPU, this path may be too slow; and it may actually be faster (profiling needed) if TextureGpuManager::checkSupport returns false when asked for AllowAutomipmaps without RenderTarget flag in macOS. eg

// on macOS only
if( textureFlags & (RenderToTexture|TextureFlags::AllowAutomipmaps) == AllowAutomipmaps )
    return false;

This will force SW mipmap generation on load.

@srmainwaring
Copy link
Contributor Author

@darksylinc Hi, and thanks for the detailed feedback and explanation. I'll make the changes you suggest and update the PR - I'm hoping that a correct fallback for copyTo should resolve the issues I'm seeing with the PbsMaterials sample.

@srmainwaring
Copy link
Contributor Author

srmainwaring commented Jul 5, 2021 •
edited
Loading

@darksylinc I'm trying to using the following in GL3PlusTextureGpu::copyTo as a fallback when GL_ARB_copy_image is not available:

else // not (support.hasMinGLVersion( 4, 3 ) || support.checkExtension( "GL_ARB_copy_image" ))
{
  if (getNextResidencyStatus() == GpuResidency::Resident)
  {
      Image2 image;
      image.convertFromTexture(this, srcMipLevel, srcMipLevel, true);
      image.uploadTo(dst, srcMipLevel, srcMipLevel);
  }
}

but see the following assertion failure at run time when copyTo is called from GenerateHwMipmaps::_executeSerial:

Assertion failed: (mMapRegionStarted && "You must call startMapRegion first!"), function mapRegion, file /Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreStagingTexture.cpp, line 90.

The exception is raised in convertFromTexture while executing texture->waitForData(), but appears to originate from another thread running TextureGpuManager::processQueuedImage.

I'm not completely clear on what is safe to call on what thread in Ogre, but from the internals of convertFromTexture thought that it would be thread safe (but blocking)?

Full details of the debug console and call stack below:

Debug console:

Launching: /Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/Contents/MacOS/Sample_PbsMaterials
Launched process 70414
Stop reason: signal SIGSTOP
2021-07-05 11:31:41.985193+0100 Sample_PbsMaterials[70414:656077] SecTaskLoadEntitlements failed error=22 cs_flags=20, pid=70414
2021-07-05 11:31:41.994883+0100 Sample_PbsMaterials[70414:656077] SecTaskCopyDebugDescription: Sample_PbsMateri[70414]/0#-1 LF=0
Creating resource group General
Creating resource group Internal
Creating resource group Autodetect
SceneManagerFactory for type 'DefaultSceneManager' registered.
Registering ResourceManager for type Material
Registering ResourceManager for type Mesh
Registering ResourceManager for type Mesh2
Registering ResourceManager for type OldSkeleton
MovableObjectFactory for type 'ParticleSystem' registered.
ArchiveFactory for archive type FileSystem registered.
ArchiveFactory for archive type Zip registered.
ArchiveFactory for archive type EmbeddedZip registered.
DDS codec registering
FreeImage version: 3.18.0
This program uses FreeImage, a free, open source image library supporting all common bitmap formats. See http://freeimage.sourceforge.net for details
Supported formats: bmp,ico,jpg,jif,jpeg,jpe,jng,koa,iff,lbm,mng,pbm,pbm,pcd,pcx,pgm,pgm,png,ppm,ppm,ras,tga,targa,tif,tiff,wap,wbmp,wbm,psd,psb,cut,xbm,xpm,gif,hdr,g3,sgi,rgb,rgba,bw,exr,j2k,j2c,jp2,pfm,pct,pict,pic,3fr,arw,bay,bmq,cap,cine,cr2,crw,cs1,dc2,dcr,drf,dsc,dng,erf,fff,ia,iiq,k25,kc2,kdc,mdc,mef,mos,mrw,nef,nrw,orf,pef,ptx,pxn,qtk,raf,raw,rdc,rw2,rwl,rwz,sr2,srf,srw,sti,x3f,webp,jxr,wdp,hdp
ETC codec registering
OITD codec registering
Registering ResourceManager for type HighLevelGpuProgram
MovableObjectFactory for type 'Decal' registered.
MovableObjectFactory for type 'InternalCubemapProbe' registered.
MovableObjectFactory for type 'Entity' registered.
MovableObjectFactory for type 'Item' registered.
MovableObjectFactory for type 'Light' registered.
MovableObjectFactory for type 'Rectangle2Dv2' registered.
MovableObjectFactory for type 'BillboardSet' registered.
MovableObjectFactory for type 'ManualObject2' registered.
MovableObjectFactory for type 'BillboardChain' registered.
MovableObjectFactory for type 'RibbonTrail' registered.
MovableObjectFactory for type 'WireAabb' registered.
Loading library /Users/rhys/Code/ogre/ogre-next2.2/build/lib/macosx/libRenderSystem_GL3Plus
Installing plugin: GL 3+ RenderSystem
OpenGL 3+ Rendering Subsystem created.
Plugin successfully installed
Loading library /Users/rhys/Code/ogre/ogre-next2.2/build/lib/macosx/libRenderSystem_Metal
Installing plugin: Metal RenderSystem
Metal: Devices Detection Starts
2021-07-05 11:31:42.855207+0100 Sample_PbsMaterials[70414:656077] +[MTLIOAccelDevice registerDevices]: Zero Metal services found
Metal: "AMD Radeon Pro W5700X"
Metal: Devices Detection Ends
Plugin successfully installed
Loading library /Users/rhys/Code/ogre/ogre-next2.2/build/lib/macosx/libPlugin_ParticleFX
Installing plugin: ParticleFX
Particle Emitter Type 'Point' registered
Particle Emitter Type 'Box' registered
Particle Emitter Type 'Ellipsoid' registered
Particle Emitter Type 'Cylinder' registered
Particle Emitter Type 'Ring' registered
Particle Emitter Type 'HollowEllipsoid' registered
Particle Affector Type 'LinearForce' registered
Particle Affector Type 'ColourFader' registered
Particle Affector Type 'ColourFader2' registered
Particle Affector Type 'ColourImage' registered
Particle Affector Type 'ColourInterpolator' registered
Particle Affector Type 'Scaler' registered
Particle Affector Type 'Rotator' registered
Particle Affector Type 'DirectionRandomiser' registered
Particle Affector Type 'DeflectorPlane' registered
Plugin successfully installed
*-*-* OGRE Initialising
*-*-* Version 2.2.5 (Cerberus)
2021-07-05 11:31:43.081727+0100 Sample_PbsMaterials[70414:656077] SecTaskLoadEntitlements failed error=22 cs_flags=20, pid=70414
2021-07-05 11:31:43.081805+0100 Sample_PbsMaterials[70414:656077] SecTaskCopyDebugDescription: Sample_PbsMateri[70414]/0#-1 LF=0
CPU Identifier & Features
-------------------------
 *   CPU ID: GenuineIntel: Intel(R) Xeon(R) W-3245 CPU @ 3.20GHz
 *   Logical cores: 32
 *      SSE: yes
 *     SSE2: yes
 *     SSE3: yes
 *      MMX: yes
 *   MMXEXT: yes
 *    3DNOW: no
 * 3DNOWEXT: no
 *     CMOV: yes
 *      TSC: yes
 *      FPU: yes
 *      PRO: yes
 *       HT: no
-------------------------
***********************************************
***  Starting Mac OS X OpenGL 3+ Subsystem  ***
***********************************************
GL3PlusRenderSystem::_createRenderWindow "PBS Materials Sample", 640x480 windowed  miscParams: FSAA=0 gamma=Yes parentWindowHandle=4358334864 reverse_depth=Yes title=PBS Materials Sample vsync= 
Creating a Cocoa Compatible Render System
Mac Cocoa Window: Rendering on an external NSView
Cocoa: Window created 640 x 480 using content scaling factor 1.0
GL Version = 4.1.0.0
GL_VERSION = 4.1 ATI-4.5.14
GL_VENDOR = ATI Technologies Inc.
GL_RENDERER = AMD Radeon Pro W5700X OpenGL Engine
GL_EXTENSIONS = 
GL_ARB_blend_func_extended
GL_ARB_draw_buffers_blend
GL_ARB_draw_indirect
GL_ARB_ES2_compatibility
GL_ARB_explicit_attrib_location
GL_ARB_gpu_shader_fp64
GL_ARB_gpu_shader5
GL_ARB_instanced_arrays
GL_ARB_internalformat_query
GL_ARB_occlusion_query2
GL_ARB_sample_shading
GL_ARB_sampler_objects
GL_ARB_separate_shader_objects
GL_ARB_shader_bit_encoding
GL_ARB_shader_subroutine
GL_ARB_shading_language_include
GL_ARB_tessellation_shader
GL_ARB_texture_buffer_object_rgb32
GL_ARB_texture_cube_map_array
GL_ARB_texture_gather
GL_ARB_texture_query_lod
GL_ARB_texture_rgb10_a2ui
GL_ARB_texture_storage
GL_ARB_texture_swizzle
GL_ARB_timer_query
GL_ARB_transform_feedback2
GL_ARB_transform_feedback3
GL_ARB_vertex_attrib_64bit
GL_ARB_vertex_type_2_10_10_10_rev
GL_ARB_viewport_array
GL_EXT_debug_label
GL_EXT_debug_marker
GL_EXT_depth_bounds_test
GL_EXT_texture_compression_s3tc
GL_EXT_texture_filter_anisotropic
GL_EXT_texture_mirror_clamp
GL_EXT_texture_sRGB_decode
GL_APPLE_client_storage
GL_APPLE_container_object_shareable
GL_APPLE_flush_render
GL_APPLE_object_purgeable
GL_APPLE_rgb_422
GL_APPLE_row_bytes
GL_APPLE_texture_range
GL_ATI_texture_mirror_once
GL_NV_texture_barrier
**************************************
***   OpenGL 3+ Renderer Started   ***
**************************************
Registering ResourceManager for type GpuProgram
RenderSystem capabilities
-------------------------
RenderSystem Name: OpenGL 3+ Rendering Subsystem
GPU Vendor: unknown
Device Name: AMD Radeon Pro W5700X OpenGL Engine
Driver Version: 4.1.0.0
 * Fixed function pipeline: no
 * Hardware generation of mipmaps: yes
 * Texture blending: yes
 * Anisotropic texture filtering: yes
 * Dot product texture operation: yes
 * Cube mapping: yes
 * Hardware stencil buffer: yes
   - Stencil depth: 8
   - Two sided stencil support: yes
   - Wrap stencil values: yes
 * Hardware vertex / index buffers: yes
 * 32-bit index buffers: yes
 * Vertex programs: yes
 * Number of floating-point constants for vertex programs: 4096
 * Number of integer constants for vertex programs: 4096
 * Number of boolean constants for vertex programs: 4096
 * Fragment programs: yes
 * Number of floating-point constants for fragment programs: 4096
 * Number of integer constants for fragment programs: 4096
 * Number of boolean constants for fragment programs: 4096
 * Geometry programs: yes
 * Number of floating-point constants for geometry programs: 4096
 * Number of integer constants for geometry programs: 4096
 * Number of boolean constants for geometry programs: 4096
 * Tessellation Hull programs: yes
 * Number of floating-point constants for tessellation hull programs: 4096
 * Number of integer constants for tessellation hull programs: 4096
 * Number of boolean constants for tessellation hull programs: 4096
 * Tessellation Domain programs: yes
 * Number of floating-point constants for tessellation domain programs: 4096
 * Number of integer constants for tessellation domain programs: 4096
 * Number of boolean constants for tessellation domain programs: 4096
 * Compute programs: no
 * Number of floating-point constants for compute programs: 0
 * Number of integer constants for compute programs: 0
 * Number of boolean constants for compute programs: 19745
 * Supported Shader Profiles: glsl glsl130 glsl140 glsl150 glsl330 glsl400 glsl410
 * Texture Compression: yes
   - DXT: yes
   - VTC: no
   - PVRTC: no
   - ATC: no
   - ETC1: no
   - ETC2: no
   - BC4/BC5: yes
   - BC6H/BC7: no
   - ASTC: no
 * Hardware Occlusion Query: yes
 * User clip planes: yes
 * VET_UBYTE4 vertex element type: yes
 * Infinite far plane projection: yes
 * Hardware render-to-texture: yes
 * Floating point textures: yes
 * Non-power-of-two textures: yes
 * 1d textures: yes
 * Volume textures: yes
 * Max Texture resolution (2D) 16384
 * Max Texture resolution (3D) 16384
 * Max Texture resolution (Cubemaps) 16384
 * Multiple Render Targets: 8
   - With different bit depths: yes
 * Point Sprites: yes
 * Extended point parameters: yes
 * Max Point Size: 8191
 * Vertex texture fetch: yes
 * Number of world matrices: 0
 * Number of texture units: 16
 * Stencil buffer depth: 8
 * Number of vertex blend matrices: 0
   - Max vertex textures: 16
   - Vertex textures shared: yes
 * Render to Vertex Buffer : yes
 * Hardware Atomic Counters: no
 * GL 1.5 without VBO workaround: no
 * Frame Buffer objects: yes
 * Frame Buffer objects (ARB extension): no
 * Frame Buffer objects (ATI extension): no
 * PBuffer support: no
 * GL 1.5 without HW-occlusion workaround: no
 * Vertex Array Objects: yes
 * Separate shader objects: no
 * Using Reverse Z: yes
DefaultWorkQueue('Root') initialising on thread main.
Particle Renderer Type 'billboard' registered
OverlayElementFactory for type Panel registered.
OverlayElementFactory for type BorderPanel registered.
OverlayElementFactory for type TextArea registered.
Registering ResourceManager for type Font
Creating resource group Essential
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/packs/DebugPack.zip' of type 'Zip' to resource group 'Essential'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/2.0/scripts/materials/Common' of type 'FileSystem' to resource group 'General'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/2.0/scripts/materials/Common/Any' of type 'FileSystem' to resource group 'General'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/2.0/scripts/materials/Common/GLSL' of type 'FileSystem' to resource group 'General'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/2.0/scripts/materials/Common/GLSLES' of type 'FileSystem' to resource group 'General'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/2.0/scripts/materials/Common/HLSL' of type 'FileSystem' to resource group 'General'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/2.0/scripts/materials/Common/Metal' of type 'FileSystem' to resource group 'General'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/Hlms/Common/Any' of type 'FileSystem' to resource group 'General'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/Hlms/Common/GLSL' of type 'FileSystem' to resource group 'General'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/Hlms/Common/HLSL' of type 'FileSystem' to resource group 'General'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/Hlms/Common/Metal' of type 'FileSystem' to resource group 'General'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/Compute/Algorithms/IBL' of type 'FileSystem' to resource group 'General'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/Compute/Tools/Any' of type 'FileSystem' to resource group 'General'
Creating resource group Popular
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/2.0/scripts/Compositors' of type 'FileSystem' to resource group 'Popular'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/models' of type 'FileSystem' to resource group 'Popular'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/materials/textures' of type 'FileSystem' to resource group 'Popular'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/materials/textures/Cubemaps' of type 'FileSystem' to resource group 'Popular'
Added resource location '/Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/../../../Samples/Media/2.0/scripts/materials/PbsMaterials' of type 'FileSystem' to resource group 'General'
[INFO] Texture cache not found at /Volumes/MacPro2_DV1/Code/ogre/ogre-next2.2/build/bin/Sample_PbsMaterials.app/Contents/Resources//textureMetadataCache.json
Parsing scripts for resource group Autodetect
Finished parsing scripts for resource group Autodetect
Creating resources for group Autodetect
All done
Parsing scripts for resource group Essential
Parsing script Materials.material
Parsing script DebugFont.fontdef
Finished parsing scripts for resource group Essential
Creating resources for group Essential
All done
Parsing scripts for resource group General
Parsing script Quad.program
Parsing script Copyback.material
Parsing script DepthUtils.material
Parsing script DPM.material
Parsing script DPSM.material
Parsing script EsmGaussianBlurLogFilter.material
Parsing script HiddenAreaMeshVr.material
Parsing script PccDepthCompressor.material
Parsing script RadialDensityMask.material
Parsing script Sky.material
Parsing script PbsMaterials.material
Parsing script EsmGaussianBlurLogFilter.material.json
Parsing script Mipmaps.material.json
Parsing script IBL.material.json
Finished parsing scripts for resource group General
Creating resources for group General
All done
Parsing scripts for resource group Internal
Finished parsing scripts for resource group Internal
Creating resources for group Internal
All done
Parsing scripts for resource group Popular
Parsing script InstancedStereo.compositor
Parsing script IrradianceFieldRaster.compositor
Parsing script LocalCubemaps.compositor
Parsing script PbsMaterials.compositor
Parsing script PlanarReflections.compositor
Parsing script Refractions.compositor
Parsing script ScreenSpaceReflections.compositor
Parsing script ShadowMapDebugging.compositor
Parsing script StaticShadowMaps.compositor
Parsing script StencilTest.compositor
Parsing script StereoRendering.compositor
Parsing script Tutorial_DynamicCubemap.compositor
Parsing script Tutorial_OpenVRWorkspace.compositor
Parsing script Tutorial_ReconstructPosFromDepth.compositor
Parsing script Tutorial_Terrain.compositor
Parsing script TutorialSky_Postprocess.compositor
Parsing script TutorialUav01_Setup.compositor
Parsing script TutorialUav02_Setup.compositor
Parsing script UvBaking.compositor
Finished parsing scripts for resource group Popular
Creating resources for group Popular
All done
Mesh: Loading Sphere1000.mesh.
WARNING: Sphere1000.mesh is an older format ([MeshSerializer_v2.1 R1]); you should upgrade it as soon as possible using the OgreMeshTool tool.
Mesh: Loading Cube_d.mesh.
WARNING: Cube_d.mesh is an older format ([MeshSerializer_v2.1 R1]); you should upgrade it as soon as possible using the OgreMeshTool tool.
Vertex Shader: 100000000VertexShader_vs
Fragment Shader: 100000000PixelShader_ps
 GLSL link result : 
WARNING: Could not find vertex shader attribute 'qtangent' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendWeights' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv5' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'secondary_colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv1' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv4' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendWeights2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv6' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'normal' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'tangent' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv3' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv7' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv0' to match BindAttributeLocation request.
Vertex Shader: 100000001VertexShader_vs
Fragment Shader: 100000001PixelShader_ps
 GLSL link result : 
WARNING: Could not find vertex shader attribute 'qtangent' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendWeights' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv5' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'secondary_colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv1' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv4' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendWeights2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv6' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'normal' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'tangent' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv3' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv7' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv0' to match BindAttributeLocation request.
Font DebugFont using texture size 512x512
Info: Freetype returned null for character 127 in font DebugFont
Info: Freetype returned null for character 128 in font DebugFont
Info: Freetype returned null for character 129 in font DebugFont
Info: Freetype returned null for character 130 in font DebugFont
Info: Freetype returned null for character 131 in font DebugFont
Info: Freetype returned null for character 132 in font DebugFont
Info: Freetype returned null for character 133 in font DebugFont
Info: Freetype returned null for character 134 in font DebugFont
Info: Freetype returned null for character 135 in font DebugFont
Info: Freetype returned null for character 136 in font DebugFont
Info: Freetype returned null for character 137 in font DebugFont
Info: Freetype returned null for character 138 in font DebugFont
Info: Freetype returned null for character 139 in font DebugFont
Info: Freetype returned null for character 140 in font DebugFont
Info: Freetype returned null for character 141 in font DebugFont
Info: Freetype returned null for character 142 in font DebugFont
Info: Freetype returned null for character 143 in font DebugFont
Info: Freetype returned null for character 144 in font DebugFont
Info: Freetype returned null for character 145 in font DebugFont
Info: Freetype returned null for character 146 in font DebugFont
Info: Freetype returned null for character 147 in font DebugFont
Info: Freetype returned null for character 148 in font DebugFont
Info: Freetype returned null for character 149 in font DebugFont
Info: Freetype returned null for character 150 in font DebugFont
Info: Freetype returned null for character 151 in font DebugFont
Info: Freetype returned null for character 152 in font DebugFont
Info: Freetype returned null for character 153 in font DebugFont
Info: Freetype returned null for character 154 in font DebugFont
Info: Freetype returned null for character 155 in font DebugFont
Info: Freetype returned null for character 156 in font DebugFont
Info: Freetype returned null for character 157 in font DebugFont
Info: Freetype returned null for character 158 in font DebugFont
Info: Freetype returned null for character 159 in font DebugFont
Info: Freetype returned null for character 160 in font DebugFont
Vertex Shader: 100000002VertexShader_vs
Fragment Shader: 100000002PixelShader_ps
 GLSL link result : 
WARNING: Could not find vertex shader attribute 'blendWeights' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv5' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'secondary_colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv1' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv4' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendWeights2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv6' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'normal' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'tangent' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv3' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv7' to match BindAttributeLocation request.
Vertex Shader: 100000002VertexShader_vs
Fragment Shader: 100000002PixelShader_ps
 GLSL validation result : 
Validation Failed: Sampler error:
  Samplers of different types use the same texture image unit.
   - or -
  A sampler's texture unit is out of range (greater than max allowed or negative).
Vertex Shader: 100000003VertexShader_vs
Fragment Shader: 100000003PixelShader_ps
 GLSL link result : 
WARNING: Could not find vertex shader attribute 'blendWeights' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv5' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'secondary_colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv1' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv4' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendWeights2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv6' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'normal' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'tangent' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv3' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv7' to match BindAttributeLocation request.
Vertex Shader: 100000003VertexShader_vs
Fragment Shader: 100000003PixelShader_ps
 GLSL validation result : 
Validation Failed: Sampler error:
  Samplers of different types use the same texture image unit.
   - or -
  A sampler's texture unit is out of range (greater than max allowed or negative).
Vertex Shader: 100000004VertexShader_vs
Fragment Shader: 100000004PixelShader_ps
 GLSL link result : 
WARNING: Could not find vertex shader attribute 'blendWeights' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv5' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'secondary_colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv1' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv4' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendWeights2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv6' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'normal' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'tangent' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv3' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv7' to match BindAttributeLocation request.
Vertex Shader: 100000004VertexShader_vs
Fragment Shader: 100000004PixelShader_ps
 GLSL validation result : 
Validation Failed: Sampler error:
  Samplers of different types use the same texture image unit.
   - or -
  A sampler's texture unit is out of range (greater than max allowed or negative).
Vertex Shader: 300000000VertexShader_vs
Fragment Shader: 300000000PixelShader_ps
 GLSL link result : 
WARNING: Could not find vertex shader attribute 'qtangent' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendWeights' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv5' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'secondary_colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv1' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv4' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendWeights2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv6' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'normal' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'tangent' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv3' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv7' to match BindAttributeLocation request.
Assertion failed: (mMapRegionStarted && "You must call startMapRegion first!"), function mapRegion, file /Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreStagingTexture.cpp, line 90.
Stop reason: hit program assert

Call stack:

__psynch_mutexwait (@__psynch_mutexwait:6)
_pthread_mutex_firstfit_lock_wait (@_pthread_mutex_firstfit_lock_wait:27)
_pthread_mutex_firstfit_lock_slow (@_pthread_mutex_firstfit_lock_slow:64)
Ogre::LightweightMutex::lock() (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/Threading/OgreLightweightMutexPThreads.cpp:47)
Ogre::TextureGpuManager::_update(bool) (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreTextureGpuManager.cpp:3079)
Ogre::TextureGpuManager::_waitFor(Ogre::TextureGpu*, bool) (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreTextureGpuManager.cpp:3203)
Ogre::TextureGpu::waitForData() (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreTextureGpu.cpp:993)
Ogre::Image2::convertFromTexture(Ogre::TextureGpu*, unsigned char, unsigned char, bool) (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreImage2.cpp:336)
Ogre::GL3PlusTextureGpu::copyTo(Ogre::TextureGpu*, Ogre::TextureBox const&, unsigned char, Ogre::TextureBox const&, unsigned char, bool) (/Users/rhys/Code/ogre/ogre-next2.2/RenderSystems/GL3Plus/src/OgreGL3PlusTextureGpu.cpp:874)
Ogre::TextureFilter::GenerateHwMipmaps::_executeSerial(Ogre::TextureGpu*) (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreTextureFilters.cpp:207)
Ogre::ObjCmdBuffer::NotifyDataIsReady::execute() (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreObjCmdBuffer.cpp:207)
Ogre::ObjCmdBuffer::execute() (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreObjCmdBuffer.cpp:108)
Ogre::TextureGpuManager::_update(bool) (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreTextureGpuManager.cpp:3133)
Ogre::RenderSystem::_update() (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreRenderSystem.cpp:1264)
Ogre::CompositorManager2::_updateImplementation() (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/Compositor/OgreCompositorManager2.cpp:738)
Ogre::RenderSystem::updateCompositorManager(Ogre::CompositorManager2*) (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreRenderSystem.cpp:1270)
Ogre::CompositorManager2::_update() (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/Compositor/OgreCompositorManager2.cpp:652)
Ogre::Root::_updateAllRenderTargets() (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreRoot.cpp:1550)
Ogre::Root::renderOneFrame() (/Users/rhys/Code/ogre/ogre-next2.2/OgreMain/src/OgreRoot.cpp:1126)
Demo::GraphicsSystem::update(float) (/Users/rhys/Code/ogre/ogre-next2.2/Samples/2.0/Common/src/GraphicsSystem.cpp:383)

@darksylinc
Copy link
Member

darksylinc commented Jul 5, 2021 •
edited
Loading

Ouch.

The bug is simple. Image2 is calling waitForData (which in this case it's not needed, i.e. it's safe) that ends up causing a call to TextureGpuManager::_update while already in that function and is left in an inconsistent state.

We'll have to do the same work in lower level objects because sadly we can't reuse that code (because we do need to wait on the async ticket before we can upload, and we can't do that)

@srmainwaring
Copy link
Contributor Author

@darksylinc

The bug is simple. Image2 is calling waitForData (which in this case it's not needed, i.e. it's safe) that ends up causing a call to TextureGpuManager::_update while already in that function and is left in an inconsistent state.

I suspected as much. I thought TextureGpuManager::_update might have been re-entrant, but I guess it's not.

It's a tricky one to resolve using the high level interfaces: this particular problem is coming from GenerateHwMipmaps::_executeSerial where functions in AsyncTextureTicket will delay because the source texture will not return isDataReady() == true until the worker thread finishes executing the commands to set up the mipmaps in ObjCmdBuffer::NotifyDataIsReady::execute. So even if TextureGpuManager::_update was reentrant and the internal state of the staging texture wasn't messed up, there'd be a logical race condition to resolve.

As you suggest, the easiest fix for the Sample_PbsMaterials at least, is to nix HW mipmap generation for macOS with OpenGL. Then the only code change needed to get it running (OpenGL wise) is the stubbing out of the glFramebufferParameteri calls (and perhaps just logging a warning in GL3PlusTextureGpu::copyTo instead of raising an exception that then exits. I have the sample running correctly at ~300 FPS with OpenGL - plenty good for my requirements.

I could not find TextureGpuManager::checkSupport to modify its behaviour - I think that's available in master but not on v2-2-egl. Should that be patched into v2-2-egl or is there an alternative approach for the earlier branch?

Rather than try to get full OpenGL fallback support for macOS in one shot perhaps I could break up the changes into a number of steps. Then I can try to convince the OSRF people to update the version of Ogre2.2 they are using that will still give PBS support for Ignition and work on macOS with improved performance to follow.

A suggested split would be:

  1. Fixes to Cocoa windows and the glFramebufferParameteri fall back.
  2. Seamless fall back to glTexStorage1D etc. (I now see what you meant and have the GL function overriding working but need to do some work to handle all cases under a single function when I don't have access to the Ogre pixel convenience functions to lookup formats and types based on the GL types rather the Ogre types).
  3. Fall back for the code in GL3PlusTextureGpu::copyTo and support for HW mipmap generation - which I'll definitely need your help with because there are a lot of cases to catch and it looks like an area for experts.

@srmainwaring srmainwaring mentioned this pull request Jul 6, 2021
Closed
6 tasks
@srmainwaring
[macOS] add GL3Plus support for macOS
e1989ac 
Reestablish support for GL3+ on maOS in Ogre2.2

Cmake

- Update cmake version
- Add extra library name for zzip (required to find homebrew installed version on macOS)
- Exclude morph animations sample that breaks compilation

Cocoa Windowing

- Remove GeneralAllocatedObject as superclass of CocoaContext
- Update class documentation in line with OgreEGL classes
- Allocate arrays for table data source in config dialog init.
  This fixes a memory access segfault that occurred when altering
  a value in the option selector.
- Remove not implemented exception from swapBuffers

TextureGpuManager

- Apply to TextureGpuManager and TextureFilters from:
    Commit: 42bbd89
    From: "Matias N. Goldberg" <[email protected]>
    Date: Wed, 9 Sep 2020 17:03:23 -0300

    Commit: 1cebf78
    Author: Matias N. Goldberg <[email protected]>
    Date:   Fri Sep 11 20:19:07 2020 -0300
- Add condition to disable HW mipmap generation on macOS

GL3Plus RenderSystem

- Add GL4.3 checks to GL3PlusRenderPassDescriptor
- Apply changes to

Signed-off-by: Rhys Mainwaring <[email protected]>
@srmainwaring srmainwaring force-pushed the feature/v2-2-egl-macos branch from 32a0297 to e1989ac Compare July 7, 2021 12:03
@srmainwaring srmainwaring changed the title WIP: add GL3Plus support for macOS add GL3Plus support for macOS Jul 7, 2021
@srmainwaring
Copy link
Contributor Author

@darksylinc - rebased and squashed ready for review

This PR will provide GL3+ support on macOS for machines that have cards with the GL_ARB_texture_storage extension.

Fallback support for glTexStorageXD and HW mipmap generation to follow separately if required.

@darksylinc darksylinc closed this Jul 10, 2021
@darksylinc darksylinc deleted the branch OGRECave:v2-2-egl July 10, 2021 23:52
@darksylinc
Copy link
Member

This PR got auto-closed because the v2-2-egl branch no longer exists (which is ​too old)

Luckily your changes merged fine with vv2-2 and I will soon merge them manually (hopefully Sunday)

@darksylinc
Copy link
Member

Your PR has been manually merged with a couple of changes.

Thanks!

@iche033 iche033 mentioned this pull request Aug 4, 2021
Merged
6 tasks
@iche033 iche033 mentioned this pull request Sep 15, 2021
Merged
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@srmainwaring @darksylinc