Add Generate LODs, Shadow Mesh and Lightmap UV2 options to OBJ mesh import #94108

Calinou · 2024-07-08T23:16:14Z

Follow-up to Add support for generating LODs and shadow mesh in PrimitiveMesh #94097
(can be merged independently).
Depends on Remove unused Optimize Mesh import option from OBJ mesh import #94100.

This puts OBJ mesh import on parity with 3D scene import. It's now possible to have the same level of optimization as imported 3D scenes while using the OBJ mesh workflow.

LOD and shadow mesh generation are enabled by default, so that users can benefit from the same performance regardless of how they're importing 3D objects into Godot. Lightmap UV2 generation is disabled by default to mimic the 3D scene import default (and PrimitiveMesh).

optimize_indices_for_cache() is now always called on import as well, similar to what the 3D scene import already does.

See Improve mesh LOD system workflow godot-proposals#5600.

Testing project: test_obj_import_lod_shadow_mesh_lightmap_uv2.zip

Preview

OBJ mesh import options

Lightmapped OBJ mesh having its LOD level changed

obj_lod_lightmap.mp4

LOD disabled

LOD enabled (Mesh LOD Threshold = 1024)

This shows the most decimated LODs for comparison purposes.

TODO

Figure out why it runs much slower than 3D scene import when LOD generation or lightmap UV2 generation is enabled. (Shadow mesh generation also suffers from this problem to a less severe extent.)
- This does not occur with 3D scene import, even on the same OBJ file switched to use the OBJ as Scene import mode.

Calinou · 2024-07-09T01:05:32Z

editor/import/3d/resource_importer_obj.cpp

+				if (p_generate_lods) {
+					// Use normal merge/split angles that match the defaults used for 3D scene importing.
+					mesh->generate_lods(60.0f, 25.0f, {});
+				}
+
+				if (p_generate_shadow_mesh) {
+					mesh->create_shadow_mesh();
+				}


Note: if you perform these steps before lightmap UV2 generation, the resulting mesh will be broken. It'll display with the wrong transform and no material (pure black color). I have no idea why this happens, but I figure I'd mention it either way. Putting up a MRP for this will likely require exposing additional ImporterMesh methods (such as create_shadow_mesh() and lightmap_unwrap_cached()).

fracteed · 2024-07-09T03:05:37Z

Nice work. I would look to see an option to not to try to import .mtl files when importing an obj. It is a constant annoyance to see errors in the console that it can't find the corresponding .mat file for an obj.

Personally, I never use the .mtl file, so only export it to stop Godot spitting out the error. Godot will see face groups fine with just the .obj itself.

lyuma · 2024-07-09T07:10:17Z

I would look to see an option to not to try to import .mat files

@fracteed To be clear, you're talking about .mtl files referenced by a mtllib statement in an obj

My opinion is that it is data which is directly referenced by the obj file, so it would be wrong not to try to load it.

I don't think we should add an option to disable an error, especially when there are cases where the error is helpful. If you really don't have the .mtl, it's also relatively easy to edit the .obj file and remove the mtllib statement.

fracteed · 2024-07-09T09:06:14Z

@lyuma yeah, I meant to write mtl. There should at least be an option to turn off the requirement for it for those that just need the mesh information. If I have hundreds of objs in a project, it is not viable to fiddle with editing obj files.

Just went and checked on a current build to see if anything had changed, as I probably wasn't very clear with my initial comment. Yes, you can export objs without an mtl and they import fine without any error messages. But the issue is that you can't get material group selections unless you have the mtl file exported from Blender.

It is a stupid quirk in the format, since if I export an obj from blender with the mtl and then delete the mtl, Godot will still import it with the material groups intact, it will however give an error message. One of the main reasons I delete the mtl is that Godot deals with them badly in general. For one, they do not show up in the file system, and secondly, when you move an obj from one folder to another, it does not move the mtl file along with the obj.

So after having hundreds of files added and deleted over the course of years in a large project you end up with a ton of errors looking for mtl files that are actually not needed, and are either in other folders or long deleted. I guess the ideal would be if Godot could actually just read the material group info directly from the obj, since the face info is in there, as you can can export an obj with material groups and no mtl.

Calinou · 2024-07-09T13:50:53Z

It is a constant annoyance to see errors in the console that it can't find the corresponding .mat file for an obj.

#94102 fixes the spurious error message if you have no MTL file referenced in the OBJ. However, I think a missing MTL file should print a message if the OBJ is trying to reference it, as it can be the source of a legitimate error.

If I have hundreds of objs in a project, it is not viable to fiddle with editing obj files.

sd should be able to automate this easily.

Something like this can be used to comment out all material file definitions in OBJ files in the current folder (recursively):

# Use this to make `**` work on Bash (not required on zsh).
shopt -s globstar nullglob

sd "mtllib" "#mtllib" **/*.obj

akien-mga · 2024-07-09T14:30:51Z

I think we should add an import option for OBJ to ignore MTL files (i.e. just skip mtllib references). IIRC Godot doesn't really support the old specular mapping they use, so only bits of it can be imported. More often than not people might want to write their own material and not use the one a thirdparty OBJ file may have come with.

zeux · 2024-07-12T01:03:51Z

I think the extra processing steps should be done in import_scene or at the end of parse obj, not inside the loop? An .obj file with multiple materials or surface groups will add multiple surfaces to the same mesh, but with this change shadow mesh or LOD or lightmap generation will be called on the entire mesh for every newly added surface. For lightmaps this might be mitigated with a cache, but for LOD/shadow meshes it looks like the code will redundantly reprocess all earlier computed surfaces.

zeux · 2024-07-12T01:17:20Z

Also after #94241 it would make sense to call ImporterMesh::optimize_indices_for_cache as well; the improvements in create_shadow_mesh will automatically apply here with no changes.

clayjohn · 2024-08-19T19:45:01Z

Since #94241 has been merged. Could you add ImporterMesh::optimize_indices_for_cache as part of this PR? I would love to include this in 4.4

…mport This puts OBJ mesh import on parity with 3D scene import. It's now possible to have the same level of optimization as imported 3D scenes while using the OBJ mesh workflow. `optimize_indices_for_cache()` is now always called on import as well, similar to what the 3D scene import already does.

Calinou · 2024-10-25T00:49:15Z

Rebased and tested again, it works as expected.

I think the extra processing steps should be done in import_scene or at the end of parse obj, not inside the loop? An .obj file with multiple materials or surface groups will add multiple surfaces to the same mesh, but with this change shadow mesh or LOD or lightmap generation will be called on the entire mesh for every newly added surface. For lightmaps this might be mitigated with a cache, but for LOD/shadow meshes it looks like the code will redundantly reprocess all earlier computed surfaces.

That was the cause of the performance issue 🙂 Fixed, thanks for the pointer!

It's now possible to use lightmap unwrap on octavus.obj in the testing project with reasonable performance:

Since #94241 has been merged. Could you add ImporterMesh::optimize_indices_for_cache as part of this PR? I would love to include this in 4.4

Done.

clayjohn

Looks great to me!

fire

The proposal makes sense to me and is wanted.

Did not technical review.

Repiteo · 2024-10-25T18:12:21Z

Thanks!

Calinou added enhancement topic:import topic:3d labels Jul 8, 2024

Calinou added this to the 4.x milestone Jul 8, 2024

Calinou requested review from a team as code owners July 8, 2024 23:16

Calinou changed the title ~~Remove unused Optimize Mesh import option from OBJ mesh import~~ Add Generate LODs, Shadow Mesh and Lightmap UV2 options to OBJ mesh import Jul 8, 2024

Calinou mentioned this pull request Jul 8, 2024

Remove unused Optimize Mesh import option from OBJ mesh import #94100

Merged

Calinou force-pushed the obj-import-add-generate-lods-shadow-mesh-lightmap-uv2 branch from 3ca15cd to 60c20bf Compare July 8, 2024 23:25

Calinou commented Jul 9, 2024

View reviewed changes

Calinou mentioned this pull request Jul 11, 2024

[TRACKER] LightmapGI (GPU lightmapper) issues #56033

Open

zeux mentioned this pull request Jul 12, 2024

Optimize base and shadow meshes for vertex cache #94241

Merged

Calinou force-pushed the obj-import-add-generate-lods-shadow-mesh-lightmap-uv2 branch from 60c20bf to 5a7a63a Compare October 24, 2024 23:55

Calinou force-pushed the obj-import-add-generate-lods-shadow-mesh-lightmap-uv2 branch from 5a7a63a to b8ee61f Compare October 25, 2024 00:48

clayjohn approved these changes Oct 25, 2024

View reviewed changes

clayjohn modified the milestones: 4.x, 4.4 Oct 25, 2024

fire approved these changes Oct 25, 2024

View reviewed changes

Repiteo merged commit de6814d into godotengine:master Oct 25, 2024
20 checks passed

Calinou mentioned this pull request Oct 25, 2024

Update meshoptimizer to 0.22 #98529

Merged

Calinou deleted the obj-import-add-generate-lods-shadow-mesh-lightmap-uv2 branch January 8, 2025 16:07

Calinou mentioned this pull request Jan 14, 2025

Add light volume nodes as a fast way to render global illumination godotengine/godot-proposals#11448

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Generate LODs, Shadow Mesh and Lightmap UV2 options to OBJ mesh import #94108

Add Generate LODs, Shadow Mesh and Lightmap UV2 options to OBJ mesh import #94108

Calinou commented Jul 8, 2024 •

edited

Loading

Calinou Jul 9, 2024 •

edited

Loading

fracteed commented Jul 9, 2024 •

edited

Loading

lyuma commented Jul 9, 2024

fracteed commented Jul 9, 2024 •

edited

Loading

Calinou commented Jul 9, 2024 •

edited

Loading

akien-mga commented Jul 9, 2024

zeux commented Jul 12, 2024

zeux commented Jul 12, 2024

clayjohn commented Aug 19, 2024

Calinou commented Oct 25, 2024 •

edited

Loading

clayjohn left a comment

fire left a comment

Repiteo commented Oct 25, 2024

Add Generate LODs, Shadow Mesh and Lightmap UV2 options to OBJ mesh import #94108

Add Generate LODs, Shadow Mesh and Lightmap UV2 options to OBJ mesh import #94108

Conversation

Calinou commented Jul 8, 2024 • edited Loading

Preview

OBJ mesh import options

Lightmapped OBJ mesh having its LOD level changed

LOD disabled

LOD enabled (Mesh LOD Threshold = 1024)

TODO

Calinou Jul 9, 2024 • edited Loading

Choose a reason for hiding this comment

fracteed commented Jul 9, 2024 • edited Loading

lyuma commented Jul 9, 2024

fracteed commented Jul 9, 2024 • edited Loading

Calinou commented Jul 9, 2024 • edited Loading

akien-mga commented Jul 9, 2024

zeux commented Jul 12, 2024

zeux commented Jul 12, 2024

clayjohn commented Aug 19, 2024

Calinou commented Oct 25, 2024 • edited Loading

clayjohn left a comment

Choose a reason for hiding this comment

fire left a comment

Choose a reason for hiding this comment

Repiteo commented Oct 25, 2024

Calinou commented Jul 8, 2024 •

edited

Loading

Calinou Jul 9, 2024 •

edited

Loading

fracteed commented Jul 9, 2024 •

edited

Loading

fracteed commented Jul 9, 2024 •

edited

Loading

Calinou commented Jul 9, 2024 •

edited

Loading

Calinou commented Oct 25, 2024 •

edited

Loading