Texture-baked exporting (princeton-vl/infinigen_internal/#103)

* Full Scene Exporter

* Regex Tweaks for Integration Testing

* Refactor and optimize export

* Move exporting README to docs/ folder and add more caveats

* Path handling tweaks for exporter

* Move export to infinigen.tools.export

* Tweak docs

* Add slurm scheduling to generate_individual_assets

* Tweak generate_individual_assets

* Small Refactor

* Glass Export and other features

Glass, Individual object, .obj vertex col export and fixes UV overrwrite

* Add --export option to generate_individual_assets, use a slurm job array per factory not for the whole set

* Add docs on generating & exporting individual assets

* Export Optimizations and Bugfixes

* Remove Unused Args

* Typo / import fixes

* Add --export option to generate_individual_assets, use a slurm job array per factory not for the whole set

* Fix kwargs

---------

Co-authored-by: Alexander Raistrick <[email protected]>

README.md

@@ -45,8 +45,9 @@ Next, see our ["Hello World" example](docs/HelloWorld.md) to generate an image &
 - ["Hello World": Generate your first Infinigen scene](docs/HelloWorld.md)
 - [Configuring Infinigen](docs/ConfiguringInfinigen.md)
 - [Downloading pre-generated data](docs/PreGeneratedData.md)
-- [Extended ground-truth](docs/GroundTruthAnnotations.md)
 - [Generating individual assets](docs/GeneratingIndividualAssets.md)
+- [Exporting to external fileformats (OBJ, OpenUSD, etc)](docs/ExportingToExternalFileFormats.md)
+- [Extended ground-truth](docs/GroundTruthAnnotations.md)
 - [Implementing new materials & assets](docs/ImplementingAssets.md)
 - [Generating fluid simulations](docs/GeneratingFluidSimulations.md)
 

docs/CHANGELOG.md

@@ -30,3 +30,7 @@ v1.3.0
 v1.3.1
 - Fix configuration bug causing massive render slowdown 
 - Create noisier video trajectories optimized for training
+
+v1.3.2
+- Bugfix USD/OBJ exporter, add export options to generate_individual_assets
+

docs/ExportingToExternalFileFormats.md

@@ -0,0 +1,56 @@
+
+# Asset Exporter
+
+This documentation details how to create an OBJ, FBX, STL, PLY or OpenUSD file from a `.blend` file, such as those produced by [Hello World](HelloWorld.md) or [Generating Individual Assets](./GeneratingIndividualAssets.md). 
+
+Blender does provide a built-in exporter, but it wont work out of the box for Infinigen since our files contain procedural materials and assets defined using shader programs. This tool's job is to "bake" all these procedural elements into more standard graphics formats (i.e, simple meshes with materials made of texture maps), before invoking the standard blender exporter. This process can be slow, since it uses a rendering engine, and lossy, since the resulting textures have a finite resolution.
+
+To convert a folder of blender files into USD files (our recommmended format), use the command below:
+```bash
+python -m infinigen.tools.export --input_folder {PATH_TO_FOLDER_OF_BLENDFILES} --output_folder outputs/my_export -f usdc -r 1024
+```
+
+If you want a different output format, please use the "--help" flag or use one of the options below:
+- `-f obj` will export in .obj format, 
+- `-f fbx` will export in .fbx format
+- `-f stl` will export in .stl format
+- `-f ply` will export in .ply format. 
+- `-f usdc` will export in .usdc format. 
+- `-v` enables per-vertex colors (only compatible with .fbx and .ply formats). 
+- `-r {INT}` controls the resolution of the baked texture maps. For instance, `-r 1024` will export 1024 x 1024 texture maps.
+- `--individual` will export each object in a scene in its own individual file.
+
+## :warning: Exporting full Infinigen scenes is only supported for USDC files. 
+
+:bulb: Note: exporting OBJ/FBX files of **single objects** *generally works fine; this discussion only refers to large-scale scenes.
+
+Infinigen uses of *instancing* to represent densely scattered objects. That is, rather than storing millions of unique high-detail pebbles or leaves to scatter on the floor, we use a smaller set of unique objects which are stored in memory only once, but are repeated all over the scene with many different transforms. 
+
+To our knowledge, no file formats except '.blend' and '.usdc' support saving 3D files that contain instanced geometry. For all file formats besides these two, instanced will be *realized*: instead of storing just a few unique meshes, the meshes will be copied, pasted and transformed thousands of times (once for each unique scatter location). This creates a simple mesh that, but the cost is so high that we do not recommend attempting it for full Infinigen scenes. 
+
+If you require OBJ/FBX/PLY files for your research, you have a few options:
+- You can use individual objects, rather than full scenes. These *generally dont contain instancing so can be exported to simple mesh formats.
+- You can use advice in [Configuring Infinigen](./ConfiguringInfinigen.md) to create a scene that has very small extent or low detail, such that the final realized mesh will still be small enough to fit in memory.
+- You can use the advice in [Configuring Infinigen](./ConfiguringInfinigen.md) to create a scene which simply doesnt contain any instanced objects. Specifically, you should turn off trees and all scattered objects.
+    - The simplest way to do this is to turn off everything except terrain, by including the config `no_assets.gin`. 
+
+*caveat for the above: Infinigen's implementation for trees uses instances to represent leaves and branches. Trees are also generally large and high detail enough to cause issues if you realize them before exporting. Therefore, exporting whole trees as OBJs also generally isnt supported, unless you do so at very low resolution, or you turn off the tree's branches / leaves first. 
+
+## Other Known Issues and Limitations
+
+* Some material features used in Infinigen are not yet supported by this exporter. Specifically, this script only handles Albedo, Roughness and Metallicity maps. Any other procedural parameters of the material will be ignored, so you should not expect complex materials (e.g skin, translucent leaves, glowing lava) to be perfectly reproduced outside of Blender. Depending on file format, there is limited support for materials with non-procedural, constant values of transmission, clearcoat, and sheen.
+
+* Exporting *animated* 3D files is generally untested and not officially supported. This includes exporting particles, articulated creatures, deforming plants, etc. These features are *in principle* supported by OpenUSD, but are untested by us and not officially supported by this export script.
+
+* Assets with transparent materials (water, glass-like materials, etc.) may have incorrect textures for all material parameters after export.
+
+* Large scenes and assets may take a long time to export and will crash Blender if you do not have a sufficiently large amount of memory. The export results may also be unusably large.
+
+* When exporting in .fbx format, the embedded roughness texture maps in the file may sometimes be too bright or too dark. The .png roughness map in the folder is accurate, however.
+
+* .fbx exports ocassionally fail due to invalid UV values on complicated geometry. Adjusting the 'island_margin' value in bpy.ops.uv.smart_project() sometimes remedies this
+
+
+
+
+

docs/GeneratingIndividualAssets.md

@@ -1,20 +1,16 @@
 # Generating Individual Assets
 
-This tutorial will help you generate images or .blend files of specific assets of your choice.
-
-Limitations (to be addressed soon): 
-- This tool only exports .blend files. [See here](../infinigen/datagen/tools/export/README.md) for a prototype tool to convert these to standard mesh file formats, but it itself has some limitations. 
-- This tool cannot currently generate or export terrain meshes.
+This tutorial will help you generate images, .blend files, or baked OBJ/USD/etc files for any asset of your choosing.
 
 ### Example Commands
 
-Shown are three examples of using our `generate_individual_assets.py` script to create images and .blend files.
+Shown are three examples of using our `generate_individual_assets.py` script to create images and .blend files. If you want to create asset files in another format (e.g. OBJ, FBX, USD) you should first generate blend files then use our [](./ExportingToExternalFileFormats.md) docs to bake them to the format of your choosing. 
 
 ```bash
 mkdir outputs
-python -m infinigen_examples.generate_individual_assets -f CoralFactory -n 8 --save_blend
-python -m infinigen_examples.generate_individual_assets -f seashells -n 1 --save_blend
-python -m infinigen_examples.generate_individual_assets -f chunkyrock -n 1 --save_blend
+python -m infinigen_examples.generate_individual_assets --output_folder outputs/corals -f CoralFactory -n 8 --save_blend
+python -m infinigen_examples.generate_individual_assets --output_folder outputs/seashells -f seashells -n 1 --save_blend
+python -m infinigen_examples.generate_individual_assets --output_folder outputs/chunkyrock -f chunkyrock -n 1 --save_blend
 ```
 
 <p align="center">
@@ -23,15 +19,26 @@ python -m infinigen_examples.generate_individual_assets -f chunkyrock -n 1 --sav
   <img src="images/individual_assets/chunkyrock.png" width="150" />
 </p>
 
-Running the above commands will save images and .blend files into your `outputs` folder.
+Running the above commands will save images and .blend files into your `outputs` folder. You can customize what object is generated by changing the `-f` argument to the name of a different AssetFactory defined in the codebase (see the file `tests/test_meshes_basic.txt` for a partial list).
 
 Please run `python -m infinigen.tools.generate_individual_assets --help` for a full list of commandline arguments.
 
-The most commonly used arguments are:
-- `-f` to specify the name(s) of assets or materials to generate. `-f NAME` can specify to generate three different types of objects:
-   - If `NAME` is the name of a class defined in `infinigen/assets`, then it will be treated as an AssetFactory and used to generate objects from scratch. For example, you can say `-f CactusFactory` or `-f CarnivoreFactory`, or use the name of any similar Factory class in the codebase.
-   - If `NAME` is the name of a file in `infinigen/assets/materials`, that material will be applied onto a sphere
-   - If `NAME` is the name of a file in `infinigen/assets/scatters`, that scatter generator will be applied nto a plane
-- `-n` adjusts the number of images / blend files to be generated.
+### Creating OBJ, USD, FBX and other file formats
+
+You can use the `--export` flag to export each generated asset to a format of your choosing. Please see [ExportingToExternalFileFormats](./ExportingToExternalFileFormats.md) for details and limitations regarding exporting. 
+
+Examples:
+
+```bash
+
+# Save a coral as an OBJ with texture maps
+python -m infinigen_examples.generate_individual_assets --output_folder outputs/corals -f CoralFactory -n 1 --render none --export obj
+
+# Save a bush as OpenUSD 
+python -m infinigen_examples.generate_individual_assets --output_folder outputs/bush -f BushFactory -n 1 --render none --export usdc
+
+# See the full list of supported formats
+python -m infinigen_examples.generate_individual_assets --help
+```
 
 

infinigen/__init__.py

@@ -1,3 +1,3 @@
 import logging
 
-__version__ = "1.3.1"
+__version__ = "1.3.2"

infinigen/assets/lighting/__init__.py

@@ -1,3 +1,2 @@
 from . import sky_lighting
-from .caustics_lamp import CausticsLampFactory
-from .glowing_rocks import GlowingRocksFactory
+from .caustics_lamp import CausticsLampFactory

infinigen/datagen/configs/compute_platform/slurm_1h.gin

@@ -1,5 +1,7 @@
 include 'compute_platform/slurm.gin'
 
+slurm_submit_cmd.slurm_niceness = 0
+
 iterate_scene_tasks.view_block_size = 3
 
 queue_combined.hours = 1