Instanced 3D Model and Composite Tile Formats

[pjcozzi]

First, see the original notes for Instanced 3D Model and Composite tile formats. To help design the Instanced 3D Model tile format, see the Batched 3D Model tile format.

We now have significant work towards instancing in Cesium. See CesiumGS/cesium#3049.

Here's some initial thoughts:

Instanced 3D Model

Header
------
magic            // uchar[4], "i3dm"
version          // uint32,   1
batchTableLength // uint32,   can be zero, length of the batch table in bytes
glTFLength       // uint32,   length of glTF section (after the Batch Table) in bytes
glTFFormat       // uint32,   0 = url, 1 = embedded binary glTF
instancesLength  // uint32,   number of instances, NOT number of bytes, can be zero I guess

Batch Table
-----------
// Same as Batched 3D Model.  Optional.

glTF
----
// UTF-8 string or binary blob depending on Header.glTFFormat
// Length in bytes is Header.glTFLength

Instances
---------
There are Header.instancesLength of these:
* x, y, z    // doubles in WGS84 coordinates
* x, y, z, w // doubles in WGS84 coordinates (quaternion)
* batchId    // uint32, index into batchTable, zero when Header.batchTableLength === 0

• We can significantly optimize the size of each instance field, e.g., positions can be quantized relative to the bounding volume (see EXT_quantized_attributes). Rotation could also be optional (part of the batchTable) and default to Y up (glTF default) in a local frame for the instance's position. Rotation could use fewer fields. RTC. Delta encoding... SOA vs. AOS... Look into point cloud compression... Instanced 3D Model updates #33
• Should instance.batchId be omitted when header.batchTableLength === 0. This will complicate the client, but save memory; however, the case it saves memory for is probably rare in practice. Instanced 3D Model updates #33
• Throughout 3D Tiles, should batchTable be renamed? Something like propertyTable, metadataTable, etc. We can also decide this later as we define more tile formats. Consistent terminology is important. Moved to Batch Table changes #32
• Throughout 3D Tiles, should the batchTable work more like glTF buffer/bufferView/accessor to avoid arrays of numbers? Moved to Batch Table changes #32

Composite

In order to have different models instanced in the same tile (or more generally, combine any number of tiles, even tiles of different formats), we can use a Composite tile format.

Perhaps:

Header
------
magic            // uchar[4], "cmpt"
version          // uint32,   1
tilesLength      // Number of tiles

Tiles
----
// header.tilesLength (potentially heterogeneous) tiles

• To make this easier to parse on the client, each tile's header could include its total length in bytes or the Composite can contain an array of lengths. I think we want to store it in each tile so other binary streams containing tiles can be easily parsed.

[pjcozzi]

We'll also want to be able to support LOD where instanced models are rendered when zoomed in close, and billboards are rendered when zoomed out. I think the right tile.json layout (e.g., with i3dm tiles at the leafs and vector tiles with billboards in non-leafs) will work.

[lilleyse]

Updated batchTableLength to be the length in bytes rather than the numbers of elements.

[lilleyse]

Some questions:

• glTF can be embedded binary or a url. Should Batched3DModel also allow both for consistency? Should Instanced3DModel only allow a url?
• Naming conventions: usually we use length to specify byte length but instancesLength refers to the number of instances. Should we use byteLength instead?

[pjcozzi]

+1 on byteLength. We should also rename it for b3dm.

[mramato]

Would we be able to specify any kind of scaling factor for each instance as part of the tile, or would that have to be done client side after-the fact? For example, I may have 100k trees that all use the same model but each tree's height and radius might be different based on their size and age. It would be convenient to bake that information into the tile rather than have to store the meta-data and do extra work to scale them every time the tile is loaded.

[pjcozzi]

Yes, scale will be a semantic in the Batch Table just like, for example, b3dm has per-building heights.

[pjcozzi]

• Initial Cesium implementation for i3dm tile: Instancing models 3d tiles cesium#3059
• Initial Cesium implementation for composite: Composite tiles cesium#3158

[pjcozzi]

• Composite spec: Composite tile format spec #31

[pjcozzi]

glTF can be embedded binary or a url. Should Batched3DModel also allow both for consistency? Should Instanced3DModel only allow a url?

Discussed offline with @lilleyse. No changes here. b3dm embeds binary glTF since it is unlikely that tiles would reuse the same model; i3dm has the uri option to share models across tiles and the embed option for a single tile use case.