Request only needed metadata

[loshjawrence]

It would be nice if a tileset.json could say what "layers" are available (extra metadata or baked pre-calculations). Up to the user's needs if they want to fetch 1 or more of these things or not. Just a way of decoupling the data so it's not all in the tile payload.

The json would list this metadata as a bunch of structs (the name, the fields and what their primitive data types are). to fetch you just append the name to the tile uri.

Maybe time could be treated as a layer to extend this to time-dynamic support.

If you wanted arbitrary amounts of per-point metadata on a pointcloud without baking it all the tile, you could break those out into layers (maybe a basis file (if lossy is OK, probably not ok for enum id's) or png) where each pixel or pixel channel corresponds to a point.

Similar to 1/0 availability subtree, if some metadata isn't available for every tile in the tree it could say so... which would indicate you need to fetch a subtree of availability for the metadata to know which tiles have it and which tiles do not.

[loshjawrence]

The block of points is a pointcloud tile. The base tile payload is just a minimum set of data for basic rendering .i.e. pos/color. The arrows are separate pieces of metadata available for in the tileset. The idea is that we decouple metadata from the tile to scale to arbitrary amounts of metadata. The metadata available to the tileset could be communicated in the tileset.json as an array of strings. These strings could just be suffixes to the base d/x/y/z uri to make it easy to request. Metadata payloads could be .png/.basis images depending on lossy tolerance for the metadata, UV computed from VBO index or whatever makes sense for the tile type. The metadata images (as ktx payloads) could store additional "mips"which could be the pre-computed min, max, ave, std.dev, summed aread table for that per-feature metadata for that tile, making analysis easier (volume/area).

Could easily have per-feature, per-tile, per-subtree, per-tree versions.

@pjcozzi what I was trying to communicate yestarday
@OmarShehata is an approach like this something that would help Uber out?

[kring]

Cool idea! I have a few random thoughts...

The first use-case of "various sets of point clouds, image files, photogrammetry, whatever" doesn't sound super likely to me. I mean, yeah, you might have those things, but I wouldn't expect them to be displayed together all that often. But maybe it's more common than I think...

Even if I'm right, though, the other use-case of a layer per discrete time makes heaps of sense. I think we can learn from the example of WMS. WMS has layers, but it also has things called dimensions. Layers are totally different datasets, while dimensions are different "slices" of the dataset. For example, time and elevation/depth are two really common dimensions (elevation is a slice because WMS is 2D). Each layer can have zero or more dimensions, and each dimension has a list of possible values (e.g. the times for which there is data). When you request some data from a WMS server, you request a particular layer with a particular value for each of the dimensions. It would be possible to achieve the same thing without any concept of dimensions, using only layers, just by advertising layers for the cartesian product of the datasets and available dimension values, but it wouldn't be as good. It would be more confusing for humans, and it would make software less easily able to present the data on the WMS server in a meaningful way. For example, our TerriaJS software treats layers as separate datasets that can be toggled on and off independently, while it automatically presents a timeline for a Time dimension (and pre-fetches the next time) and dropdowns for other dimensions.

I think a single tileset.json should be just one layer, but that it should support multiple dimensions. The tile URL would include placeholders for the dimensions in the same way it will (presumably) include placeholders for l/x/y/[z] in implicit tiling.

Like I said, I think the use-case for simultaneously displaying multiple sets of tiles with the same tiling scheme and availability is pretty limited, especially if those sets of tiles must be packaged together in the same tileset.json. But on the other hand, optimizing traversal in the case where two different tilesets with different availability (from different data providers, even) use the same implicit tiling scheme and implicit error values could be worthwhile. E.g. two buildings tilesets for each of two nearby cities. It could get tricky, though, so we'd have to weight the complexity against the actual benefit. For example, in my two cities example, the two tilesets would really only have shared traversal for the low-detail tiles that contain both cities.

re: metadata: For some stuff I'm working on right now, we've "solved" this problem by storing only a feature ID in the batch table. Then when the user clicks a feature, we request a separate JSON file from a URL templated with that feature ID, and that JSON file contains the rest of the metadata. Obviously this only works if we don't need to style based on the metadata. Which I think would be true of a metadata layer in tileset.json too, right? But having an external metadata mechanism as a core part of 3D Tiles would be nice. If the feature metadata is chunked by tile (rather than by feature as I'm currently doing) but not part of the main 3D content tile (so it must be downloaded separately), that's supporting a use-case where the metadata isn't always needed but, when it is, we're going to want it for a bunch of features in the tile not just for one. Is there such a use-case?

I can't quite think of what I would use per-tile (rather than per-feature) metadata for, other than maybe debugging. The way I've used 3D Tiles, at least, the tiles have no particular meaning to the user, they're just an implementation detail.

Kind of just a stream of consciousness here, hope some of it is useful.

[loshjawrence]

Thanks for taking the time to write feedback. I need to know more about real use cases but the motivation for this was a hypothetical situation for pointclouds where you have a lot of metadata that could be represented in a texture.

[loshjawrence]

Consider this for storing metadata.
https://arrow.apache.org/

[lilleyse]

One idea: could metadata availability be encoded in implicit tiling?

Also CC #136 - storing metadata in external buffers