Support for more 2D and 3D primitives #1361
Comments
jleibs
added
enhancement
|
I think we should add a minimal version of this ASAP - it can be something that just generates meshes during scene construction for now. This is very limiting at the moment, and we need this to test other things like transparency, 2D layering, and styling. |
|
An interesting note: a For 2D we support it via specifying its min/max corners, or center and size, and some other options. We will also add support for 2D transforms (#349). For 3D we may want the same 2D interface plus a 3D transform, treating the 2D rect as a 3D shape with an implicit Z=0. |
emilk
changed the title
|
Rect/Box we have, but needs styling (Separate issue). Cylinders and capsules I think are very useful (e.g. limbs) 2D circles/discs are very useful. So are spheres. What is the difference with point? Point radii can be auto and ui-based, but spheres/circles need a real radius. |
|
It would be great to have a way to visualize the grid or ground. This could be achieved through a loggable object or simply as a visual artifact. |
|
@jeongseok-meta yeah I think this should be builtin feature of the viewer that doesn't require logging anything (other than maybe logging the settings itself on the space view once code configurable views are there). There's already a separate issue for this: |
|
Hi there. Just curious, has there been any progress or updates on this topic? Would love to know what's going on. |
|
@mont1004 Unfortunately the next release will have little new on this front, but this is still something that's high on our priority list. Do you have any particular primitive in mind that you would need most? |
|
@abey79 Thank you for the quick reply! My team has been developing Visual SLAM, and we've been eyeing the rerun as a top candidate for our new visualizer. As I mentioned earlier, we currently represent the covariance of the estimated camera poses with ellipsoids. Having ellipsoids that can be scaled and compared in size with one another would be very helpful, as this capability is crucial for us. |
|
Want to add like #6012 that adding text to an image, similar to OpenCV would be appreciated, oftentimes I don't want to log something in textlog/textdocument as I would like it to show in the image itself. As a work around I log a point2d or linestrip with 0 radius and add a label. This works, but I can't easily change the size of the font like cv2.drawtext would allow |
|
@pablovela5620: Are you adding the text to a particular place in the image (i.e. tying it to a geometric location), or is the text more of a description of the image (i.e. it could alternatively be below the image for instance)? |
|
Generally to a geometric location, for example detecting someone's face and placing the text next to or above their detected face. Then telling them to either get closer or farther from the screen or to move their face to a specific location. Below an image could also be helpful (though I think this would already be possible with a textdoc + blueprint?) |
|
Rays and planes as per #2179 would be great |
|
For the 3D primitives, we should have a style option to either render them as solid, as wireframe, or both. The spatial space-view can convert them to a re_renderer mesh and/or lines as needed. Initially we could keep the coarseness of these primitives hard-coded (i.e. how many subdivisions of a sphere). To keep things simple, I suggest we use the Which axis should "height" be along? I doesn't really matter, but once we picked something we should stick with it forever, and also use it for all other shapes (i.e. the axis along which a capsule and cylinder stretches). It's worth spending some time investigating what is the most common choice in other visualization tools. (VRML uses Y as primary axis) We should still have nice helpers to construct these primitives. For instance, defining a capsule by two points and a radius (a fat line segment) but then logging that as a single component Capsule { radius, length } plus an For the 3D ellipsoids: I did some preliminary experimentation with these in #4991 |
We use the (PS: don't mind my edit on your comment, I inadvertently pressed |
|
A less important, but very nice, primitive, is the tapered capsule:
|
### What Implement an `Ellipsoids` archetype and visualizer. This may be used to render ellipsoids, or spheres where `Points3D` is unsuitable because the actual boundary in space is significant. It is intended as a first step towards completing #1361, by adding ellipsoids and by sketching out a general mechanism for rendering procedurally-generated shapes, which will be reusable for cylinders and anything which needs many vertices to render. The archetype, extensions, etc. are largely forked off of `Boxes3d`. Unfinished parts / future work: * Currently, the spheres are always drawn as wireframes (with an automatically chosen mesh subdivision level). We will want to also support solid rendering (i.e. triangles in place of lines), but many use cases for spheres might want them to be transparent, which isn't yet properly supported in 3D space views (#702), so I chose wireframe as the first version. * Currently, all of the wireframe lines are piped through the `LineDrawableBuilder` every frame. Instead, they should be drawn as instanced meshes; that will require implementing a new renderer that invokes the existing line shader with the different data layout. However, this current implementation should not be worse than if the user were to create a similar number of lines directly. * The sphere mesh needs a choice of subdivision level. Currently, it is automatically derived from the size of the sphere in the scene, which may be inappropriate depending on the scale of scene units or the specific application. I'd like feedback on the best way to handle this. Just add another component, perhaps kind of like the `Radius` component? * I‘m not sure what the best way to handle the (future) errors in generating meshes is. Should it return a `SpaceViewSystemExecutionError` (which hides unrelated data), just draw nothing and continue (which could lead to silent missing data), or something else?  ### Checklist * [X] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [ ] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6853?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6853?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [X] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [x] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide - [PR Build Summary](https://build.rerun.io/pr/6853) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`. --------- Co-authored-by: Emil Ernerfeldt <[email protected]>
…`Ellipsoids`. (#6953) ### What With these changes, `Boxes3D` and `Ellipsoids` can now be viewed as solid objects. This is part of the work on #1361 and the mechanisms added here will generalize to other shapes. * Add new component type `SolidColor`. This is identical to `Color` except that, on shapes where it applies, it sets the color of surfaces instead of lines. Whether its color is transparent controls whether the surfaces are drawn at all. * Add `SolidColor` to the `Boxes3D` and `Ellipsoids` archetypes. * Add support for solid colors to those archetypes’ visualizers. * Add `proc_mesh::SolidCache` to cache the calculation of triangle meshes. * Add `proc_mesh::ProcMeshKey::Cube` to allow the cached mech mechanism to generate solid cubes.   Future work (things this PR *doesn't* do): * Viewer UI needs a version of the color picker that lets you pick "fully transparent". * The line renderer does not play well with adjacent faces, causing line width to be halved some of the time. This could be fixed with something like #6508, simple depth offsets, or something else. * Getting naming and semantics right: * Should the `colors` of `Boxes3D` be renamed `line_colors`, or something like that? I think so, but that would be a breaking change, so I didn't in this PR. * Should `Color` be renamed? * Should there be all 3 of `SolidColor`, LineColor`, and `Color` with some override policy between them? * Should `SolidColor` be called `SurfaceColor` instead? * How should `SolidColor` interact with annotation contexts? * Figure out whether instanced meshes are the right choice for performance. ### Checklist * [X] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [X] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6953?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6953?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [X] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [X] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [X] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide * The `Ellipsoids` archetype has a renamed field but that isn't released yet, so doesn't need noting. - [PR Build Summary](https://build.rerun.io/pr/6953) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`. --------- Co-authored-by: Andreas Reich <[email protected]>
|
Progress update:
More 3D shapes to come! |
### What `ProcMeshVisBuffer` extracts common elements from `Boxes3DVisualizer` and `Ellipsoids3DVisualizer`. This will be useful to minimize duplicated code when further proc-mesh shapes are added (#1361). Additionally, `ComponentFallbackProvider` is no longer a supertrait of `VisualizerSystem`. Instead, the fallback provider is always accessed through `as_fallback_provider()` (which may return `&self` if it likes). The advantage of this is that a visualizer may access its desired fallback logic even while parts of itself are exclusively borrowed, by putting the `TypedComponentFallbackProvider` implementations on a separate type. (Strictly speaking, the supertrait change is not needed, if we instead let a `ComponentFallbackProvider` implementation be made from `TypedComponentFallbackProvider`s on a different type, but that seems like a worse design to me.) ### Checklist * [X] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [X] I've included a screenshot or gif (if applicable) * [ ] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7507?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7507?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [X] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [X] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [X] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide - [PR Build Summary](https://build.rerun.io/pr/7507) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.
What is the main axis of a cylinder/capsule?Finding this information online wasn't easy. These are the only ones I've managed to confirm (so far): X
Y
Z
|
|
ROS Viz-marker aligns with urdf/xacro spec in using Z for the length/height axis: https://wiki.ros.org/rviz/DisplayTypes/Marker#Cylinder_.28CYLINDER.3D3.29 |
|
A lot of 3D libraries have a configurable main-axis, that the user can set to any Vec3. I think we should provide the same. That way a user can easily configure where a cylinder points, without having to calculate a quaternion and read our docs about what our default is. So I suggest we introduce a |
### What * Part of #1361 Adds the archetype `Capsules3D`, component `Length`, and a matching visualizer. Capsules are defined by their length and radius, and then transformed in the usual fashion to set their orientation. There is also a constructor for joining a pair of endpoints. <img width="483" alt="image" src="https://github.com/user-attachments/assets/28bd3104-0c14-41c0-aeb4-6e15b4547b0f"> Missing parts that should make it into this PR before it leaves draft: * Needs an example image — I assume I don't have the permissions to upload one. Further work not included in this PR, that will add complications: * C++ and Python have no `from_endpoints` constructors, because there wasn’t the quaternion math handy. Should we add an equivalent to `glam::Quat::from_rotation_arc()` to the minimal Rerun quaternion types? * There is no wireframe mesh support (expressed as having no `FillMode` component). This is because it was recommended that I use `re_math`’s capsules, but `re_math` has no support for generating wireframes. In the long run, we will want to support tapered capsules and cylinders and a new or substantially revised mesh generator to handle these cases will be needed anyway. * The support for setting the length of a capsule is a horrible kludge which is hard on the mesh cache — each length:radius ratio is cached separately. The idea I have for fixing this is adding “bone” support — the ability for selected vertices to be displaced by an externally controlled offset. This allows taking a single capsule mesh and stretching it without distorting the endcaps. (Tapered capsules will require multiple meshes to account for the angle at which the cone meets the endcap, but those can be fairly well approximated by a finite set of meshes.) ### Checklist * [X] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [X] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7574?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7574?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [X] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [X] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [X] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide - [PR Build Summary](https://build.rerun.io/pr/7574) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`. --------- Co-authored-by: Emil Ernerfeldt <[email protected]>
It would be nice to be able to log the common 2D/3D geometry building blocks in order to enable building up simple 3D models without going all the way to fully specified meshes.
Some basics:
Maybe more, but good enough for VRML, good enough for me?
These should all consistently support components for
OutOfTreeTransform3D, Color, and Fill.The text was updated successfully, but these errors were encountered: