-
Notifications
You must be signed in to change notification settings - Fork 402
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize out unnecessary joins across autogenerated instance keys #3107
Comments
My proposed solution here would be to completely drop the notion of autogenerated instance keys altogether (which would fix a bunch of other issues). When we get two components to join and none of them have keys, then we know we can blindly pair them. |
The solution proposed above induces way too many cascading changes resulting from having to make cluster keys optional (internally), since everything has been built on the assumption that they should always be present (and rightly so). A less invasive solution would be to indicate in the query results whether or not the returned instance keys were autogenerated (which we know since all autogenerated keys of a given length point to the same arrow buffer). |
I think this is high-priority for 0.10, since we have introduces potential performance regressions in 0.9 because of it (#3363) |
What the title says; see the commits for details, it's pretty trivial. Going from a ~62ms space view build time to ~40ms in the OPF example, so pretty nice gains overall. OPF, before: ![image (17)](https://github.com/rerun-io/rerun/assets/2910679/700fe397-7fa2-4f4c-b661-07b9dfb30937) OPF, after: ![image](https://github.com/rerun-io/rerun/assets/2910679/4b4ae9db-ee6b-4318-a0ed-ea25e08facdd) * Fixes #3107
The store will auto-generate monotically increasing instance keys when they are not passed in by the user. This allows
re_query
's join logic to "just work" no matter what but it also means that in most cases we're computing a costly join across 2 components of the same size, both with autogenerated instance keys, even though we already know the answer to that join.The text was updated successfully, but these errors were encountered: