Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New types and traits for (co-existing!) eager serialization #8642

Merged
merged 5 commits into from
Jan 13, 2025

Conversation

teh-cmc
Copy link
Member

@teh-cmc teh-cmc commented Jan 10, 2025

This introduces SerializedComponentBatch, which will become the main type we use to carry user data around internally.

/// The serialized contents of a [`ComponentBatch`] with associated [`ComponentDescriptor`].
///
/// This is what gets logged into Rerun:
/// * See [`ComponentBatch`] to easily serialize component data.
/// * See [`AsComponents`] for logging serialized data.
///
/// [`AsComponents`]: [crate::AsComponents]
#[derive(Debug, Clone)]
pub struct SerializedComponentBatch {
    pub array: arrow::array::ArrayRef,

    // TODO(cmc): Maybe Cow<> this one if it grows bigger. Or intern descriptors altogether, most likely.
    pub descriptor: ComponentDescriptor,
}

The goal is to keep the ComponentBatch trait isolated at the edge, where it is used as a means of easily converting any data into arrow arrays, instead of simultaneously being used as a means of transporting data around through the internals.
ComponentBatch is here to stay, if only for its conversion capabilities.

This opens a lot of opportunities of improvements in terms of DX, UX and future features (e.g. generics).

The two code paths will co-exist for the foreseeable future, until all archetypes have been made eager.

Copy link

github-actions bot commented Jan 10, 2025

Web viewer built successfully. If applicable, you should also test it:

  • I have tested the web viewer
Result Commit Link Manifest
7d74213 https://rerun.io/viewer/pr/8642 +nightly +main

Note: This comment is updated whenever you push a commit.

Copy link

github-actions bot commented Jan 10, 2025

Latest documentation preview deployed successfully.

Result Commit Link
7d74213 https://landing-k2akvml6q-rerun.vercel.app/docs

Note: This comment is updated whenever you push a commit.

@teh-cmc teh-cmc added enhancement New feature or request 🦀 Rust API Rust logging API 🍏 primitives Relating to Rerun primitives include in changelog labels Jan 10, 2025
@teh-cmc teh-cmc marked this pull request as ready for review January 10, 2025 16:01
@emilk emilk self-requested a review January 10, 2025 17:40
@teh-cmc teh-cmc merged commit c171b04 into main Jan 13, 2025
10 of 16 checks passed
@teh-cmc teh-cmc deleted the cmc/eager_partial_0_new_traits branch January 13, 2025 08:18
teh-cmc added a commit that referenced this pull request Jan 13, 2025
…8643)

Generate methods for all archetypes that allow retrieving the
fully-qualified descriptor for any of the archetype's field.
E.g. this method returns the descriptor for the `radii` field of
`Points3D`:
```rust
impl Points3D {
    /// Returns the [`ComponentDescriptor`] for [`Self::radii`].
    #[inline]
    pub fn descriptor_radii() -> ComponentDescriptor {
        ComponentDescriptor {
            archetype_name: Some("rerun.archetypes.Points3D".into()),
            component_name: "rerun.components.Radius".into(),
            archetype_field_name: Some("radii".into()),
        }
    }
}
```

This becomes a must as we start require tags in more and more places
(e.g. partial updates APIs).

* DNM: requires #8642 
* Part of #8581
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request include in changelog 🍏 primitives Relating to Rerun primitives 🦀 Rust API Rust logging API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants