Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zarr3 shard user block / offset #186

Open
mkitti opened this issue Aug 5, 2024 · 0 comments
Open

zarr3 shard user block / offset #186

mkitti opened this issue Aug 5, 2024 · 0 comments

Comments

@mkitti
Copy link

mkitti commented Aug 5, 2024

It would nice if for Zarr version 3 shards, one could specify a byte offset for tensorstore to start writing at. This way one could store additional binary data at the beginning of a zarr shard. This is conceptually similar to a HDF5 User Block. The idea would be to allow an arbitrary number of X bytes at the beginning of a shard file that tensorstore will not touch.

A modification to the Zarr v3 standard is not needed since the offset itself is encoded in the Zarr v3 shard index. This is an implementation option.

With this change the user block could contain HDF5 meta data or TIFF IFDs, allowing Zarr shards to also be HDF5 or Zarr v3 files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant