Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GeoZarr Extension #124

Closed
christophenoel opened this issue Jan 17, 2022 · 14 comments
Closed

GeoZarr Extension #124

christophenoel opened this issue Jan 17, 2022 · 14 comments
Labels
protocol-extension Protocol extension related issue

Comments

@christophenoel
Copy link

christophenoel commented Jan 17, 2022 •
edited
Loading

Dear Community,

In the context of an European Space Agency (ESA) GSTP (General Support Technology Programme) we prepared a first draft for a geospatial extension to Zarr (GeoZarr).

This specification is an early draft (v0.1) and a prototype implementation of GeoZarr-based (and Cloud based) geospatial data store has been demonstrated to the Agency in the frame of the project. We are also developing a map viewer (based on OpenLayers) to demonstrate multiscaling and integration of the store in an Earth Observation catalogue.

If you have any interest in Space domain, we would be very interested to get your feedback and ideas to consolidate this extension. We mostly based the specification on what xarray already supports.

Here is the repository: https://github.com/christophenoel/geozarr-spec/blob/main/geozarr-spec.md

Kind regards,

Christophe Noël, Spacebel.
@joshmoore
Copy link
Member

First off, big 👍 for this, @christophenoel.

an alternative channel to share this information

Probably https://github.com/zarr-developers/zarr-specs would be best for the moment. Happy to transfer if there are no objections. (I'll point out though that as with #50 it may be that portions are intended for the core spec while portions are more an extension or a community convention)

This specification is an early draft (v0.1) ...

It looks like there's also some overlap with upcoming proposals for NGFF (the proposal that developed out of #50) like use of UDUNITS (ome/ngff#57 (comment)) which is re-assuringly complementary.

We are also developing ...

Links welcome.

If you have any interest in Space domain, we would be very interested to get your feedback and ideas to consolidate this extension. We mostly based the specification on what xarray already supports.

👍 A push to better unify xarray and zarr is just getting underway so this is really great timing. Thanks again!

cc: @jhamman @aurghs etc.

@rabernat
Copy link
Contributor

This is a great effort @christophenoel!

I am leading the Zarr OGC community standard effort. The google document linked below is a "preamble" which will soon be voted on by OGC - https://docs.google.com/document/d/199yucgyJbU3tTZcO1lEbCEWHIvkUXoR1/edit?usp=sharing&ouid=102969561847050549657&rtpof=true&sd=true. It mentions geospatial data in several ways. I would love to have your comments.

It would be great to align efforts. I will try to review your spec over the next week.

@jhamman
Copy link
Member

jhamman commented Jan 17, 2022

+1 to transferring this issue to zarr-specs

And +1 to hearing more about the map viewer work (perhaps that could use its own issue).

@christophenoel
Copy link
Author

Thanks a lot for your friendly encouragement.

Some feedback:

  • Multiscale: I used draft 50 ideas as a baseline. It was migrated to image.sc but misses the geospatial aspects. From my point of view, resolution property is relevant for geospatial data to ease the computation of the scale according to zoom level (instead of computing the value from the coordinates variables)
  • Demonstrators: the dissemination phase is not started yet but presentations are planned with the Agency and related organization such as the OGC. I will keep you inform if any public meeting is scheduled during the next months. In particular, we have worked with ScanWorld company who developed real use cases examples consuming the (serverless) cloud-native data store. I will share the link if they become available publicly.

@christophenoel
Copy link
Author

I am leading the Zarr OGC community standard effort. The google document linked below is a "preamble" which will soon be voted on by OGC - https://docs.google.com/document/d/199yucgyJbU3tTZcO1lEbCEWHIvkUXoR1/edit?usp=sharing&ouid=102969561847050549657&rtpof=true&sd=true. It mentions geospatial data in several ways. I would love to have your comments.

Very interesting. I have read very carefully the outcome from the COG/Zarr report from OGC Testbed 17. I keep in touch with you.

@jakirkham jakirkham transferred this issue from zarr-developers/zarr-python Jan 18, 2022
@jakirkham
Copy link
Member

Have gone ahead and transferred this issue to zarr-specs since it seems like we are ok with that

@jhamman jhamman mentioned this issue Jan 18, 2022
Open
@christophenoel christophenoel mentioned this issue Jan 19, 2022
Closed
@joshmoore joshmoore mentioned this issue Jan 21, 2022
Open
@joshmoore
Copy link
Member

@christophenoel, I've just added #125 which will be ramping up over the course of the next week or two. Hopefully this would help us to refactor any of the shared issues from the multiscale conventions into a shared document.

@christophenoel
Copy link
Author

christophenoel commented Jan 25, 2022 •
edited
Loading

It would be great to align efforts. I will try to review your spec over the next week.

Hi Ryan, it tried to reach you by email unsuccessfully. I will be working on this by mid february.

@rabernat
Copy link
Contributor

rabernat commented Jan 25, 2022 via email

@jhamman jhamman mentioned this issue Jan 26, 2022
Open
@christophenoel
Copy link
Author

christophenoel commented Mar 21, 2022 •
edited
Loading

Dear all,

I'm very happy to share with you a first bunch of resources (presentations, demos) presenting the project, and showing the assets of GeoZarr.

We finally refined the multiscales support and recently introduced visual portrayals and symbology (see OGC WMS SLD) directly in the data to facilitate the display on map viewers.

I hope this helps !

Christophe.

Specification Document - Change Log

Demonstration Videos (Youtube channel):

OpenLayers extension prototype: https://github.com/spacebel/geozarr-openlayers

@aurghs
Copy link

aurghs commented Mar 21, 2022

That's very interesting!

@joshmoore
Copy link
Member

Thanks, @christophenoel. 👍 I've passed your comment on to the github.com/ome/ngff folks as well. You might be interested in further iterations to the OME-Zarr spec built on top of multiscales, e.g. Transformation types from @bogovicj. If you have any thoughts or just general interest in discussing ways of sharing updates to a base spec, let me/us know, here or possibly at a regular community call (which happens to be today).

All the best,
~Josh

@joshmoore
Copy link
Member

@christophenoel: you might be interested in ome/ngff#114 which resulted from discussion with various xarray-affine folks.

cc: @thewtex @malmans2 @TomNicholas @aurghs

@jstriebel jstriebel added the protocol-extension Protocol extension related issue label Nov 16, 2022
@briannapagan
Copy link

Can this be closed in light of: https://github.com/zarr-developers/geozarr-spec

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
protocol-extension Protocol extension related issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@joshmoore @rabernat @jhamman @jakirkham @jstriebel @christophenoel @briannapagan @aurghs