Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OGC Charter #23

Merged
merged 33 commits into from
Jul 17, 2023
Merged
Changes from 9 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
ba47de4
OGC SWG Chart initial draft
christophenoel Apr 12, 2023
81a460e
Added business value prop
briannapagan May 10, 2023
866f3dc
updating remaining empty headers
briannapagan May 24, 2023
2b1a33f
minor formatting fixes
briannapagan May 24, 2023
dafbe8e
adding authors
briannapagan May 24, 2023
8fc7e7f
Update CHARTER.adoc
christophenoel May 24, 2023
1c9c286
adding Alexey to supporters
briannapagan May 26, 2023
e296f30
adding relation to geodatacube swg
briannapagan May 26, 2023
e3974c6
typo in convention
briannapagan May 26, 2023
9a588b6
Update CHARTER.adoc
christophenoel May 26, 2023
2564ee2
mention of the OGC netCDF S>G
christophenoel May 26, 2023
c3c8bea
adding Tyler Erickson to list of supporters
briannapagan May 31, 2023
94b41fb
remove accidental geoparquet reference and update list of supporters
briannapagan May 31, 2023
ff091bb
Relaxing the work on CF
christophenoel Jun 15, 2023
b91ed5e
typo cloud-native
christophenoel Jun 15, 2023
cadfcfc
rewording of data discovery
christophenoel Jun 15, 2023
5fafcf2
update multiple projections line
briannapagan Jun 21, 2023
c0a316f
update format on xcube line
briannapagan Jun 21, 2023
e62b316
Update CHARTER.adoc
briannapagan Jun 21, 2023
c162329
specify rechunking examples in space and time
briannapagan Jun 21, 2023
e6840a1
add ZEP 4 reference
briannapagan Jun 21, 2023
de2711e
add clarification on geodatacube specifying an API
briannapagan Jun 21, 2023
747a08f
Update CHARTER.adoc
christophenoel Jul 3, 2023
dfe623e
remove visualization as a specific item, included conformance class t…
briannapagan Jul 5, 2023
ca90100
fix typo and remove reference to multiple projections
briannapagan Jul 5, 2023
ee8ffdb
removing the line regarding encoding of time series
briannapagan Jul 5, 2023
6006017
change wording of multiple resolutions line
briannapagan Jul 5, 2023
7e83fd4
add line about portability
briannapagan Jul 5, 2023
9b0d97c
format change on xcube line
briannapagan Jul 6, 2023
ed96db6
indent to make xcube line clearer
briannapagan Jul 6, 2023
0897f0f
type in tyler's association
briannapagan Jul 6, 2023
f473687
updated start date
briannapagan Jul 17, 2023
75acf3c
switch 'will' to 'can' define a standard cloud-native format
briannapagan Jul 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 198 additions & 0 deletions CHARTER.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
:Title: OGC GeoZarr Standards Working Group Charter
:titletext: {Title}
:doctype: book
:encoding: utf-8
:lang: en
:toc:
:toc-placement!:
:toclevels: 4
:numbered:
:sectanchors:
:source-highlighter: pygments

<<<
[cols = ">",frame = "none",grid = "none"]
|===
|{set:cellbgcolor:#FFFFFF}
|[big]*Open Geospatial Consortium*
|Submission Date: 2023-05-01
|Approval Date: 
|Internal reference number of this OGC(R) document: 23-xxx
|Category: OGC(R) Standards Working Group Charter
|Authors: Christophe Noel, Brianna R. Pagán
|===

[cols = "^", frame = "none"]
|===
|[big]*{titletext}*
|===

[cols = "^", frame = "none", grid = "none"]
|===
|*Copyright notice*
|Copyright (C) 2021 Open Geospatial Consortium
|To obtain additional rights of use, visit http://www.opengeospatial.org/legal/
|===

<<<

To: OGC members & interested parties

A new OGC Standards Working Group (SWG) is being formed. The OGC members listed below have proposed the OGC GeoZarr SWG. The SWG proposal provided in this document meets the requirements of the OGC Technical Committee (TC) Policies and Procedures.

The SWG name, statement of purpose, scope, list of deliverables, audience, and language specified in the proposal will constitute the SWG's official charter. Technical discussions may occur no sooner than the SWG's first meeting.

This SWG will operate under the OGC IPR Policy. The eligibility requirements for becoming a participant in the SWG at the first meeting (see details below) are that:
christophenoel marked this conversation as resolved.
Show resolved Hide resolved

* You must be an employee of an OGC member organization or an individual
member of OGC;
christophenoel marked this conversation as resolved.
Show resolved Hide resolved

* The OGC member must have signed the OGC Membership agreement;

* You must notify the SWG chair of your intent to participate to the first meeting. Members may do so by logging onto the OGC Portal and navigating to the Observer page and clicking on the link for the SWG they wish to join and;

* You must attend meetings of the SWG. The first meeting of this SWG is at the time and date fixed below. Attendance may be by teleconference.

Of course, participants also may join the SWG at any time. The OGC and the SWG welcomes all interested parties.

Non-OGC members who wish to participate may contact us about joining the OGC. In addition, the public may access some of the resources maintained for each SWG: the SWG public description, the SWG Charter, Change Requests, and public comments, which will be linked from the SWG’s page.
christophenoel marked this conversation as resolved.
Show resolved Hide resolved

Please feel free to forward this announcement to any other appropriate lists. The OGC is an open standards organization; we encourage your feedback.

== Purpose of the Standards Working Group

The GeoZarr Standard Working Group (SWG) is chartered to develop a Zarr encoding for geospatial gridded data in the form of Zarr conventions (based on the approach described in the draft Zarr Enhancement Proposal 4). Zarr specifies a protocol and format used for storing Zarr arrays, while GeoZarr defines **conventions** and recommendations for storing **multidimensional georeferenced grid** of geospatial observations (including rasters). The GeoZarr SWG will also work on improving the Climate and Forecast (CF) conventions if necessary, particularly for alternative coordinate reference system (CRS) encoding if relevant.
christophenoel marked this conversation as resolved.
Show resolved Hide resolved
christophenoel marked this conversation as resolved.
Show resolved Hide resolved



== Business Value Proposition

In the geospatial world, new cloud native data formats are emerging. Zarr is a generic data format for n-dimensional arrays that enables access to data in compressed chunks of the original array and has become increasingly popular to use for geospatial purposes. In June 2022, the OGC endorsed a community standard of Zarr V2.0 (https://zarr.readthedocs.io/en/stable/spec/v2.html). The purpose of this charter is to adopt a more explicit GeoZarr as an OGC Standard which would provide guidance to standardize the approach for encoding various aspects of geospatial data in zarrs.
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
christophenoel marked this conversation as resolved.
Show resolved Hide resolved

== Scope of Work

The goal of the GeoZarr specification is to establish flexible and inclusive conventions for the Zarr cloud-native format, specifically designed to meet the diverse requirements within the geospatial domain. These conventions aim to provide a clear and standardized framework for organizing and describing data, ensuring unambiguous representation.

The objectives of GeoZarr conventions includes:
christophenoel marked this conversation as resolved.
Show resolved Hide resolved

1. Compatibility: Ensuring easy compatibility with popular mapping and data analysis tools such as GDAL, Xarray, ArcGIS, and QGIS, enabling seamless integration into existing workflows.
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
2. Visualisation: Simplifying the creation and display of geospatial data in web browsers without the need for complex workarounds, making geospatial information more accessible to users.
3. Dimensions: Supporting multidimensional data, such as hyperspectral and altitude information, to address diverse geospatial data requirements.
4. Data Discovery: Providing metadata for discovering, accessing, and retrieving the data, including subsets or different arrangements of the data.
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
christophenoel marked this conversation as resolved.
Show resolved Hide resolved
5. Mixing Data: Facilitating the combination of different types of geospatial data, including satellite images, elevation maps, and weather models, to create comprehensive and informative datasets.
8. Flexibilty: Allowing scientists and researchers to work with diverse data types and projections in their preferred software and programming languages, promoting flexibility and adaptability in geospatial data processing and analysis.
briannapagan marked this conversation as resolved.
Show resolved Hide resolved

Specifically, the convention should provide guidance to standardize the approach for encoding various aspects of geospatial data, including:
christophenoel marked this conversation as resolved.
Show resolved Hide resolved

* Multiple related variables with heterogeneous coordinates (e.g., children or linked datasets)
* Multiple resolutions of the data, possibly leveraging multiscale drafts
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
* Data subsets that are only available at certain resolutions
* Multiple projections
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
* Multi-dimensional optimizations (rechunking)
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
* Supporting typical Earth observation (EO) products (for example, how to encode multispectral bands)
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
* Encoding of time series that have not been normalized
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
* Accessing the symbology of the corresponding data
briannapagan marked this conversation as resolved.
Show resolved Hide resolved

=== Statement of relationship of planned work to the current OGC standards baseline
As the existing draft GeoZarr metadata utilizes Climate and Forecast (CF) attributes, there is an expected relationship with the OGC CF-netCDF Data Model Extension Standard. If necessary, any adjustments or enhancements required for the GeoZarr specification will be considered as a proposal to improve CF conventions as well.
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
christophenoel marked this conversation as resolved.
Show resolved Hide resolved

briannapagan marked this conversation as resolved.
Show resolved Hide resolved

=== What is Out of Scope?
In early conversations around creating a draft GeoZarr specification, concerns arose multiple times around the CF encoding of CRS which may pose issues, see https://github.com/zarr-developers/geozarr-spec/issues/20. While these concerns will be discussed and suggestions created for potentially updating CF conventions, if resolutions cannot be made with the GeoZarr specification, we consider out of scope waiting on any subsequent updates to CF conventions to reflect these suggestions.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The out of scope section should be much stronger to provide guidance for and focus the SWG activities.

Will dimensions of l model data (like ensemble) be in scope? I think the answer is no?

Will data access patterns for GeoZARR be part of the SWG activities or will this be strictly limited to a ZARR data package? i.e. will the conformance target be a zarr dataset standard or will there be other conformance targets like web data access? I see some stuff above that says that yes, access patterns are in scope, but would shy away from that being part of the normative spec and leave it as informative / guidance on how a person might use GeoZARR.

Perhaps some things called out from the discussion around visualization and symbology? Other things that should be out?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree the charter should determine that. I also disagree about descoping those topics (which are covered in the original draft).

=== Specific Existing Work Used as Starting Point
* GeoZarr draft specification: https://github.com/zarr-developers/geozarr-spec/

=== Is This a Persistent SWG

[x] YES

[ ] NO

=== When can the SWG be Inactivated

The SWG can be inactivated once the SWG identifies no new tasks for the SWG and there are no open Change Requests.

== Description of deliverables
The GeoZarr SWG will deliver a candidate Standard and associated developer resources.

The SWG expects to have a candidate Standard ready for OGC Architecture Board (OAB) review and public comment within nine months of creation of the SWG. Because example implementations will be developed at the same time the candidate Standard is formalized, reference implementations that fully use GeoParquet should be documented at the same time the candidate Standard goes to vote.
briannapagan marked this conversation as resolved.
Show resolved Hide resolved

=== Initial Deliverables

The following deliverables will be the initial results of work of the SWG.

* OGC GeoZarr Standard

* GeoZarr developer resources

The targeted start date for this SWG is
briannapagan marked this conversation as resolved.
Show resolved Hide resolved

=== Additional SWG Tasks

No specific additional tasks are currently planned for the SWG.

== IPR Policy for this SWG

[x] RAND-Royalty Free

[ ] RAND for fee

== Anticipated Audience / Participants

This SWG will develop a Standard for general use in the geospatial community and suitable for data exchange beyond this community. Geospatial data providers and software implementers will be interested in assisting with the development of this Standard as well as the output of the SWG.

== Domain Working Group Endorsement

The SWG convenors will discuss the charter with potentially interested Domain Working Groups (DWGs) at the first opportunity.
briannapagan marked this conversation as resolved.
Show resolved Hide resolved

== Other informative information about the work of this SWG

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some narrative about the relationship to the CF and ZARR communities could go here?

=== Collaboration

All work in the Standards Working Group will be public and the SWG solicits contributions and feedback from OGC members and non-OGC members to the extent that is supported by the OGC Technical Committee Policies and Procedures.

The OGC GeoZarr SWG will collaborate on Standard development using a public GitHub repository and a Gitter channel. Development of the Standard will include the use of Issues and other project tools in GitHub.

=== Similar or Applicable Standards Work (OGC and Elsewhere)
briannapagan marked this conversation as resolved.
Show resolved Hide resolved

* The OGC endorsed a community standard of Zarr V2.0 (https://zarr.readthedocs.io/en/stable/spec/v2.html) in June 2022.
briannapagan marked this conversation as resolved.
Show resolved Hide resolved

* This SWG is closely related to the newly announced [Geodatacube SWG](https://www.ogc.org/press-release/ogc-forms-new-geodatacube-standards-working-group/). Essentially, GeoZarr will define a standard for a serverless datacube. Therefore, close coordination between these SWGs seems needed.
briannapagan marked this conversation as resolved.
Show resolved Hide resolved

The XCube project has potential synergies with the GeoZarr specification as it already relies and complies with CF conventions:
briannapagan marked this conversation as resolved.
Show resolved Hide resolved

* xcube Dataset Convention: https://github.com/dcs4cop/xcube/blob/master/docs/source/cubespec.md
briannapagan marked this conversation as resolved.
Show resolved Hide resolved

* xcube Multi-Resolution Datasets: https://github.com/dcs4cop/xcube/blob/master/docs/source/mldatasets.md
briannapagan marked this conversation as resolved.
Show resolved Hide resolved

=== Details of first meeting

The first meeting of the SWG will occur within four weeks of approval of the SWG charter.

=== Projected on-going meeting schedule

The work of this SWG will be carried out primarily on GitHub and via email, web conferences / calls, and at face-to-face sessions at OGC Member Meetings as agreed to by the SWG members. The web conferences / calls will be scheduled as needed and posted to the OGC portal. Voting on OGC GeoZarr Conventions content will be limited to SWG members only.

=== Supporters of this Charter

The following people support this proposal and are committed to the Charter and projected meeting schedule. These members are known as SWG Founding or Charter members. The charter members agree to the SoW and IPR terms as defined in this charter. The charter members have voting rights beginning the day the SWG is officially formed. Charter Members are shown on the public SWG page.

|===
|Name |Organization

|Christophe Noel | Spacebel
|Brianna R. Pagán | NASA GES DISC
|Alexey N. Shiklomanov | NASA Goddard Space Flight Center

briannapagan marked this conversation as resolved.
Show resolved Hide resolved
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
briannapagan marked this conversation as resolved.
Show resolved Hide resolved
|===

=== Conveners

xxx

[bibliography]
== References

- [[[gj,1]]] IETF: IETF RFC 7946, The GeoJSON Format, 2016

briannapagan marked this conversation as resolved.
Show resolved Hide resolved