Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add global attribute for data processing level #980

Closed
emiliom opened this issue Mar 13, 2023 · 7 comments
Closed

Add global attribute for data processing level #980

emiliom opened this issue Mar 13, 2023 · 7 comments
Assignees
Labels
enhancement This makes echopype better
Milestone

Comments

@emiliom
Copy link
Collaborator

emiliom commented Mar 13, 2023

Add a single global attribute that specifies the data processing/product level for the dataset.

The ACDD convention defines the Recommended global attribute processing_level, "A textual description of the processing (or quality control) level of the data". Usage of this attribute is also recommended (possibly as part of its ACDD implementation?) in the NASA Earth Science Data and Information System Standards Office document, "Data Product Development Guide (DPDG) for Data Producers version1.1", https://doi.org/10.5067/DOC/ESO/RFC-041VERSION1

Another relevant ACDD global attribute is product_version, "Version identifier of the data file or product as assigned by the data creator. For example, a new algorithm or methodology could result in a new product_version." ACDD classifies it as Suggested.

Related to #817

@emiliom emiliom added enhancement This makes echopype better product levels labels Mar 13, 2023
@emiliom emiliom added this to the 0.7.0 milestone Mar 13, 2023
@emiliom emiliom self-assigned this Mar 13, 2023
@emiliom emiliom added this to Echopype Mar 13, 2023
@github-project-automation github-project-automation bot moved this to Todo in Echopype Mar 13, 2023
@leewujung
Copy link
Member

leewujung commented Mar 17, 2023

A minimal example of decorator to add attribute:

In [29]: def test_add_attrs(func):
    ...:     def wrapper():
    ...:         da = func()
    ...:         da = da.assign_attrs({"test_attr": "some attribute"})
    ...:         return da
    ...:     return wrapper
    ...: 

In [30]: @test_add_attrs
    ...: def build_da():
    ...:     return xr.DataArray([1, 2, 3], coords={"channel": ["chA","chB","chC"]})
    ...: 

In [31]: da = build_da()

In [32]: da
Out[32]: 
<xarray.DataArray (channel: 3)>
array([1, 2, 3])
Coordinates:
  * channel  (channel) <U3 'chA' 'chB' 'chC'
Attributes:
    test_attr:  some attribute

@leewujung
Copy link
Member

Out of curiosity, I tried another one with parameters:

In [50]: def test_add_attrs(some_str):
    ...:     def inner(func):
    ...:        def wrapper(*args, **kwargs):
    ...:           da = func()
    ...:           da = da.assign_attrs({"input_str": some_str})
    ...:           da = da.assign_attrs({"test_attr": "some attribute"})
    ...:           return da
    ...:        return wrapper
    ...:     return inner
    ...: 

In [51]: @test_add_attrs(some_str="add this string!")
    ...: def build_da():
    ...:     return xr.DataArray([1, 2, 3], coords={"channel": ["chA","chB","chC"]})
    ...: 

In [52]: da = build_da()

In [53]: da
Out[53]: 
<xarray.DataArray (channel: 3)>
array([1, 2, 3])
Coordinates:
  * channel  (channel) <U3 'chA' 'chB' 'chC'
Attributes:
    input_str:  add this string!
    test_attr:  some attribute

@emiliom
Copy link
Collaborator Author

emiliom commented Mar 17, 2023

Well, thanks for doing my work for me! Adding a parameter was my first question/to-do 😆 . Though, that inner/wrapper hierarchy plus the use of *args, **kwargs is kind of ugly, but that'll be a hidden ugliness.

It does look like a decorator will work. And I completely agree that from a code usability perspective, it's perfect. Hopefully it will work.

@leewujung
Copy link
Member

This is the result of my pattern matching 😁 so there are probably redundancy that can be cut down, but it's fun to make it work, haha.

Hopefully it will work.

🤞

@emiliom
Copy link
Collaborator Author

emiliom commented Mar 20, 2023

ACDD defines a couple of global attributes that end in the suffix _url, to specify a url corresponding to the global attribute it complements: publisher_url and creator_url. While it doesn't define a processing_level_url attribute, I thin we should add one that'll point to the compact data processing levels page we discussed.

@emiliom
Copy link
Collaborator Author

emiliom commented Mar 20, 2023

Product level attributes will be added only if lat-lon coordinates (and depth for L2 and higher?) are present.

@emiliom
Copy link
Collaborator Author

emiliom commented Mar 25, 2023

Addressed in #1001

@emiliom emiliom closed this as completed Mar 25, 2023
@github-project-automation github-project-automation bot moved this from In Progress to Done in Echopype Mar 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement This makes echopype better
Projects
Status: Done
Development

No branches or pull requests

2 participants