Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling missing metadata fields in pyart.aux_io.read_odim_h5() reader #1599

Open
jorahu opened this issue Jun 21, 2024 · 4 comments
Open

Handling missing metadata fields in pyart.aux_io.read_odim_h5() reader #1599

jorahu opened this issue Jun 21, 2024 · 4 comments

Comments

@jorahu
Copy link

jorahu commented Jun 21, 2024

  • Py-ART version: 1.18.3
  • Python version: 3.10.1
  • Operating System: Linux

Description

Importing h5 file with pyart.aux_io.read_odim_h5() could not complete due to missing "version" or "source" field in file metadata.
The problem is with the lines 169 and 170 in https://github.com/ARM-DOE/pyart/blob/main/pyart/aux_io/odim_h5.py
There should be a check like there is for example lines 177 and 178.

What I Did

First, line 164 should be deleted as it is duplicated at line 170.

The lines 168-170 could be changed as follows:

h_what = hfile["what"].attrs 
if "version" in h_what:
    metadata["version"] = _to_str(h_what["version"])
if "source" in h_what:
    metadata["source"] = _to_str(h_what["source"])

I am not sure how this complies with ODIM structure, but I tested it with 2 completely different files (from different parts of the world), one had missing "source", other had missing "version" for unknown reasons. One missing (potentially unimportant) metadata field should not stop the whole reading process.

@mgrover1
Copy link
Collaborator

@jorahu - any interest in submitting a PR with the fix here? Thanks for highlighting the current limitation.

Would you be interested in supplying those sample files so we can add that to the test suite?

@jorahu
Copy link
Author

jorahu commented Jun 21, 2024

I was looking into making the pull request before raising the issue, but unfortunately I am too noob with github to do that. Sorry :(

One of the test files got another error (but the first error was fixed with this previous code), I also checked with wradlib importer, that's why I knew about the missing fields. But I used this file just to test this issue, don't have time to track down the other problem at the moment. And the initial file I tried was sent to me for testing, so I need to ask if it's okay to share it.

@jorahu jorahu closed this as completed Jun 21, 2024
@jorahu jorahu reopened this Jun 21, 2024
@jorahu
Copy link
Author

jorahu commented Jun 21, 2024

Yeah, see... can't even comment without messing it up. I'm pretty new to github. Apologies.

@mgrover1
Copy link
Collaborator

No worries! If you could share some of the files to test, that would be ideal, to make sure that the fix works generically!

Feel free to share it with my email, [email protected]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants