Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option ignore_keys (similar to filter_keys) #285

Closed
floriankrb opened this issue Feb 18, 2022 · 2 comments · Fixed by #382
Closed

Add option ignore_keys (similar to filter_keys) #285

floriankrb opened this issue Feb 18, 2022 · 2 comments · Fixed by #382

Comments

@floriankrb
Copy link
Member

When a user wants to open a grib file containing multiple gribs with inconsistent values, cfgrib refuses (rightfully) to merge them, leading to an error such as:

DatasetBuildError: multiple values for unique key, try re-open the file with one of:
    filter_by_keys={'dataType': 'pf'}
    filter_by_keys={'dataType': 'cf'}

While this solves some issues, it is not enough when the user does not want to filter, but prefers to ignore this key. I would like to extend the filter_keys option and add a new ignore_keys option.

For a key in the list provided by the user in ignore_keys, cfgrib should force the value to None when reading these keys. And the error message above could become:

DatasetBuildError: multiple values for unique key, try re-open the file with one of:
    filter_by_keys={'dataType': 'pf'}
    filter_by_keys={'dataType': 'cf'}
    ignore_keys=['dataType']

This is related to ecmwf/climetlab#33 (and maybe others, such as #263 and #268 and https://gis.stackexchange.com/questions/372729/unable-to-read-grib-datas-with-xarray).

@EddyCMWF
Copy link
Contributor

Hi @floriankrb , I also could do with this feature, so I've linked a branch I have been working on. If you still have an active use case for this, maybe you could give it a go?
https://github.com/ecmwf/cfgrib/tree/ignore_keys

@floriankrb
Copy link
Member Author

@sandorkertesz would these ignore_keys still be needed in earthkit.data ?

@EddyCMWF EddyCMWF mentioned this issue Jun 21, 2024
@EddyCMWF EddyCMWF linked a pull request Jun 21, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants