Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add values_dtype backend option to load values at full precision #407

Merged
merged 2 commits into from
Dec 18, 2024

Conversation

juntyr
Copy link
Contributor

@juntyr juntyr commented Nov 22, 2024

Supersedes #395

#395 (comment)

Hi @juntyr and @dmey, thanks for this change! However, I think this is something we would prefer to be an option since it will double the memory requirement (and we do have GRIB fields with over 1 billion values).

Potential API:

ds = xr.open_dataset('data.grib', engine='cfgrib', backend_kwargs={'values_dtype': 'float64'})

Do you think you could make this modification to the PR, or would you rather leave it as a request issue?

@juntyr
Copy link
Contributor Author

juntyr commented Nov 30, 2024

@iainrussell What are your thoughts on the updated PR?

@juntyr
Copy link
Contributor Author

juntyr commented Dec 9, 2024

Ping @iainrussell

@juntyr juntyr mentioned this pull request Dec 12, 2024
27 tasks
@juntyr
Copy link
Contributor Author

juntyr commented Dec 17, 2024

Cc @SF-N

@iainrussell
Copy link
Member

Hi @juntyr, sorry for the long delay, but I've got back to this now - your PR looks good, I just added a couple of tests and plan to make a release tomorrow.

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 84.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 41.85%. Comparing base (11acc76) to head (21ec8e8).

Files with missing lines Patch % Lines
cfgrib/dataset.py 0.00% 3 Missing ⚠️
cfgrib/messages.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #407      +/-   ##
==========================================
+ Coverage   41.34%   41.85%   +0.50%     
==========================================
  Files          26       26              
  Lines        2201     2222      +21     
  Branches      194      194              
==========================================
+ Hits          910      930      +20     
- Misses       1291     1292       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@juntyr
Copy link
Contributor Author

juntyr commented Dec 17, 2024

Thank you very much for your help @iainrussell!

@iainrussell iainrussell merged commit bda2944 into ecmwf:master Dec 18, 2024
111 of 113 checks passed
@iainrussell
Copy link
Member

cfgrib 0.9.15.0 has been released with this new feature - thank you @juntyr!

@juntyr juntyr deleted the float64 branch December 18, 2024 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants