-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix wetday frequency correction #174
Fix wetday frequency correction #174
Conversation
@dgergel I requested your review for the method check. @brews, in the computational side. I tested this on both Jupyter and Argo, and this ran within 3 mins using around 15GB of memory (that's because of the huge numpy sample array that I have to create, which has the shape of the GCM data). That fits in ~20GB resources for that step. We might need to increase slightly this number so that the ERA-5 data fits in. But it will fit in the node and run fast. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for cleaning this up, @emileten.
My one suggestion is to rename this new var
argument to variable
. This makes it consistent with the other functions and methods that grab a variable name. ...I see "var" and I think "variance" but that's an aside....
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good, thanks @emileten for fixing this bug!
ds > threshold, np.random.uniform(low=low, high=threshold) | ||
ds[variable] = ds[variable].where( | ||
ds[variable] >= threshold, | ||
np.random.uniform(low=low, high=threshold, size=ds[variable].shape), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the issue is here - changing from np.random.uniform(low, high)
to np.random.uniform(low, high, size)
changes the returned type from pure python float
to ndarray[np.float64]
. xarray coerces the former to ds[variable].dtype
, but defers to the higher precision data type of the latter
essential changes :
process='pre'
process='pre'
.seed
), smaller and more precise.secondary changes :
var
parameter -- default value is'pr'
, but in tests we have other variable names. Need that parameter in the service as well.