Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sdba - Broadcasting with two new coordinates of differing dimensions fails #449

Closed
aulemahal opened this issue May 12, 2020 · 6 comments · Fixed by #567
Closed

sdba - Broadcasting with two new coordinates of differing dimensions fails #449

aulemahal opened this issue May 12, 2020 · 6 comments · Fixed by #567
Assignees
Labels
bug Something isn't working

Comments

@aulemahal
Copy link
Collaborator

  • xclim version: master
  • Python version: 3.8
  • Operating System: Linux

Description

In QuantileDeltaMapping.adjust with grouping (here 'time.month') there is a broadcasting operation that fails when sim and hist have additional dimensions (other then quantiles and time).
In fact, calls of xclim.sdba.utils.broadcast fails when grouping is used, interp is not 'nearest' and both grouped and the coordinate in sel have dimensions that are not trageted by the broadcasting operation.

What I Did

from xclim.sdba.adjustment import QuantileDeltaMapping
QDM = QuantileDeltaMapping(group='time.month', interp='linear'
QDM.train(ref, hist)
scen = QDM.adjust(sim)

Where, ref has one spatial dim + 'time' and hist and sim have the same spatial dim, 'time' and an additional dim, in my case realization.

What I Received

ValueError: axes don't match array

The traceback is quite long, but it all comes down to xr.interp.

@aulemahal aulemahal self-assigned this May 12, 2020
@aulemahal aulemahal added bug Something isn't working priority Immediate priority labels May 12, 2020
@aulemahal aulemahal added this to the v0.17 milestone May 12, 2020
@aulemahal
Copy link
Collaborator Author

aulemahal commented May 13, 2020

xarray's interp doesn't understand multidimensionnal interpolation when the new coordinates have differing set of coords, so I tried to broadcast all arrays of the sel dictionary together which makes it work but enlarges the arrays a lot. Hopefully, no data is created so memory should be ok. However, dask arrays chunked along one of the additional dimensions will make broadcast fail. So, that's not a solution.

Kinda stuck on this. A way would be to call a xr.apply_ufunc with a wrapper of scipy.interpolate.interpn, but that's a lot of machinery just because xarray as kind of a bug hasn't implemented the thing yet.

@huard
Copy link
Collaborator

huard commented May 13, 2020

Is there an open issue about this ?

@aulemahal
Copy link
Collaborator Author

Not that I can found.

@huard
Copy link
Collaborator

huard commented May 13, 2020

I suggest you design a short, self-contained example illustrating the issue and open an issue with it.

@aulemahal
Copy link
Collaborator Author

I didn't realize up to now, but it is worth noting that xarray raises a "NotImplementedError". It's not a real bug.

@huard huard modified the milestones: v0.17, v0.18 May 15, 2020
@huard huard modified the milestones: v0.18, v0.19 Jun 26, 2020
@Zeitsperre Zeitsperre modified the milestones: v0.19, v0.20 Aug 5, 2020
@aulemahal
Copy link
Collaborator Author

Corresponds to issue pydata/xarray#4058.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants