Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems with xarray broadcasting/dimensions #2069

Closed
dopplershift opened this issue Aug 31, 2021 · 3 comments · Fixed by #2073
Closed

Problems with xarray broadcasting/dimensions #2069

dopplershift opened this issue Aug 31, 2021 · 3 comments · Fixed by #2073
Labels
Area: Calc Pertains to calculations Area: Xarray Pertains to xarray integration Type: Bug Something is not working like it should
Milestone

Comments

@dopplershift
Copy link
Member

I've encountered pretty easily some challenges trying to use XArray-based data, specifically when pulling out 1D pressure coordinates to use as a data field. I was able to easily fix heat_index by adding broadcast to the preprocess_and_wrap decorator, but I haven't quickly figured out this one with dewpoint_from_specific_humidity:

import xarray as xr
import metpy.calc as mpcalc
from metpy.cbook import get_test_data

narr = xr.open_dataset(get_test_data('narr_example.nc', as_file_obj=False))
dewp = mpcalc.dewpoint_from_specific_humidity(narr.isobaric, narr.Temperature, narr.Specific_humidity)

This gives:

ValueError                                Traceback (most recent call last)
/var/folders/r8/bhrhn4m56lgdckj491rm6_w40000gp/T/ipykernel_48975/3495402994.py in <module>
----> 1 dewp = mpcalc.dewpoint_from_specific_humidity(narr.isobaric, narr.Temperature, narr.Specific_humidity)

~/repos/metpy/src/metpy/xarray.py in wrapper(*args, **kwargs)
   1232                     return tuple(wrapping(*args) for args in zip(result, match))
   1233                 else:
-> 1234                     return wrapping(result, match)
   1235         return wrapper
   1236     return decorator

~/repos/metpy/src/metpy/xarray.py in _wrap_output_like_not_matching_units(result, match)
   1288             result = units.Quantity(result)
   1289         return (
-> 1290             xr.DataArray(result, coords=match.coords, dims=match.dims) if output_xarray
   1291             else result
   1292         )

~/miniconda3/envs/py39/lib/python3.9/site-packages/xarray/core/dataarray.py in __init__(self, data, coords, dims, name, attrs, indexes, fastpath)
    408             data = _check_data_shape(data, coords, dims)
    409             data = as_compatible_data(data)
--> 410             coords, dims = _infer_coords_and_dims(data.shape, coords, dims)
    411             variable = Variable(dims, data, attrs, fastpath=True)
    412             indexes = dict(

~/miniconda3/envs/py39/lib/python3.9/site-packages/xarray/core/dataarray.py in _infer_coords_and_dims(shape, coords, dims)
    156         for d, s in zip(v.dims, v.shape):
    157             if s != sizes[d]:
--> 158                 raise ValueError(
    159                     f"conflicting sizes for dimension {d!r}: "
    160                     f"length {sizes[d]} on the data but length {s} on "

ValueError: conflicting sizes for dimension 'time': length 29 on the data but length 1 on coordinate 'time'

It's also disconcerting to see this warning:

/Users/rmay/repos/metpy/src/metpy/calc/thermo.py:1047: UserWarning: Relative humidity >120%, ensure proper units.
  warnings.warn('Relative humidity >120%, ensure proper units.')
@dopplershift dopplershift added Type: Bug Something is not working like it should Area: Calc Pertains to calculations Area: Xarray Pertains to xarray integration labels Aug 31, 2021
@dopplershift dopplershift added this to the 1.2.0 milestone Aug 31, 2021
@jthielen
Copy link
Collaborator

So this is embarrassing...here's the fix:

broadcast=('pressure', 'temperature', 'specific_humdiity')

    broadcast=('pressure', 'temperature', 'specific_humidity')

Do we need a fancy linter to check for spelling/variable mismatch mistakes?

@jthielen
Copy link
Collaborator

That being said, given that the order of broadcast matters for the dimension order of the final result, it may be worth reviewing all the choices made for that (for example, looking at this now, I think the order for dewpoint_from_specific_humidity should be ('temperature', 'specific_humidity', 'pressure')...the wrap like target should generally come first and inputs most likely to come from coordinates should go last).

@dopplershift
Copy link
Member Author

Is there any reason we shouldn't just raise a ValueError if something is listed in broadcast that doesn't match to a parameter name? Users can't do anything about it, but it would help us catch a typo like that right away.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Calc Pertains to calculations Area: Xarray Pertains to xarray integration Type: Bug Something is not working like it should
Projects
None yet
2 participants