Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add names for test failures #1690

Closed
wants to merge 10 commits into from
Closed
9 changes: 9 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,15 @@ What's New
Changes since v0.10.0 rc1 (Unreleased)
--------------------------------------

Enhancements
~~~~~~~~~~~~

- Add name parameter and extra formatting to
:py:func:`xarray.testing.assert_allclose`,
:py:func:`xarray.testing.assert_identical` and
:py:func:`xarray.testing.assert_equal` for nicer failure printouts.
By `Matti Eskelinen <https://github.com/maaleske>`_

Bug fixes
~~~~~~~~~

Expand Down
31 changes: 21 additions & 10 deletions xarray/testing.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ def _data_allclose_or_equiv(arr1, arr2, rtol=1e-05, atol=1e-08,
arr1, arr2, rtol=rtol, atol=atol)


def assert_equal(a, b):
def assert_equal(a, b, name=''):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None is probably a better default value. It's valid (though obviously not a great idea) to have an xarray object with name given by an empty string.

"""Like :py:func:`numpy.testing.assert_array_equal`, but for xarray
objects.

Expand All @@ -42,6 +42,8 @@ def assert_equal(a, b):
The first object to compare.
b : xarray.Dataset, xarray.DataArray or xarray.Variable
The second object to compare.
name : str, optional
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should have a private function, e.g., assert_variable_equal, that accepts an optional name and call that from assert_equal? It's a little weird to allow names for Dataset objects.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't consider it as a name for either object, but rather an identifier for the specific assertion. Maybe it should be named/documented better as such?

Descriptive name to be shown in test failures.

See also
--------
Expand All @@ -52,13 +54,14 @@ def assert_equal(a, b):
__tracebackhide__ = True # noqa: F841
assert type(a) == type(b)
if isinstance(a, (xr.Variable, xr.DataArray, xr.Dataset)):
assert a.equals(b), '{}\n{}'.format(a, b)
assert a.equals(b), \
'{}:{}\n{}'.format(name, a, b) if name else '{}\n{}'.format(a, b)
else:
raise TypeError('{} not supported by assertion comparison'
.format(type(a)))


def assert_identical(a, b):
def assert_identical(a, b, name=''):
"""Like :py:func:`xarray.testing.assert_equal`, but also matches the
objects' names and attributes.

Expand All @@ -70,6 +73,8 @@ def assert_identical(a, b):
The first object to compare.
b : xarray.Dataset, xarray.DataArray or xarray.Variable
The second object to compare.
name : str, optional
Descriptive name to be shown in test failures.

See also
--------
Expand All @@ -80,15 +85,16 @@ def assert_identical(a, b):
assert type(a) == type(b)
if isinstance(a, xr.DataArray):
assert a.name == b.name
assert_identical(a._to_temp_dataset(), b._to_temp_dataset())
assert_identical(a._to_temp_dataset(), b._to_temp_dataset(), name=name)
elif isinstance(a, (xr.Dataset, xr.Variable)):
assert a.identical(b), '{}\n{}'.format(a, b)
assert a.identical(b), \
'{}:{}\n{}'.format(name, a, b) if name else '{}\n{}'.format(a, b)
else:
raise TypeError('{} not supported by assertion comparison'
.format(type(a)))


def assert_allclose(a, b, rtol=1e-05, atol=1e-08, decode_bytes=True):
def assert_allclose(a, b, rtol=1e-05, atol=1e-08, decode_bytes=True, name=''):
"""Like :py:func:`numpy.testing.assert_allclose`, but for xarray objects.

Raises an AssertionError if two objects are not equal up to desired
Expand All @@ -108,6 +114,8 @@ def assert_allclose(a, b, rtol=1e-05, atol=1e-08, decode_bytes=True):
Whether byte dtypes should be decoded to strings as UTF-8 or not.
This is useful for testing serialization methods on Python 3 that
return saved strings as bytes.
name : str, optional
Descriptive name to be shown in test failures.

See also
--------
Expand All @@ -120,21 +128,24 @@ def assert_allclose(a, b, rtol=1e-05, atol=1e-08, decode_bytes=True):
if isinstance(a, xr.Variable):
assert a.dims == b.dims
allclose = _data_allclose_or_equiv(a.values, b.values, **kwargs)
assert allclose, '{}\n{}'.format(a.values, b.values)
assert allclose, '{}:{}\n{}'.format(name, a.values, b.values)
elif isinstance(a, xr.DataArray):
assert_allclose(a.variable, b.variable, **kwargs)
assert_allclose(a.variable, b.variable, name=name, **kwargs)
assert set(a.coords) == set(b.coords)
for v in a.coords.variables:
# can't recurse with this function as coord is sometimes a
# DataArray, so call into _data_allclose_or_equiv directly
allclose = _data_allclose_or_equiv(a.coords[v].values,
b.coords[v].values, **kwargs)
assert allclose, '{}\n{}'.format(a.coords[v].values,
b.coords[v].values)
assert allclose, '{}:{}\n{}'.format(
'{} in {}'.format(v, name) if name else v,
a.coords[v].values,
b.coords[v].values)
elif isinstance(a, xr.Dataset):
assert set(a.data_vars) == set(b.data_vars)
assert set(a.coords) == set(b.coords)
for k in list(a.variables) + list(a.coords):
kwargs['name'] = '{} in {}'.format(k, name) if name else k
assert_allclose(a[k], b[k], **kwargs)

else:
Expand Down