Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Starter property-based test suite #1972

Merged
merged 1 commit into from
Mar 20, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
*.py[cod]
__pycache__

# example caches from Hypothesis
.hypothesis/

# temp files from docs build
doc/auto_gallery
doc/example.nc
Expand Down
4 changes: 4 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ matrix:
env: CONDA_ENV=py36-zarr-dev
- python: 3.5
env: CONDA_ENV=docs
- python: 3.6
env: CONDA_ENV=py36-hypothesis
allow_failures:
- python: 3.6
env:
Expand Down Expand Up @@ -104,6 +106,8 @@ script:
- if [[ "$CONDA_ENV" == "docs" ]]; then
conda install -c conda-forge sphinx sphinx_rtd_theme sphinx-gallery numpydoc;
sphinx-build -n -j auto -b html -d _build/doctrees doc _build/html;
elif [[ "$CONDA_ENV" == "py36-hypothesis" ]]; then
pytest properties ;
else
py.test xarray --cov=xarray --cov-config ci/.coveragerc --cov-report term-missing --verbose $EXTRA_FLAGS;
fi
Expand Down
27 changes: 27 additions & 0 deletions ci/requirements-py36-hypothesis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: test_env
channels:
- conda-forge
dependencies:
- python=3.6
- dask
- distributed
- h5py
- h5netcdf
- matplotlib
- netcdf4
- pytest
- flake8
- numpy
- pandas
- scipy
- seaborn
- toolz
- rasterio
- bottleneck
- zarr
- pip:
- coveralls
- pytest-cov
- pydap
- lxml
- hypothesis
22 changes: 22 additions & 0 deletions properties/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Property-based tests using Hypothesis

This directory contains property-based tests using a library
called [Hypothesis](https://github.com/HypothesisWorks/hypothesis-python).

The property tests for Xarray are a work in progress - more are always welcome.
They are stored in a separate directory because they tend to run more examples
and thus take longer, and so that local development can run a test suite
without needing to `pip install hypothesis`.

## Hang on, "property-based" tests?

Instead of making assertions about operations on a particular piece of
data, you use Hypothesis to describe a *kind* of data, then make assertions
that should hold for *any* example of this kind.

For example: "given a 2d ndarray of dtype uint8 `arr`,
`xr.DataArray(arr).plot.imshow()` never raises an exception".

Hypothesis will then try many random examples, and report a minimised
failing input for each error it finds.
[See the docs for more info.](https://hypothesis.readthedocs.io/en/master/)
46 changes: 46 additions & 0 deletions properties/test_encode_decode.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
"""
Property-based tests for encoding/decoding methods.

These ones pass, just as you'd hope!

"""
from __future__ import absolute_import, division, print_function

from hypothesis import given, settings
import hypothesis.strategies as st
import hypothesis.extra.numpy as npst

import xarray as xr

# Run for a while - arrays are a bigger search space than usual
settings.deadline = None


an_array = npst.arrays(
dtype=st.one_of(
npst.unsigned_integer_dtypes(),
npst.integer_dtypes(),
npst.floating_dtypes(),
),
shape=npst.array_shapes(max_side=3), # max_side specified for performance
)


@given(st.data(), an_array)
def test_CFMask_coder_roundtrip(data, arr):
names = data.draw(st.lists(st.text(), min_size=arr.ndim,
max_size=arr.ndim, unique=True).map(tuple))
original = xr.Variable(names, arr)
coder = xr.coding.variables.CFMaskCoder()
roundtripped = coder.decode(coder.encode(original))
xr.testing.assert_identical(original, roundtripped)


@given(st.data(), an_array)
def test_CFScaleOffset_coder_roundtrip(data, arr):
names = data.draw(st.lists(st.text(), min_size=arr.ndim,
max_size=arr.ndim, unique=True).map(tuple))
original = xr.Variable(names, arr)
coder = xr.coding.variables.CFScaleOffsetCoder()
roundtripped = coder.decode(coder.encode(original))
xr.testing.assert_identical(original, roundtripped)