Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrap grdgradient #1269

Merged
merged 30 commits into from
May 25, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
31d5347
create grdgradient.py
willschlitzer May 10, 2021
e95cc99
add grdgradient to init files
willschlitzer May 10, 2021
f9ae69e
add grdgradient to API docs
willschlitzer May 10, 2021
8eb8ecd
add azimuth, direction, and radiance
willschlitzer May 11, 2021
e9d4ddf
add azimuth docstring
willschlitzer May 11, 2021
4d59891
add direction docstring
willschlitzer May 11, 2021
198dce3
add radiance docstring
willschlitzer May 11, 2021
9aeb560
run make format
willschlitzer May 11, 2021
e79adb7
Merge branch 'master' into wrap-grdgradient
willschlitzer May 12, 2021
e53201f
add args_in_kwargs for required argument in grdgradient
willschlitzer May 12, 2021
e22bd57
add test_grdgradient.py
willschlitzer May 12, 2021
429e7ff
Merge remote-tracking branch 'origin/wrap-grdgradient' into wrap-grdg…
willschlitzer May 12, 2021
43585a0
remove unused imports
willschlitzer May 12, 2021
77d4cc9
update grdgradient docstring
willschlitzer May 12, 2021
a3794dd
Merge branch 'master' into wrap-grdgradient
willschlitzer May 14, 2021
0d26dc1
split up lead in docstring
willschlitzer May 14, 2021
f433363
change "ingrid" to "grid"
willschlitzer May 14, 2021
801684a
Apply suggestions from code review
willschlitzer May 15, 2021
1451ff1
Merge branch 'master' into wrap-grdgradient
willschlitzer May 15, 2021
ce45929
style fix
willschlitzer May 15, 2021
9287fc6
add test_grdgradient_outgrid()
willschlitzer May 15, 2021
1fd6d32
fix imports for test_grdgradient.py
willschlitzer May 15, 2021
fb751e1
Update pygmt/src/grdgradient.py
willschlitzer May 17, 2021
939375d
Merge branch 'master' into wrap-grdgradient
willschlitzer May 17, 2021
9f02faf
Merge branch 'master' into wrap-grdgradient
willschlitzer May 18, 2021
37bb1e2
Merge branch 'master' into wrap-grdgradient
willschlitzer May 20, 2021
8b2a772
Apply suggestions from code review
willschlitzer May 21, 2021
9f3e083
Merge branch 'master' into wrap-grdgradient
willschlitzer May 21, 2021
39c3d27
Merge branch 'master' into wrap-grdgradient
willschlitzer May 22, 2021
7d6d8ab
Merge branch 'master' into wrap-grdgradient
willschlitzer May 24, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/api/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ Operations on grids:
grdcut
grdfill
grdfilter
grdgradient
grdtrack

Crossover analysis with x2sys:
Expand Down
1 change: 1 addition & 0 deletions pygmt/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
grdcut,
grdfill,
grdfilter,
grdgradient,
grdinfo,
grdtrack,
info,
Expand Down
1 change: 1 addition & 0 deletions pygmt/src/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
from pygmt.src.grdcut import grdcut
from pygmt.src.grdfill import grdfill
from pygmt.src.grdfilter import grdfilter
from pygmt.src.grdgradient import grdgradient
from pygmt.src.grdimage import grdimage
from pygmt.src.grdinfo import grdinfo
from pygmt.src.grdtrack import grdtrack
Expand Down
118 changes: 118 additions & 0 deletions pygmt/src/grdgradient.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
"""
grdgradient - Compute directional gradients from a grid.
"""

import xarray as xr
from pygmt.clib import Session
from pygmt.exceptions import GMTInvalidInput
from pygmt.helpers import (
GMTTempFile,
args_in_kwargs,
build_arg_string,
fmt_docstring,
kwargs_to_strings,
use_alias,
)


@fmt_docstring
@use_alias(
A="azimuth",
D="direction",
E="radiance",
G="outgrid",
R="region",
V="verbose",
n="interpolation",
)
@kwargs_to_strings(A="sequence", E="sequence", R="sequence")
def grdgradient(grid, **kwargs):
r"""
Compute the directional derivative of the vector gradient of the data.

Can accept ``azimuth``, ``direction``, and ``radiance`` input to create
the resulting gradient.

Full option list at :gmt-docs:`grdgradient.html`

{aliases}

Parameters
----------
grid : str or xarray.DataArray
The file name of the input grid or the grid loaded as a DataArray.
outgrid : str or None
The name of the output netCDF file with extension .nc to store the grid
in.
azimuth : int or float or str or list
*azim*\ [/*azim2*].
Azimuthal direction for a directional derivative; *azim* is the
angle in the x,y plane measured in degrees positive clockwise from
north (the +y direction) toward east (the +x direction). The
negative of the directional derivative, -[dz/dx\*sin(*azim*) +
dz/dy\*cos(\ *azim*)], is found; negation yields positive values
when the slope of z(x,y) is downhill in the *azim* direction, the
correct sense for shading the illumination of an image by a light
source above the x,y plane shining from the *azim* direction.
Optionally, supply two azimuths, *azim*/*azim2*, in which case the
gradients in each of these directions are calculated and the one
larger in magnitude is retained; this is useful for illuminating data
with two directions of lineated structures, e.g., *0*/*270*
illuminates from the north (top) and west (left). Finally, if *azim*
is a file it must be a grid of the same domain, spacing and
registration as *grid* that will update the azimuth at each output
node when computing the directional derivatives.
direction : str
[**a**][**c**][**o**][**n**].
Find the direction of the positive (up-slope) gradient of the data.
To instead find the aspect (the down-slope direction), use **a**.
By default, directions are measured clockwise from north, as *azim*
in ``azimuth``. Append **c** to use conventional Cartesian angles
measured counterclockwise from the positive x (east) direction.
Append **o** to report orientations (0-180) rather than
directions (0-360). Append **n** to add 90 degrees to all angles
(e.g., to give local strikes of the surface).
radiance : str or list
[**m**\|\ **s**\|\ **p**]\ *azim/elev*\ [**+a**\ *ambient*][**+d**\
*diffuse*][**+p**\ *specular*][**+s**\ *shine*].
Compute Lambertian radiance appropriate to use with ``grdimage``
and ``grdview``. The Lambertian Reflection assumes an ideal surface
that reflects all the light that strikes it and the surface appears
equally bright from all viewing directions. Here, *azim* and *elev* are
the azimuth and elevation of the light vector. Optionally, supply
*ambient* [0.55], *diffuse* [0.6], *specular* [0.4], or *shine* [10],
which are parameters that control the reflectance properties of the
surface. Default values are given in the brackets. Use **s** for a
simpler Lambertian algorithm. Note that with this form you only have
to provide azimuth and elevation. Alternatively, use **p** for
the Peucker piecewise linear approximation (simpler but faster
algorithm; in this case the *azim* and *elev* are hardwired to 315
and 45 degrees. This means that even if you provide other values
they will be ignored.)
{R}
{V}
{n}
"""
with GMTTempFile(suffix=".nc") as tmpfile:
if not args_in_kwargs(args=["A", "D", "E"], kwargs=kwargs):
raise GMTInvalidInput(
"""At least one of the following parameters must be specified:
azimuth, direction, or radiance"""
)
with Session() as lib:
file_context = lib.virtualfile_from_data(check_kind="raster", data=grid)
with file_context as infile:
if "G" not in kwargs.keys(): # if outgrid is unset, output to tempfile
kwargs.update({"G": tmpfile.name})
outgrid = kwargs["G"]
arg_str = " ".join([infile, build_arg_string(kwargs)])
lib.call_module("grdgradient", arg_str)

if outgrid == tmpfile.name: # if user did not set outgrid, return DataArray
with xr.open_dataarray(outgrid) as dataarray:
result = dataarray.load()
_ = result.gmt # load GMTDataArray accessor information
else:
result = None # if user sets an outgrid, return None

return result
62 changes: 62 additions & 0 deletions pygmt/tests/test_grdgradient.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
"""
Tests for grdgradient.
"""
import os

import numpy.testing as npt
import pytest
from pygmt import grdgradient, grdinfo
from pygmt.datasets import load_earth_relief
from pygmt.exceptions import GMTInvalidInput
from pygmt.helpers import GMTTempFile


@pytest.fixture(scope="module", name="grid")
def fixture_grid():
"""
Load the grid data from the sample earth_relief file.
"""
return load_earth_relief(resolution="01d", region=[-5, 5, -5, 5])


def test_grdgradient_outgrid(grid):
"""
Test the azimuth and direction parameters for grdgradient with a set
outgrid.
"""
with GMTTempFile(suffix=".nc") as tmpfile:
result = grdgradient(grid=grid, outgrid=tmpfile.name, azimuth=10, direction="c")
assert result is None # return value is None
assert os.path.exists(path=tmpfile.name) # check that outgrid exists
result = (
grdinfo(grid=tmpfile.name, force_scan="a", per_column="n").strip().split()
)
npt.assert_allclose(float(result[4]), -0.0045060496) # min
npt.assert_allclose(float(result[5]), 0.0575332976) # max
# Check spherically weighted statistics below
npt.assert_allclose(float(result[10]), 0.000384754501283) # median
npt.assert_allclose(float(result[12]), 0.00285958005568) # mean


def test_grdgradient_no_outgrid(grid):
"""
Test the azimuth and direction parameters for grdgradient with no set
outgrid.
"""
temp_grid = grdgradient(grid=grid, azimuth=10, direction="c")
assert temp_grid.dims == ("lat", "lon")
assert temp_grid.gmt.gtype == 1 # Geographic grid
assert temp_grid.gmt.registration == 1 # Pixel registration
npt.assert_allclose(temp_grid.min(), -0.0045060496)
npt.assert_allclose(temp_grid.max(), 0.0575332976)
willschlitzer marked this conversation as resolved.
Show resolved Hide resolved
npt.assert_allclose(temp_grid.median(), 0.0004889865522272885)
npt.assert_allclose(temp_grid.mean(), 0.0028633063193410635)


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add add an extra test using e.g. grdgradient(..., radiance=[135, 45]) that produces an outgrid. Similar in style to your test_grdclip_outgrid example

def test_grdclip_outgrid(grid):
"""
Test the below and above parameters for grdclip and creates a test outgrid.
"""
with GMTTempFile(suffix=".nc") as tmpfile:
result = grdclip(
grid=grid, outgrid=tmpfile.name, below=[-1500, -1800], above=[-200, 40]
)
assert result is None # return value is None
assert os.path.exists(path=tmpfile.name) # check that outgrid exists
result = (
grdinfo(grid=tmpfile.name, force_scan=0, per_column="n").strip().split()
)
assert int(result[4]) == -1800 # minimum value
assert int(result[5]) == 40 # maximum value
.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reasons I don't understand, both test_grdgradient_outgrid() and test_grdgradient_no_outgrid() fail when I run them on my computer, but they pass on the GitHub Action. For both of them, I get an AssertionError that tells me that the assert_allclose is not equal within tolerance. Any idea why this would be a problem when running the test_grdgradient.py on some machines by not others?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you report the full error message? I'm not surprised that this did not work, this might be due to flakiness (#1242) and/or something with grdgradient itself. The shading parameter in grdimage/grdview (which uses grdgradient behind the scenes) has always been a bit sensitive to crashes and bugs, but we haven't managed to track down the true cause of it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the failure for test_grdgradient_outgrid():
image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the failure for test_grdgradient_no_outgrid():
image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weiji14 What I'm a little confused about with the issue you mention above is that this test fails when I run test_grdgradient.py by itself and as part of make test on my computer , but consistently passes when the entire test suite is run on GitHub Actions. I would think the issue wouldn't affect a single file run and the issue is that all of the testing is run under one GMT instance.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you provide the output of pygmt.show_versions() please? You're right that it's funny breaking the other way around. I'll try to see if I cam reproduce it on my end tomorrow.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyGMT information:
version: v0.3.2.dev143+g6bc7fa28
System information:
python: 3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 05:02:46) [GCC 9.3.0]
executable: /home/will/miniconda3/envs/gmt62rc/bin/python
machine: Linux-5.8.0-53-generic-x86_64-with-glibc2.31
Dependency information:
numpy: 1.20.2
pandas: 1.2.4
xarray: 0.17.0
netCDF4: 1.5.6
packaging: 20.9
ghostscript: 9.53.3
gmt: 6.2.0rc1
GMT library information:
binary dir: /home/will/miniconda3/envs/gmt62rc/bin
cores: 12
grid layout: rows
library path: /home/will/miniconda3/envs/gmt62rc/lib/libgmt.so
padding: 2
plugin dir: /home/will/miniconda3/envs/gmt62rc/lib/gmt/plugins
share dir: /home/will/miniconda3/envs/gmt62rc/share/gmt
version: 6.2.0rc1

I'm not too smart on using conda environments in PyCharm, but I changed the default conda environment to a different environment and changed it back, and now my test appear to be passing when both run alone and in make test. Not sure if it was a PyCharm error (unlikely) or human error (very likely), but the problem seems to be fixed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't seem to reproduce your error above when running pytest pygmt/tests/test_grdgradient.py, so maybe you're right that there isn't any problem. My pygmt.show_versions() is as below for reference, just did a fresh install yesterday. I only see a difference in xarray and ghostscript versions but those shouldn't matter much I think.

PyGMT information:
  version: v0.3.2.dev145+ga2cdce2c.d20210520
System information:
  python: 3.9.4 | packaged by conda-forge | (default, May 10 2021, 22:13:33)  [GCC 9.3.0]
  executable: /home/username/miniconda3/envs/pygmt/bin/python
  machine: Linux-5.4.0-73-generic-x86_64-with-glibc2.31
Dependency information:
  numpy: 1.20.2
  pandas: 1.2.4
  xarray: 0.18.2
  netCDF4: 1.5.6
  packaging: 20.9
  ghostscript: 9.54.0
  gmt: 6.2.0rc1
GMT library information:
  binary dir: /home/username/miniconda3/envs/pygmt/bin
  cores: 6
  grid layout: rows
  library path: /home/username/miniconda3/envs/pygmt/lib/libgmt.so
  padding: 2
  plugin dir: /home/username/miniconda3/envs/pygmt/lib/gmt/plugins
  share dir: /home/username/miniconda3/envs/pygmt/share/gmt
  version: 6.2.0rc1

def test_grdgradient_fails(grid):
"""
Check that grdgradient fails correctly when neither of azimuth, direction
or radiance is given.
"""
with pytest.raises(GMTInvalidInput):
grdgradient(grid=grid)