Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Climatological op #290

Merged
merged 73 commits into from
Dec 19, 2023
Merged
Changes from 1 commit
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
9318cf1
filter indicator list for variables available in a dataset
vindelico Oct 12, 2023
bd37066
climatological_op 1st version; enhanced docs only
vindelico Oct 12, 2023
379decc
filter indicator list for variables available in a dataset
vindelico Oct 12, 2023
77564af
update xscen version from remote
vindelico Oct 13, 2023
ffdcb08
test for select_inds_for_avail_vars, first draft
vindelico Oct 16, 2023
5931257
snapshot; climatological_op working for linregress, doing window(ing)…
vindelico Nov 6, 2023
fcff131
snapshot; rmoved xscen/aggregate_old.py intermediate file
vindelico Nov 6, 2023
7de6829
climatological_op, working v1
vindelico Nov 9, 2023
f21aa05
debug test_indicators
vindelico Nov 9, 2023
539c9d5
all tests for climatological_op passing
vindelico Nov 13, 2023
364c680
change regressors in linregress to periods years, updated test, all t…
vindelico Nov 13, 2023
5fb2fef
Updated HISTORY; climatological_mean calls climatological_op, TestCli…
vindelico Nov 15, 2023
33c3ad0
Draft output option periods_as_dim for climatological_op + test
vindelico Nov 15, 2023
675fcad
periods_as_dim final and updated to xarray=2023.9; added unstacked ti…
vindelico Nov 17, 2023
677be08
Merge branch 'main' of github.com:Ouranosinc/xscen into select_indica…
vindelico Nov 17, 2023
6ad3176
merged with climatological_op
vindelico Nov 17, 2023
9c7e916
removed select_inds_for_avail_vars + test for PR
vindelico Nov 17, 2023
a6e5440
Update HISTORY.rst
vindelico Nov 17, 2023
9e1497e
Updated docstring, pre-commit passed
vindelico Nov 19, 2023
51ff13a
Changed Getting Started notebook to use climatological_op
vindelico Nov 19, 2023
757be8e
Merged climatological_op with origin/main
vindelico Nov 21, 2023
2a03ba6
Comments from PR addressed, round 1.
vindelico Nov 22, 2023
d98f082
Changed call of climatological_mean within compute_horizon to calling…
vindelico Nov 22, 2023
5276bb3
Fixed text in Getting Started docs.
vindelico Nov 22, 2023
851de09
Update aggregate.py
vindelico Nov 22, 2023
4d56597
Merged with updated climatological_op
vindelico Nov 22, 2023
75babcf
Update select_ind_for_avail_vars and test.
vindelico Nov 22, 2023
d5fa354
Test select_inds_for_avail_vars; draft
vindelico Nov 23, 2023
0d4ca1f
Test select_inds_for_avail_vars passes.
vindelico Nov 23, 2023
ffabb74
Updated HISTORY.rst
vindelico Nov 23, 2023
3f5028b
Updated to xarray's new way of handling pandas.MultiIndex objects
vindelico Nov 27, 2023
3fcea88
Merge branch 'climatological_op' into select_indicators_for_available…
vindelico Nov 27, 2023
4cc6928
Merge branch 'main' into climatological_op
aulemahal Nov 28, 2023
5a0abdb
Merge branch 'climatological_op' into select_indicators_for_available…
RondeauG Nov 29, 2023
1518814
Update aggregate.py
vindelico Dec 1, 2023
2ec9855
Apply suggestions from code review
vindelico Dec 1, 2023
e8ef834
Update HISTORY.rst
vindelico Dec 1, 2023
9b629e6
Apply suggestions from code review
vindelico Dec 1, 2023
96bca11
Merge branch 'main' of github.com:Ouranosinc/xscen into select_indica…
vindelico Dec 1, 2023
d419b79
Merge branch 'select_indicators_for_available_vars' of github.com:Our…
vindelico Dec 1, 2023
a7a727e
Merge branch 'climatological_op' of github.com:Ouranosinc/xscen into …
vindelico Dec 1, 2023
40a9afc
Merge remote-tracking branch 'origin/main' into climatological_op
vindelico Dec 1, 2023
d74adcc
Review comments integrated, test for empty module returned added.
vindelico Dec 1, 2023
e2277dd
Updated tests from code review; try/except to adapt to FutureWarning'…
vindelico Dec 2, 2023
ac206a3
Updated notebooks 2_getting_started and 6_config with associated temp…
vindelico Dec 2, 2023
af032a8
Fixed some warnings in test_aggregate
vindelico Dec 2, 2023
c81817c
Merge branch 'climatological_op' into select_indicators_for_available…
vindelico Dec 2, 2023
0fa8382
Correct kernel in notebook 6_config
vindelico Dec 2, 2023
1ae4981
Fix bug in notebook 6_config
vindelico Dec 3, 2023
0a2f292
Merge branch 'climatological_op' into select_indicators_for_available…
vindelico Dec 3, 2023
256a333
Correct docstring select_inds_for_avail_vars
vindelico Dec 5, 2023
53f08d5
Select the set of indicators for available variables (#291)
vindelico Dec 5, 2023
2b9e299
Integrations from review; removed redundant tests for climatological_…
vindelico Dec 5, 2023
76250a7
Merge branch 'climatological_op' of github.com:Ouranosinc/xscen into …
vindelico Dec 5, 2023
ab1c226
Minor edits
vindelico Dec 5, 2023
71f374b
Update CHANGES.rst
RondeauG Dec 5, 2023
49a33a5
Fix CHANGES.rst
vindelico Dec 5, 2023
8940f68
Merge branch 'climatological_op' of github.com:Ouranosinc/xscen into …
vindelico Dec 5, 2023
b5644ac
produce_horizon uses climatological_op with option horizons_as_dim=Tr…
vindelico Dec 7, 2023
19b2675
horizons_as_dim changed to singular: horizon_as_dim
vindelico Dec 7, 2023
58de8bb
Fixed test_aggregate
vindelico Dec 8, 2023
a2919b5
Notebook 2_getting_started adjusted
vindelico Dec 8, 2023
fd4c350
Remove time coord from output from produce_horizon
vindelico Dec 8, 2023
49f493a
Do remove with call to climatological_op in produce_horizon
vindelico Dec 8, 2023
dacff59
Place removal of time coord after the call to climatological_op withi…
vindelico Dec 8, 2023
f831b38
Implementation of horizons_as_dim (previously periods_as_dim) using x…
vindelico Dec 9, 2023
2774148
Implementation of horizons_as_dim (previously periods_as_dim) cleaned
vindelico Dec 9, 2023
9c44e9a
Merge branch 'climatological_op' of github.com:Ouranosinc/xscen into …
vindelico Dec 9, 2023
ef03413
Merge branch 'main' into climatological_op
vindelico Dec 9, 2023
1a444da
Correct order of rename and set_coords in aggregate.py, remove void c…
vindelico Dec 18, 2023
faeb686
Merge branch 'climatological_op' of github.com:Ouranosinc/xscen into …
vindelico Dec 18, 2023
76959d1
Corrections notebook 2, aggregate.py
vindelico Dec 19, 2023
199a796
Merge branch 'main' into climatological_op
vindelico Dec 19, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
removed select_inds_for_avail_vars + test for PR
vindelico committed Nov 17, 2023
commit 9c7e916a000aa26822031e096a89b0e81e1b70c7
32 changes: 0 additions & 32 deletions tests/test_indicators.py
Original file line number Diff line number Diff line change
@@ -150,35 +150,3 @@ def test_as_jul(self, restrict_years):
assert len(out.time) == 4
assert out.time[0].dt.strftime("%Y-%m-%d").item() == "2000-07-01"
assert out.time[-1].dt.strftime("%Y-%m-%d").item() == "2003-07-01"

def test_select_inds_for_avail_vars(self):
# Test that select_inds_for_avail_vars filters a list of indicators to only those that can be computed with the
# variables available in the input dataset.
ds = xs.testing.datablock_3d(
np.ones((365, 3, 3)),
variable="tas",
x="lon",
x_start=-75,
y="lat",
y_start=45,
x_step=1,
y_step=1,
start="2001-01-01",
freq="D",
units="K",
as_dataset=True,
)
indicators = [
xclim.core.indicator.Indicator.from_dict(
data={"base": "tg_min", "parameters": {"freq": "QS-DEC"}},
identifier="tg_min_qs",
module="tests"),
xclim.core.indicator.Indicator.from_dict(
data={"base": "days_over_precip_thresh", "parameters": {"freq": "MS"}},
identifier="precip_average_ms",
module="tests")
]
inds = xs.indicators.select_inds_for_avail_vars(ds=ds, indicators=indicators)
assert len(inds) == 1 # the precip-based indicator must be skipped
assert inds[0][0] == "tg_min_qs"
assert inds[0][1] == indicators[0]
41 changes: 0 additions & 41 deletions xscen/indicators.py
Original file line number Diff line number Diff line change
@@ -19,7 +19,6 @@

logger = logging.getLogger(__name__)


__all__ = ["compute_indicators", "load_xclim_module"]


@@ -231,46 +230,6 @@ def _infer_freq_from_meta(ind):
return out_dict


def select_inds_for_avail_vars(
ds: xr.Dataset,
indicators: Union[
str, Path, Sequence[Indicator], Sequence[tuple[str, Indicator]], ModuleType
],
) -> Sequence[Indicator]:
"""Filter the indicators from a YAML file for which the necessary variables are available.
Parameters
----------
ds : xr.Dataset
Dataset to use for the indicators.
indicators : Union[str, Path, Sequence[Indicator], Sequence[Tuple[str, Indicator]]]
Path to a YAML file that instructs on how to calculate missing variables.
Can also be only the "stem", if translations and custom indices are implemented.
Can be the indicator module directly, or a sequence of indicators or a sequence of
tuples (indicator name, indicator) as returned by `iter_indicators()`.
Returns
-------
Sequence[Indicator]
A sequence of tuples of (indicator name, indicator).
See Also
--------
xclim.indicators, xclim.core.indicator.build_indicator_module_from_yaml
"""
if isinstance(indicators, (str, Path)):
logger.debug("Loading indicator module.")
indicators = load_xclim_module(indicators, reload=True)

if hasattr(indicators, "iter_indicators"):
indicators = [(name, ind) for name, ind in indicators.iter_indicators()]

available_vars = {var for var in ds.data_vars if var in xc.core.utils.VARIABLES.keys()}
available_ind = [(name, ind) for var in available_vars
for name, ind in indicators if var in ind.parameters.keys()]
return available_ind


def registry_from_module(
module, registry=None, variable_column="variable"
) -> DerivedVariableRegistry: