Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

duplicate method does not work as expected #361

Closed
huard opened this issue May 7, 2024 · 2 comments
Closed

duplicate method does not work as expected #361

huard opened this issue May 7, 2024 · 2 comments

Comments

@huard
Copy link
Collaborator

huard commented May 7, 2024

  • RavenPy version:
  • Python version: 3.11
  • Operating System:

Description

Config.duplicate does not correctly ingest an array of parameters.

@huard
Copy link
Collaborator Author

huard commented May 7, 2024

I think the bugs founds in the PAVICS nbs when updating to Py3.11 are bugs dating back from the switch to pydantic v2 that were not exposed by the test suite.

tlvu added a commit to Ouranosinc/PAVICS-e2e-workflow-tests that referenced this issue May 9, 2024
…t xclim and ravenpy to smooth transition (#121)

# Overview

This new full build has latest of almost everything except `xclim` and
`ravenpy` as intermediate step to smooth transition to `pandas` 2.2 freq
strings changes.

## Changes

- New: save conda env export, DockerHub build logs and Jenkins test
result in the repo to track changes much more easily between releases

- Jenkins: add `SAVE_RESULTING_NOTEBOOK_TIMEOUT` for slow notebooks or
slow machine

- Jupyter env changes:
- add `conda-pack` so we can export the conda env outside of the docker
image if need to run locally without docker
  - upgrade from Python 3.9 to 3.11
  - Relevant changes (alphabetical order):
```diff
-  - birdy=0.8.4=pyh1a96a4e_0
+      - birdhouse-birdy==0.8.7

# major upgrade from v2 to v3
-  - bokeh=2.4.3=pyhd8ed1ab_3
+  - bokeh=3.4.1=pyhd8ed1ab_0

-  - cartopy=0.21.1=py39h6e7ad6e_0
+  - cartopy=0.23.0=py311h320fe9a_0

-  - cf_xarray=0.8.0=pyhd8ed1ab_0
+  - cf_xarray=0.9.0=pyhd8ed1ab_0

-  - cfgrib=0.9.10.4=pyhd8ed1ab_0
+  - cfgrib=0.9.11.0=pyhd8ed1ab_0

-  - cftime=1.6.2=py39h2ae25f5_1
+  - cftime=1.6.3=py311h1f0f07a_0

-  - climpred=2.3.0=pyhd8ed1ab_0
+  - climpred=2.4.0=pyhd8ed1ab_0

-  - clisops=0.9.6=pyh1a96a4e_0
+  - clisops=0.13.0=pyhca7485f_0

-  - dask=2023.5.1=pyhd8ed1ab_0
+  - dask=2024.5.0=pyhd8ed1ab_0

-  - geopandas=0.13.0=pyhd8ed1ab_0
+  - geopandas=0.14.4=pyhd8ed1ab_0

-  - hvplot=0.8.3=pyhd8ed1ab_0
+  - hvplot=0.9.2=pyhd8ed1ab_0

-  - numpy=1.23.5=py39h3d75532_0
+  - numpy=1.24.4=py311h64a7726_0

-  - numba=0.57.0=py39hb75a051_1
+  - numba=0.59.1=py311h96b013e_0

# major upgrade from v1 to v2
-  - pandas=1.3.5=py39hde0f152_0
+  - pandas=2.1.4=py311h320fe9a_0

# major upgrade to v1
-  - panel=0.14.4=pyhd8ed1ab_0
+  - panel=1.4.2=pyhd8ed1ab_0

# major upgrade from v1 to v2
-  - pydantic=1.10.8=py39hd1e30aa_0
+  - pydantic=2.7.1=pyhd8ed1ab_0

# Python 3.9 to 3.11
-  - python=3.9.16=h2782a2a_0_cpython
+  - python=3.11.6=hab00c5b_0_cpython

-  - raven-hydro=0.2.1=py39h8e2dbb5_1
+  - raven-hydro=0.2.4=py311h64a4d7b_0

-  - ravenpy=0.12.1=py39hf3d152e_0
+      - ravenpy==0.13.1

-  - rioxarray=0.14.1=pyhd8ed1ab_0
+  - rioxarray=0.15.5=pyhd8ed1ab_0

-  - roocs-utils=0.6.4=pyh1a96a4e_0
+  - roocs-utils=0.6.8=pyhd8ed1ab_0

-  - scipy=1.9.1=py39h8ba3f38_0
+  - scipy=1.13.0=py311h517d4fd_1

-  - xarray=2023.1.0=pyhd8ed1ab_0
+  - xarray=2023.8.0=pyhd8ed1ab_0

-  - xclim=0.43.0=py39hf3d152e_1
+  - xclim=0.47.0=py311h38be061_0

-  - xesmf=0.7.1=pyhd8ed1ab_0
+  - xesmf=0.8.5=pyhd8ed1ab_0

-  - xskillscore=0.0.24=pyhd8ed1ab_0
+  - xskillscore=0.0.26=pyhd8ed1ab_0

+  - xscen=0.8.2=pyhd8ed1ab_0

+      - figanos==0.3.0

-      - xncml==0.2
+      - xncml==0.4.0

```


## Test

- Deployed as "beta" image in production for bokeh visualization
performance regression testing.
- Manual test notebook
https://github.com/Ouranosinc/PAVICS-landing/blob/master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb
for bokeh visualization performance and it looks fine.
- Jenkins build:
- Default notebooks, all passed:
https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/54792e6510adfcd1bb21e1bd31fdfa36c5c634e0/docker/saved_buildout/jenkins-buildlogs-default.txt
- Raven notebooks, only known `HydroShare_integration.ipynb` failing:
https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/931cfc924a147d07b59e88badff9f170e852a03b/docker/saved_buildout/jenkins-buildlogs-raven.txt


## Related Issue / Discussion

- Matching notebook fixes:
  - Pavics-sdi: PR Ouranosinc/pavics-sdi#321
  - Finch: PR url: None
- PAVICS-landing: PR
Ouranosinc/PAVICS-landing#78
  - RavenPy: PR CSHS-CWRA/RavenPy#356
  - Resolves Ouranosinc/PAVICS-landing#65
  - Resolves Ouranosinc/PAVICS-landing#66

- Deployment to PAVICS:
bird-house/birdhouse-deploy#453

- Jenkins-config changes for new notebooks: PR url: None

- Other issues found while working on this one
  - computationalmodelling/nbval#204
  - jupyterlab-contrib/jupyter-archive#132
  - CSHS-CWRA/RavenPy#357
  - CSHS-CWRA/RavenPy#361
  - CSHS-CWRA/RavenPy#362

- Previous release: PR
#134


## Additional Information

Full diff conda env export:

81deb99...931cfc9#diff-e8f2a6a53085ae29bb7cedc701c1d345a330651ae971555e85a5c005e94f4cd9


Full new conda env export:

https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/931cfc924a147d07b59e88badff9f170e852a03b/docker/saved_buildout/conda-env-export.yml


DockerHub build log

https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/931cfc924a147d07b59e88badff9f170e852a03b/docker/saved_buildout/docker-buildlogs.txt
@huard
Copy link
Collaborator Author

huard commented May 31, 2024

Fixed by #367

@huard huard closed this as completed May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant