Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve resample_in_space and rectify_dataset #1098

Merged
merged 9 commits into from
Jan 2, 2025

Conversation

konstntokas
Copy link
Contributor

@konstntokas konstntokas commented Dec 20, 2024

This PR improves the resampling_in_space method, if another CRS is required and therefore a reprojection needs to be done.

  • The method xcube.core.GridMapping.transform now supports lazy execution. If
    computations based on actual data are required—such as determining whether the
    grid mapping is regular or estimating the resolution in the x or y direction—only a
    single chunk is accessed whenever possible, ensuring faster performance.
    -> see large example examples/notebooks/resampling/reproject_large_esa_cci_landcover.ipynb execution time of cell 5 drops from 1.42min to ~800ms.

  • The function xcube.core.resampling.rectify_dataset now supports xarray.Datasets
    containing multi-dimensional data variables structured as var(..., y_dim, x_dim).
    The two spatial dimensions (y_dim and x_dim) must occupy the last two positions
    in the variable's dimensions.
    -> see large example examples/notebooks/resampling/reproject_large_esa_cci_landcover.ipynb execution time of cell 6 drops from 4.51min to ~2.16ms, even though now 5 time steps are selected compared to just one.

Checklist:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/source/*
  • Changes documented in CHANGES.md
  • GitHub CI passes
  • AppVeyor CI passes
  • Test coverage remains or increases (target 100%)

@konstntokas konstntokas force-pushed the konstntokas-xxx-improve_resample_in_space branch from b78cca8 to 92ffe7b Compare December 23, 2024 11:36
@konstntokas
Copy link
Contributor Author

CI fails due to two tests failures in test/core/gen2/test_progress, which is not related to this PR.

@konstntokas konstntokas requested a review from forman December 23, 2024 12:00
@konstntokas konstntokas marked this pull request as ready for review December 23, 2024 12:05
Copy link
Member

@forman forman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks! Please take care of my suggestions, then merge.

xcube/core/resampling/rectify.py Show resolved Hide resolved
xcube/core/resampling/rectify.py Show resolved Hide resolved
xcube/core/resampling/rectify.py Show resolved Hide resolved
xcube/core/resampling/rectify.py Outdated Show resolved Hide resolved
xcube/core/resampling/rectify.py Outdated Show resolved Hide resolved
@konstntokas konstntokas force-pushed the konstntokas-xxx-improve_resample_in_space branch from 3f38055 to 27b4ae2 Compare January 2, 2025 11:52
@konstntokas konstntokas merged commit f3184e3 into main Jan 2, 2025
1 of 3 checks passed
@konstntokas konstntokas deleted the konstntokas-xxx-improve_resample_in_space branch January 2, 2025 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants