Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add local caching of slicing for data reduction #635

Merged
merged 4 commits into from
Mar 4, 2019

Conversation

mraspaud
Copy link
Member

@mraspaud mraspaud commented Feb 28, 2019

This PR avoids computing the slices for data reduction repeatedly when a bunch of datasets are resampled from the same area to the same area at the same time

  • Tests added
  • Tests passed
  • Passes git diff origin/master -- "*py" | flake8 --diff
  • Fully documented

@mraspaud mraspaud self-assigned this Feb 28, 2019
Copy link
Member

@djhoese djhoese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this (the use of a reductions dictionary) be cleaned up by using iter_by_area or something similar? Could the reduction functionality shown here then be shared with the other similar methods like crop and slice? Maybe something like an iterated slicing/reducing method?

@coveralls
Copy link

coveralls commented Mar 4, 2019

Coverage Status

Coverage increased (+0.01%) to 78.303% when pulling 49604aa on mraspaud:feature-data-reduction-cache into ab4956a on pytroll:master.

@codecov
Copy link

codecov bot commented Mar 4, 2019

Codecov Report

Merging #635 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #635      +/-   ##
=========================================
+ Coverage   78.28%   78.3%   +0.01%     
=========================================
  Files         136     136              
  Lines       19723   19735      +12     
=========================================
+ Hits        15441   15453      +12     
  Misses       4282    4282
Impacted Files Coverage Δ
satpy/scene.py 88.13% <100%> (+0.13%) ⬆️
satpy/tests/test_scene.py 99.45% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ab4956a...49604aa. Read the comment docs.

@mraspaud mraspaud merged commit ccacdf4 into pytroll:master Mar 4, 2019
@mraspaud mraspaud deleted the feature-data-reduction-cache branch March 4, 2019 21:38
@mraspaud
Copy link
Member Author

mraspaud commented Mar 5, 2019

@djhoese 's comments are recorded in issue #637

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:resampling component:scene enhancement code enhancements, features, improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants