Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache validation ignore pyaerocom version; use mtime instead of ctime #1243

Merged

Conversation

heikoklein
Copy link
Member

@heikoklein heikoklein commented Jul 5, 2024

Change Summary

The cache validation of ungridded data was using the pyaercom_version to identify invalid caches. Since caches and pyaerocom_version are independent, this is now only logged, but no longer used to invalidate caches.

In addition, cache validation was using ctime to identify newest files, but ctime is metadata change time, i.e. when permissions/ownerships of data changed last. This has been changed to mtime which is data modification time.

Related issue number

fixes #1242

Checklist

  • Start with a draft-PR
  • The PR title is a good summary of the changes
  • PR is set to AeroTools and a tentative milestone
  • Documentation reflects the changes where applicable
  • Tests for the changes exist where applicable
  • Tests pass locally
  • Tests pass on CI
  • At least 1 reviewer is selected
  • Make PR ready to review

@heikoklein heikoklein linked an issue Jul 5, 2024 that may be closed by this pull request
@heikoklein heikoklein added this to the m2024-08 milestone Jul 5, 2024
Copy link

codecov bot commented Jul 5, 2024

Codecov Report

Attention: Patch coverage is 70.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 79.33%. Comparing base (f244a51) to head (b1fa238).
Report is 449 commits behind head on main-dev.

Files with missing lines Patch % Lines
pyaerocom/io/cachehandler_ungridded.py 70.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##           main-dev    #1243   +/-   ##
=========================================
  Coverage     79.33%   79.33%           
=========================================
  Files           133      133           
  Lines         20284    20289    +5     
=========================================
+ Hits          16092    16096    +4     
- Misses         4192     4193    +1     
Flag Coverage Δ
unittests 79.33% <70.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@heikoklein heikoklein marked this pull request as ready for review July 5, 2024 10:53
Copy link
Member

@jgriesfeller jgriesfeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@heikoklein heikoklein merged commit 7458cfc into main-dev Jul 5, 2024
7 of 8 checks passed
@heikoklein heikoklein deleted the 1242-update-caching-strategy-for-ungridded-data branch July 5, 2024 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update caching strategy for ungridded data
2 participants