Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cache management feature #841

Merged
merged 5 commits into from
Nov 18, 2023
Merged

Add cache management feature #841

merged 5 commits into from
Nov 18, 2023

Conversation

felddy
Copy link
Owner

@felddy felddy commented Nov 17, 2023

🗣 Description

This PR adds the ability to set a maximum container cache size using the CONTAINER_CACHE_SIZE environment variable. After installation of a FoundryVTT distribution, if CONTAINER_CACHE_SIZE is set, the CONTAINER_CACHE directory contents (if any) will be reduced the specified number of FoundryVTT distribution archives.

💭 Motivation and context

This feature was requested by @madereddy in:

A PR was also submitted:

I think this feature makes sense. I took cues from #840 but added some additional safety to the removal procedure. This is also an "opt in" feature as there are folks (me included) that do keep a large number of previous versions for testing.

Closes #830
Supersedes and closes #840

🧪 Testing

I've tested this modification on my development machine, and in continuous integration. Below are sample logs of a large cache being culled:

foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:02 | [info] Installing Foundry Virtual Tabletop 11.315
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:02 | [debug] Checking mime-type of release file.
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:02 | [debug] Found mime-type: application/zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:02 | [debug] Extracting release file.
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [debug] Installation completed.
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [info] Preserving release archive file in cache.
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [info] Cleaning up cache directory: /data/container_cache
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [info] Keeping 3 latest versions.
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.311.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.309.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.308.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.307.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.305.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.304.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.302.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.301.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.300.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.299.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.298.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.297.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.296.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.295.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.294.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-11.292.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.303.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.291.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.290.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.288.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.287.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.286.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.285.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.284.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.283.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.283-a.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.282.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.278.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.277.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.276.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.274.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.273.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.272.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.270.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.265.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.264.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.263.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.262.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-10.260.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.269.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.268.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.266.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.255.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.254.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.251.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.249.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.242.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.240.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.238.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.236.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.235.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.232.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.231.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.226.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.225.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.224.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-9.220.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.9.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.8.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.7.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.7-a.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.6.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.6-b.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.6-a.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.5.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.5-a.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.4.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.3.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.2.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.2-a.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.1.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.8.0.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.7.10.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.7.9.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.7.8.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.7.7.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.7.6.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.7.5.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.7.4.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.7.3.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.7.2.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.7.1.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.7.0.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.6.6.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.6.5.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.6.5-a.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [warn] Removing: /data/container_cache/foundryvtt-0.6.4.zip
foundryvtt-cache-foundry-1  | Entrypoint | 2023-11-16 21:39:04 | [info] Completed cache cleanup. Removed 87 files.

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • Changes are limited to a single goal - eschew scope creep!
  • All relevant type-of-change labels have been added.
  • I have read the CONTRIBUTING document.
  • These code changes follow cisagov code standards.
  • All relevant repo and/or project documentation has been updated
    to reflect the changes in this PR.
  • All new and existing tests pass.

@felddy felddy self-assigned this Nov 17, 2023
@felddy felddy marked this pull request as ready for review November 17, 2023 14:38
@felddy felddy force-pushed the improvement/cache-clean branch 2 times, most recently from 2fc678f to a6ff6b5 Compare November 17, 2023 21:28
@felddy felddy force-pushed the improvement/cache-clean branch from a6ff6b5 to c342cd4 Compare November 17, 2023 21:29
@madereddy
Copy link

@felddy Looks good from my testing. Tested with no variable set and the variable from 1-5 and it handled it perfectly.

@madereddy madereddy mentioned this pull request Nov 18, 2023
9 tasks
@felddy felddy merged commit 325b1b6 into develop Nov 18, 2023
38 checks passed
@felddy felddy deleted the improvement/cache-clean branch November 18, 2023 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automate container_cache cleanup
2 participants