Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-63539] Use additional repo URL variants to find cache #947

Closed
wants to merge 27 commits into from
Closed

[JENKINS-63539] Use additional repo URL variants to find cache #947

Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
cfed0dd
Use JGit for caching decision
MarkEWaite Aug 26, 2020
410f70e
[JENKINS-63539] Use additional repo URL variants to find cache
MarkEWaite Aug 27, 2020
be4edc4
Move array initialization inside method
MarkEWaite Aug 29, 2020
1b11caf
Log early exit from alternatives generator
MarkEWaite Aug 29, 2020
9fd24bb
Use size estimate from largest cache
MarkEWaite Aug 29, 2020
abe2421
Use random sizes within range
MarkEWaite Aug 29, 2020
31ecea6
Mention repo URL is git tool log message
MarkEWaite Aug 29, 2020
c87d025
No benefit from initial capacity of LinkedHashSet
MarkEWaite Aug 29, 2020
afb270f
Cache variants were not handling URLs that ended with '/'
MarkEWaite Aug 29, 2020
ff84573
Cache repository size in memory
MarkEWaite Aug 29, 2020
9cc9dc7
Check the remoteAlternatives empty string case
MarkEWaite Aug 29, 2020
e3af88b
Do not include credentials in checkout unless required
MarkEWaite Sep 1, 2020
7176254
[JENKINS-63572] Test for NPE if no remote configs are defined
MarkEWaite Sep 1, 2020
ea58c75
[JENKINS-63572] Avoid NPE if remote configs is empty
MarkEWaite Sep 1, 2020
d2cd2d7
Add GitToolChooser logging to diagnose test failures
MarkEWaite Sep 7, 2020
ae3f031
Use unique project name in GitToolChooserTest
MarkEWaite Sep 7, 2020
1ec4598
Fix compilation error from prior check
MarkEWaite Sep 7, 2020
7a13a85
Fix workflow syntax error from merge
MarkEWaite Sep 7, 2020
da79822
Better formatting of logging
MarkEWaite Sep 7, 2020
cf9aa8c
Report cache dir location when found
MarkEWaite Sep 7, 2020
8f6058d
Better log message when cache entry found
MarkEWaite Sep 7, 2020
d670743
Add call to clear repository cache
MarkEWaite Sep 7, 2020
987b3c0
Clear repository size cache on test entry
MarkEWaite Sep 7, 2020
dbd2c7a
Merge branch 'master' into find-more-caches
MarkEWaite Sep 11, 2020
f74d8ed
Merge branch 'master' into find-more-caches
MarkEWaite Sep 12, 2020
ee2cce4
Remove System.out from test
MarkEWaite Sep 12, 2020
ea73f1e
Fix merge mistakes
MarkEWaite Sep 12, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions src/main/java/jenkins/plugins/git/GitToolChooser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,17 +153,23 @@ private boolean decideAndUseCache(String remoteName) throws IOException, Interru
Git git = Git.with(TaskListener.NULL, new EnvVars(EnvVars.masterEnvVars)).in(cacheDir).using("jgit");
GitClient client = git.getClient();
if (client.hasGitRepo()) {
sizeOfRepo = FileUtils.sizeOfDirectory(cacheDir);
sizeOfRepo = (sizeOfRepo/1000); // Conversion from Bytes to Kilo Bytes
long clientRepoSize = FileUtils.sizeOfDirectory(cacheDir) / 1024;
if (clientRepoSize > sizeOfRepo) {
Copy link
Contributor

@rishabhBudhouliya rishabhBudhouliya Sep 7, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which case would we have multiple caches for the same git repository?
Would it happen if two different multi branch projects work on a different version of the same git repository?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that one case that might happen is two jobs that refer to a remote repository with different URLs and with different update frequencies. If one of the caches is only updated once a week and the other is updated once an hour, the "once an hour" repository may have a larger size than the "once a week" repository. This assumes the larger of the two values is the better approximation of repository size.

/* Use the size of the largest cache */
if (sizeOfRepo > 0) {
LOGGER.log(Level.FINE, "Replacing prior size estimate {0} with new size estimate {1} for remote {2} from cache {3}",
new Object[]{sizeOfRepo, clientRepoSize, remoteName, cacheDir});
}
sizeOfRepo = clientRepoSize;
}
useCache = true;
if (remoteName.equals(repoUrl)) {
LOGGER.log(Level.FINE, "Remote URL {0} using cache {1} with size {2}",
LOGGER.log(Level.FINE, "Remote URL {0} found cache {1} with size {2}",
new Object[]{remoteName, cacheDir, sizeOfRepo});
} else {
LOGGER.log(Level.FINE, "Remote URL {0} using cache {1} with size {2}, alternative URL {3}",
LOGGER.log(Level.FINE, "Remote URL {0} found cache {1} with size {2}, alternative URL {3}",
new Object[]{remoteName, cacheDir, sizeOfRepo, repoUrl});
}
break;
} else {
// Log the surprise but continue looking for a cache
LOGGER.log(Level.FINE, "Remote URL {0} cache {1} has no git dir", new Object[]{remoteName, cacheDir});
Expand Down