Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-65612] Support for immediately cleaning @libs workspaces #110

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

basil
Copy link
Member

@basil basil commented May 11, 2021

See JENKINS-65612. SCMRetriever and SCMSourceRetriever create a @libs workspace for retrieving the shared library before copying it into the build directory. While old build directories are eventually discarded based on the global build discarder configuration, these @libs workspaces tend to stick around for a long time and eventually need to be removed. It would be nice if a flag could be set on SCMRetriever and SCMSourceRetriever to clean these @libs workspaces immediately after we are done with them.

@basil basil closed this May 11, 2021
@basil basil reopened this May 11, 2021
@basil basil closed this May 12, 2021
@basil basil reopened this May 12, 2021
@jglick
Copy link
Member

jglick commented May 13, 2021

Why not just implement ItemListener.onDeleted so workspace/xxx@libs/ gets deleted when jobs/xxx/ is deleted?

@basil
Copy link
Member Author

basil commented May 13, 2021

just

I could see an argument for also implementing that by default under the general principle of not leaking resources. But I think this PR has independent value. Let me explain why.

We (unfortunately) use a lot of static agents, and we want to keep builds as isolated as possible. So our shared library has a vars/nodeWithCleanup.groovy with

def call(Map params = [:], Closure closure) {
  def cleanupClosure = {
    try {
      it()
    } finally {
      cleanWs()
    }
  }

  node(params.label) {
    if (params.workspace) {
      ws(params.workspace) {
        cleanupClosure(closure)
      }
    } else {
      cleanupClosure(closure)
    }
  }
}

which we use universally as a generic substitute for node. This allows us to reclaim the workspace immediately when the run ends, since we were going to wipe it on the next run anyway. This avoids wasting space while jobs are at rest, which is useful for jobs that run very infrequently (we have thousands of jobs that run a few times a month to cover various portions of our test matrix).

We've found the above quite useful, so I also think we would find the feature in this PR useful for the same reason (opt-in, of course, rather than by default).

@basil
Copy link
Member Author

basil commented Sep 13, 2021

Why not just implement ItemListener.onDeleted so workspace/xxx@libs/ gets deleted when jobs/xxx/ is deleted?

Done in #115.

@car-roll
Copy link
Contributor

car-roll commented Nov 4, 2021

#115 merged, closing this pr

@car-roll car-roll closed this Nov 4, 2021
@jglick
Copy link
Member

jglick commented Nov 4, 2021

@car-roll read #110 (comment)

@jglick jglick reopened this Nov 4, 2021
@jglick
Copy link
Member

jglick commented May 23, 2022

As of #172 most of the code in this plugin has been moved to another plugin repository so this PR must be closed. If this change is still needed, please

git clone https://github.com/jenkinsci/pipeline-groovy-lib-plugin
cd pipeline-groovy-lib-plugin
git checkout -b JENKINS-65612
git pull https://github.com/basil/workflow-cps-global-lib-plugin JENKINS-65612

resolve any merge conflicts, and file a fresh PR on the new repository. Be sure to paste a link to this old PR to enable bidirectional navigation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants