Add garbage collection to jobs #586
Conversation
|
Some thoughts about the design - a. I am wondering why we have exposed the gc duration configurable for an end user of Nomad. GCing jobs out of Nomad when they are no longer going to be referenced by the system doesn't feel like something an end user needs to configure via the job defn. b. We could just GC any job which won't be referenced again in the system. Batch Jobs by definition feels GC-able. Service jobs which has count > 1 and num of Tasks > 1 should never be GC-able until the moment the user stops the job. Setting count = 0 or num Tasks = 0 is almost same as the job getting stopped. System jobs, the way they are currently implemented shouldn't be GC-ed as well. c. Periodic Jobs are interesting - If a periodic job won't be invoked again to submit a child jobs because of the configuration of it's spec(which seems highly unlikely), it's a candidate for being GC-ed, otherwise there shouldn't be any need for GC-ing it. d. Any of the jobs - periodic, batch or service jobs which are stopped can immediately be GCed. I think (b) and (c) just explains that when a job can be GC-ed from the system could be inferred by the state of it's current allocations and whether any future evaluations would be created for that job without needing any additional configuration from the end user whose job is running on Nomad. |
|
@diptanu: Updated to address comments. |
| c.srv.logger.Printf("[DEBUG] sched.core: job GC: scanning before index %d (%v)", | ||
| oldThreshold, c.srv.config.JobGCThreshold) | ||
|
|
||
| // Collect the allocations and evaluations to GC |
There was a problem hiding this comment.
allocations, evaluations and jobs
|
LGTM overall. |
|
I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions. |
This PR adds garbage collection to jobs and marks batch jobs as eligible for garbage collection