Add a configurable cap on total pantsd memory usage. #10003
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Delete this line to force CI to run Clippy and the Rust tests. [ci skip-rust-tests] # Delete this line to force CI to run the JVM tests. [ci skip-jvm-tests]
Eric-Arellano
approved these changes
Jun 8, 2020
| @@ -399,6 +399,22 @@ def test_pantsd_memory_usage(self): | |||
| ), | |||
| ) | |||
|
|
|||
| def test_pantsd_max_memory_usage(self): | |||
There was a problem hiding this comment.
Will probably want to sanity check that test_pantsd_memory_usage is still relevant. I know they do different things, but possible bit rot not. NB that we skip it due to flakiness.
There was a problem hiding this comment.
It is still relevant (see the comments on the ticket linked in the description of this PR), but I have not been able to follow up on re-enabling it.
Co-authored-by: Eric Arellano <[email protected]>
gshuflin
approved these changes
Jun 9, 2020
Closed
This was referenced Jul 6, 2020
stuhood
added a commit
that referenced
this pull request
Jul 7, 2020
### Problem The default max memory usage from #10003 was chosen with larger monorepos in mind, and didn't match the expectation of consumers in smaller repos. ### Solution Very large repos will have folks who are able/willing to adjust limits like this to optimize for their repo size, so adjust the default down in favor of having good defaults. This relates to #7675, but it isn't the time to dive in there. ### Result Fixes #10264. [ci skip-rust-tests]
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
pantsddoes not implement garbage collection of theGraph(see #7675), but additionally, there are likely a few Python-level reference cycles beyond those that we have already discovered.Solution
We will eventually implement #7675, and it will need a config value to control how much it collects. But in the meantime, having a configurable built-in cap on total memory usage is generally useful, and can consume the same flag that our eventual collection will.
Result
pantsdwill restart itself when it uses more than the configured amount of memory (defaulting to 4GB). As mentioned in the test comment, until #8200 is fixed, the message rendered when we restart will not be particularly friendly, so we should likely not cherry-pick this to 1.29.x, which will not receive #8200.This is not a complete fix for #9999, but I'm going to resolve it in favor of tracking followup in #7675.
[ci skip-rust-tests]
[ci skip-jvm-tests]