-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
artifact: protect against unbounded artifact decompression (1.4.x) #16126
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
shoenig
force-pushed
the
14x-go-getter-limits
branch
from
February 10, 2023 15:23
d572abd
to
1d7fd37
Compare
shoenig
force-pushed
the
14x-go-getter-limits
branch
2 times, most recently
from
February 10, 2023 16:52
96937b1
to
c39101e
Compare
shoenig
force-pushed
the
14x-go-getter-limits
branch
from
February 10, 2023 19:00
c39101e
to
d813024
Compare
shoenig
force-pushed
the
14x-go-getter-limits
branch
from
February 10, 2023 19:07
d813024
to
c4c9c03
Compare
shoenig
force-pushed
the
14x-go-getter-limits
branch
from
February 10, 2023 19:12
c4c9c03
to
18652fd
Compare
Spot check nomad.hclserver {
enabled = true
}
client {
enabled = true
artifact {
decompression_size_limit = "1M"
decompression_file_count_limit = 1024
}
} bomb.nomadjob "bomb" {
datacenters = ["dc1"]
group "group" {
task "task" {
driver = "exec"
artifact {
source = "https://github.com/hashicorp/go-getter/raw/main/testdata/decompress-zip/bomb.zip"
destination = "local/"
}
config {
command = "/bin/sleep"
args = ["infinity"]
}
resources {
cpu = 100
memory = 256
}
}
}
} agent logs
no count limitclient {
enabled = true
artifact {
decompression_size_limit = "1M"
decompression_file_count_limit = 0
}
} agent logs 2023-02-10T19:19:33.560Z [ERROR] client.alloc_runner.task_runner: prestart failed: alloc_id=dcfa7246-9e18-d510-c1d1-d52d33528d81 task=task error="prestart hook \"artifacts\" failed: failed to download artifact \"https://github.com/hashicorp/go-getter/raw/main/testdata/decompress-zip/bomb.zip\": zip archive larger than limit: 1000000" |
This PR enables mitigations provided by go-getter against payloads which decompress into an unbounded size or file count. There are two new client config options under the artifact block: artifact.decompression_size_limit (e.g. "10GB") - the maximum amount of data that will be decompressed before triggering an error and cancelling the operation artifact.decompression_file_count_limit (e.g. 1024) - the maximum number of files that will be decompressed before triggering ana error and cancelling the operation.
shoenig
force-pushed
the
14x-go-getter-limits
branch
from
February 10, 2023 19:24
18652fd
to
0db43aa
Compare
shoenig
commented
Feb 10, 2023
go.mod
Outdated
@@ -54,7 +54,7 @@ require ( | |||
// versions. | |||
github.com/hashicorp/go-discover v0.0.0-20220621183603-a413e131e836 | |||
github.com/hashicorp/go-envparse v0.0.0-20180119215841-310ca1881b22 | |||
github.com/hashicorp/go-getter v1.6.2 | |||
github.com/hashicorp/go-getter v1.6.3-0.20230210143508-0edab8534827 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will switch to v1.7.0
once the tag exists
shoenig
added
backport/1.2.x
backport to 1.1.x release line
backport/1.3.x
backport to 1.3.x release line
labels
Feb 10, 2023
tgross
approved these changes
Feb 10, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
picatz
approved these changes
Feb 10, 2023
shoenig
changed the title
artifact: protect against unbounded artifact decompression
artifact: protect against unbounded artifact decompression (1.4.x)
Feb 13, 2023
shoenig
added a commit
that referenced
this pull request
Feb 13, 2023
…16126) * artifact: protect against unbounded artifact decompression This PR enables mitigations provided by go-getter against payloads which decompress into an unbounded size or file count. There are two new client config options under the artifact block: artifact.decompression_size_limit (e.g. "10GB") - the maximum amount of data that will be decompressed before triggering an error and cancelling the operation artifact.decompression_file_count_limit (e.g. 1024) - the maximum number of files that will be decompressed before triggering ana error and cancelling the operation. * fixup CR comments * deps: update to go-getter 1.7.0
shoenig
added a commit
that referenced
this pull request
Feb 13, 2023
…16126) * artifact: protect against unbounded artifact decompression This PR enables mitigations provided by go-getter against payloads which decompress into an unbounded size or file count. There are two new client config options under the artifact block: artifact.decompression_size_limit (e.g. "10GB") - the maximum amount of data that will be decompressed before triggering an error and cancelling the operation artifact.decompression_file_count_limit (e.g. 1024) - the maximum number of files that will be decompressed before triggering ana error and cancelling the operation. * fixup CR comments * deps: update to go-getter 1.7.0
This was referenced Feb 13, 2023
shoenig
added a commit
that referenced
this pull request
Feb 13, 2023
…16126) (#16157) * artifact: protect against unbounded artifact decompression This PR enables mitigations provided by go-getter against payloads which decompress into an unbounded size or file count. There are two new client config options under the artifact block: artifact.decompression_size_limit (e.g. "10GB") - the maximum amount of data that will be decompressed before triggering an error and cancelling the operation artifact.decompression_file_count_limit (e.g. 1024) - the maximum number of files that will be decompressed before triggering ana error and cancelling the operation. * fixup CR comments * deps: update to go-getter 1.7.0
shoenig
added a commit
that referenced
this pull request
Feb 13, 2023
…16126) (#16158) * artifact: protect against unbounded artifact decompression This PR enables mitigations provided by go-getter against payloads which decompress into an unbounded size or file count. There are two new client config options under the artifact block: artifact.decompression_size_limit (e.g. "10GB") - the maximum amount of data that will be decompressed before triggering an error and cancelling the operation artifact.decompression_file_count_limit (e.g. 1024) - the maximum number of files that will be decompressed before triggering ana error and cancelling the operation. * fixup CR comments * deps: update to go-getter 1.7.0
tgross
pushed a commit
that referenced
this pull request
Feb 14, 2023
…16126) (#16158) * artifact: protect against unbounded artifact decompression This PR enables mitigations provided by go-getter against payloads which decompress into an unbounded size or file count. There are two new client config options under the artifact block: artifact.decompression_size_limit (e.g. "10GB") - the maximum amount of data that will be decompressed before triggering an error and cancelling the operation artifact.decompression_file_count_limit (e.g. 1024) - the maximum number of files that will be decompressed before triggering ana error and cancelling the operation. * fixup CR comments * deps: update to go-getter 1.7.0
tgross
pushed a commit
that referenced
this pull request
Feb 14, 2023
…16126) (#16157) * artifact: protect against unbounded artifact decompression This PR enables mitigations provided by go-getter against payloads which decompress into an unbounded size or file count. There are two new client config options under the artifact block: artifact.decompression_size_limit (e.g. "10GB") - the maximum amount of data that will be decompressed before triggering an error and cancelling the operation artifact.decompression_file_count_limit (e.g. 1024) - the maximum number of files that will be decompressed before triggering ana error and cancelling the operation. * fixup CR comments * deps: update to go-getter 1.7.0
tgross
pushed a commit
that referenced
this pull request
Feb 14, 2023
…16126) * artifact: protect against unbounded artifact decompression This PR enables mitigations provided by go-getter against payloads which decompress into an unbounded size or file count. There are two new client config options under the artifact block: artifact.decompression_size_limit (e.g. "10GB") - the maximum amount of data that will be decompressed before triggering an error and cancelling the operation artifact.decompression_file_count_limit (e.g. 1024) - the maximum number of files that will be decompressed before triggering ana error and cancelling the operation. * fixup CR comments * deps: update to go-getter 1.7.0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR enables mitigation provided by go-getter against payloads which decompress into an unbounded size. Nomad Client now exposes two new fields in the
artifact
block,Note that for the 1.4.x, 1.3.x and 1.2.x these values are left
unset
, meaning nolimit will be applied. Operators will need to opt-in to the mitigation if they choose
to do so.
In Nomad 1.5 these limits will start defaulting to some value (which may be a breaking
change for some users).
Targets:
release/1.4.x
, Backport to 1.3.x and 1.2.x(not main, which changed all of this code and will get its own PR)