VAULT-32206: verify audit log and systemd journal secret integrity #28932
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ryancragun
added
pr/no-changelog
pr/no-milestone
backport/ent/1.16.x+ent
github-actions
bot
added
the
hashicorp-contributed-pr
|
Build Results: |
|
CI Results: |
ryancragun
force-pushed
the
vault-32208
branch
from
f3464c5 to
ee96498
Compare
|
Ent test of changes looks good: https://github.com/hashicorp/vault-enterprise/pull/7008 |
ryancragun
force-pushed
the
vault-32208
branch
from
cec6830 to
9f6bd12
Compare
tvo0813
approved these changes
Nov 21, 2024
Signed-off-by: Ryan Cragun <[email protected]>
Signed-off-by: Ryan Cragun <[email protected]>
Integrate the newly created log scanning into the CI pipeline. As part of adding our new secrets I also took the time to migrate our enterprise secrets to use the hosted Vault instance. Signed-off-by: Ryan Cragun <[email protected]>
Signed-off-by: Ryan Cragun <[email protected]>
ryancragun
force-pushed
the
vault-32208
branch
from
9f6bd12 to
17f99f7
Compare
ryancragun
changed the title
6 tasks
Monkeychip
pushed a commit
that referenced
this pull request
Nov 27, 2024
…28932) Verify vault secret integrity in unauthenticated I/O streams (audit log, STDOUT/STDERR via the systemd journal) by scanning the text with Vault Radar. We search for both known and unknown secrets by using an index of KVV2 values and also by radar's built-in heuristics for credentials, secrets, and keys. The verification has been added to many scenarios where a slight time increase is allowed, as we now have to install Vault Radar and scan the text. In practice this adds less than 10 seconds to the overall duration of a scenario. In the in-place upgrade scenario we explicitly exclude this verification when upgrading from a version that we know will fail the check. We also make the verification opt-in so as to not require a Vault Radar license to run Enos scenarios, though it will always be enabled in CI. As part of this we also update our enos workflow to utilize secret values from our self-hosted Vault when executing in the vault-enterprise repo context. Signed-off-by: Ryan Cragun <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
In our effort to ensure that no unhashed secret values are ever present in Vault unauthenticated output streams, we now utilize Vault Radars scanning capabilities to look for both known and unknown secret values in both the Vault audit log and it's service log, the systemd journal. We utilize Radar's kvv2 index scanning feature to scan for known kvv2 secret values in addition to things that look like sensitive credentials.
As part of out integration we also move away from using Github secrets in the enterprise context and instead utilize values in the hosted vault instances.
This has been tested against versions listed in VAULT-30557 and properly detects token values.
TODO: We'll need to remove those as possible initial version targets, or perhaps skip the verification step using a semver constaint.
TODO only if you're a HashiCorp employee
backport/label that matches the desired release branch. Note that in the CE repo, the latest release branch will look likebackport/x.x.x, but older release branches will bebackport/ent/x.x.x+ent.of a public function, even if that change is in a CE file, double check that
applying the patch for this PR to the ENT repo and running tests doesn't
break any tests. Sometimes ENT only tests rely on public functions in CE
files.
in the PR description, commit message, or branch name.
description. Also, make sure the changelog is in this PR, not in your ENT PR.