Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x-pack/filebeat/input/cel: make redact configuration recommended #36008

Merged
merged 2 commits into from
Jul 11, 2023
Merged

x-pack/filebeat/input/cel: make redact configuration recommended #36008

merged 2 commits into from
Jul 11, 2023

Conversation

efd6
Copy link
Contributor

@efd6 efd6 commented Jul 6, 2023

What does this PR do?

This detects when a user has not configured the redact options in the CEL input for filebeat and logs it as missing with a link to the documentation.

Ideally this would be a hard requirement, but that would be a breaking change, so just log at WARN if the configuration is missing.

Why is it important?

By design the CEL input can log the entire input state and its evaluation during debug logging. This may leak secrets, so we provide an option to allow users to specify fields that are sensitive to be redacted or deleted. If the user is not aware of this behaviour or the option to redact they may put themselves at risk. The change here it designed to help them identify this risk.

Backported to 8.9 despite being an enhancement because of risk.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works — manually checked
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

@efd6 efd6 requested a review from a team as a code owner July 6, 2023 01:20
@efd6 efd6 self-assigned this Jul 6, 2023
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 6, 2023
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jul 6, 2023
@botelastic
Copy link

botelastic bot commented Jul 6, 2023

This pull request doesn't have a Team:<team> label.

@efd6 efd6 force-pushed the cel-redact-recommended branch from 62b1fa2 to b2488a0 Compare July 6, 2023 01:21
efd6
efd6 commented Jul 6, 2023
View reviewed changes
x-pack/filebeat/input/cel/config.go Outdated
@@ -65,6 +66,10 @@ type redact struct {
}

func (c config) Validate() error {
if c.Redact == nil {
logp.L().Named("input.cel").Warn("missing recommended 'redact' configuration: " +
"see documentation for details: https://www.elastic.co/guide/en/beats/filebeat/8.9/filebeat-input-cel.html#_redact_fields")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link here is to 8.9 because the docs currently list "current" as 8.8.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When this is released "current" will point to 8.9 so I think it would be safe to use it here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. In that case I'll change to _redact rather than _redact_fields, since that will have the actual claims.

@elasticmachine
Copy link
Collaborator

elasticmachine commented Jul 6, 2023 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-07-11T21:15:17.254+0000

  • Duration: 72 min 34 sec

Test stats 🧪

Test Results
Failed 0
Passed 3050
Skipped 178
Total 3228

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@mergify

This comment was marked as outdated.

Sign in to view
@efd6 efd6 force-pushed the cel-redact-recommended branch from b2488a0 to 5504cc2 Compare July 10, 2023 21:41
@mergify
Copy link
Contributor

mergify bot commented Jul 11, 2023

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b cel-redact-recommended upstream/cel-redact-recommended
git merge upstream/main
git push upstream cel-redact-recommended

@efd6
x-pack/filebeat/input/cel: make redact configuration recommended
5e2fe86 
Ideally this would be a hard requirement, but that would be a breaking
change, so just log at WARN if the configuration is missing.
@efd6 efd6 force-pushed the cel-redact-recommended branch from 5504cc2 to 5e2fe86 Compare July 11, 2023 08:55
andrewkroh
andrewkroh approved these changes Jul 11, 2023
View reviewed changes
x-pack/filebeat/input/cel/config.go Outdated
@@ -65,6 +66,10 @@ type redact struct {
}

func (c config) Validate() error {
if c.Redact == nil {
logp.L().Named("input.cel").Warn("missing recommended 'redact' configuration: " +
"see documentation for details: https://www.elastic.co/guide/en/beats/filebeat/8.9/filebeat-input-cel.html#_redact_fields")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When this is released "current" will point to 8.9 so I think it would be safe to use it here.

@efd6 efd6 merged commit ae923ba into elastic:main Jul 11, 2023
mergify bot pushed a commit that referenced this pull request Jul 11, 2023
@efd6 @mergify
x-pack/filebeat/input/cel: make redact configuration recommended (#36008
7c9afbc 
)

Ideally this would be a hard requirement, but that would be a breaking
change, so just log at WARN if the configuration is missing.

(cherry picked from commit ae923ba)
@mergify mergify bot mentioned this pull request Jul 11, 2023
Merged
efd6 added a commit that referenced this pull request Jul 12, 2023
@mergify @efd6
x-pack/filebeat/input/cel: make redact configuration recommended (#36008
0dbf0eb 
) (#36046)

Ideally this would be a hard requirement, but that would be a breaking
change, so just log at WARN if the configuration is missing.

(cherry picked from commit ae923ba)

Co-authored-by: Dan Kortschak <[email protected]>
Scholar-Li pushed a commit to Scholar-Li/beats that referenced this pull request Feb 5, 2024
@efd6 @Scholar-Li
x-pack/filebeat/input/cel: make redact configuration recommended (ela…
aa76408 
…stic#36008)

Ideally this would be a hard requirement, but that would be a breaking
change, so just log at WARN if the configuration is missing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewkroh andrewkroh andrewkroh approved these changes

Assignees

@efd6 efd6

Labels
8.10-candidate backport-v8.9.0 Automated backport with mergify enhancement Filebeat Filebeat
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@efd6 @elasticmachine @andrewkroh