Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #18467 to 7.x: [Journalbeat] Add ID config option input #18590

Merged
merged 1 commit into from
May 19, 2020

Conversation

andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented May 15, 2020

Cherry-pick of PR #18467 to 7.x branch. Original message:

What does this PR do?

Add an id configuration option to the Journalbeat input to make it possible
to start multiple inputs on the same journal.

I also fixed a test that was modifying a file under source control by copying it
into the build directory before applying changes.

Why is it important?

As an administrator this provides more flexibility in how you can deploy
Journalbeat. It gives you the configuration grainularity you need to be able
to have an input for each service or log type. You can apply different
include_matches to each input. And you can easily add new log types at any
point and make Journalbeat read all matching logs from the "head" without
affecting existing input cursor positions.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Relates: #10985

Use cases

The configuration I push out to Journalbeat depends on what particular services are deployed on that node. This allows me to setup one input for each service.

journalbeat.inputs:
- id: vault.service
  input_matches: [_SYSTEMD_UNIT=vault.service]
  processors:
    - disect: ...

- id: coredns.service
  input_matches: [_SYSTEMD_UNIT=coredns.service]
  processors:
    - disect: ...

- id: iptables
  input_matches:
    - _TRANSPORT=kernel
    - SYSLOG_IDENTIFIER=kernel
  processors:
    - drop_event:
        when.not.regexp.message: '^iptables'
    - dissect: ...

Add an `id` configuration option to the Journalbeat input to make it possible
to start multiple inputs on the same journal.

As an administrator this provides more flexibility in how you can deploy
Journalbeat. It gives you the configuration grainularity you need to be able
to have an input for each service or log type. You can apply different
`include_matches` to each input. And you can easily add new log types at any
point and make Journalbeat read all matching logs from the "head" without
affecting existing input cursor positions.

I also fixed a test that was modifying a file under source control by copying it
into the build directory before applying changes.

(cherry picked from commit 74a5bd2)
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 15, 2020
@andrewkroh andrewkroh added the Team:Services (Deprecated) Label for the former Integrations-Services team label May 15, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 15, 2020
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

@andrewkroh andrewkroh merged commit 98972eb into elastic:7.x May 19, 2020
@andrewkroh andrewkroh deleted the backport_18467_7.x branch January 14, 2022 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport review Team:Services (Deprecated) Label for the former Integrations-Services team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants