Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Journalbeat] Add ID config option input #18467

Merged
merged 1 commit into from
May 15, 2020

Conversation

andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented May 12, 2020

What does this PR do?

Add an id configuration option to the Journalbeat input to make it possible
to start multiple inputs on the same journal.

I also fixed a test that was modifying a file under source control by copying it
into the build directory before applying changes.

Why is it important?

As an administrator this provides more flexibility in how you can deploy
Journalbeat. It gives you the configuration grainularity you need to be able
to have an input for each service or log type. You can apply different
include_matches to each input. And you can easily add new log types at any
point and make Journalbeat read all matching logs from the "head" without
affecting existing input cursor positions.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Relates: #10985

Use cases

The configuration I push out to Journalbeat depends on what particular services are deployed on that node. This allows me to setup one input for each service.

journalbeat.inputs:
- id: vault.service
  input_matches: [_SYSTEMD_UNIT=vault.service]
  processors:
    - disect: ...

- id: coredns.service
  input_matches: [_SYSTEMD_UNIT=coredns.service]
  processors:
    - disect: ...

- id: iptables
  input_matches:
    - _TRANSPORT=kernel
    - SYSLOG_IDENTIFIER=kernel
  processors:
    - drop_event:
        when.not.regexp.message: '^iptables'
    - dissect: ...

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 12, 2020
@andrewkroh andrewkroh added the Team:Services (Deprecated) Label for the former Integrations-Services team label May 12, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 12, 2020
@andrewkroh andrewkroh force-pushed the feature/jb/input-id branch from 6db8ad3 to bdbd1b3 Compare May 12, 2020 23:01
@elasticmachine
Copy link
Collaborator

elasticmachine commented May 12, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [andrewkroh commented: run tests]

  • Start Time: 2020-05-14T17:58:52.616+0000

  • Duration: 34 min 33 sec (2013130)

Add an `id` configuration option to the Journalbeat input to make it possible
to start multiple inputs on the same journal.

As an administrator this provides more flexibility in how you can deploy
Journalbeat. It gives you the configuration grainularity you need to be able
to have an input for each service or log type. You can apply different
`include_matches` to each input. And you can easily add new log types at any
point and make Journalbeat read all matching logs from the "head" without
affecting existing input cursor positions.

I also fixed a test that was modifying a file under source control by copying it
into the build directory before applying changes.
@andrewkroh andrewkroh force-pushed the feature/jb/input-id branch from bdbd1b3 to 2b34160 Compare May 13, 2020 00:49
@andrewkroh andrewkroh marked this pull request as ready for review May 13, 2020 01:27
@urso urso requested a review from kvch May 13, 2020 12:00
@andrewkroh
Copy link
Member Author

run tests

@andrewkroh andrewkroh merged commit 74a5bd2 into elastic:master May 15, 2020
andrewkroh added a commit to andrewkroh/beats that referenced this pull request May 15, 2020
Add an `id` configuration option to the Journalbeat input to make it possible
to start multiple inputs on the same journal.

As an administrator this provides more flexibility in how you can deploy
Journalbeat. It gives you the configuration grainularity you need to be able
to have an input for each service or log type. You can apply different
`include_matches` to each input. And you can easily add new log types at any
point and make Journalbeat read all matching logs from the "head" without
affecting existing input cursor positions.

I also fixed a test that was modifying a file under source control by copying it
into the build directory before applying changes.

(cherry picked from commit 74a5bd2)
andrewkroh added a commit that referenced this pull request May 19, 2020
Add an `id` configuration option to the Journalbeat input to make it possible
to start multiple inputs on the same journal.

As an administrator this provides more flexibility in how you can deploy
Journalbeat. It gives you the configuration grainularity you need to be able
to have an input for each service or log type. You can apply different
`include_matches` to each input. And you can easily add new log types at any
point and make Journalbeat read all matching logs from the "head" without
affecting existing input cursor positions.

I also fixed a test that was modifying a file under source control by copying it
into the build directory before applying changes.

(cherry picked from commit 74a5bd2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Journalbeat review Team:Services (Deprecated) Label for the former Integrations-Services team v7.9.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants