-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix syslog RFC 5424 parsing in CheckPoint module #21854
Merged
andrewkroh
merged 1 commit into
elastic:master
from
andrewkroh:bugfix/fb/checkpoint-syslog-rfc5424
Oct 15, 2020
Merged
Fix syslog RFC 5424 parsing in CheckPoint module #21854
andrewkroh
merged 1 commit into
elastic:master
from
andrewkroh:bugfix/fb/checkpoint-syslog-rfc5424
Oct 15, 2020
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andrewkroh
added
bug
review
Filebeat
Filebeat
Team:Security-External Integrations
labels
Oct 15, 2020
botelastic
bot
added
needs_team
Indicates that the issue/PR needs a Team:* label
and removed
needs_team
Indicates that the issue/PR needs a Team:* label
labels
Oct 15, 2020
andrewkroh
force-pushed
the
bugfix/fb/checkpoint-syslog-rfc5424
branch
from
October 15, 2020 14:24
b4265dc
to
4135c42
Compare
Change the input type in the CheckPoint module to `udp` from `syslog` so the syslog parsing happens in the ingest node pipeline rather than in the Filebeat syslog input that only support RFC 3164.
andrewkroh
force-pushed
the
bugfix/fb/checkpoint-syslog-rfc5424
branch
from
October 15, 2020 14:28
4135c42
to
cc73439
Compare
andrewkroh
added
the
needs_backport
PR is waiting to be backported to other branches.
label
Oct 15, 2020
Collaborator
marc-gr
approved these changes
Oct 15, 2020
4 tasks
andrewkroh
added a commit
to andrewkroh/beats
that referenced
this pull request
Oct 15, 2020
Change the input type in the CheckPoint module to `udp` from `syslog` so the syslog parsing happens in the ingest node pipeline rather than in the Filebeat syslog input that only support RFC 3164. (cherry picked from commit f2e161f)
andrewkroh
added
v7.11.0
and removed
needs_backport
PR is waiting to be backported to other branches.
labels
Oct 15, 2020
andrewkroh
added a commit
to andrewkroh/beats
that referenced
this pull request
Oct 15, 2020
Change the input type in the CheckPoint module to `udp` from `syslog` so the syslog parsing happens in the ingest node pipeline rather than in the Filebeat syslog input that only support RFC 3164. (cherry picked from commit f2e161f)
4 tasks
4 tasks
andrewkroh
added a commit
to andrewkroh/beats
that referenced
this pull request
Oct 15, 2020
Change the input type in the CheckPoint module to `udp` from `syslog` so the syslog parsing happens in the ingest node pipeline rather than in the Filebeat syslog input that only support RFC 3164. (cherry picked from commit f2e161f)
v1v
added a commit
to v1v/beats
that referenced
this pull request
Oct 19, 2020
* upstream/master: (23 commits) [Ingest Manager] Prevent reporting ecs version twice (elastic#21616) [CI] Use google storage to keep artifacts (elastic#21910) Update docs.asciidoc (elastic#21849) Kubernetes leaderelection improvements (elastic#21896) Apply name changes to elastic agent docs (elastic#21549) Add 7.7.1 relnotes to 7.8 docs (elastic#21937) (elastic#21941) [libbeat] Fix potential deadlock in the disk queue + add more unit tests (elastic#21930) Refactor docker watcher to fix flaky test and other small issues (elastic#21851) [CI] Add stage name in the step (elastic#21887) [docs] Remove extra word in autodiscover docs (elastic#21871) [CI] lint stage doesn't produce test reports (elastic#21888) Add tests of reader of filestream input (elastic#21814) [Ingest Manager] Use local temp instead of system one (elastic#21883) chore: delegate variant pushes to the right method (elastic#21861) [CI] kind setup fails sometimes (elastic#21857) Fix panic on add_docker_metadata close (elastic#21882) Add tests for fileProspector in filestream input (elastic#21712) [Filebeat][okta] Fix okta pagination (elastic#21797) Add cloud.account.id into add_cloud_metadata for gcp (elastic#21776) Fix syslog RFC 5424 parsing in CheckPoint module (elastic#21854) ...
v1v
added a commit
to v1v/beats
that referenced
this pull request
Oct 19, 2020
* upstream/master: (23 commits) [Ingest Manager] Prevent reporting ecs version twice (elastic#21616) [CI] Use google storage to keep artifacts (elastic#21910) Update docs.asciidoc (elastic#21849) Kubernetes leaderelection improvements (elastic#21896) Apply name changes to elastic agent docs (elastic#21549) Add 7.7.1 relnotes to 7.8 docs (elastic#21937) (elastic#21941) [libbeat] Fix potential deadlock in the disk queue + add more unit tests (elastic#21930) Refactor docker watcher to fix flaky test and other small issues (elastic#21851) [CI] Add stage name in the step (elastic#21887) [docs] Remove extra word in autodiscover docs (elastic#21871) [CI] lint stage doesn't produce test reports (elastic#21888) Add tests of reader of filestream input (elastic#21814) [Ingest Manager] Use local temp instead of system one (elastic#21883) chore: delegate variant pushes to the right method (elastic#21861) [CI] kind setup fails sometimes (elastic#21857) Fix panic on add_docker_metadata close (elastic#21882) Add tests for fileProspector in filestream input (elastic#21712) [Filebeat][okta] Fix okta pagination (elastic#21797) Add cloud.account.id into add_cloud_metadata for gcp (elastic#21776) Fix syslog RFC 5424 parsing in CheckPoint module (elastic#21854) ...
v1v
added a commit
to v1v/beats
that referenced
this pull request
Oct 19, 2020
…laky-test-analyser * upstream/master: (22 commits) [Ingest Manager] Prevent reporting ecs version twice (elastic#21616) [CI] Use google storage to keep artifacts (elastic#21910) Update docs.asciidoc (elastic#21849) Kubernetes leaderelection improvements (elastic#21896) Apply name changes to elastic agent docs (elastic#21549) Add 7.7.1 relnotes to 7.8 docs (elastic#21937) (elastic#21941) [libbeat] Fix potential deadlock in the disk queue + add more unit tests (elastic#21930) Refactor docker watcher to fix flaky test and other small issues (elastic#21851) [CI] Add stage name in the step (elastic#21887) [docs] Remove extra word in autodiscover docs (elastic#21871) [CI] lint stage doesn't produce test reports (elastic#21888) Add tests of reader of filestream input (elastic#21814) [Ingest Manager] Use local temp instead of system one (elastic#21883) chore: delegate variant pushes to the right method (elastic#21861) [CI] kind setup fails sometimes (elastic#21857) Fix panic on add_docker_metadata close (elastic#21882) Add tests for fileProspector in filestream input (elastic#21712) [Filebeat][okta] Fix okta pagination (elastic#21797) Add cloud.account.id into add_cloud_metadata for gcp (elastic#21776) Fix syslog RFC 5424 parsing in CheckPoint module (elastic#21854) ...
@andrewkroh just to confirm. Looking into the just released 7.9.3 Filebeat, it seems the change didn't make to this release. I still see
Could you please confirm if this is correct, or am I missing something? Thanks a lot. |
andrewkroh
added a commit
that referenced
this pull request
Oct 27, 2020
Change the input type in the CheckPoint module to `udp` from `syslog` so the syslog parsing happens in the ingest node pipeline rather than in the Filebeat syslog input that only support RFC 3164. (cherry picked from commit f2e161f)
andrewkroh
added a commit
that referenced
this pull request
Oct 27, 2020
Change the input type in the CheckPoint module to `udp` from `syslog` so the syslog parsing happens in the ingest node pipeline rather than in the Filebeat syslog input that only support RFC 3164. (cherry picked from commit f2e161f)
2 tasks
andrewkroh
added a commit
to andrewkroh/integrations
that referenced
this pull request
Oct 28, 2020
Sync the change from elastic/beats#21854 to use the UDP input instead of syslog input to allow for RFC 5424 syslog parsing.
andrewkroh
added a commit
to elastic/integrations
that referenced
this pull request
Oct 29, 2020
* Update CheckPoint package with RFC 5424 parsing fix Sync the change from elastic/beats#21854 to use the UDP input instead of syslog input to allow for RFC 5424 syslog parsing. * paths is required for logfile
eyalkraft
pushed a commit
to build-security/integrations
that referenced
this pull request
Mar 30, 2022
* Update CheckPoint package with RFC 5424 parsing fix Sync the change from elastic/beats#21854 to use the UDP input instead of syslog input to allow for RFC 5424 syslog parsing. * paths is required for logfile
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Change the input type in the CheckPoint module to
udp
fromsyslog
so the syslog parsing happens in the ingest node pipeline rather than
in the Filebeat syslog input that only support RFC 3164.
Why is it important?
The module was causing warnings while parsing data.
Checklist
CHANGELOG.next.asciidoc
orCHANGELOG-developer.next.asciidoc
.How to test this PR locally
echo -n '<134>1 2020-03-29T13:19:21Z gw-da58d3 CheckPoint 1930 - [flags:"133440"; ifdir:"inbound"; ifname:"daemon"; loguid:"{0x5e80a059,0x2,0x6401a8c0,0x3c7878a}"; origin:"192.168.1.100"; sequencenum:"2"; version:"5"; product:"System Monitor"; sys_message::"installed Standard"]' | nc -4u -w1 localhost 9001
Related issues
Logs
I manually tested the syslog parsing using the steps above and this is the event that landed in ES.