[Auditbeat] fim(ebpf): enrich file events with process data #38199

mmat11 · 2024-03-06T15:08:40Z

Proposed commit message

fim(ebpf): enrich file events with process data

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

Make sure entity ID is calculated consistently everywhere // copied from https://github.com/elastic/beats/blob/main/x-pack/auditbeat/module/system/process/process.go#L139

Related issues

elastic/integrations#7401

Screenshot

mergify · 2024-03-06T15:09:50Z

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @mmat11? 🙏.
For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

elasticmachine · 2024-03-06T15:15:54Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Duration: 180 min 13 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-03-08T12:45:04Z

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

auditbeat/_meta/fields.common.yml

auditbeat/docs/fields.asciidoc

auditbeat/module/file_integrity/event.go

nicholasberlin

LGTM FWIW

elasticmachine · 2024-03-11T16:33:47Z

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

leehinman

Overall looks good.

Just a couple of questions. process.pid matching the ECS type is the biggest question.

auditbeat/_meta/fields.common.yml

auditbeat/module/file_integrity/event.go

auditbeat/module/file_integrity/event_linux.go

…user-data

pkoutsovasilis · 2024-04-05T09:34:00Z

run docs-build rebuild

alexsapran · 2024-04-05T09:36:38Z

run docs-build

pierrehilbert

Thx for your work and investigations!

dliappis · 2024-04-05T10:20:10Z

metricbeat/Dockerfile

@@ -1,4 +1,5 @@
 FROM golang:1.21.8
+COPY --from=docker:26.0.0-alpine3.19 /usr/local/bin/docker  /usr/local/bin/


Can we leave a comment (with a TODO perhaps that it will be addressed via #38678) ?

* fim(ebpf): enrich file events with process data * apply review suggestions * apply review suggestions * fix(fim/ebpf): move process fields to event root and insert them so keys do not contain dots * fix(fim/ebpf): refactor HostID to utilise sync.OnceValue and expose boot time * fix(fim/ebpf): refactor TicksPerSecond to utilise sync.OnceValue * fix(fim/ebpf): remove empty slice allocation * chore: go mod tidy * fix: explicitly set go 1.21.8 in go.mod * fix(fim/ebpf): nil slice of errors in TestNewEventFromEbpfEvent * fix(fim/ebpf): remove re-declaration of already ecs included fields * fix(fim/ebpf): utilise OnceValues to declutter the code * fix(fim/ebpf): remove x-pack import from OSS package * fix(fim/ebpf): propagate process fields changes to integration tests * chore: go mod tidy * ci: temporary solution to outdated docker compose python library * ci: transition to a fixed tag for docker image instead of a rolling one --------- Co-authored-by: Panos Koutsovasilis <[email protected]> Co-authored-by: Pierre HILBERT <[email protected]> (cherry picked from commit dbdaac3) # Conflicts: # go.mod # go.sum

…38743) * Manual port of docker CI fix from #38199 * Fix order in requirements.txt.

…38743) * Manual port of docker CI fix from #38199 * Fix order in requirements.txt. (cherry picked from commit 33b776a) # Conflicts: # libbeat/tests/system/requirements.txt # libbeat/tests/system/requirements_aix.txt # metricbeat/Dockerfile

…ocker-compose package (#38746) * [7.17] Fix Python systems tests with forked docker-compose package (#38743) * Manual port of docker CI fix from #38199 * Fix order in requirements.txt. (cherry picked from commit 33b776a) # Conflicts: # libbeat/tests/system/requirements.txt # libbeat/tests/system/requirements_aix.txt # metricbeat/Dockerfile * Resolve conflicts * Restore uintentionally removed packages * Remove duplicate package. * Add dropped docker copy --------- Co-authored-by: Craig MacKenzie <[email protected]>

…38199) * fim(ebpf): enrich file events with process data * apply review suggestions * apply review suggestions * fix(fim/ebpf): move process fields to event root and insert them so keys do not contain dots * fix(fim/ebpf): refactor HostID to utilise sync.OnceValue and expose boot time * fix(fim/ebpf): refactor TicksPerSecond to utilise sync.OnceValue * fix(fim/ebpf): remove empty slice allocation * chore: go mod tidy * fix: explicitly set go 1.21.8 in go.mod * fix(fim/ebpf): nil slice of errors in TestNewEventFromEbpfEvent * fix(fim/ebpf): remove re-declaration of already ecs included fields * fix(fim/ebpf): utilise OnceValues to declutter the code * fix(fim/ebpf): remove x-pack import from OSS package * fix(fim/ebpf): propagate process fields changes to integration tests * chore: go mod tidy * ci: temporary solution to outdated docker compose python library * ci: transition to a fixed tag for docker image instead of a rolling one --------- Co-authored-by: Panos Koutsovasilis <[email protected]> Co-authored-by: Pierre HILBERT <[email protected]>

…38743) * Manual port of docker CI fix from #38199 * Fix order in requirements.txt. (cherry picked from commit 33b776a) # Conflicts: # libbeat/tests/system/requirements.txt # libbeat/tests/system/requirements_aix.txt # metricbeat/Dockerfile

…h process data (#38742) * [Auditbeat] fim(ebpf): enrich file events with process data (#38199) * fim(ebpf): enrich file events with process data * apply review suggestions * apply review suggestions * fix(fim/ebpf): move process fields to event root and insert them so keys do not contain dots * fix(fim/ebpf): refactor HostID to utilise sync.OnceValue and expose boot time * fix(fim/ebpf): refactor TicksPerSecond to utilise sync.OnceValue * fix(fim/ebpf): remove empty slice allocation * chore: go mod tidy * fix: explicitly set go 1.21.8 in go.mod * fix(fim/ebpf): nil slice of errors in TestNewEventFromEbpfEvent * fix(fim/ebpf): remove re-declaration of already ecs included fields * fix(fim/ebpf): utilise OnceValues to declutter the code * fix(fim/ebpf): remove x-pack import from OSS package * fix(fim/ebpf): propagate process fields changes to integration tests * chore: go mod tidy * ci: temporary solution to outdated docker compose python library * ci: transition to a fixed tag for docker image instead of a rolling one --------- Co-authored-by: Panos Koutsovasilis <[email protected]> Co-authored-by: Pierre HILBERT <[email protected]> (cherry picked from commit dbdaac3) # Conflicts: # go.mod # go.sum * fix: resolve conflicts --------- Co-authored-by: Mattia Meleleo <[email protected]> Co-authored-by: Panos Koutsovasilis <[email protected]>

mmat11 requested review from a team as code owners March 6, 2024 15:08

mmat11 requested review from fearful-symmetry and leehinman March 6, 2024 15:08

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 6, 2024

mergify bot assigned mmat11 Mar 6, 2024

mmat11 force-pushed the matt/fim-user-data branch from 83d9ec6 to 36c6ecd Compare March 6, 2024 15:12

mmat11 force-pushed the matt/fim-user-data branch 8 times, most recently from ae5dd9b to f11bcbd Compare March 7, 2024 18:39

mmat11 added the Team:Security-Linux Platform Linux Platform Team in Security Solution label Mar 8, 2024

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Mar 8, 2024

mmat11 requested a review from andrewkroh March 8, 2024 12:45

nicholasberlin reviewed Mar 11, 2024

View reviewed changes

auditbeat/_meta/fields.common.yml Outdated Show resolved Hide resolved

auditbeat/docs/fields.asciidoc Outdated Show resolved Hide resolved

auditbeat/module/file_integrity/event.go Show resolved Hide resolved

auditbeat/module/file_integrity/event.go Show resolved Hide resolved

mmat11 force-pushed the matt/fim-user-data branch from afe8971 to 7ada154 Compare March 11, 2024 15:01

nicholasberlin approved these changes Mar 11, 2024

View reviewed changes

pierrehilbert added the Team:Elastic-Agent Label for the Agent team label Mar 11, 2024

leehinman requested changes Mar 11, 2024

View reviewed changes

auditbeat/_meta/fields.common.yml Outdated Show resolved Hide resolved

auditbeat/module/file_integrity/event.go Outdated Show resolved Hide resolved

auditbeat/module/file_integrity/event_linux.go Show resolved Hide resolved

mmat11 force-pushed the matt/fim-user-data branch 2 times, most recently from c98f693 to 9bc78d4 Compare March 12, 2024 13:35

pkoutsovasilis force-pushed the matt/fim-user-data branch from eae0bda to 5a87cee Compare April 3, 2024 21:17

pkoutsovasilis and others added 2 commits April 4, 2024 00:17

chore: go mod tidy

4bd33cf

Merge branch 'main' into matt/fim-user-data

2941987

pkoutsovasilis force-pushed the matt/fim-user-data branch 2 times, most recently from 58cd307 to bd4cf98 Compare April 4, 2024 18:47

ci: temporary solution to outdated docker compose python library

e37a6d3

pkoutsovasilis force-pushed the matt/fim-user-data branch from bd4cf98 to e37a6d3 Compare April 4, 2024 18:53

pkoutsovasilis added 2 commits April 5, 2024 02:58

ci: transition to a fixed tag for docker image instead of a rolling one

1ca5f8b

Merge remote-tracking branch 'refs/remotes/beats/main' into matt/fim-…

4c385e3

…user-data

pierrehilbert approved these changes Apr 5, 2024

View reviewed changes

pkoutsovasilis merged commit dbdaac3 into main Apr 5, 2024
125 checks passed

pkoutsovasilis deleted the matt/fim-user-data branch April 5, 2024 10:19

dliappis reviewed Apr 5, 2024

View reviewed changes

faec added the backport-v8.13.0 Automated backport with mergify label Apr 5, 2024

mergify bot mentioned this pull request Apr 5, 2024

[8.13](backport #38199) [Auditbeat] fim(ebpf): enrich file events with process data #38742

Merged

7 tasks

cmacknz added a commit to cmacknz/beats that referenced this pull request Apr 5, 2024

Manual port of docker CI fix from elastic#38199

7ab0e0a

cmacknz mentioned this pull request Apr 5, 2024

[7.17] Fix Python systems tests with forked docker-compose package #38743

Merged

cmacknz added a commit that referenced this pull request Apr 5, 2024

[7.17] Fix Python systems tests with forked docker-compose package (#…

33b776a

…38743) * Manual port of docker CI fix from #38199 * Fix order in requirements.txt.

mergify bot mentioned this pull request Apr 5, 2024

[8.13](backport #38743) [7.17] Fix Python systems tests with forked docker-compose package #38746

Merged

pkoutsovasilis mentioned this pull request Apr 8, 2024

File Integrity Monitoring | User Information - Linux | KProbe Backend elastic/integrations#8692

Closed

This was referenced Apr 8, 2024

[8.12](backport #38743) [7.17] Fix Python systems tests with forked docker-compose package #38766

Closed

[8.13](backport #38328) [Auditbeat] fim(ebpf): enrich file events with container id #38775

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Auditbeat] fim(ebpf): enrich file events with process data #38199

[Auditbeat] fim(ebpf): enrich file events with process data #38199

mmat11 commented Mar 6, 2024 •

edited by pkoutsovasilis

Loading

mergify bot commented Mar 6, 2024

elasticmachine commented Mar 6, 2024 •

edited

Loading

Build stats

elasticmachine commented Mar 8, 2024

nicholasberlin left a comment

elasticmachine commented Mar 11, 2024

leehinman left a comment

pkoutsovasilis commented Apr 5, 2024

alexsapran commented Apr 5, 2024

pierrehilbert left a comment

dliappis Apr 5, 2024

		@@ -1,4 +1,5 @@
		FROM golang:1.21.8
		COPY --from=docker:26.0.0-alpine3.19 /usr/local/bin/docker /usr/local/bin/

[Auditbeat] fim(ebpf): enrich file events with process data #38199

[Auditbeat] fim(ebpf): enrich file events with process data #38199

Conversation

mmat11 commented Mar 6, 2024 • edited by pkoutsovasilis Loading

Proposed commit message

Checklist

Author's Checklist

Related issues

Screenshot

mergify bot commented Mar 6, 2024

elasticmachine commented Mar 6, 2024 • edited Loading

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

elasticmachine commented Mar 8, 2024

nicholasberlin left a comment

Choose a reason for hiding this comment

elasticmachine commented Mar 11, 2024

leehinman left a comment

Choose a reason for hiding this comment

pkoutsovasilis commented Apr 5, 2024

alexsapran commented Apr 5, 2024

pierrehilbert left a comment

Choose a reason for hiding this comment

dliappis Apr 5, 2024

Choose a reason for hiding this comment

mmat11 commented Mar 6, 2024 •

edited by pkoutsovasilis

Loading

elasticmachine commented Mar 6, 2024 •

edited

Loading