[filebeat] add netflow input metrics #38055

pkoutsovasilis · 2024-02-19T11:33:47Z

Proposed commit message

This PR captures introduces input/netmetrics and extends the netflow input metrics and exposes them per input instance basis

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~I have made corresponding changes to the documentation~~
~~I have made corresponding change to the default configuration files~~
~~I have added tests that prove my fix is effective or that my feature works~~
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

How to test this PR locally

Related issues

Closes [Filebeat] Netflow input - add input metrics #37762

Use cases

Screenshots

Logs

mergify · 2024-02-19T11:34:24Z

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @pkoutsovasilis? 🙏.
For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

filebeat/input/netmetrics/common.go

filebeat/input/netmetrics/procnet.go

filebeat/input/netmetrics/procnet_test.go

filebeat/input/netmetrics/tcp.go

filebeat/input/netmetrics/udp.go

filebeat/input/netmetrics/tcp.go

filebeat/input/netmetrics/udp.go

filebeat/input/tcp/input.go

… same tcp and/or udp metrics

…s_total, decode_errors_total

botelastic · 2024-02-20T11:46:27Z

This pull request doesn't have a Team:<team> label.

elasticmachine · 2024-02-20T13:03:55Z

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)

elasticmachine · 2024-02-20T14:00:13Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2024-02-22T14:59:55.857+0000
Duration: 138 min 50 sec

Test stats 🧪

Test	Results
Failed	0
Passed	8386
Skipped	759
Total	9145

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

filebeat/input/netmetrics/tcp.go

filebeat/input/tcp/input.go

filebeat/input/netmetrics/tcp.go

filebeat/input/netmetrics/udp.go

filebeat/input/netmetrics/tcp.go

andrewkroh

The asciidocs for the netflow input need a new "Metrics" section added. Here's a template you may follow eca6d05

…promote idiomatic go

pkoutsovasilis · 2024-02-22T11:06:38Z

The asciidocs for the netflow input need a new "Metrics" section added. Here's a template you may follow eca6d05

Oh nice catch ty @andrewkroh

x-pack/filebeat/docs/inputs/input-netflow.asciidoc

pkoutsovasilis · 2024-02-22T15:01:33Z

@andrewkroh another question, what do you think should this PR be back-ported? I added backport-skip but I am not entirely sure that was the right choice

elasticmachine · 2024-02-22T15:02:16Z

💚 Build Succeeded

Buildkite Build
Commit: a72aab3

History

💚 Build #1566 succeeded 213c2b1
💚 Build #1521 succeeded 14ff7ea
💚 Build #1495 succeeded 47aea5b
💚 Build #1491 succeeded 6cf69cb
💚 Build #1486 succeeded 5068d10
💚 Build #1479 succeeded 018ad5c

cc @pkoutsovasilis

elasticmachine · 2024-02-22T15:02:20Z

💚 Build Succeeded

Buildkite Build
Commit: a72aab3

History

💚 Build #2781 succeeded 213c2b1
💚 Build #2735 succeeded 14ff7ea
💚 Build #2709 succeeded 47aea5b

cc @pkoutsovasilis

elasticmachine · 2024-02-22T15:02:24Z

💚 Build Succeeded

Buildkite Build
Commit: a72aab3

History

💚 Build #1574 succeeded 213c2b1
💚 Build #1529 succeeded 14ff7ea
💚 Build #1503 succeeded 47aea5b
💚 Build #1499 succeeded 6cf69cb
💚 Build #1494 succeeded 5068d10
💚 Build #1486 succeeded 018ad5c

cc @pkoutsovasilis

elasticmachine · 2024-02-22T15:02:41Z

💚 Build Succeeded

Buildkite Build
Commit: a72aab3

History

💚 Build #730 succeeded 213c2b1
💚 Build #685 succeeded 14ff7ea
💚 Build #659 succeeded 47aea5b

cc @pkoutsovasilis

elasticmachine · 2024-02-22T15:20:44Z

💚 Build Succeeded

Buildkite Build
Commit: a72aab3

History

💚 Build #1863 succeeded 213c2b1
💚 Build #1821 succeeded 14ff7ea
💚 Build #1798 succeeded 47aea5b

cc @pkoutsovasilis

elasticmachine · 2024-02-22T16:00:09Z

💚 Build Succeeded

Buildkite Build
Commit: a72aab3

History

💚 Build #3202 succeeded 213c2b1
💔 Build #3160 failed 14ff7ea
💚 Build #3137 succeeded 47aea5b

cc @pkoutsovasilis

andrewkroh · 2024-02-22T16:00:43Z

another question, what do you think should this PR be back-ported? I added backport-skip but I am not entirely sure that was the right choice

That's the right choice IMO. This will land in 8.14. It's past the feature freeze for 8.13 so we shouldn't backport unless it's a bugfix or some low-risk, critical feature (like for helping to debug an issue).

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 19, 2024

mergify bot assigned pkoutsovasilis Feb 19, 2024

pkoutsovasilis force-pushed the pkoutsovasilis/netflow_metrics branch from cc6073c to fa059a3 Compare February 19, 2024 11:37

pkoutsovasilis changed the title ~~feat: refactor tcp and udp metrics to allow multiple inputs to incorp…~~ [filebeat] add netflow input metrics Feb 19, 2024

pkoutsovasilis force-pushed the pkoutsovasilis/netflow_metrics branch 3 times, most recently from 018ad5c to 5068d10 Compare February 19, 2024 21:33

efd6 reviewed Feb 19, 2024

View reviewed changes

feat: introduce input/netmetrics to allow multiple inputs utilise the…

de72a9c

… same tcp and/or udp metrics

pkoutsovasilis force-pushed the pkoutsovasilis/netflow_metrics branch from 5068d10 to 6cf69cb Compare February 20, 2024 09:14

pkoutsovasilis added 9 commits February 20, 2024 13:32

feat: add netflow input specific metrics discarded_events_total, flow…

1616fc6

…s_total, decode_errors_total

feat: add netflow input specific metric open_connections

fc0b5d1

fix: omit the name of unused receivers

fa87cb9

fix: remove deprecated ioutil with io

27c70a5

fix: remove redundant type conversion

c991a63

fix: address naked return

dbf6ab9

fix: proper equality check in TestSessionState

54626b1

fix: ignore explicitly the error from registering v9 protocol

f9897e2

doc: update changelog

47aea5b

pkoutsovasilis force-pushed the pkoutsovasilis/netflow_metrics branch from 6cf69cb to 47aea5b Compare February 20, 2024 11:42

elastic deleted a comment from elasticmachine Feb 20, 2024

pkoutsovasilis added the Team:Security-Deployment and Devices Deployment and Devices Team in Security Solution label Feb 20, 2024

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 20, 2024

pkoutsovasilis added the enhancement label Feb 20, 2024

pkoutsovasilis marked this pull request as ready for review February 20, 2024 13:03

pkoutsovasilis requested review from a team as code owners February 20, 2024 13:03

pkoutsovasilis requested review from belimawr and fearful-symmetry February 20, 2024 13:03

efd6 reviewed Feb 20, 2024

View reviewed changes

filebeat/input/netmetrics/tcp.go Outdated Show resolved Hide resolved

filebeat/input/tcp/input.go Outdated Show resolved Hide resolved

filebeat/input/netmetrics/tcp.go Outdated Show resolved Hide resolved

filebeat/input/netmetrics/udp.go Outdated Show resolved Hide resolved

feat: refactor netmetrics to instantiate and expose monitoring registry

14ff7ea

belimawr approved these changes Feb 21, 2024

View reviewed changes

efd6 approved these changes Feb 21, 2024

View reviewed changes

filebeat/input/netmetrics/udp.go Outdated Show resolved Hide resolved

filebeat/input/netmetrics/tcp.go Outdated Show resolved Hide resolved

andrewkroh reviewed Feb 22, 2024

View reviewed changes

feat: rename GetRegistry to Registry in TCP and UDP of netmetrics to …

c36c819

…promote idiomatic go

doc: update input-netflow.asciidoc to capture the new metrics

213c2b1

pkoutsovasilis requested a review from andrewkroh February 22, 2024 11:34

andrewkroh reviewed Feb 22, 2024

View reviewed changes

x-pack/filebeat/docs/inputs/input-netflow.asciidoc Outdated Show resolved Hide resolved

andrewkroh approved these changes Feb 22, 2024

View reviewed changes

fix: move metrics last in input-netflow.asciidoc

a72aab3

pkoutsovasilis merged commit a0a567c into elastic:main Feb 22, 2024
45 of 48 checks passed

pkoutsovasilis deleted the pkoutsovasilis/netflow_metrics branch February 22, 2024 22:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[filebeat] add netflow input metrics #38055

[filebeat] add netflow input metrics #38055

pkoutsovasilis commented Feb 19, 2024 •

edited

Loading

mergify bot commented Feb 19, 2024

botelastic bot commented Feb 20, 2024

elasticmachine commented Feb 20, 2024

elasticmachine commented Feb 20, 2024 •

edited

Loading

Build stats

Test stats 🧪

andrewkroh left a comment

pkoutsovasilis commented Feb 22, 2024

pkoutsovasilis commented Feb 22, 2024

elasticmachine commented Feb 22, 2024

elasticmachine commented Feb 22, 2024

elasticmachine commented Feb 22, 2024

elasticmachine commented Feb 22, 2024

elasticmachine commented Feb 22, 2024

elasticmachine commented Feb 22, 2024

andrewkroh commented Feb 22, 2024 •

edited

Loading

[filebeat] add netflow input metrics #38055

[filebeat] add netflow input metrics #38055

Conversation

pkoutsovasilis commented Feb 19, 2024 • edited Loading

Proposed commit message

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

mergify bot commented Feb 19, 2024

botelastic bot commented Feb 20, 2024

elasticmachine commented Feb 20, 2024

elasticmachine commented Feb 20, 2024 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

andrewkroh left a comment

Choose a reason for hiding this comment

pkoutsovasilis commented Feb 22, 2024

pkoutsovasilis commented Feb 22, 2024

elasticmachine commented Feb 22, 2024

💚 Build Succeeded

History

elasticmachine commented Feb 22, 2024

💚 Build Succeeded

History

elasticmachine commented Feb 22, 2024

💚 Build Succeeded

History

elasticmachine commented Feb 22, 2024

💚 Build Succeeded

History

elasticmachine commented Feb 22, 2024

💚 Build Succeeded

History

elasticmachine commented Feb 22, 2024

💚 Build Succeeded

History

andrewkroh commented Feb 22, 2024 • edited Loading

pkoutsovasilis commented Feb 19, 2024 •

edited

Loading

elasticmachine commented Feb 20, 2024 •

edited

Loading

andrewkroh commented Feb 22, 2024 •

edited

Loading