Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Greatly trim what netstat collector exposes by default #876

Merged
merged 1 commit into from
Mar 30, 2018

Conversation

brian-brazil
Copy link
Contributor

Netstat is 40% of the metrics on my laptop, many of which
are highly detailed information about IP internals in the kernel.
~300 such metrics on every machine in your fleet is excessive,
so focus on key metrics by default, overridable by the user.

Fixes #515

@SuperQ I think we should aim for about 20-30 metrics from this collector, I've selected what I think are the main ones but there's undoubtedly more. Unfortunately these metrics have basically no documentation, so I'm working off experience.

@discordianfish
Copy link
Member

Yeah it's annoying that these are not well documented, guess we won't get around tweaking this once we realize something is needed that got disabled. So LGTM for now.

@SuperQ
Copy link
Member

SuperQ commented Mar 30, 2018

How about keeping IP/IP6 In/Out Octets?

@brian-brazil
Copy link
Contributor Author

My thought was that we already have that from netdev.

@SuperQ
Copy link
Member

SuperQ commented Mar 30, 2018

But we don't get IP vs IP6 from netdev.

@SuperQ
Copy link
Member

SuperQ commented Mar 30, 2018

Let's also keep Udp Udp6 In/Out Datagrams.

@brian-brazil brian-brazil force-pushed the netstat-trim branch 2 times, most recently from f5aa239 to 6f315e0 Compare March 30, 2018 16:51
@SuperQ
Copy link
Member

SuperQ commented Mar 30, 2018

There a few overall error counters that might be useful:

node_netstat_Icmp6_InErrors
node_netstat_Icmp_InErrors
node_netstat_Tcp_InErrs
node_netstat_Udp6_InErrors
node_netstat_UdpLite6_InErrors
node_netstat_UdpLite_InErrors
node_netstat_Udp_InErrors

@SuperQ
Copy link
Member

SuperQ commented Mar 30, 2018

Udp/Udp6 NoPorts is a counter of packets that don't hit a UDP listener. A good indicator of spammy / scanning traffic. Possibly worth including.

Netstat is 40% of the metrics on my laptop, many of which
are highly detailed information about IP internals in the kernel.
~300 such metrics on every machine in your fleet is excessive,
so focus on key metrics by default, overridable by the user.

Fixes #515

Signed-off-by: Brian Brazil <[email protected]>
@brian-brazil
Copy link
Contributor Author

We're at 31 now, so that's probably enough.

Copy link
Member

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@brian-brazil brian-brazil merged commit 31ce32f into master Mar 30, 2018
@brian-brazil brian-brazil deleted the netstat-trim branch March 30, 2018 18:28
@swade1987
Copy link

Does this mean metrics such as the following no longer exist in any capacity ...

node_netstat_TcpExt_ListenOverflows
node_netstat_TcpExt_TCPTimeouts
node_netstat_Tcp_AttemptFails
node_netstat_Tcp_OutSegs

@hoffie
Copy link
Contributor

hoffie commented Jun 29, 2018

Does this mean metrics such as the following no longer exist in any capacity ...

I don't think so. As Brian noted ("overridable by the user"), this can still be configured:

# The following command line is from --help with your cited metrics added to the whitelist:
$ ./node_exporter --collector.netstat.fields="^(.*_(InErrors|InErrs)|Ip_Forwarding|Ip(6|Ext)_(InOctets|OutOctets)|Icmp6?_(InMsgs|OutMsgs)|TcpExt_(Listen.*|Syncookies.*|TCPTimeouts)|Tcp_(ActiveOpens|PassiveOpens|RetransSegs|CurrEstab|AttemptFails|OutSegs)|Udp6?_(InDatagrams|OutDatagrams|NoPorts))$" &
$ curl -s localhost:9100/metrics | grep -P '^[^#]+(ListenOverflows|TCPTimeouts|AttemptFails|OutSegs)'
node_netstat_TcpExt_ListenOverflows 0
node_netstat_TcpExt_TCPTimeouts 2
node_netstat_Tcp_AttemptFails 556
node_netstat_Tcp_OutSegs 302001

This PR is just about changing the defaults (node_netstat_TcpExt_ListenOverflows still seems to be part of the defaults, btw).

oblitorum pushed a commit to shatteredsilicon/node_exporter that referenced this pull request Apr 9, 2024
Netstat is 40% of the metrics on my laptop, many of which
are highly detailed information about IP internals in the kernel.
~300 such metrics on every machine in your fleet is excessive,
so focus on key metrics by default, overridable by the user.

Fixes prometheus#515

Signed-off-by: Brian Brazil <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants