Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error walking target request timeout #713

Closed
subuk opened this issue Dec 12, 2021 · 2 comments
Closed

Error walking target request timeout #713

subuk opened this issue Dec 12, 2021 · 2 comments

Comments

@subuk
Copy link
Contributor

subuk commented Dec 12, 2021

Host operating system: output of uname -a

Linux infra 4.18.0-305.12.1.el8_4.x86_64 #1 SMP Wed Aug 11 01:59:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

snmp_exporter version: output of snmp_exporter -version

snmp_exporter-0.20.0-4.el8.x86_64 from https://packagecloud.io/prometheus-rpm/release/el/8/

Device

Keenetic Giga (KN-1011)

Configs

generator.yml

modules:
  keenetic:
    max_repetitions: 50
    auth:
      community: public
    walk:
      - ifXTable

The problem

Timeout while scrape:

$ curl 'http://localhost:9116/snmp?module=keenetic&target=192.168.84.1'
An error has occurred while serving metrics:

error collecting metric Desc{fqName: "snmp_error", help: "Error scraping target", constLabels: {}, variableLabels: []}: error walking target 192.168.84.1: request timeout (after 3 retries)
snmp_exporter[153939]: level=info ts=2021-12-12T12:45:16.552Z caller=collector.go:224 module=keenetic target=192.168.84.1 msg="Error scraping target" err="error walking target 192.168.84.1: request timeout (after 3 retries)"

Despite snmpbulkwalk works:

$ snmpbulkwalk -Cr50 -v2c -c public 192.168.84.1 IF-MIB::ifXTable | head -10
IF-MIB::ifName.1 = STRING: Loopback
IF-MIB::ifName.2 = STRING: GigabitEthernet0
IF-MIB::ifName.3 = STRING: 1
IF-MIB::ifName.4 = STRING: 2
IF-MIB::ifName.5 = STRING: 3
IF-MIB::ifName.6 = STRING: 4
IF-MIB::ifName.7 = STRING: GigabitEthernet0/Vlan1
IF-MIB::ifName.8 = STRING: GigabitEthernet0/Vlan3
IF-MIB::ifName.9 = STRING: ISP
IF-MIB::ifName.10 = STRING: 0

I captured traffic using tcpdump for snmpbulkwalk and snmp_exporter, and looks like device successfully responds to GetBulk requests, but snmp_exporter does not parse it and tries the same query until timeout.
snmp_exporter_pcaps.zip

@subuk
Copy link
Contributor Author

subuk commented Dec 12, 2021

I looked through pcaps one more time and noticed that router respond from another ip address. Net-SNMP handles this situation correctly, but snmp_exporter does not, so I think I should close this issue and open another one with vendor tech support.

@subuk subuk closed this as completed Dec 12, 2021
@SuperQ
Copy link
Member

SuperQ commented Dec 13, 2021

Yes, getting things to respond on the correct interface/IP is best. But we're looking to support workarounds for broken devices.

#699

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants