Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for parsing SNMP transport from target #914

Merged
merged 5 commits into from
Jul 15, 2023
Merged

Add support for parsing SNMP transport from target #914

merged 5 commits into from
Jul 15, 2023

Conversation

hhromic
Copy link
Contributor

@hhromic hhromic commented Jul 12, 2023

This PR implements support for parsing the SNMP transport to use from the target query parameter.

  • Accepted syntax for target: [transport://]host[:port]
  • New dedicated function: configures a gosnmp.GoSNMP instance from a parsed target string.
  • If transport:// is not provided, Transport is left unconfigured and GoSNMP defaults to udp.
  • Setting Target and Port in GoSNMP is left as it was in the original implementation.
  • There is no transport validation during parsing, this is delegated to GoSNMP during connect.
  • Added unit test and cases for the new dedicated parsing function.
  • Added better documentation for the target parameter in README that includes transport support.

I tested connectivity as well. The snmp exporter can now connect via TCP to an SNMP server.

Closes #555

hhromic added 3 commits July 12, 2023 22:31
* Syntax: `[transport://]host[:port]`

Signed-off-by: Hugo Hromic <[email protected]>
Signed-off-by: Hugo Hromic <[email protected]>
Copy link
Member

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, thanks.

@SuperQ SuperQ requested a review from RichiH July 13, 2023 05:25
@SuperQ
Copy link
Member

SuperQ commented Jul 13, 2023

@RichiH Any comments on the syntax here?

Copy link
Member

@RichiH RichiH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remain torn on "URL via SD" vs "config", but have come to accept that URL is probably Least Bad overall.

Feature itself is very nice, even though I have never needed it myself.

I suspect some tests are missing, see comment. Other than that, LGTM.

collector/collector_test.go Show resolved Hide resolved
@hhromic
Copy link
Contributor Author

hhromic commented Jul 13, 2023

I remain torn on "URL via SD" vs "config", but have come to accept that URL is probably Least Bad overall.

As a further argument, take as an example the approach from the blackbox exporter:

scrape_configs:
  - job_name: 'blackbox'
    metrics_path: /probe
    params:
      module: [http_2xx]  # Look for a HTTP 200 response.
    static_configs:
      - targets:
        - http://prometheus.io    # Target to probe with http.
        - https://prometheus.io   # Target to probe with https.
        - http://example.com:8080 # Target to probe with http on port 8080.
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 127.0.0.1:9115  # The blackbox exporter's real hostname:port.

As you can see, the targets contain the scheme (http:// or https://) in the target parameter as well for the http probe.

@RichiH
Copy link
Member

RichiH commented Jul 13, 2023

Yes, thinking about blackbox_exporter yesterday night is what made me flip over to URL scheme. I had a whole thing written on why config is slightly preferable, but... I was wrong and didn't send it :)

@hhromic
Copy link
Contributor Author

hhromic commented Jul 13, 2023

I now added an example SNMP device via TCP for Prometheus to the README file as well.

@SuperQ
Copy link
Member

SuperQ commented Jul 15, 2023

I still think config is slightly preferable. But I also don't object to this format.

@SuperQ SuperQ merged commit 03bc586 into prometheus:main Jul 15, 2023
@hhromic
Copy link
Contributor Author

hhromic commented Jul 15, 2023

I still think config is slightly preferable. But I also don't object to this format.

I still don't see how that would work :). The transport is a simple property of each target, like the port number to use, independent of the auth or module used. In the same way insecure or secure HTTP (the scheme) is a property of an URL.

In any case, thanks for merging this contribution!
I'm just curious how you envision it as a config and still being practical.

@hhromic hhromic deleted the add-tgt-transport branch July 15, 2023 14:31
@SuperQ
Copy link
Member

SuperQ commented Jul 15, 2023

It would simply be a property of either the auth or module, like the SNMP version or the walk params (max_repititions, etc). Which is also part of the "Transport" IMO.

The thing is, the module is meant to represent a device or device class. So having some devices with UDP and some with TCP represent different modules.

@hhromic
Copy link
Contributor Author

hhromic commented Jul 15, 2023

It would simply be a property of either the auth or module, like the SNMP version or the walk params (max_repititions, etc). Which is also part of the "Transport" IMO.

Ah I see, yes seems closer to module via WalkParams:

type WalkParams struct {
MaxRepetitions uint32 `yaml:"max_repetitions,omitempty"`
Retries *int `yaml:"retries,omitempty"`
Timeout time.Duration `yaml:"timeout,omitempty"`
UseUnconnectedUDPSocket bool `yaml:"use_unconnected_udp_socket,omitempty"`
AllowNonIncreasingOIDs bool `yaml:"allow_nonincreasing_oids,omitempty"`
}

The thing is, the module is meant to represent a device or device class. So having some devices with UDP and some with TCP represent different modules.

I always have seen module as a representation of a device class indeed but with the target query parameter representing a concrete device of a certain class (module query parameter) to be scraped using certain parameters (e.g. transport). In my humble opinion, that a device uses UDP or TCP, or both, should not mean different modules. The same device can even be reconfigured to change its transport at different times, never changing its class.

The same could be argued about timeouts and retries even. In my humble opinion, most of these walk params are more a specific target-related set of properties than the class of the target (module). Especially when changing those. For example, if we want a different transport or timeout for the same device, we would have to modify the huge snmp.yaml file. In the case of Docker, even maybe regenerate the Docker image.

SuperQ added a commit that referenced this pull request Jul 19, 2023
BREAKING CHANGES:

This version of the exporter introduces a new configuration file format. This
new format separates the walk and metric mappings from the connection and
authentication settings. This allows for easier configuration of different
auth params without having to duplicate the full walk and metric mapping.

See auth-split-migration.md for more details.

* [CHANGE] Split config of auth and modules #859
* [FEATURE] Improved Lookup process for label information #908
* [ENHANCEMENT] Add support for parsing SNMP transport from target #914
* [BUGFIX] Fix metrics path not using command-line argument value #904

Signed-off-by: SuperQ <[email protected]>
@SuperQ SuperQ mentioned this pull request Jul 19, 2023
SuperQ added a commit that referenced this pull request Jul 19, 2023
BREAKING CHANGES:

This version of the exporter introduces a new configuration file format. This
new format separates the walk and metric mappings from the connection and
authentication settings. This allows for easier configuration of different
auth params without having to duplicate the full walk and metric mapping.

See auth-split-migration.md for more details.

* [CHANGE] Split config of auth and modules #859
* [FEATURE] Improved Lookup process for label information #908
* [ENHANCEMENT] Add support for parsing SNMP transport from target #914
* [BUGFIX] Fix metrics path not using command-line argument value #904

Signed-off-by: SuperQ <[email protected]>
SuperQ added a commit that referenced this pull request Jul 19, 2023
BREAKING CHANGES:

This version of the exporter introduces a new configuration file format. This
new format separates the walk and metric mappings from the connection and
authentication settings. This allows for easier configuration of different
auth params without having to duplicate the full walk and metric mapping.

See auth-split-migration.md for more details.

* [CHANGE] Split config of auth and modules #859
* [FEATURE] Add support for parsing SNMP transport from target #914
* [ENHANCEMENT] Improved Lookup process for label information #908
* [BUGFIX] Fix metrics path not using command-line argument value #904

Signed-off-by: SuperQ <[email protected]>
SuperQ added a commit that referenced this pull request Jul 20, 2023
BREAKING CHANGES:

This version of the exporter introduces a new configuration file format. This
new format separates the walk and metric mappings from the connection and
authentication settings. This allows for easier configuration of different
auth params without having to duplicate the full walk and metric mapping.

See auth-split-migration.md for more details.

* [CHANGE] Split config of auth and modules #859
* [FEATURE] Add support for parsing SNMP transport from target #914
* [ENHANCEMENT] Improved Lookup process for label information #908
* [BUGFIX] Fix metrics path not using command-line argument value #904

Signed-off-by: SuperQ <[email protected]>
SuperQ added a commit that referenced this pull request Jul 20, 2023
BREAKING CHANGES:

This version of the exporter introduces a new configuration file format. This
new format separates the walk and metric mappings from the connection and
authentication settings. This allows for easier configuration of different
auth params without having to duplicate the full walk and metric mapping.

See auth-split-migration.md for more details.

* [CHANGE] Split config of auth and modules #859
* [FEATURE] Add support for parsing SNMP transport from target #914
* [ENHANCEMENT] Improved Lookup process for label information #908
* [BUGFIX] Fix metrics path not using command-line argument value #904

Signed-off-by: SuperQ <[email protected]>
SuperQ added a commit that referenced this pull request Aug 9, 2023
BREAKING CHANGES:

This version of the exporter introduces a new configuration file format. This
new format separates the walk and metric mappings from the connection and
authentication settings. This allows for easier configuration of different
auth params without having to duplicate the full walk and metric mapping.

See auth-split-migration.md for more details.

* [CHANGE] Split config of auth and modules #859
* [FEATURE] Add support for parsing SNMP transport from target #914
* [ENHANCEMENT] Improved Lookup process for label information #908
* [BUGFIX] Fix metrics path not using command-line argument value #904

Signed-off-by: SuperQ <[email protected]>
@SuperQ SuperQ mentioned this pull request Aug 9, 2023
SuperQ added a commit that referenced this pull request Aug 16, 2023
* Release v0.23.0

BREAKING CHANGES:

This version of the exporter introduces a new configuration file format. This
new format separates the walk and metric mappings from the connection and
authentication settings. This allows for easier configuration of different
auth params without having to duplicate the full walk and metric mapping.

See auth-split-migration.md for more details.

* [CHANGE] Split config of auth and modules #859
* [FEATURE] Add support for parsing SNMP transport from target #914
* [ENHANCEMENT] Improved Lookup process for label information #908
* [BUGFIX] Fix metrics path not using command-line argument value #904

Signed-off-by: SuperQ <[email protected]>

* Lint dashboards

Update dashboards to latest mixin linting.

Signed-off-by: SuperQ <[email protected]>

---------

Signed-off-by: SuperQ <[email protected]>
stephan-windischmann-sky pushed a commit to stephan-windischmann-sky/snmp_exporter that referenced this pull request Oct 27, 2023
* Add support for parsing SNMP transport from target
* Syntax: `[transport://]host[:port]`

---------

Signed-off-by: Hugo Hromic <[email protected]>
Signed-off-by: Stephan Windischmann <[email protected]>
stephan-windischmann-sky pushed a commit to stephan-windischmann-sky/snmp_exporter that referenced this pull request Oct 27, 2023
BREAKING CHANGES:

This version of the exporter introduces a new configuration file format. This
new format separates the walk and metric mappings from the connection and
authentication settings. This allows for easier configuration of different
auth params without having to duplicate the full walk and metric mapping.

See auth-split-migration.md for more details.

* [CHANGE] Split config of auth and modules prometheus#859
* [FEATURE] Add support for parsing SNMP transport from target prometheus#914
* [ENHANCEMENT] Improved Lookup process for label information prometheus#908
* [BUGFIX] Fix metrics path not using command-line argument value prometheus#904

Signed-off-by: SuperQ <[email protected]>
Signed-off-by: Stephan Windischmann <[email protected]>
stephan-windischmann-sky pushed a commit to stephan-windischmann-sky/snmp_exporter that referenced this pull request Oct 27, 2023
* Release v0.23.0

BREAKING CHANGES:

This version of the exporter introduces a new configuration file format. This
new format separates the walk and metric mappings from the connection and
authentication settings. This allows for easier configuration of different
auth params without having to duplicate the full walk and metric mapping.

See auth-split-migration.md for more details.

* [CHANGE] Split config of auth and modules prometheus#859
* [FEATURE] Add support for parsing SNMP transport from target prometheus#914
* [ENHANCEMENT] Improved Lookup process for label information prometheus#908
* [BUGFIX] Fix metrics path not using command-line argument value prometheus#904

Signed-off-by: SuperQ <[email protected]>

* Lint dashboards

Update dashboards to latest mixin linting.

Signed-off-by: SuperQ <[email protected]>

---------

Signed-off-by: SuperQ <[email protected]>
Signed-off-by: Stephan Windischmann <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support RFC 3430
3 participants