Split config of auth and modules #859

SuperQ · 2023-03-26T12:53:32Z

Allow configuration of auth/version parameters separately from the walk and metrics in the generator and exporter configuration.

Simplify startup with ReloadConfig()
Make sure to init metrics on config reload.

Fixes: #619

SuperQ · 2023-04-04T17:32:26Z

CC @sebastien-coavoux

sebastien-coavoux · 2023-04-05T01:28:25Z

Hey thanks for the tag, the only thing that might not be straighforward would be what to do with the auth.
This is supposed to be a get parameter received by l'exporter from prometheus from the code

            <label>Target:</label> <input type="text" name="target" placeholder="X.X.X.X" value="1.2.3.4"><br>
            <label>Auth:</label> <input type="text" name="auth" placeholder="auth" value="public"><br>
            <label>Module:</label> <input type="text" name="module" placeholder="module" value="if_mib"><br>

But that might not be clear when reading the parameter in the Readme

SuperQ · 2023-04-05T07:03:22Z

Also CC @dswarbrick for review.

SuperQ · 2023-04-05T08:21:01Z

@sebastien-coavoux Thanks, yea, I forgot to update the top-level README. I've updated that with some more information on how to use the the split auth/module.

I've also renamed the default auth to public_v2 to make it more clear that it's not directly a community string, but rather a configuration identifier. Kinda like how we changed default walk module to if_mib.

README.md

main.go

dswarbrick

LGTM

dswarbrick · 2023-04-15T03:20:43Z

It seems that this would also address what #601 attempted to solve.

marctc

LGTM! I've been working on adapting the changes to Grafana Agent's SNMP integration and overall I didn't find any problems. It will introduce breaking changes but for sake of simplicity. If you curious how is looking, here's WIP branch (cc @RichiH).

matthiasr

What would a typical rollout of this version look like? It seems like it requires both changes to the configuration, and to the scrape parameters, at the same time?

How could we ease this transition? For example, could the exporter understand the legacy configuration for a while? Or could we still support modules referencing a certain auth, so that the scrape URL doesn't have to change atomically?

README.md

ZoldanYao · 2023-06-21T03:30:23Z

This function is very very useful. I hope I can use this function asap.

SuperQ · 2023-06-22T13:15:10Z

Any last thoughts? I plan to merge this soon.

RichiH · 2023-06-22T13:47:13Z

config/config.go

@@ -60,17 +63,21 @@ var (
 	DefaultRegexpExtract = RegexpExtract{
 		Value: "$1",
 	}
+
+	ErrUnsupportedVersion = errors.New("unsupported config version")


Suggested change

ErrUnsupportedVersion = errors.New("unsupported config version")

ErrUnsupportedVersion = errors.New("Unsupported config version, we only support config version 2. Please see https://github.com/prometheus/snmp_exporter/blob/main/README.md#debugging")

This is a Go error, it shouldn't be this verbose.

Will the user see it on CLI?

Yes, but basically no. That's the whole reason for the "Possible old config file" error. The user won't directly see this error. It's for internal errors.Is() use.

RichiH · 2023-06-22T13:48:54Z

There's no in-review way to add the

#Debugging

## Unsupported config version

There was a change from SNMP exporter 0.22 to 0.23. The `auth` block is now split
out from the scrape config. You will need to change both the configuration of the
SNMP exporter and Prometheus' scrape config. For details, please see our
[config upgrading doc](https://github.com/prometheus/snmp_exporter/blob/main/auth-split-migration.md).

to the end of the README.

SuperQ · 2023-06-22T15:42:43Z

@RichiH That change is not needed as we already give a URL on error.

RichiH · 2023-06-23T11:54:20Z

Unless there's an active reason not to, I would prefer to leave more rather than fewer breadcrumbs for users hitting a wall.

SuperQ · 2023-06-23T12:05:18Z

There already is "Possible old config file". As I explained previously, due to the current strict yaml parsing, we can't do much about this. We would have to rewrite the yaml strict parser with our own validator. Frankly, I don't have the time or energy to do this.

candlerb · 2023-06-29T13:21:31Z

auth-split-migration.md

@@ -0,0 +1,59 @@
+# Module and Auth Split Migration
+
+In [version 0.23.0](https://github.com/prometheus/snmp_exporter/releases/tag/v0.23.0) the configuration for the `snmp_epxorter` the configuration file format has changed. Configuration files for versions v0.22.0 and before will not work. The configuration was split from a flat list of modules to separate metric walking/mapping modules and authentication configuratoins.


s/configuratoins/configurations/

BREAKING CHANGES: This version of the exporter introduces a new configuration file format. This new format separates the walk and metric mappings from the connection and authentication settings. This allows for easier configuration of different auth params without having to duplicate the full walk and metric mapping. See auth-split-migration.md for more details. * [CHANGE] Split config of auth and modules #859 * [BUGFIX] Fix metrics path not using command-line argument value #904 Signed-off-by: SuperQ <[email protected]>

BREAKING CHANGES: This version of the exporter introduces a new configuration file format. This new format separates the walk and metric mappings from the connection and authentication settings. This allows for easier configuration of different auth params without having to duplicate the full walk and metric mapping. See auth-split-migration.md for more details. * [CHANGE] Split config of auth and modules #859 * [ENHANCEMENT] Improved Lookup process for label information #908 * [BUGFIX] Fix metrics path not using command-line argument value #904 Signed-off-by: SuperQ <[email protected]>

BREAKING CHANGES: This version of the exporter introduces a new configuration file format. This new format separates the walk and metric mappings from the connection and authentication settings. This allows for easier configuration of different auth params without having to duplicate the full walk and metric mapping. See auth-split-migration.md for more details. * [CHANGE] Split config of auth and modules #859 * [FEATURE] Improved Lookup process for label information #908 * [ENHANCEMENT] Add support for parsing SNMP transport from target #914 * [BUGFIX] Fix metrics path not using command-line argument value #904 Signed-off-by: SuperQ <[email protected]>

BREAKING CHANGES: This version of the exporter introduces a new configuration file format. This new format separates the walk and metric mappings from the connection and authentication settings. This allows for easier configuration of different auth params without having to duplicate the full walk and metric mapping. See auth-split-migration.md for more details. * [CHANGE] Split config of auth and modules #859 * [FEATURE] Add support for parsing SNMP transport from target #914 * [ENHANCEMENT] Improved Lookup process for label information #908 * [BUGFIX] Fix metrics path not using command-line argument value #904 Signed-off-by: SuperQ <[email protected]>

* Release v0.23.0 BREAKING CHANGES: This version of the exporter introduces a new configuration file format. This new format separates the walk and metric mappings from the connection and authentication settings. This allows for easier configuration of different auth params without having to duplicate the full walk and metric mapping. See auth-split-migration.md for more details. * [CHANGE] Split config of auth and modules #859 * [FEATURE] Add support for parsing SNMP transport from target #914 * [ENHANCEMENT] Improved Lookup process for label information #908 * [BUGFIX] Fix metrics path not using command-line argument value #904 Signed-off-by: SuperQ <[email protected]> * Lint dashboards Update dashboards to latest mixin linting. Signed-off-by: SuperQ <[email protected]> --------- Signed-off-by: SuperQ <[email protected]>

BREAKING CHANGES: This version of the exporter introduces a new configuration file format. This new format separates the walk and metric mappings from the connection and authentication settings. This allows for easier configuration of different auth params without having to duplicate the full walk and metric mapping. See auth-split-migration.md for more details. * [CHANGE] Split config of auth and modules prometheus#859 * [ENHANCEMENT] Improved Lookup process for label information prometheus#908 * [BUGFIX] Fix metrics path not using command-line argument value prometheus#904 Signed-off-by: SuperQ <[email protected]> Signed-off-by: Stephan Windischmann <[email protected]>

BREAKING CHANGES: This version of the exporter introduces a new configuration file format. This new format separates the walk and metric mappings from the connection and authentication settings. This allows for easier configuration of different auth params without having to duplicate the full walk and metric mapping. See auth-split-migration.md for more details. * [CHANGE] Split config of auth and modules prometheus#859 * [FEATURE] Add support for parsing SNMP transport from target prometheus#914 * [ENHANCEMENT] Improved Lookup process for label information prometheus#908 * [BUGFIX] Fix metrics path not using command-line argument value prometheus#904 Signed-off-by: SuperQ <[email protected]> Signed-off-by: Stephan Windischmann <[email protected]>

* Release v0.23.0 BREAKING CHANGES: This version of the exporter introduces a new configuration file format. This new format separates the walk and metric mappings from the connection and authentication settings. This allows for easier configuration of different auth params without having to duplicate the full walk and metric mapping. See auth-split-migration.md for more details. * [CHANGE] Split config of auth and modules prometheus#859 * [FEATURE] Add support for parsing SNMP transport from target prometheus#914 * [ENHANCEMENT] Improved Lookup process for label information prometheus#908 * [BUGFIX] Fix metrics path not using command-line argument value prometheus#904 Signed-off-by: SuperQ <[email protected]> * Lint dashboards Update dashboards to latest mixin linting. Signed-off-by: SuperQ <[email protected]> --------- Signed-off-by: SuperQ <[email protected]> Signed-off-by: Stephan Windischmann <[email protected]>

SuperQ requested a review from RichiH March 26, 2023 12:53

SuperQ force-pushed the superq/auth_config branch from 3fb0b34 to bcd8ebf Compare March 26, 2023 13:00

SuperQ requested a review from roidelapluie March 26, 2023 13:16

SuperQ force-pushed the superq/auth_config branch 2 times, most recently from 503e1f4 to 9b3b0e9 Compare March 26, 2023 13:56

SuperQ marked this pull request as ready for review March 26, 2023 14:02

SuperQ force-pushed the superq/auth_config branch 2 times, most recently from f4ee7d2 to 4a7a6f0 Compare March 27, 2023 08:54

SuperQ force-pushed the superq/auth_config branch from 4a7a6f0 to ea8444c Compare April 4, 2023 17:31

SuperQ force-pushed the superq/auth_config branch from ea8444c to 97e2206 Compare April 5, 2023 08:15

dswarbrick reviewed Apr 5, 2023

View reviewed changes

README.md Outdated Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

main.go Outdated Show resolved Hide resolved

SuperQ force-pushed the superq/auth_config branch 4 times, most recently from 053ff00 to cf42453 Compare April 6, 2023 17:18

SuperQ requested a review from dswarbrick April 6, 2023 17:20

dswarbrick approved these changes Apr 8, 2023

View reviewed changes

SuperQ mentioned this pull request Apr 15, 2023

an optional config file and a new HTTP query string to override walk param #601

Closed

This was referenced Apr 19, 2023

[FEATURE] Separation of MIB definitions and auth info #221

Closed

Support auth alongside target & module as distinct property #619

Closed

SuperQ force-pushed the superq/auth_config branch 2 times, most recently from 37a8099 to 8db8ef0 Compare May 3, 2023 16:37

SuperQ mentioned this pull request May 4, 2023

Allow snmp_context to be passed as optional parameter to override the snmp.ContextName #845

Closed

marctc approved these changes May 31, 2023

View reviewed changes

matthiasr reviewed Jun 1, 2023

View reviewed changes

README.md Outdated Show resolved Hide resolved

RichiH reviewed Jun 22, 2023

View reviewed changes

SuperQ merged commit 9644498 into main Jun 23, 2023

SuperQ deleted the superq/auth_config branch June 23, 2023 12:27

marctc mentioned this pull request Jun 27, 2023

Adapt SNMP flow component/integration to new SNMP exporter config changes grafana/agent#4265

Merged

3 tasks

candlerb reviewed Jun 29, 2023

View reviewed changes

SuperQ mentioned this pull request Jul 4, 2023

Release 0.23.0-rc.0 #906

Merged

frittentheke mentioned this pull request Jul 10, 2023

Allow auth config to be kept in an independent config file (or maybe even add conf.d subfolder support) #912

Closed

SuperQ mentioned this pull request Jul 10, 2023

Allow configuring SNMP version and credentials through parameters #762

Open

SuperQ mentioned this pull request Jul 19, 2023

Release v0.23.0-rc.1 #924

Merged

SuperQ mentioned this pull request Aug 9, 2023

Release v0.23.0 #940

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split config of auth and modules #859

Split config of auth and modules #859

SuperQ commented Mar 26, 2023 •

edited

Loading

SuperQ commented Apr 4, 2023

sebastien-coavoux commented Apr 5, 2023

SuperQ commented Apr 5, 2023

SuperQ commented Apr 5, 2023

dswarbrick left a comment

dswarbrick commented Apr 15, 2023

marctc left a comment

matthiasr left a comment

ZoldanYao commented Jun 21, 2023

SuperQ commented Jun 22, 2023

RichiH Jun 22, 2023

SuperQ Jun 22, 2023

RichiH Jun 23, 2023

SuperQ Jun 23, 2023

RichiH commented Jun 22, 2023

SuperQ commented Jun 22, 2023

RichiH commented Jun 23, 2023

SuperQ commented Jun 23, 2023

candlerb Jun 29, 2023

	ErrUnsupportedVersion = errors.New("unsupported config version")
	ErrUnsupportedVersion = errors.New("Unsupported config version, we only support config version 2. Please see https://github.com/prometheus/snmp_exporter/blob/main/README.md#debugging")

		@@ -0,0 +1,59 @@
		# Module and Auth Split Migration

		In [version 0.23.0](https://github.com/prometheus/snmp_exporter/releases/tag/v0.23.0) the configuration for the `snmp_epxorter` the configuration file format has changed. Configuration files for versions v0.22.0 and before will not work. The configuration was split from a flat list of modules to separate metric walking/mapping modules and authentication configuratoins.

Split config of auth and modules #859

Split config of auth and modules #859

Conversation

SuperQ commented Mar 26, 2023 • edited Loading

SuperQ commented Apr 4, 2023

sebastien-coavoux commented Apr 5, 2023

SuperQ commented Apr 5, 2023

SuperQ commented Apr 5, 2023

dswarbrick left a comment

Choose a reason for hiding this comment

dswarbrick commented Apr 15, 2023

marctc left a comment

Choose a reason for hiding this comment

matthiasr left a comment

Choose a reason for hiding this comment

ZoldanYao commented Jun 21, 2023

SuperQ commented Jun 22, 2023

RichiH Jun 22, 2023

Choose a reason for hiding this comment

SuperQ Jun 22, 2023

Choose a reason for hiding this comment

RichiH Jun 23, 2023

Choose a reason for hiding this comment

SuperQ Jun 23, 2023

Choose a reason for hiding this comment

RichiH commented Jun 22, 2023

SuperQ commented Jun 22, 2023

RichiH commented Jun 23, 2023

SuperQ commented Jun 23, 2023

candlerb Jun 29, 2023

Choose a reason for hiding this comment

SuperQ commented Mar 26, 2023 •

edited

Loading