feature: Prometheus node_exporter compatible output

[greenpau]

The output produced by bombardier cannot be consumed by Prometheus.

It would be nice to have metrics like this:

Prometheus' node_exporter has textfile collector. The collector scans directory for .prom files and adds the metrics found in the file to its own metrics set.

This way, I can run bombardier every minute, output results to bombardier.prom file and the metrics will be picked up by Prometheus server.

I would like to come up with a number of metrics and have an option to output results as a Prometheus metric.

For example, the following output:

Statistics        Avg      Stdev        Max
  Reqs/sec      6607.00     524.56       7109
  Latency       29.86ms     5.36ms   305.02ms

would result in the following metrics:

• bombardier_http_request_rate_avg
• bombardier_http_request_rate_max
• bombardier_http_request_rate_stdev
• bombardier_http_latency_avg
• bombardier_http_latency_stdev
• bombardier_http_latency_max

Each metric could have one or more labels associated with it.

For example, the following command:

bombardier -c 200 -d 10s -l http://ya.ru

would result in the following metrics:

conn: 200
duration: 10
url: http://ya.ru

Further, it would be helpful to output UUID associated with each test.

[greenpau]

@codesenberg , I was thinking about creating a PR for this. Do you have any view on this one?

[codesenberg]

I was thinking about adding two flags --print (-p) and --format (-f), so users could do something like:
bombardier [other flags...] --print=result --format=json <url>
and get something like:

{
	"attack": {
		// attack parameters, i.e url, number of conns., etc.
	},
	"requests_per_second": {
		"avg": 1000,
		"stdev": 200,
		"max": 2000
	},
	"latency": {
		// durations are in microseconds
		"avg": 20000.2,
		"stdev": 5000.0,
		"max": 100000.0,
		// if --latencies flag is used
		"percentiles": {
			"50": 19200.0,
			"75": 21000.0,
			"90": 23000.0,
			"99": 31000.0,
		},
	},
	// throughput is in bytes per second
	"throughput": 10000.0,
	"codes": { /* HTTP status codes */ },
	// some other things I could forget
}

With this you could transform said JSON into the format you need. Sounds good?
As for the UUID, you could run some script, like this one:

(python -c "import sys,uuid; sys.stdout.write("test_uuid: " + uuid.uuid4().hex + '\n')" && bombardier --print=result --format=json <url> | transform-bombardier-output-to-prometheus-format) > bombardier.prom

[codesenberg]

Just to clarify a little bit --print=result means that only the stats will be printed. No intro and no progress bar.

[codesenberg]

@greenpau, you know, we could actually step a bit further and just add prometheus to the list of formats understood by the --format flag, saving users from the hassle of transforming json into prometheus's text format.

[greenpau]

add prometheus to the list of formats understood by the --format flag,

vot eto razgovor 👍

[codesenberg]

Hey, @greenpau, how important that UUID thing for your use case? I plan to implement user-defined templates, so if it's somewhat important (granted I will in fact implement them), I could add some func UUID() (string|something) helper.

[greenpau]

UUID thing for your use case?

@codesenberg , it is pretty important, because it allows to store that transaction in other systems (and later correlate it), e.g. elasticsearch. Consider reading https://discuss.elastic.co/t/elastic-search--id-uuid-format/28801/4 for insights.

[codesenberg]

The I'll probably add helpers for all five versions of it from this library.
Sounds good to you?

[greenpau]

The I'll probably add helpers for all five versions of it from this library.

That's sounds great! 👍

[codesenberg]

Ok, user-defined templates landed on master just now. Let me know if everything is in place to generate the output you want.

[greenpau]

Ok, user-defined templates landed on master just now. Let me know if everything is in place to generate the output you want.

@codesenberg , 👍

[tkanos]

Hello @codesenberg,
Let me know if I can help you with the prometheus side.

[codesenberg]

@tkanos, well, I don't know if there is some reasonable default output format for prometheus that we could use. Mainly, because I'm not too familiar with prometheus in general, but if there is one, then sure, we could add it.

[codesenberg]

By the way. @greenpau, have you succeeded in your endeavor to generate prometheus compatible output by using user-defined output templates?

[tkanos]

@codesenberg,
Well we can define all the output of bombardier to be scraped by Prometheus, but Prometheus is a time series database, and benchmark is done time to time (or in the CI/CD path), so there is (for me) no real sense.

And the guy that wants to have it, will have to configure Prometheus to scrap the endpoint that we will have to provide. And if he configure it to scrap each 15 sec, and do a benchmark of 5s, He will not have any information, and then after the benchmark he will have to remove his config.

It could have sense in a benchmark Suite, where all benchmark are managed by a web application, that runs all benchmark of all projects, and provide the info to prometheus, but it's not the goal of Bombardier.

Or do what @greenpau is doing, a prom generated, that we can generate during the CI/CD and provide it to prometheus to alert if we have an issue.

[greenpau]

By the way. @greenpau, have you succeeded in your endeavor to generate prometheus compatible output by using user-defined output templates?

@codesenberg , unfortunately, I am tied doing other things at the moment. I think the way you did is with templates is great, because then, using that same templating technique, you could generate Elasticsearch inserts. Once I get back to app testing, I will post in here.

[codesenberg]

@tkanos, let's leave things as they are then. (At least for the time being.)

[codesenberg]

I'll close this one, since more than two years passed already. Unlikely that we'll hear any feedback 😄