Accept HTTP headers in request parameters

[anemyte]

Hello,

I've come to a problem when I need to setup blackbox monitoring for each edge server under a domain name. I found this way to acheive that:

• use dns_sd_config to get a list of server IP addresses;
• create a module with hard-coded Host header in it:

modules:
  http_2xx_example:
    http:
      headers:
        Host: vhost.example.com

• disable SSL check (still fails with CloudFlare) or try to set httpClientConfig.TLSConfig.ServerName via module configuration (I saw it as a part of the module but found no example).

Having several dozens domains to monitor, I considered this approach unsatisfying and so I made this feature. See the updated README.md for examples.

Also, this can be used to change User-Agent (#557).

[mem]

Looks good to me, but I'll defer to @roidelapluie for approval.

[roidelapluie]

I'd rather go to have a hostname parameter directly, see https://docs.google.com/document/d/1VwqXi2TOb5KXaZY6Iio7411x64pJao3GusX8MqYsJ2g/edit#

[anemyte]

I'd rather go to have a hostname parameter directly, see https://docs.google.com/document/d/1VwqXi2TOb5KXaZY6Iio7411x64pJao3GusX8MqYsJ2g/edit#

I don't mind changing this to a hostname parameter to set Host header and TLS SNI, since those two are the most dynamic part that requires a module per each domain name. However, there is a comment left by Joaquin:

Sorry if this has been talked about somewhere else, but what do you think of generalizing this to any header? header[XXX]=YYY or something like this

Which makes me think that not only I could benefit from the current general-case implementation. Allowing any header as a parameter makes it possible to use the default http_2xx module for most HTTP probe use cases. This is convenient because you have one config less to manage and the probe configuration is kept within scrape config, so you don't need to look for details somewhere else.

Let me know if you are adamant on this and I'll work on fix.

[roidelapluie]

i have not changed my mind. HTTP headers hidden in Prometheus would be very complex to think about as users.

[anemyte]

Closing this in favor of #823