Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blackbox Exporter adding ip to url instead dns name #886

Open
ghost opened this issue Feb 28, 2022 · 10 comments
Open

Blackbox Exporter adding ip to url instead dns name #886

ghost opened this issue Feb 28, 2022 · 10 comments

Comments

@ghost
Copy link

ghost commented Feb 28, 2022

Host operating system: output of uname -a

Running blackbox exporter in a k8s cluster

blackbox_exporter version: output of blackbox_exporter --version

0.19.0

What is the blackbox.yml module config.

   modules:
      http_2xx:
        http:
          follow_redirects: true
          preferred_ip_protocol: ip4
          valid_http_versions:
          - HTTP/1.1
          - HTTP/2.0
        prober: http
        timeout: 5s
      tcp_connect:
        prober: tcp
      icmp:
        prober: icmp
        icmp:
          preferred_ip_protocol: "ip4"

What is the prometheus.yml scrape config.

  - interval: 5m
    metricRelabelings:
    - sourceLabels:
      - __address__
      targetLabel: __param_target
    - replacement: prometheus-blackbox-exporter:9115
      sourceLabels:
      - __param_target
      targetLabel: instance
    relabelings:
    - sourceLabels: [job]
      regex: (.*)
      targetLabel: job
      replacement: "my.super.host"
      action: replace
    params:
      module:
      - http_2xx
      target:
      - my.super.host
    path: /probe
    port: http
    scheme: http
    scrapeTimeout: 30s

What logging output did you get from adding &debug=true to the probe URL?

X

What did you do that produced an error?

I have added a new scrape target to the configuration which uses the http_2xx module to do http checks.

What did you expect to see?

ts=2022-02-28T13:27:12.202701182Z caller=client.go:251 module=http_2xx target=https://my.super.host/ level=info msg="Making HTTP request" url=https://my.super.host/ host=my.super.host

What did you see instead?

ts=2022-02-28T13:27:12.202701182Z caller=client.go:251 module=http_2xx target=https://my.super.host/ level=info msg="Making HTTP request" url=https://IP-ADRESS/ host=my.super.host

I was wondering if there is something I can do to bring blackbox exporter to call the host name from target instead using the ip address. The target host is using cloud flare and I get a 503 instead of 200 when going on the ip.

@bru73f0rc3
Copy link

I'm having the same issue, did you manage to resolve this by chance?

@ghost
Copy link
Author

ghost commented Mar 17, 2022

@bru73f0rc3 no, we were looking into it again and again but no resolution yet. The only option we had is to get whitelisted by cloud flare but that was not possible from the customer side.

@Rajendraladkat1919
Copy link

I am facing the same issue. I don't think so we can use anything in blackbox.yaml . I guess we need to fix it at go level.

@roidelapluie
Copy link
Member

roidelapluie commented Mar 17, 2022

We should provide cloudflare with the appropriate HOST headers and SNI . What's going on?

@anemyte
Copy link
Contributor

anemyte commented Mar 24, 2022

I have some hosts behind CloudFlare but I'm not able to reproduce the problem with the exporter v0.19.0 or v0.20.0:

ts=2022-03-24T22:08:44.808129138Z caller=client.go:252 module=http_2xx target=https://foo.bar/ level=info msg="Making HTTP request" url=https://188.114.99.128/ host=foo.bar
ts=2022-03-24T22:08:45.016320736Z caller=main.go:140 module=http_2xx target=https://foo.bar/ level=info msg="Received HTTP response" status_code=200

@wikitops
Copy link

wikitops commented Apr 1, 2022

We have the same problem on our side too when we try to run http checks on external domains. We are behind a proxy and it is complicated/ not our policy to allow traffic on public IP. Even more so when they can be dynamic.

I tried to update the header to force the host with a specific dns name but still the same. Black box tries to access the public IP.

@roidelapluie
Copy link
Member

I would be open to a new http_probe, http_proxy which would forcibly use a proxy in "an expected way".

@stewartn782
Copy link

We've having the same problem with AWS services where the IP changes randomly so we hit 403 errors when we should be seeing 200's. We've configured Blackbox to pass through the host header but that seems to have no effect either.

@roidelapluie
Copy link
Member

The fix for this is #938

@senemamourr
Copy link

senemamourr commented Oct 11, 2023

having same issue with api health check target=https://dnsname/health

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants