Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

swarm install alertmanager cluster failed #1383

Closed
like-inspur opened this issue May 15, 2018 · 3 comments
Closed

swarm install alertmanager cluster failed #1383

like-inspur opened this issue May 15, 2018 · 3 comments

Comments

@like-inspur
Copy link

like-inspur commented May 15, 2018

docker version: 17.09.1-ce,alertmanager image version: prom/alertmanager:v0.15.0-rc.
I config docker stack yml referenced alertmanager#high-availability, include three nodes three services
Three alertmanager are alike, alertmanager1 config is like this:
alertmanager1:
 image: prom/alertmanager:v0.15.0-rc.1
 volumes:
  - /monitor/alertmanager:/alertmanager
  - /etc/localtime:/etc/localtime:ro
 command: --config.file=/etc/alertmanager/config.yml --storage.path=/alertmanager --cluster.listen-address=0.0.0.0:9094 --cluster.peer=alertmanager2:9094 --cluster.peer=alertmanager3:9094
 ports:
   - target: 9093
  published: 9093
  protocol: tcp
  mode: host
 configs:
   - source: alertmanager_config
  target: /etc/alertmanager/config.yml
  mode: 0664
 networks:
   - monitor
I found peers is only one that output from url, container log is like this:
level=info ts=2018-05-15T09:03:32.989325663Z caller=main.go:140 msg="Starting Alertmanager" version="(version=0.15.0-rc.1, branch=HEAD, revision=acb111e812530bec1ac6d908bc14725793e07cf3)"
level=info ts=2018-05-15T09:03:32.989460303Z caller=main.go:141 build_context="(go=go1.10, user=root@f278953f13ef, date=20180323-13:05:10)"
level=warn ts=2018-05-15T09:03:33.274080488Z caller=cluster.go:129 component=cluster msg="failed to join cluster" err="2 errors occurred:\n\n* Failed to resolve alertmanager2:9094: lookup alertmanager2 on 127.0.0.11:53: no such host\n* Failed to resolve alertmanager3:9094: lookup alertmanager3 on 127.0.0.11:53: no such host"
level=info ts=2018-05-15T09:03:33.27433325Z caller=cluster.go:249 component=cluster msg="Waiting for gossip to settle..." interval=2s
level=info ts=2018-05-15T09:03:33.274972301Z caller=main.go:270 msg="Loading configuration file" file=/etc/alertmanager/config.yml
level=info ts=2018-05-15T09:03:33.286000647Z caller=main.go:346 msg=Listening address=:9093
level=info ts=2018-05-15T09:03:35.274784642Z caller=cluster.go:274 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000239866s

@stuartnelson3
Copy link
Contributor

It's probably related to #1307

Please format your issue next time.

@like-inspur
Copy link
Author

I use overlay network, so container has overlay and docker_gwbridge addresses. I found alertmanager cluster peers address is docker_gwbridge, not the real overlay address that containers can connect.

@stuartnelson3
Copy link
Contributor

this is a usage question, and would be better directed to the users mailing list [email protected]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants