haproxy-ingress locks up attempting leader election

[zapman449]

E0912 17:17:48.511187       6 leaderelection.go:258] Failed to update lock: Operation cannot be fulfilled on configmaps "ingress-controller-leader-haproxy": the object has been modified; please apply your changes to the latest version and try again

haproxy-ingress version haproxy-ingress:v0.4-snapshot.4
Kubernetes version 1.5.5

Works great, other than this lock up. After that happens, no new changes in the cluster get registered.

[jcmoraisjr]

Couldn't reproduce this (yet). How many instances do you have running at the same time? Any other detail of your deployment which worth to mention?

[zapman449]

2 worker nodes, 3 masters built by kops.

It MIGHT have been triggered by a lurking nginx-ingress we had (thought we) deleted. We haven't replicated it since actually deleting it. If it survives the night, I think we can close the ticket.

[Scukerman]

I have the same messages in log only if I run controller as a DaemonSet.

[zapman449]

we are running as a daemonset...

[zapman449]

And we locked up overnight

[Scukerman]

kubernetes/ingress-nginx#1297

So it seems like Ingress update is required.
Merged PR: kubernetes/ingress-nginx#1338

[zapman449]

@Scukerman Dumb question: What is the relationship between kubernetes/ingress/haproxy and jcmoraisjr/haproxy-ingress?

[Scukerman]

It is the same project (repo is here, docs are there).

Check the README.md and you'll see that some of the links point to the Ingress repo docs.

Here is a list of available and known Ingress Controllers:
https://github.com/kubernetes/ingress/blob/master/docs/catalog.md

nginx and gce controllers are not in the list because they are official.

[mbreitung]

I can confirm that kubernetes/ingress-nginx#1338 fixes this issue.

[jcmoraisjr]

Sorry the long delay, vacation here.

@zapman449 b66c7f9 should fix this. I'll provide a new image in the following days. Let us know if you still have any problem.

@Scukerman and @mbreitung thanks for sharing your findings, this way fix HAProxy Ingress is a lot easier.

[jcmoraisjr]

@zapman449 please report back - commenting and/or closing the issue - if the problem persists. Fyi the canary tag was just updated with this fix.

[zapman449]

Building from master allowed us to move past this problem. Thank you.

[zapman449]

Out of curiosity, when will a snapshot release be forthcoming?

[Scukerman]

@zapman449 docker pull quay.io/jcmoraisjr/haproxy-ingress:v0.4-beta.1
Doesn't it work yet?

[zapman449]

It does. I missed the shift from snapshot to beta