Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cloudflare provider keeps updating every ~10 seconds #840

Closed
materemias opened this issue Jan 4, 2019 · 13 comments
Closed

cloudflare provider keeps updating every ~10 seconds #840

materemias opened this issue Jan 4, 2019 · 13 comments

Comments

@materemias
Copy link

my ingresses do have multiple loadbalancers defined

...
status:
  loadBalancer:
    ingress:
    - ip: 18.197.33.96
    - ip: 3.121.26.154
    - ip: 35.158.122.112

and cloudflare provider keeps updating A and TXT records like crazy. the dns service kinda works but I am afraid I will hit an api rate limit at cloudflare

time="2019-01-04T14:16:06Z" level=info msg="Changing record." action=UPDATE record=alertmanager.mydomain.com ttl=1 type=A zone=98ee028f99adf9e763e83c1757c0fe
time="2019-01-04T14:16:07Z" level=info msg="Changing record." action=UPDATE record=prometheus.mydomain.com ttl=1 type=A zone=98ee028f99adf9e763e83c1757c0fe

could there be a solution to only use the first ip of a loadbalancer 'cluster'?

@Evesy
Copy link
Contributor

Evesy commented Feb 6, 2019

This doesn't seem limited to ingresses with multiple addresses. Every minute (Default polling interval) I can see external DNS changing the same record despite it being correctly sync'd:

NAME                      HOSTS                       ADDRESS        PORTS     AGE
platform-testing-public   neg-test.domain.co.uk   35.xxx.xx.xx   80, 443   5d
2019-02-06T18:18:06.76689818Z time="2019-02-06T18:18:06Z" level=info msg="Changing record." action=UPDATE record=neg-test.domain.co.uk ttl=1 type=A zone=c9ed3d64d4fcd40000e38d4724c05808
2019-02-06T18:18:07.386830847Z time="2019-02-06T18:18:07Z" level=info msg="Changing record." action=UPDATE record=neg-test.domain.co.uk ttl=1 type=TXT zone=c9ed3d64d4fcd40000e38d4724c05808
2019-02-06T18:19:05.630554878Z time="2019-02-06T18:19:05Z" level=info msg="Changing record." action=UPDATE record=neg-test.domain.co.uk ttl=1 type=A zone=c9ed3d64d4fcd40000e38d4724c05808
2019-02-06T18:19:06.231534976Z time="2019-02-06T18:19:06Z" level=info msg="Changing record." action=UPDATE record=neg-test.domain.co.uk ttl=1 type=TXT zone=c9ed3d64d4fcd40000e38d4724c05808

@Evesy
Copy link
Contributor

Evesy commented Feb 6, 2019

UpdateOld:[neg-test.domain.co.uk 120 IN A 35.xxx.xx.xx [{external-dns.alpha.kubernetes.io/cloudflare-proxied false}]] UpdateNew:[neg-test.domain.co.uk 120 IN A 35.xxx.xx.xx []]

It seems as though the ProviderSpecific configuration is always causing a diff between the current & desired state.

As a side note it also appears setting the TTL does not work if you have cloudflare proxying enabled (Need to confirm this) Edit: With proxying enabled Cloudflare manages the TTL

@Evesy
Copy link
Contributor

Evesy commented Feb 6, 2019

@materemias - Can you confirm if you're using Cloudflare proxying for your records or not (I'll create a separate issue if this doesn't match your case)

Endpoints with Cloudflare proxying enabled via the deployment argument (--cloudflare-proxied) are constantly seen as needing to be updated, endpoints that have the proxy annotation set (external-dns.alpha.kubernetes.io/cloudflare-proxied: "true") correctly display no changes

@Evesy
Copy link
Contributor

Evesy commented Feb 7, 2019

@eswets @amolenaar I don't suppose you'd be able to offer any insight here, I'm struggling to track down what's causing the behaviour described above

In my case I'm building against master since there's some Cloudflare changes (support pagination) that haven't made it into a release yet. It seems unrelated to the regressions introduced in 0.5.10 (Proposed fix: #886 which I've tried pulling in to my build)

@Evesy
Copy link
Contributor

Evesy commented Feb 8, 2019

So the generated Endpoints only include the provider-specific configuration (in this case external-dns.alpha.kubernetes.io/cloudflare-proxied: "true") if it's been specified as an annotation.

Setting --cloudflare-proxied on the deployment has no bearing on the resultant Endpoints returned, however, it is correctly used when records are applied to the provider -- This causes a constant diff here https://github.com/kubernetes-incubator/external-dns/blob/master/plan/plan.go#L188

Not sure if this affects other providers (Not sure if any other providers actually have provider specific annotations)

@freeyoung
Copy link

I'm having the same issue. external-dns keeps saying the same record gets updated msg="Changing record." action=UPDATE every minute.

I tried adding the annotation external-dns.alpha.kubernetes.io/cloudflare-proxied: "true" to my Ingress but this made no difference.

@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 26, 2019
@chapati23
Copy link

/remove-lifecycle stale

This is a pretty big issue for us. We're often running into CloudFlares rate limit as a result of this.

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 24, 2019
@Constantin07
Copy link

Same happening for me. Had to request API calls limit increase meanwhile ...

@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 7, 2019
@fejta-bot
Copy link

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 7, 2019
@fejta-bot
Copy link

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

@k8s-ci-robot
Copy link
Contributor

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Feb 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants