PowerDNS regex domain filter stopped working

[matusf]

What happened:
--regex-domain-filter was working in v0.13.1 but in it fails in latest version (on master) and v0.13.5.

What you expected to happen:
Successful domain synchronization.

How to reproduce it (as minimally and precisely as possible):
Run service according to the tutorial:

---
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
    - name: nginx
      image: nginx
      ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  annotations:
    external-dns.alpha.kubernetes.io/internal-hostname: nginx.ednstest.example.com.
spec:
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: 80
  selector:
    run: nginx

Run external dns (latest)

./external-dns --provider pdns --pdns-api-key=$PDNS_API_KEY --source service --pdns-server=https://pdns.example.com --once --tls-ca=/etc/ssl/certs/ca-bundle.crt --regex-domain-filter=ednstest.example.com --log-level=debug

Logs:

INFO[0000] Created Kubernetes client https://192.168.39.181:8443 
DEBU[0000] Skipping TLS for PDNS Provider.              
DEBU[0000] Unable to fetch zone Status: 422 Unprocessable Entity, Body: {"domain":"example.com.","error":"Fatality! Error checking domain/key. Token or domain might be wrong."} 
DEBU[0000] Retrying ListZone() ... 0                    
DEBU[0001] Unable to fetch zone Status: 422 Unprocessable Entity, Body: {"domain":"example.com.","error":"Fatality! Error checking domain/key. Token or domain might be wrong."} 
DEBU[0001] Retrying ListZone() ... 1                    
DEBU[0001] Unable to fetch zone Status: 422 Unprocessable Entity, Body: {"domain":"example.com.","error":"Fatality! Error checking domain/key. Token or domain might be wrong."} 
DEBU[0001] Retrying ListZone() ... 2                    
ERRO[0002] Unable to list zone. Status: 422 Unprocessable Entity, Body: {"domain":"example.com.","error":"Fatality! Error checking domain/key. Token or domain might be wrong."} 
WARN[0002] Unable to fetch Records                      
FATA[0002] Status: 422 Unprocessable Entity, Body: {"domain":"example.com.","error":"Fatality! Error checking domain/key. Token or domain might be wrong."} 
exit status 1

Anything else we need to know?:
Here are logs from successful run of external-dns on v0.13.1 with the same setup as above. I just checked out on the tag and recompiled.

Run external-dns v0.13.1

~/c/g/m/external-dns ((v0.13.1))> ./external-dns --provider pdns --pdns-api-key=$PDNS_API_KEY --source service --pdns-server=https://pdns.example.com --once --tls-ca=/etc/ssl/certs/ca-bundle.crt --regex-domain-filter=ednstest.example.com --log-level=debug

Logs:

INFO[0000] config: {APIServerURL: KubeConfig: RequestTimeout:30s DefaultTargets:[] ContourLoadBalancerService:heptio-contour/contour GlooNamespace:gloo-system SkipperRouteGroupVersion:zalando.org/v1 Sources:[service] Namespace: AnnotationFilter: LabelFilter: FQDNTemplate: CombineFQDNAndAnnotation:false IgnoreHostnameAnnotation:false IgnoreIngressTLSSpec:false IgnoreIngressRulesSpec:false GatewayNamespace: GatewayLabelFilter: Compatibility: PublishInternal:false PublishHostIP:false AlwaysPublishNotReadyAddresses:false ConnectorSourceServer:localhost:8080 Provider:pdns GoogleProject: GoogleBatchChangeSize:1000 GoogleBatchChangeInterval:1s GoogleZoneVisibility: DomainFilter:[] ExcludeDomains:[] RegexDomainFilter:ednstest.example.com RegexDomainExclusion: ZoneNameFilter:[] ZoneIDFilter:[] TargetNetFilter:[] ExcludeTargetNets:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType: AWSZoneTagFilter:[] AWSAssumeRole: AWSAssumeRoleExternalID: AWSBatchChangeSize:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AWSPreferCNAME:false AWSZoneCacheDuration:0s AWSSDServiceCleanup:false AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup: AzureSubscriptionID: AzureUserAssignedIdentityClientID: BluecatDNSConfiguration: BluecatConfigFile:/etc/kubernetes/bluecat.json BluecatDNSView: BluecatGatewayHost: BluecatRootZone: BluecatDNSServerName: BluecatDNSDeployType:no-deploy BluecatSkipTLSVerify:false CloudflareProxied:false CloudflareZonesPerPage:50 CoreDNSPrefix:/skydns/ RcodezeroTXTEncrypt:false AkamaiServiceConsumerDomain: AkamaiClientToken: AkamaiClientSecret: AkamaiAccessToken: AkamaiEdgercPath: AkamaiEdgercSection: InfobloxGridHost: InfobloxWapiPort:443 InfobloxWapiUsername:admin InfobloxWapiPassword: InfobloxWapiVersion:2.3.1 InfobloxSSLVerify:true InfobloxView: InfobloxMaxResults:0 InfobloxFQDNRegEx: InfobloxCreatePTR:false InfobloxCacheDuration:0 DynCustomerName: DynUsername: DynPassword: DynMinTTLSeconds:0 OCIConfigFile:/etc/kubernetes/oci.yaml InMemoryZones:[] OVHEndpoint:ovh-eu OVHApiRateLimit:20 PDNSServer:https://pdns.example.com PDNSAPIKey:****** PDNSTLSEnabled:false TLSCA:/etc/ssl/certs/ca-bundle.crt TLSClientCert: TLSClientCertKey: Policy:sync Registry:txt TXTOwnerID:default TXTPrefix: TXTSuffix: Interval:1m0s MinEventSyncInterval:5s Once:true DryRun:false UpdateEvents:false LogFormat:text MetricsAddress::7979 LogLevel:debug TXTCacheInterval:0s TXTWildcardReplacement: ExoscaleEndpoint:https://api.exoscale.ch/dns ExoscaleAPIKey: ExoscaleAPISecret: CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] CFAPIEndpoint: CFUsername: CFPassword: RFC2136Host: RFC2136Port:0 RFC2136Zone: RFC2136Insecure:false RFC2136GSSTSIG:false RFC2136KerberosRealm: RFC2136KerberosUsername: RFC2136KerberosPassword: RFC2136TSIGKeyName: RFC2136TSIGSecret: RFC2136TSIGSecretAlg: RFC2136TAXFR:false RFC2136MinTTL:0s RFC2136BatchChangeSize:50 NS1Endpoint: NS1IgnoreSSL:false NS1MinTTLSeconds:0 TransIPAccountName: TransIPPrivateKeyFile: DigitalOceanAPIPageSize:50 ManagedDNSRecordTypes:[A CNAME] GoDaddyAPIKey: GoDaddySecretKey: GoDaddyTTL:0 GoDaddyOTE:false OCPRouterName: IBMCloudProxied:false IBMCloudConfigFile:/etc/kubernetes/ibmcloud.json TencentCloudConfigFile:/etc/kubernetes/tencent-cloud.json TencentCloudZoneType: PluralCluster: PluralProvider:} 
INFO[0000] Instantiating new Kubernetes client          
DEBU[0000] apiServerURL:                                
DEBU[0000] kubeConfig: /home/matus/.kube/config         
INFO[0000] Using kubeConfig                             
INFO[0000] Created Kubernetes client https://192.168.39.181:8443 
DEBU[0000] Skipping TLS for PDNS Provider.              
DEBU[0000] Records fetched:
[a-nginx.ednstest.example.com 300 IN TXT  "heritage=external-dns,external-dns/owner=default,external-dns/resource=service/default/nginx" [] nginx.ednstest.example.com 300 IN TXT  "heritage=external-dns,external-dns/owner=default,external-dns/resource=service/default/nginx" [] nginx.ednstest.example.com 300 IN A  10.111.145.5 [] ednstest.example.com 3600 IN SOA  ns1.example.com. hostmaster.ednstest.example.com. 2023072403 10800 3600 604800 3600 [] ednstest.example.com 3600 IN NS  ns1.in.example.com;ns2.in.example.com []] 
DEBU[0000] No endpoints could be generated from service default/kubernetes 
DEBU[0000] Endpoints generated from service: default/nginx: [nginx.ednstest.example.com 0 IN A  10.111.145.5 []] 
DEBU[0000] No endpoints could be generated from service kube-system/kube-dns 
INFO[0000] All records are already up to date 

dig for verification

~/c/g/m/external-dns ((v0.13.1))> dig nginx.ednstest.example.com
nginx.ednstest.example.com. 300 IN A	10.111.145.5

Environment:

• External-DNS version (use external-dns --version): latest (master)
• DNS provider: PowerDNS
• Others: linux fedora 38

---

might be related to: #3284, #3073