Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strange number of replicas/API RateLimit hit on Hetzner #155

Closed
madbbb opened this issue Mar 26, 2023 · 0 comments · Fixed by #157
Closed

Strange number of replicas/API RateLimit hit on Hetzner #155

madbbb opened this issue Mar 26, 2023 · 0 comments · Fixed by #157

Comments

@madbbb
Copy link

madbbb commented Mar 26, 2023

Hello! When I try to create a cluster with more then 1 replica it starts to report strange number of replicas (824649122032 instead of 3) in clusterctl describe cluster unovi --show-conditions all (all fine with 1 replica):

NAME                                                                   READY  SEVERITY  REASON                      SINCE  MESSAGE                                                                                                  
Cluster/unovi                                                          False  Warning   ScalingDown                 5s     Scaling down control plane to 824649122032 replicas (actual 3)                                            
│           ├─ControlPlaneInitialized                                  True                                         19h                                                                                                              
│           ├─ControlPlaneReady                                        False  Warning   ScalingDown                 5s     Scaling down control plane to 824649122032 replicas (actual 3)                                            
│           └─InfrastructureReady                                      True                                         19h                                                                                                              
├─ClusterInfrastructure - HetznerCluster/unovi                                                                                                                                                                                       
│             ├─LoadBalancerAttached                                   True                                         36m                                                                                                              
│             ├─NetworkAttached                                        True                                         36m                                                                                                              
│             ├─PlacementGroupsSynced                                  False  Error     PlacementGroupsUnreachable  2m38s  failed to find placement group: failed to list placement groups: limit of 3600 requests per hour rea ...  
│             └─RateLimitExceeded                                      True                                         2m38s                                                                                                            
├─ControlPlane - TalosControlPlane/unovi-control-plane                 False  Warning   ScalingDown                 5s     Scaling down control plane to 824649122032 replicas (actual 3)                                            
│ │           ├─Available                                              True                                         19h                                                                                                              
│ │           ├─ControlPlaneComponentsHealthy                          True                                         34m                                                                                                              
│ │           ├─EtcdClusterHealthyCondition                            False  Warning   EtcdClusterUnhealthy        61s    : expected to have 3 members, got 2                                                                       
│ │           ├─MachinesBootstrapped                                   True                                         19h                                                                                                              
│ │           ├─MachinesCreated                                        True                                         19h                                                                                                              
│ │           ├─MachinesReady                                          True                                         5s                                                                                                               
│ │           └─Resized                                                False  Warning   ScalingDown                 5s     Scaling down control plane to 824649122032 replicas (actual 3)                                            
│ ├─Machine/unovi-control-plane-h6xgz                                  False  Warning   NodeNotFound                56s                                                                                                              

This is scale down process which is stuck, but I had the same problem with initial creation and you can see that it reached API limits on Hetzner. I'm happy to provide any logs, just tell what you want to see.

Unix4ever added a commit to Unix4ever/cluster-api-control-plane-provider-talos that referenced this issue Apr 13, 2023
Fixes: siderolabs#155

We were reading it from `*int` pointer without dereferencing it, so it
was writing pointer address to the status instead of the actual value.

Signed-off-by: Artem Chernyshev <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant