Support autoscaling

[okozachenko1203]

Fix #3

[mnaser]

I believe in this case you've actually ran the auto scaler on the workload cluster with a kubeconfig file to talk to the management cluster

even if things can be secure and locked down, this could be a security issue down the road. Therefore, I think it's better to run the auto scaler on the management cluster talking to the workload instead.

So I think you'll have to remove it from the ClusterResourceSet and get rid of the kubeconfig option, for the actual deployment in the management cluster, you can mount the clustername-kubeconfig secret to get access to the cluster directly.

[mnaser]

See commet

[okozachenko1203]

I believe in this case you've actually ran the auto scaler on the workload cluster with a kubeconfig file to talk to the management cluster

even if things can be secure and locked down, this could be a security issue down the road. Therefore, I think it's better to run the auto scaler on the management cluster talking to the workload instead.

So I think you'll have to remove it from the ClusterResourceSet and get rid of the kubeconfig option, for the actual deployment in the management cluster, you can mount the clustername-kubeconfig secret to get access to the cluster directly.

when we run autoscaler in mgmt cluster, i wonder it means we need to deploy one autoscaler per one magnum cluster
That was my concern and the reason why i chose autosacler in workload cluster and tried to create clusterrole with resourceName binding.

[mnaser]

I believe in this case you've actually ran the auto scaler on the workload cluster with a kubeconfig file to talk to the management cluster
even if things can be secure and locked down, this could be a security issue down the road. Therefore, I think it's better to run the auto scaler on the management cluster talking to the workload instead.
So I think you'll have to remove it from the ClusterResourceSet and get rid of the kubeconfig option, for the actual deployment in the management cluster, you can mount the clustername-kubeconfig secret to get access to the cluster directly.

when we run autoscaler in mgmt cluster, i wonder it means we need to deploy one autoscaler per one magnum cluster That was my concern and the reason why i chose autosacler in workload cluster and tried to create clusterrole with resourceName binding.

yes, indeed, actually we will need to create a autoscaler per CAPI cluster on the control plane, but I think the load added of this is kinda negligible so we can be OK with it.

[okozachenko1203]

Ok, I have changed the clusterAPIMode as you mentioned.
Also I used flux helmrelease for autoscaler deployment.

[mnaser]

@okozachenko1203 good progress

[mnaser]

@okozachenko1203 I pushed some changes.

I think the issue is that we're being hit by kubernetes-sigs/cluster-api#7088 -- without that in place, we're unable to use the metadata the way we want to.

I think what we should do is instead take advantage of update_cluster_status (or maybe update_node_group_status) to mutate and reconcile the annotations on the MachineDeployment.

Let's leave a comment pointing to the PR.