🌱 templates for hcloud remediation.

[guettli]

What this PR does / why we need it:

Add templates for hcloud remediation.

[guettli]

---

Protocol of testing the remediation:

I stopped a node via ssh and systemctl stop kubelet.

Logs:

Apr 03 12:19:41 testing-md-0-hjztf systemd[1]: Stopping kubelet: The Kubernetes Node Agent...
Apr 03 12:19:41 testing-md-0-hjztf systemd[1]: kubelet.service: Deactivated successfully.
Apr 03 12:19:41 testing-md-0-hjztf systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Apr 03 12:19:41 testing-md-0-hjztf systemd[1]: kubelet.service: Consumed 19.256s CPU time.
-- Boot 7bed8d86e44c4345a8c99a74256fcb1d --
Apr 03 12:23:44 testing-md-0-hjztf systemd[1]: Started kubelet: The Kubernetes Node Agent.

❯ k get machines
NAME                            CLUSTER   NODENAME                      PROVIDERID          PHASE     AGE   VERSION
testing-md-0-7d45d88d47-tzf7c   testing   testing-md-0-hjztf            hcloud://30749989   Running   72m   v1.25.2
(removed other lines)

relevant part of k logs -n caph-system caph-controller-manager-6cf7b5d796-8k2jq

{"level":"INFO","time":"2023-04-03T12:20:21.138Z","file":"controllers/hcloudmachine_controller.go:181","message":"Reconciling HCloudMachine","controller":"hcloudmachine","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudMachine","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"namespace":"default","name":"testing-md-0-hjztf","reconcileID":"b53ba1d0-c133-4f1b-b7d3-828ee79b6d1c","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"}}
{"level":"DEBUG","time":"2023-04-03T12:23:22.007Z","logger":"controller-runtime.webhook.webhooks","file":"admission/http.go:96","message":"received request","webhook":"/mutate-infrastructure-cluster-x-k8s-io-v1beta1-hcloudremediation","UID":"15ecec64-39cf-4a4c-8cfa-6734cd074a6e","kind":"infrastructure.cluster.x-k8s.io/v1beta1, Kind=HCloudRemediation","resource":{"group":"infrastructure.cluster.x-k8s.io","version":"v1beta1","resource":"hcloudremediations"}}
{"level":"DEBUG","time":"2023-04-03T12:23:22.007Z","logger":"controller-runtime.webhook.webhooks","file":"admission/http.go:143","message":"wrote response","webhook":"/mutate-infrastructure-cluster-x-k8s-io-v1beta1-hcloudremediation","code":200,"reason":"","UID":"15ecec64-39cf-4a4c-8cfa-6734cd074a6e","allowed":true}
{"level":"DEBUG","time":"2023-04-03T12:23:22.010Z","logger":"controller-runtime.webhook.webhooks","file":"admission/http.go:96","message":"received request","webhook":"/validate-infrastructure-cluster-x-k8s-io-v1beta1-hcloudremediation","UID":"55bd863b-233f-4613-b47e-25f36d1d79dd","kind":"infrastructure.cluster.x-k8s.io/v1beta1, Kind=HCloudRemediation","resource":{"group":"infrastructure.cluster.x-k8s.io","version":"v1beta1","resource":"hcloudremediations"}}
{"level":"DEBUG","time":"2023-04-03T12:23:22.010Z","logger":"controller-runtime.webhook.webhooks","file":"admission/http.go:143","message":"wrote response","webhook":"/validate-infrastructure-cluster-x-k8s-io-v1beta1-hcloudremediation","code":200,"reason":"","UID":"55bd863b-233f-4613-b47e-25f36d1d79dd","allowed":true}
{"level":"INFO","time":"2023-04-03T12:23:22.015Z","file":"controllers/hcloudremediation_controller.go:175","message":"Reconciling HCloudRemediation","controller":"hcloudremediation","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudRemediation","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"namespace":"default","name":"testing-md-0-7d45d88d47-tzf7c","reconcileID":"026048d9-1e54-43fc-8413-fd2aa5e78860","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"}}
{"level":"INFO","time":"2023-04-03T12:23:22.015Z","file":"remediation/remediation.go:49","message":"Reconciling hcloud remediation","controller":"hcloudremediation","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudRemediation","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"namespace":"default","name":"testing-md-0-7d45d88d47-tzf7c","reconcileID":"026048d9-1e54-43fc-8413-fd2aa5e78860","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"},"name":"testing-md-0-7d45d88d47-tzf7c"}
{"level":"INFO","time":"2023-04-03T12:23:22.018Z","file":"controllers/hcloudmachine_controller.go:181","message":"Reconciling HCloudMachine","controller":"hcloudmachine","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudMachine","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"namespace":"default","name":"testing-md-0-hjztf","reconcileID":"e4d5c663-a451-4a7b-9f4a-be45a0ef7103","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"}}
{"level":"INFO","time":"2023-04-03T12:23:32.270Z","file":"remediation/remediation.go:84","message":"Rebooting the server","controller":"hcloudremediation","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudRemediation","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"namespace":"default","name":"testing-md-0-7d45d88d47-tzf7c","reconcileID":"026048d9-1e54-43fc-8413-fd2aa5e78860","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"}}
{"level":"INFO","time":"2023-04-03T12:23:32.270Z","file":"remediation/remediation.go:166","message":"Rebooting server","controller":"hcloudremediation","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudRemediation","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"namespace":"default","name":"testing-md-0-7d45d88d47-tzf7c","reconcileID":"026048d9-1e54-43fc-8413-fd2aa5e78860","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"},"server":30749989}
{"level":"INFO","time":"2023-04-03T12:23:32.339Z","file":"controllers/hcloudmachine_controller.go:181","message":"Reconciling HCloudMachine","controller":"hcloudmachine","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudMachine","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"namespace":"default","name":"testing-md-0-hjztf","reconcileID":"7e3a1a1d-d2e9-4a61-a9ae-2fe706eba2f8","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"}}
{"level":"INFO","time":"2023-04-03T12:23:32.686Z","file":"controllers/hcloudremediation_controller.go:175","message":"Reconciling HCloudRemediation","controller":"hcloudremediation","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudRemediation","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"namespace":"default","name":"testing-md-0-7d45d88d47-tzf7c","reconcileID":"818a3175-1c51-4430-8ce2-1ba46c8f5a5a","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"}}
{"level":"INFO","time":"2023-04-03T12:23:32.686Z","file":"remediation/remediation.go:49","message":"Reconciling hcloud remediation","controller":"hcloudremediation","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudRemediation","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"namespace":"default","name":"testing-md-0-7d45d88d47-tzf7c","reconcileID":"818a3175-1c51-4430-8ce2-1ba46c8f5a5a","HCloudRemediation":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"},"name":"testing-md-0-7d45d88d47-tzf7c"}
{"level":"INFO","time":"2023-04-03T12:23:44.319Z","file":"controllers/hcloudmachine_controller.go:181","message":"Reconciling HCloudMachine","controller":"hcloudmachine","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudMachine","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"namespace":"default","name":"testing-md-0-hjztf","reconcileID":"01bc55d2-adbb-4719-8346-5389b49fd1ff","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"}}
{"level":"INFO","time":"2023-04-03T12:23:44.503Z","file":"controllers/hcloudmachine_controller.go:181","message":"Reconciling HCloudMachine","controller":"hcloudmachine","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudMachine","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"namespace":"default","name":"testing-md-0-hjztf","reconcileID":"7f9ea85b-ff17-4ac2-83f2-d01005488d74","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"}}
{"level":"INFO","time":"2023-04-03T12:23:54.805Z","file":"controllers/hcloudmachine_controller.go:181","message":"Reconciling HCloudMachine","controller":"hcloudmachine","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudMachine","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"namespace":"default","name":"testing-md-0-hjztf","reconcileID":"d50e9fb4-a692-441a-beff-5544e909f192","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"}}
{"level":"INFO","time":"2023-04-03T12:23:54.991Z","file":"controllers/hcloudmachine_controller.go:181","message":"Reconciling HCloudMachine","controller":"hcloudmachine","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudMachine","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"namespace":"default","name":"testing-md-0-hjztf","reconcileID":"4e81a8cb-2f34-413a-8558-f07cceffd7ab","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"}}
{"level":"INFO","time":"2023-04-03T12:23:55.142Z","file":"controllers/hcloudmachine_controller.go:181","message":"Reconciling HCloudMachine","controller":"hcloudmachine","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HCloudMachine","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"namespace":"default","name":"testing-md-0-hjztf","reconcileID":"a58b9b9c-2b33-4f6d-9ed2-ce439c0876e1","HCloudMachine":{"name":"testing-md-0-hjztf","namespace":"default"},"Machine":{"name":"testing-md-0-7d45d88d47-tzf7c","namespace":"default"},"Cluster":{"name":"testing","namespace":"default"},"HetznerCluster":{"name":"testing","namespace":"default"}}

only these lines don't contain hjztf:

{"level":"DEBUG","time":"2023-04-03T12:23:22.007Z","logger":"controller-runtime.webhook.webhooks","file":"admission/http.go:96","message":"received request","webhook":"/mutate-infrastructure-cluster-x-k8s-io-v1beta1-hcloudremediation","UID":"15ecec64-39cf-4a4c-8cfa-6734cd074a6e","kind":"infrastructure.cluster.x-k8s.io/v1beta1, Kind=HCloudRemediation","resource":{"group":"infrastructure.cluster.x-k8s.io","version":"v1beta1","resource":"hcloudremediations"}}
{"level":"DEBUG","time":"2023-04-03T12:23:22.007Z","logger":"controller-runtime.webhook.webhooks","file":"admission/http.go:143","message":"wrote response","webhook":"/mutate-infrastructure-cluster-x-k8s-io-v1beta1-hcloudremediation","code":200,"reason":"","UID":"15ecec64-39cf-4a4c-8cfa-6734cd074a6e","allowed":true}
{"level":"DEBUG","time":"2023-04-03T12:23:22.010Z","logger":"controller-runtime.webhook.webhooks","file":"admission/http.go:96","message":"received request","webhook":"/validate-infrastructure-cluster-x-k8s-io-v1beta1-hcloudremediation","UID":"55bd863b-233f-4613-b47e-25f36d1d79dd","kind":"infrastructure.cluster.x-k8s.io/v1beta1, Kind=HCloudRemediation","resource":{"group":"infrastructure.cluster.x-k8s.io","version":"v1beta1","resource":"hcloudremediations"}}
{"level":"DEBUG","time":"2023-04-03T12:23:22.010Z","logger":"controller-runtime.webhook.webhooks","file":"admission/http.go:143","message":"wrote response","webhook":"/validate-infrastructure-cluster-x-k8s-io-v1beta1-hcloudremediation","code":200,"reason":"","UID":"55bd863b-233f-4613-b47e-25f36d1d79dd","allowed":true}

Conclusion: The logging is ok. All relevant lines contain the node-name

change was implemented.