Update AWS EKS terraform jinja templates to so we can use k8s 1.24

[consideRatio]

I wrote this issue quickly as I think it should be blocking #2017. I can get it done myself I think.

Background

In #2044 deploying a AWS EKS hub I did some changes related to "EBS" so that we could deploy k8s 1.24 instead of be stuck at k8s 1.22.

I did not do those changes to a eksctl related jinja template though, so it needs to be re-done for future hubs we setup. So, I'm creating this issue to track the need to update the terraform template now that we can conclude that the changes I made did the trick.

Related

k8s version should be 1.24 (latest supported by eksctl atm I think)

infrastructure/eksctl/template.jsonnet

Lines 35 to 47 in ad9179c

	kind: 'ClusterConfig',
	metadata+: {
	name: "<< cluster_name >>",
	region: clusterRegion,
	// Warning: version 1.23 introduces some breaking changes
	// Checkout the docs before upgrading
	// ref: https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi-migration-faq.html
	version: '1.22'
	},
	availabilityZones: masterAzs,
	iam: {
	withOIDC: true,
	},

We need the EBS addon + iam permissions for nodes

Without this, what happens is that PVC (request for storage) is not given storage, and pods that wants to mount this storages doesn't get scheduled.

infrastructure/eksctl/ubc-eoas.jsonnet

Lines 45 to 52 in ad9179c

	addons: [
	{
	name: 'aws-ebs-csi-driver',
	wellKnownPolicies: {
	ebsCSIController: true,
	},
	},
	],

infrastructure/eksctl/ubc-eoas.jsonnet

Lines 67 to 71 in ad9179c

	iam+: {
	withAddonPolicies+: {
	ebs: true,
	},
	},