Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Karpenter v1beta1 compatibility #2800

Merged

Conversation

mbarrien
Copy link
Contributor

Description

Add IAM support for Karpenter v1beta1 resources.

Motivation and Context

The recent Karpenter v0.32 release added support for v1beta1 resources, as described in https://github.com/aws/karpenter/blob/main/designs/v1beta1-full-changelist.md. As part of this change, new permissions are needed such as the ability to create instance profiles.

The new controller IAM policy is described in https://github.com/aws/karpenter/blob/main/website/content/en/preview/upgrading/v1beta1-controller-policy.json

This PR adds a minimal set of changes for v0.32 compatibility, specifically around instance profiles. This gives the IRSA role the ability to create any instance profile in the account, without attempting to scope it down based on tags as aws/karpenter-provider-aws#3948 would have done.

Breaking Changes

No breaking changes, only adds permissions.

How Has This Been Tested?

  • I have updated at least one of the examples/* to demonstrate and validate my change(s)
  • I have tested and validated these changes using one or more of the provided examples/* projects
  • I have executed pre-commit run -a on my pull request

@bryantbiggs
Copy link
Member

I still need to re-test, I was getting some errors

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x1f1f97c]

goroutine 492 [running]:
github.com/aws/karpenter/pkg/cloudprovider.(*CloudProvider).resolveNodeClassFromNodePool(0xc00086a370, {0x308db90, 0xc001042b10}, 0xc0017541c0)
	github.com/aws/karpenter/pkg/cloudprovider/cloudprovider.go:269 +0x19c
github.com/aws/karpenter/pkg/cloudprovider.(*CloudProvider).GetInstanceTypes(0xc00086a370, {0x308db90, 0xc001042b10}, 0xc0017541c0)
	github.com/aws/karpenter/pkg/cloudprovider/cloudprovider.go:175 +0x4b
github.com/aws/karpenter-core/pkg/cloudprovider/metrics.(*decorator).GetInstanceTypes(0xc00047abe0, {0x308db90, 0xc001042b10}, 0xc00065a6c0?)
	github.com/aws/[email protected]/pkg/cloudprovider/metrics/cloudprovider.go:139 +0x15c
github.com/aws/karpenter-core/pkg/controllers/disruption.buildNodePoolMap({0x308db90, 0xc001042b10}, {0x309d980, 0xc00065a6c0}, {0x3094388, 0xc00047abe0})
	github.com/aws/[email protected]/pkg/controllers/disruption/helpers.go:203 +0x1fa
github.com/aws/karpenter-core/pkg/controllers/disruption.GetCandidates({0x308db90, 0xc001042b10}, 0x0?, {0x309d980, 0xc00065a6c0}, {0x30612a0, 0xc0006254b8}, {0x3091a68, 0x47a3ca0}, {0x3094388, ...}, ...)
	github.com/aws/[email protected]/pkg/controllers/disruption/helpers.go:178 +0x7a
github.com/aws/karpenter-core/pkg/controllers/disruption.(*Controller).disrupt(0xc000492f00, {0x308db90, 0xc001042b10}, {0x308e9b8?, 0xc000a3aa40?})
	github.com/aws/[email protected]/pkg/controllers/disruption/controller.go:156 +0x3c5
github.com/aws/karpenter-core/pkg/controllers/disruption.(*Controller).Reconcile(0xc000492f00, {0x308db90, 0xc001042b10}, {{{0xc0006a2c00?, 0xc0012aeeec?}, {0x2?, 0xc0006a2c00?}}})
	github.com/aws/[email protected]/pkg/controllers/disruption/controller.go:137 +0x36c
github.com/aws/karpenter-core/pkg/operator/controller.(*Singleton).reconcile(0xc000666b00, {0x308db90, 0xc001042b10})
	github.com/aws/[email protected]/pkg/operator/controller/singleton.go:98 +0x31c
github.com/aws/karpenter-core/pkg/operator/controller.(*Singleton).Start(0xc000666b00, {0x308dbc8, 0xc0007bf860})
	github.com/aws/[email protected]/pkg/operator/controller/singleton.go:86 +0x1f0
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1(0xc000666ce0)
	sigs.k8s.io/[email protected]/pkg/manager/runnable_group.go:223 +0xc8
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile in goroutine 488
	sigs.k8s.io/[email protected]/pkg/manager/runnable_group.go:207 +0x19d

@bryantbiggs bryantbiggs merged commit aec2bab into terraform-aws-modules:master Nov 1, 2023
17 checks passed
antonbabenko pushed a commit that referenced this pull request Nov 1, 2023
## [19.18.0](v19.17.4...v19.18.0) (2023-11-01)

### Features

* Add Karpenter v1beta1 compatibility ([#2800](#2800)) ([aec2bab](aec2bab))
@antonbabenko
Copy link
Member

This PR is included in version 19.18.0 🎉

@mbarrien mbarrien deleted the karpenter-v1beta1-minimal branch November 1, 2023 18:55
@tl-alex-nicot
Copy link

is this the module where the karpenter policy will now be managed and if so does that mean the karpenter-controller policy in iam-role-for-service-accounts-eks is being deprecated ?

Copy link

github-actions bot commented Dec 3, 2023

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants