Add new AWS podIdentity (#5061)

* Add new AWS podIdentity Co-authored-by: Sami S <[email protected]> Signed-off-by: Jorge Turrado <[email protected]> * use new resources Signed-off-by: Jorge Turrado <[email protected]> * fix logs Signed-off-by: Jorge Turrado <[email protected]> * rename scalerIndex to triggerIndex and ScalerUniqueKey to TriggerUniqueKey Signed-off-by: Jorge Turrado <[email protected]> * add missing headers Signed-off-by: Jorge Turrado <[email protected]> * propagate ctx Signed-off-by: Jorge Turrado <[email protected]> * Add documentation commments Signed-off-by: Jorge Turrado <[email protected]> * Add explanation to new files Signed-off-by: Jorge Turrado <[email protected]> * Add semgrep exclusion Signed-off-by: Jorge Turrado <[email protected]> * go mod updates Signed-off-by: Jorge Turrado <[email protected]> * update comment Signed-off-by: Jorge Turrado <[email protected]> * update clientset Signed-off-by: Jorge Turrado <[email protected]> * apply feedback Signed-off-by: Jorge Turrado <[email protected]> --------- Signed-off-by: Jorge Turrado <[email protected]> Signed-off-by: Jorge Turrado <[email protected]> Signed-off-by: Jorge Turrado Ferrero <[email protected]> Co-authored-by: Sami S <[email protected]>
kedacore · Jan 9, 2024 · c132117 · c132117
1 parent 3118fbc
commit c132117
Show file tree

Hide file tree

Showing 193 changed files with 3,065 additions and 795 deletions.
diff --git a/.github/workflows/pr-e2e.yml b/.github/workflows/pr-e2e.yml
@@ -22,7 +22,7 @@ jobs:
         id: checkUserMember
         with:
           username: ${{ github.actor }}
-          team: 'keda-e2e-test-executors'
+          team: "keda-e2e-test-executors"
           GITHUB_TOKEN: ${{ secrets.GH_CHECKING_USER_AUTH }}
 
       - name: Update comment with the execution url
@@ -221,5 +221,5 @@ jobs:
         uses: actions/upload-artifact@c7d193f32edcb7bfad88892161225aeda64e9392 # v4
         with:
           name: e2e-test-logs
-          path: '${{ github.workspace }}/tests/**/*.log'
+          path: "${{ github.workspace }}/**/*.log"
           if-no-files-found: ignore
diff --git a/.github/workflows/template-main-e2e-test.yml b/.github/workflows/template-main-e2e-test.yml
@@ -51,5 +51,5 @@ jobs:
         if: ${{ always() }}
         with:
           name: e2e-test-logs
-          path: '${{ github.workspace }}/tests/**/*.log'
+          path: "${{ github.workspace }}/**/*.log"
           if-no-files-found: ignore
diff --git a/.github/workflows/template-smoke-tests.yml b/.github/workflows/template-smoke-tests.yml
@@ -48,5 +48,5 @@ jobs:
         if: ${{ always() }}
         with:
           name: smoke-test-logs ${{ inputs.runs-on }}-${{ inputs.kubernetesVersion }}
-          path: "${{ github.workspace }}/tests/**/*.log"
+          path: "${{ github.workspace }}/**/*.log"
           if-no-files-found: ignore
diff --git a/.gitignore b/.gitignore
@@ -45,3 +45,6 @@ __debug_bin
 
 # GO Test result
 report.xml
+
+# KEDA Certs
+certs/*
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -51,7 +51,7 @@ To learn more about active deprecations, we recommend checking [GitHub Discussio
 
 ### New
 
-- **General**: TODO ([#XXX](https://github.com/kedacore/keda/issues/XXX))
+- **General**: Introduce new AWS Authentication ([#4134](https://github.com/kedacore/keda/issues/4134))
 
 #### Experimental
 
@@ -65,7 +65,9 @@ Here is an overview of all new **experimental** features:
 - **General**: Add parameter queryParameters to prometheus-scaler ([#4962](https://github.com/kedacore/keda/issues/4962))
 - **General**: Add validations for replica counts when creating ScaledObjects ([#5288](https://github.com/kedacore/keda/issues/5288))
 - **General**: Bubble up AuthRef TriggerAuthentication errors as ScaledObject events ([#5190](https://github.com/kedacore/keda/issues/5190))
+- **General**: Enhance podIdentity Role Assumption in AWS by Direct Integration with OIDC/Federation ([#5178](https://github.com/kedacore/keda/issues/5178))
 - **General**: Fix issue where paused annotation being set to false still leads to scaled objects/jobs being paused ([#5215](https://github.com/kedacore/keda/issues/5215))
+- **General**: Implement Credentials Cache for AWS Roles to reduce AWS API calls ([#5297](https://github.com/kedacore/keda/issues/5297))
 - **General**: Support TriggerAuthentication properties from ConfigMap ([#4830](https://github.com/kedacore/keda/issues/4830))
 - **General**: Use client-side round-robin load balancing for grpc calls ([#5224](https://github.com/kedacore/keda/issues/5224))
 - **GCP pubsub scaler**: Support distribution-valued metrics and metrics from topics ([#5070](https://github.com/kedacore/keda/issues/5070))

diff --git a/CREATE-NEW-SCALER.md b/CREATE-NEW-SCALER.md
@@ -65,18 +65,18 @@ The return type of this function is `MetricSpec`, but in KEDA's case we will mos
 - `TargetValue`: is the value of the metric we want to reach at all times at all costs. As long as the current metric doesn't match TargetValue, HPA will increase the number of the pods until it reaches the maximum number of pods allowed to scale to.
 - `TargetAverageValue`: the value of the metric for which we require one pod to handle. e.g. if we have a scaler based on the length of a message queue, and we specificy 10 for `TargetAverageValue`, we are saying that each pod will handle 10 messages. So if the length of the queue becomes 30, we expect that we have 3 pods in our cluster. (`TargetAverage` and `TargetValue` are mutually exclusive).
 
-All scalers receive a parameter named `scalerIndex` as part of `ScalerConfig`. This value is the index of the current scaler in a ScaledObject. All metric names have to start with `sX-` (where `X` is `scalerIndex`). This convention makes the metric name unique in the ScaledObject and brings the option to have more than 1 "similar metric name" defined in a ScaledObject.
+All scalers receive a parameter named `triggerIndex` as part of `ScalerConfig`. This value is the index of the current scaler in a ScaledObject. All metric names have to start with `sX-` (where `X` is `triggerIndex`). This convention makes the metric name unique in the ScaledObject and brings the option to have more than 1 "similar metric name" defined in a ScaledObject.
 
 For example:
 - s0-redis-mylist
 - s1-redis-mylist
 
->**Note:** There is a naming helper function `GenerateMetricNameWithIndex(scalerIndex int, metricName string)`, that receives the current index and the original metric name (without the prefix) and returns the concatenated string using the convention (please use this function).<br>Next lines are an example about how to use it:
+>**Note:** There is a naming helper function `GenerateMetricNameWithIndex(triggerIndex int, metricName string)`, that receives the current index and the original metric name (without the prefix) and returns the concatenated string using the convention (please use this function).<br>Next lines are an example about how to use it:
 >```golang
 >func (s *artemisScaler) GetMetricSpecForScaling() []v2.MetricSpec {
 >	externalMetric := &v2.ExternalMetricSource{
 >		Metric: v2.MetricIdentifier{
->			Name: GenerateMetricNameWithIndex(s.metadata.scalerIndex, kedautil.NormalizeString(fmt.Sprintf("%s-%s-%s", "artemis", s.metadata.brokerName, s.metadata.queueName))),
+>			Name: GenerateMetricNameWithIndex(s.metadata.triggerIndex, kedautil.NormalizeString(fmt.Sprintf("%s-%s-%s", "artemis", s.metadata.brokerName, s.metadata.queueName))),
 >		},
 >		Target: GetMetricTarget(s.metricType, s.metadata.queueLength),
 >	}

diff --git a/Makefile b/Makefile
@@ -281,7 +281,7 @@ deploy: install ## Deploy controller to the K8s cluster specified in ~/.kube/con
 	fi
 	if [ "$(AWS_RUN_IDENTITY_TESTS)" = true ]; then \
 		cd config/service_account && \
-		$(KUSTOMIZE) edit add annotation --force eks.amazonaws.com/role-arn:arn:aws:iam::${TF_AWS_ACCOUNT_ID}:role/${TEST_CLUSTER_NAME}-role; \
+		$(KUSTOMIZE) edit add annotation --force eks.amazonaws.com/role-arn:${TF_AWS_KEDA_ROLE}; \
 	fi
 	if [ "$(GCP_RUN_IDENTITY_TESTS)" = true ]; then \
 		cd config/service_account && \

diff --git a/apis/keda/v1alpha1/triggerauthentication_types.go b/apis/keda/v1alpha1/triggerauthentication_types.go
@@ -118,9 +118,9 @@ const (
 	PodIdentityProviderAzure         PodIdentityProvider = "azure"
 	PodIdentityProviderAzureWorkload PodIdentityProvider = "azure-workload"
 	PodIdentityProviderGCP           PodIdentityProvider = "gcp"
-	PodIdentityProviderSpiffe        PodIdentityProvider = "spiffe"
 	PodIdentityProviderAwsEKS        PodIdentityProvider = "aws-eks"
 	PodIdentityProviderAwsKiam       PodIdentityProvider = "aws-kiam"
+	PodIdentityProviderAws           PodIdentityProvider = "aws"
 )
 
 // PodIdentityAnnotationEKS specifies aws role arn for aws-eks Identity Provider
@@ -133,9 +133,17 @@ const (
 // AuthPodIdentity allows users to select the platform native identity
 // mechanism
 type AuthPodIdentity struct {
+	// +kubebuilder:validation:Enum=azure;azure-workload;gcp;aws;aws-eks;aws-kiam
 	Provider PodIdentityProvider `json:"provider"`
 	// +optional
 	IdentityID *string `json:"identityId"`
+	// +optional
+	// RoleArn sets the AWS RoleArn to be used. Mutually exclusive with IdentityOwner
+	RoleArn string `json:"roleArn"`
+	// +kubebuilder:validation:Enum=keda;workload
+	// +optional
+	// IdentityOwner configures which identity has to be used during auto discovery, keda or the scaled workload. Mutually exclusive with roleArn
+	IdentityOwner *string `json:"identityOwner"`
 }
 
 func (a *AuthPodIdentity) GetIdentityID() string {
@@ -145,6 +153,13 @@ func (a *AuthPodIdentity) GetIdentityID() string {
 	return *a.IdentityID
 }
 
+func (a *AuthPodIdentity) IsWorkloadIdentityOwner() bool {
+	if a.IdentityOwner == nil {
+		return false
+	}
+	return *a.IdentityOwner == workloadString
+}
+
 // AuthConfigMapTargetRef is used to authenticate using a reference to a config map
 type AuthConfigMapTargetRef AuthTargetRef
 

diff --git a/apis/keda/v1alpha1/triggerauthentication_webhook.go b/apis/keda/v1alpha1/triggerauthentication_webhook.go
@@ -28,6 +28,11 @@ import (
 	"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
 )
 
+const (
+	kedaString     = "keda"
+	workloadString = "workload"
+)
+
 var triggerauthenticationlog = logf.Log.WithName("triggerauthentication-validation-webhook")
 
 func (ta *TriggerAuthentication) SetupWebhookWithManager(mgr ctrl.Manager) error {
@@ -113,6 +118,10 @@ func validateSpec(spec *TriggerAuthenticationSpec) (admission.Warnings, error) {
 			if spec.PodIdentity.IdentityID != nil && *spec.PodIdentity.IdentityID == "" {
 				return nil, fmt.Errorf("identityid of PodIdentity should not be empty. If it's set, identityId has to be different than \"\"")
 			}
+		case PodIdentityProviderAws:
+			if spec.PodIdentity.RoleArn != "" && spec.PodIdentity.IsWorkloadIdentityOwner() {
+				return nil, fmt.Errorf("roleArn of PodIdentity can't be set if KEDA isn't identityOwner")
+			}
 		default:
 			return nil, nil
 		}

diff --git a/apis/keda/v1alpha1/triggerauthentication_webhook_test.go b/apis/keda/v1alpha1/triggerauthentication_webhook_test.go
@@ -24,13 +24,13 @@ import (
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )
 
-var _ = It("validate triggerauthentication when IdentityID is nil", func() {
+var _ = It("validate triggerauthentication when IdentityID is nil, roleArn is empty and identityOwner is nil", func() {
 	namespaceName := "nilidentityid"
 	namespace := createNamespace(namespaceName)
 	err := k8sClient.Create(context.Background(), namespace)
 	Expect(err).ToNot(HaveOccurred())
 
-	spec := createTriggerAuthenticationSpecWithPodIdentity(nil)
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAzure, "", nil, nil)
 	ta := createTriggerAuthentication("nilidentityidta", namespaceName, "TriggerAuthentication", spec)
 	Eventually(func() error {
 		return k8sClient.Create(context.Background(), ta)
@@ -44,7 +44,7 @@ var _ = It("validate triggerauthentication when IdentityID is empty", func() {
 	Expect(err).ToNot(HaveOccurred())
 
 	identityID := ""
-	spec := createTriggerAuthenticationSpecWithPodIdentity(&identityID)
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAzure, "", &identityID, nil)
 	ta := createTriggerAuthentication("emptyidentityidta", namespaceName, "TriggerAuthentication", spec)
 	Eventually(func() error {
 		return k8sClient.Create(context.Background(), ta)
@@ -58,7 +58,76 @@ var _ = It("validate triggerauthentication when IdentityID is not empty", func()
 	Expect(err).ToNot(HaveOccurred())
 
 	identityID := "12345"
-	spec := createTriggerAuthenticationSpecWithPodIdentity(&identityID)
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAzure, "", &identityID, nil)
+	ta := createTriggerAuthentication("identityidta", namespaceName, "TriggerAuthentication", spec)
+	Eventually(func() error {
+		return k8sClient.Create(context.Background(), ta)
+	}).ShouldNot(HaveOccurred())
+})
+
+var _ = It("validate triggerauthentication when RoleArn is not empty and IdentityOwner is nil", func() {
+	namespaceName := "rolearn"
+	namespace := createNamespace(namespaceName)
+	err := k8sClient.Create(context.Background(), namespace)
+	Expect(err).ToNot(HaveOccurred())
+
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAws, "Helo", nil, nil)
+	ta := createTriggerAuthentication("identityidta", namespaceName, "TriggerAuthentication", spec)
+	Eventually(func() error {
+		return k8sClient.Create(context.Background(), ta)
+	}).ShouldNot(HaveOccurred())
+})
+
+var _ = It("validate triggerauthentication when RoleArn is not empty and IdentityOwner is keda", func() {
+	namespaceName := "rolearnandkedaowner"
+	namespace := createNamespace(namespaceName)
+	err := k8sClient.Create(context.Background(), namespace)
+	Expect(err).ToNot(HaveOccurred())
+
+	identityOwner := kedaString
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAws, "Helo", nil, &identityOwner)
+	ta := createTriggerAuthentication("identityidta", namespaceName, "TriggerAuthentication", spec)
+	Eventually(func() error {
+		return k8sClient.Create(context.Background(), ta)
+	}).ShouldNot(HaveOccurred())
+})
+
+var _ = It("validate triggerauthentication when RoleArn is not empty and IdentityOwner is workload", func() {
+	namespaceName := "rolearnandworkloadowner"
+	namespace := createNamespace(namespaceName)
+	err := k8sClient.Create(context.Background(), namespace)
+	Expect(err).ToNot(HaveOccurred())
+
+	identityOwner := workloadString
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAws, "Helo", nil, &identityOwner)
+	ta := createTriggerAuthentication("identityidta", namespaceName, "TriggerAuthentication", spec)
+	Eventually(func() error {
+		return k8sClient.Create(context.Background(), ta)
+	}).Should(HaveOccurred())
+})
+
+var _ = It("validate triggerauthentication when RoleArn is empty and IdentityOwner is keda", func() {
+	namespaceName := "kedaowner"
+	namespace := createNamespace(namespaceName)
+	err := k8sClient.Create(context.Background(), namespace)
+	Expect(err).ToNot(HaveOccurred())
+
+	identityOwner := kedaString
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAws, "", nil, &identityOwner)
+	ta := createTriggerAuthentication("identityidta", namespaceName, "TriggerAuthentication", spec)
+	Eventually(func() error {
+		return k8sClient.Create(context.Background(), ta)
+	}).ShouldNot(HaveOccurred())
+})
+
+var _ = It("validate triggerauthentication when RoleArn is not empty and IdentityOwner is workload", func() {
+	namespaceName := "workloadowner"
+	namespace := createNamespace(namespaceName)
+	err := k8sClient.Create(context.Background(), namespace)
+	Expect(err).ToNot(HaveOccurred())
+
+	identityOwner := workloadString
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAws, "", nil, &identityOwner)
 	ta := createTriggerAuthentication("identityidta", namespaceName, "TriggerAuthentication", spec)
 	Eventually(func() error {
 		return k8sClient.Create(context.Background(), ta)
@@ -71,7 +140,7 @@ var _ = It("validate clustertriggerauthentication when IdentityID is nil", func(
 	err := k8sClient.Create(context.Background(), namespace)
 	Expect(err).ToNot(HaveOccurred())
 
-	spec := createTriggerAuthenticationSpecWithPodIdentity(nil)
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAzure, "", nil, nil)
 	ta := createTriggerAuthentication("clusternilidentityidta", namespaceName, "ClusterTriggerAuthentication", spec)
 	Eventually(func() error {
 		return k8sClient.Create(context.Background(), ta)
@@ -85,7 +154,7 @@ var _ = It("validate clustertriggerauthentication when IdentityID is empty", fun
 	Expect(err).ToNot(HaveOccurred())
 
 	identityID := ""
-	spec := createTriggerAuthenticationSpecWithPodIdentity(&identityID)
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAzure, "", &identityID, nil)
 	ta := createTriggerAuthentication("clusteremptyidentityidta", namespaceName, "ClusterTriggerAuthentication", spec)
 	Eventually(func() error {
 		return k8sClient.Create(context.Background(), ta)
@@ -99,18 +168,89 @@ var _ = It("validate clustertriggerauthentication when IdentityID is not empty",
 	Expect(err).ToNot(HaveOccurred())
 
 	identityID := "12345"
-	spec := createTriggerAuthenticationSpecWithPodIdentity(&identityID)
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAzure, "", &identityID, nil)
 	ta := createTriggerAuthentication("clusteridentityidta", namespaceName, "ClusterTriggerAuthentication", spec)
 	Eventually(func() error {
 		return k8sClient.Create(context.Background(), ta)
 	}).ShouldNot(HaveOccurred())
 })
 
-func createTriggerAuthenticationSpecWithPodIdentity(identityID *string) TriggerAuthenticationSpec {
+var _ = It("validate clustertriggerauthentication when RoleArn is not empty and IdentityOwner is nil", func() {
+	namespaceName := "clusterrolearn"
+	namespace := createNamespace(namespaceName)
+	err := k8sClient.Create(context.Background(), namespace)
+	Expect(err).ToNot(HaveOccurred())
+
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAws, "Helo", nil, nil)
+	ta := createTriggerAuthentication("clusteridentityidta", namespaceName, "ClusterTriggerAuthentication", spec)
+	Eventually(func() error {
+		return k8sClient.Create(context.Background(), ta)
+	}).ShouldNot(HaveOccurred())
+})
+
+var _ = It("validate clustertriggerauthentication when RoleArn is not empty and IdentityOwner is keda", func() {
+	namespaceName := "clusterrolearnandkedaowner"
+	namespace := createNamespace(namespaceName)
+	err := k8sClient.Create(context.Background(), namespace)
+	Expect(err).ToNot(HaveOccurred())
+
+	identityOwner := kedaString
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAws, "Helo", nil, &identityOwner)
+	ta := createTriggerAuthentication("clusteridentityidta", namespaceName, "ClusterTriggerAuthentication", spec)
+	Eventually(func() error {
+		return k8sClient.Create(context.Background(), ta)
+	}).ShouldNot(HaveOccurred())
+})
+
+var _ = It("validate clustertriggerauthentication when RoleArn is not empty and IdentityOwner is workload", func() {
+	namespaceName := "clusterrolearnandworkloadowner"
+	namespace := createNamespace(namespaceName)
+	err := k8sClient.Create(context.Background(), namespace)
+	Expect(err).ToNot(HaveOccurred())
+
+	identityOwner := workloadString
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAws, "Helo", nil, &identityOwner)
+	ta := createTriggerAuthentication("clusteridentityidta", namespaceName, "ClusterTriggerAuthentication", spec)
+	Eventually(func() error {
+		return k8sClient.Create(context.Background(), ta)
+	}).Should(HaveOccurred())
+})
+
+var _ = It("validate clustertriggerauthentication when RoleArn is empty and IdentityOwner is keda", func() {
+	namespaceName := "clusterandkedaowner"
+	namespace := createNamespace(namespaceName)
+	err := k8sClient.Create(context.Background(), namespace)
+	Expect(err).ToNot(HaveOccurred())
+
+	identityOwner := kedaString
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAws, "", nil, &identityOwner)
+	ta := createTriggerAuthentication("clusteridentityidta", namespaceName, "ClusterTriggerAuthentication", spec)
+	Eventually(func() error {
+		return k8sClient.Create(context.Background(), ta)
+	}).ShouldNot(HaveOccurred())
+})
+
+var _ = It("validate clustertriggerauthentication when RoleArn is not empty and IdentityOwner is workload", func() {
+	namespaceName := "clusterandworkloadowner"
+	namespace := createNamespace(namespaceName)
+	err := k8sClient.Create(context.Background(), namespace)
+	Expect(err).ToNot(HaveOccurred())
+
+	identityOwner := workloadString
+	spec := createTriggerAuthenticationSpecWithPodIdentity(PodIdentityProviderAws, "", nil, &identityOwner)
+	ta := createTriggerAuthentication("clusteridentityidta", namespaceName, "TriggerAuthentication", spec)
+	Eventually(func() error {
+		return k8sClient.Create(context.Background(), ta)
+	}).ShouldNot(HaveOccurred())
+})
+
+func createTriggerAuthenticationSpecWithPodIdentity(provider PodIdentityProvider, roleArn string, identityID, identityOwner *string) TriggerAuthenticationSpec {
 	return TriggerAuthenticationSpec{
 		PodIdentity: &AuthPodIdentity{
-			Provider:   PodIdentityProviderAzure,
-			IdentityID: identityID,
+			Provider:      provider,
+			IdentityID:    identityID,
+			RoleArn:       roleArn,
+			IdentityOwner: identityOwner,
 		},
 	}
 }

diff --git a/apis/keda/v1alpha1/zz_generated.deepcopy.go b/apis/keda/v1alpha1/zz_generated.deepcopy.go