diff --git a/enhancements/installer/aws-cross-account-dns-zone.md b/enhancements/installer/aws-cross-account-dns-zone.md index 1a63f5402e3..2de37e65c53 100644 --- a/enhancements/installer/aws-cross-account-dns-zone.md +++ b/enhancements/installer/aws-cross-account-dns-zone.md @@ -14,7 +14,7 @@ api-approvers: - "@deads2k" - "@JoelSpeed" creation-date: 2023-05-08 -last-updated: 2023-05-08 +last-updated: 2023-07-17 tracking-link: # link to the tracking ticket (for example: Jira Feature or Epic ticket) that corresponds to this enhancement - https://issues.redhat.com/browse/CORS-2613 - https://issues.redhat.com/browse/OCPBU-558 @@ -139,8 +139,6 @@ type AWSDNSSpec struct { ``` -The name of the field is discussed further in open questions. - #### Install Config `platform.aws.hostedZoneRole` would be added to complement the existing `platform.aws.hostedZone` field. @@ -162,9 +160,55 @@ platform: hostedZone: Z00147933I3NWOQ6M4699 hostedZoneRole: arn:aws:iam:::role/ ``` -Please see Open Questions for further discussion of the install config. +### API: ExternalDNS Operator + +[ExternalDNS Operator](https://github.com/openshift/external-dns-operator) allows you to deploy and manage [ExternalDNS](https://github.com/kubernetes-sigs/external-dns), +a cluster-internal component which makes Kubernetes resources discoverable through public DNS servers. This enhancement +extends the ExternalDNS Operator to support cross-account DNS zones in AWS. We extend the`ExternalDNS` API object by adding +the field `RoleARN` to `ExternalDNSAWSProviderOptions` struct (`spec.provider.aws`). +Refer to [`externaldns_types.go`](https://github.com/openshift/external-dns-operator/blob/main/api/v1beta1/externaldns_types.go) +for the existing API structure of the `ExternalDNS` object. + +In addition to adding `RoleARN`, the `Credentials` field was updated to be optional. Prior to this change, the `Credentials` +field was required if `spec.provider.aws` was specified. However, OpenShift clusters are exempt from explicitly +providing `Credentials` since the ExternalDNS Operator generates a `CredentialRequest` automatically. Consequently, to +enable OpenShift clusters to specify `RoleARN` without `Credentials`, it is necessary to make the `Credentials` field +optional. The validation webhook takes care of guaranteeing that non-OpenShift clusters include credentials through the +API. + +The ExternalDNS Operator's validation webhook will use the existing [IsARN](https://pkg.go.dev/github.com/aws/aws-sdk-go/service/s3/internal/arn#IsARN) +function to validate the `RoleARN` field. The approach mentioned above differs from the API design documented in the +[API: DNS](#API-DNS) section, which employs CRD validation utilizing a regular expression. Using the existing `IsARN` +function provides more flexibility and reliability since it is maintained by AWS themselves, which helps to mitigate +potential issues with edge cases. +```go +type ExternalDNSAWSProviderOptions struct { + // Credentials is a reference to a secret containing + // the following keys (with corresponding values): + // + // * aws_access_key_id + // * aws_secret_access_key + // + // See + // https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/aws.md + // for more information. + // + // +kubebuilder:validation:Optional + // +optional + Credentials SecretReference `json:"credentials"` + + // RoleARN contains the ARN of a IAM role that will be assumed when using the AWS API. + // It provides the ability to use a hosted zone in another AWS account. + // + // +kubebuilder:validation:Optional + // +optional + RoleARN *string `json:"roleARN,omitempty"` +} +``` + +See [ExternalDNS Operator Details](#External-DNS-Operator-Details) for more implementation details. ### Implementation Details/Notes/Constraints [optional] @@ -218,9 +262,30 @@ An example IAM Trust Policy: ``` +#### ExternalDNS Operator Details + +Prior to this enhancement, the [ExternalDNS Operator API](https://github.com/openshift/external-dns-operator/blob/main/api) +did not allow users to configure an AWS role ARN field. As a result, there wasn't a supported way to use the External +DNS Operator to create DNS records in another AWS account within a shared VPC. + +The ExternalDNS Operator configures and runs the [ExternalDNS](https://github.com/openshift/external-dns) binary. +Conveniently, ExternalDNS already supports the [`--aws-assume-role`](https://github.com/openshift/external-dns/blob/fe00b4b83c2263282a9068655e8e3fbbc167b653/docs/faq.md#can-external-dns-manageaddremove-records-in-a-hosted-zone-which-is-setup-in-different-aws-account) +argument, which uses the specified AWS role ARN when creating new DNS records. Therefore, to support shared VPCs with +the ExternalDNS Operator, we update the [API](#API-External-DNS-Operator) to expose the `--aws-assume-role` argument +for the ExternalDNS binary. + +By design, ExternalDNS can update both public and private DNS zones using credentials that the cluster-admin provides. +Therefore, ExternalDNS Operator users are able to use the new [API](#API-External-DNS-Operator) to update both public +and private DNS zones in other accounts, provided the role ARN has appropriate permissions. + +More details on using a role ARN to create DNS records in another AWS account will be documented in ExternalDNS +Operator's [Docs](https://github.com/openshift/external-dns-operator/tree/main/docs). + ### Risks and Mitigations -Add-on operators needing permissions will not work unless updated to use the role ARN. +Add-on operators needing permissions will not work unless updated to use the role ARN. The following add-on operators +have been addressed: +- ExternalDNS Operator ### Drawbacks @@ -235,14 +300,23 @@ platform dependencies in this config, but it should be carefully considered. An e2e test will be setup which utilizes both AWS CI accounts. +For the ExternalDNS Operator, we will add e2e tests utilizing both AWS CI accounts using the RoleARN to create DNS +records. + ### Graduation Criteria -This functionality is targeted for 4.14 GA and for backporting to previous releases. +This functionality is targeted for 4.14 GA and for backporting to previous releases. + +Note that the ExternalDNS Operator updates to support Shared VPC are out of payload and are therefore not aligned with +OpenShift releases and will not need backporting. #### Dev Preview -> Tech Preview Behind `PrivateHostedZoneAWS` feature gate. +Updates to the ExternalDNS Operator will be made available without being restricted by a feature gate. These updates +will be considered GA at the time of merging the feature. + #### Tech Preview -> GA - E2E CI coverage with satisfactory quality signal is needed to lift feature gate. @@ -258,7 +332,12 @@ TODO: there should be minimal impact here. ### Version Skew Strategy -n/a +There are potential version skew scenarios for the ExternalDNS Operator support of Shared VPC. The updates to the +ExternalDNS Operator are decoupled from the standard release schedule of OpenShift Container Platform. Consequently, in +a Shared VPC cluster using cross-account DNS record creation with the Ingress Operator, there is a possibility +of installing a version of the ExternalDNS Operator that lacks support for this same feature. However, it is important +to note that the ExternalDNS Operator and the installer or Ingress Operator do not rely on each other's logic. +Hence, this mismatch does not pose a concern. ### Operational Aspects of API Extensions @@ -267,7 +346,7 @@ n/a #### Failure Modes - Installation will fail if there are any issues with role during install. -- If there are issues with the role during day-2 operations, the cluster ingress operator will not +- If there are issues with the role during day-2 operations, the Cluster Ingress Operator will not be able to create DNS records and should log appropriate error messages. #### Support Procedures @@ -286,6 +365,16 @@ profile directly in the AWS creds. The implementation suggested in this enhancem preferred as it allows users to declare intent of a cross account install, rather than OpenShift trying to infer that from the credentials. +### External ID + +We considered supporting configuration of an [external ID](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html) +for both the Ingress Operator and the ExternalDNS Operator when using an assumed role. ExternalDNS provides support for +an external ID via the [`--aws-assume-role-external-id`](https://github.com/kubernetes-sigs/external-dns/blob/22da9f231dbc6faa3a668b507a4a06823a129609/pkg/apis/externaldns/types.go#L460) +command line argument. AWS suggests the use of an external ID in specific scenarios to mitigate the confused deputy +problem. However, for this effort, we opted not to include this functionality at the moment. The reason behind this +decision is that incorporating support for external ID throughout all the components that enable Shared VPC (including +API, Install, and Ingress Operator) would be necessary, but there hasn't been a specific customer need for it thus far. + ### API: Infrastructure A `privateHostedZoneRole` field containing the role ARN would be added to `AWSPlatformStatus` in the [cluster infrastructure object](https://github.com/openshift/api/blob/894b49f57a15cbce3869961e20cd9d52df6f8b0f/config/v1/types_infrastructure.go#L424).