Skip to content

Commit

Permalink
feat(eks): default masters role (#9464)
Browse files Browse the repository at this point in the history 
If `mastersRole` is not specified, we now define a default IAM role that can be assumed by anyone (with permissions) in the account.
        
This will allow users to interact with the cluster through `kubectl` by issuing the `aws eks update-kubeconfig` command with the appropriate `--role-arn` option, as specified in the CFN output.
    
Fixes #9463


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
Elad Ben-Israel authored Aug 5, 2020
1 parent 93657b7 commit b80c271
Show file tree
Hide file tree
Showing 6 changed files with 167 additions and 101 deletions.
128 changes: 42 additions & 86 deletions packages/@aws-cdk/aws-eks/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
## Amazon EKS Construct Library

<!--BEGIN STABILITY BANNER-->
---

Expand Down Expand Up @@ -29,6 +30,7 @@ const cluster = new eks.Cluster(this, 'hello-eks', {
version: eks.KubernetesVersion.V1_16,
});

// apply a kubernetes manifest to the cluster
cluster.addResource('mypod', {
apiVersion: 'v1',
kind: 'Pod',
Expand All @@ -45,6 +47,46 @@ cluster.addResource('mypod', {
});
```

In order to interact with your cluster through `kubectl`, you can use the `aws
eks update-kubeconfig` [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/eks/update-kubeconfig.html) command
to configure your local kubeconfig.

The EKS module will define a CloudFormation output in your stack which contains
the command to run. For example:

```
Outputs:
ClusterConfigCommand43AAE40F = aws eks update-kubeconfig --name cluster-xxxxx --role-arn arn:aws:iam::112233445566:role/yyyyy
```

> The IAM role specified in this command is called the "**masters role**". This is
> an IAM role that is associated with the `system:masters` [RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)
> group and has super-user access to the cluster.
>
> You can specify this role using the `mastersRole` option, or otherwise a role will be
> automatically created for you. This role can be assumed by anyone in the account with
> `sts:AssumeRole` permissions for this role.
Execute the `aws eks update-kubeconfig ...` command in your terminal to create a
local kubeconfig:

```console
$ aws eks update-kubeconfig --name cluster-xxxxx --role-arn arn:aws:iam::112233445566:role/yyyyy
Added new context arn:aws:eks:rrrrr:112233445566:cluster/cluster-xxxxx to /home/boom/.kube/config
```

And now you can simply use `kubectl`:

```console
$ kubectl get all -n kube-system
NAME READY STATUS RESTARTS AGE
pod/aws-node-fpmwv 1/1 Running 0 21m
pod/aws-node-m9htf 1/1 Running 0 21m
pod/coredns-5cb4fb54c7-q222j 1/1 Running 0 23m
pod/coredns-5cb4fb54c7-v9nxx 1/1 Running 0 23m
...
```

### Endpoint Access

You can configure the [cluster endpoint access](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html) by using the `endpointAccess` property:
Expand All @@ -58,7 +100,6 @@ const cluster = new eks.Cluster(this, 'hello-eks', {

The default value is `eks.EndpointAccess.PUBLIC_AND_PRIVATE`. Which means the cluster endpoint is accessible from outside of your VPC, and worker node traffic to the endpoint will stay within your VPC.


### Capacity

By default, `eks.Cluster` is created with a managed nodegroup with x2 `m5.large` instances. You must specify the kubernetes version for the cluster with the `version` property.
Expand Down Expand Up @@ -208,7 +249,6 @@ When adding capacity, you can specify options for
which is responsible for associating the node to the EKS cluster. For example,
you can use `kubeletExtraArgs` to add custom node labels or taints.


```ts
// up to ten spot instances
cluster.addCapacity('spot', {
Expand All @@ -224,90 +264,6 @@ cluster.addCapacity('spot', {
To disable bootstrapping altogether (i.e. to fully customize user-data), set `bootstrapEnabled` to `false` when you add
the capacity.

### Masters Role

The Amazon EKS construct library allows you to specify an IAM role that will be
granted `system:masters` privileges on your cluster.

Without specifying a `mastersRole`, you will not be able to interact manually
with the cluster.

The following example defines an IAM role that can be assumed by all users
in the account and shows how to use the `mastersRole` property to map this
role to the Kubernetes `system:masters` group:

```ts
// first define the role
const clusterAdmin = new iam.Role(this, 'AdminRole', {
assumedBy: new iam.AccountRootPrincipal()
});

// now define the cluster and map role to "masters" RBAC group
new eks.Cluster(this, 'Cluster', {
mastersRole: clusterAdmin,
version: eks.KubernetesVersion.V1_16,
});
```

When you `cdk deploy` this CDK app, you will notice that an output will be printed
with the `update-kubeconfig` command.

Something like this:

```
Outputs:
eks-integ-defaults.ClusterConfigCommand43AAE40F = aws eks update-kubeconfig --name cluster-ba7c166b-c4f3-421c-bf8a-6812e4036a33 --role-arn arn:aws:iam::112233445566:role/eks-integ-defaults-Role1ABCC5F0-1EFK2W5ZJD98Y
```

Copy & paste the "`aws eks update-kubeconfig ...`" command to your shell in
order to connect to your EKS cluster with the "masters" role.

Now, given [AWS CLI](https://aws.amazon.com/cli/) is configured to use AWS
credentials for a user that is trusted by the masters role, you should be able
to interact with your cluster through `kubectl` (the above example will trust
all users in the account).

For example:

```console
$ aws eks update-kubeconfig --name cluster-ba7c166b-c4f3-421c-bf8a-6812e4036a33 --role-arn arn:aws:iam::112233445566:role/eks-integ-defaults-Role1ABCC5F0-1EFK2W5ZJD98Y
Added new context arn:aws:eks:eu-west-2:112233445566:cluster/cluster-ba7c166b-c4f3-421c-bf8a-6812e4036a33 to /Users/boom/.kube/config

$ kubectl get nodes # list all nodes
NAME STATUS ROLES AGE VERSION
ip-10-0-147-66.eu-west-2.compute.internal Ready <none> 21m v1.13.7-eks-c57ff8
ip-10-0-169-151.eu-west-2.compute.internal Ready <none> 21m v1.13.7-eks-c57ff8

$ kubectl get all -n kube-system
NAME READY STATUS RESTARTS AGE
pod/aws-node-fpmwv 1/1 Running 0 21m
pod/aws-node-m9htf 1/1 Running 0 21m
pod/coredns-5cb4fb54c7-q222j 1/1 Running 0 23m
pod/coredns-5cb4fb54c7-v9nxx 1/1 Running 0 23m
pod/kube-proxy-d4jrh 1/1 Running 0 21m
pod/kube-proxy-q7hh7 1/1 Running 0 21m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kube-dns ClusterIP 172.20.0.10 <none> 53/UDP,53/TCP 23m

NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/aws-node 2 2 2 2 2 <none> 23m
daemonset.apps/kube-proxy 2 2 2 2 2 <none> 23m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/coredns 2/2 2 2 23m

NAME DESIRED CURRENT READY AGE
replicaset.apps/coredns-5cb4fb54c7 2 2 2 23m
```

For your convenience, an AWS CloudFormation output will automatically be
included in your template and will be printed when running `cdk deploy`.

**NOTE**: if the cluster is configured with `kubectlEnabled: false`, it
will be created with the role/user that created the AWS CloudFormation
stack. See [Kubectl Support](#kubectl-support) for details.

### Kubernetes Resources

The `KubernetesResource` construct or `cluster.addResource` method can be used
Expand Down
25 changes: 14 additions & 11 deletions packages/@aws-cdk/aws-eks/lib/cluster.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -175,11 +175,8 @@ export interface ClusterOptions {
*
* @see https://kubernetes.io/docs/reference/access-authn-authz/rbac/#default-roles-and-role-bindings
*
* @default - By default, it will only possible to update this Kubernetes
* system by adding resources to this cluster via `addResource` or
* by defining `KubernetesResource` resources in your AWS CDK app.
* Use this if you wish to grant cluster administration privileges
* to another role.
* @default - a role that assumable by anyone with permissions in the same
* account will automatically be defined
*/
readonly mastersRole?: iam.IRole;

Expand Down Expand Up @@ -689,19 +686,25 @@ export class Cluster extends Resource implements ICluster {
new CfnOutput(this, 'ClusterName', { value: this.clusterName });
}

// map the IAM role to the `system:masters` group.
if (props.mastersRole) {
if (!this.kubectlEnabled) {
if (!this.kubectlEnabled) {
if (props.mastersRole) {
throw new Error('Cannot specify a "masters" role if kubectl is disabled');
}
} else {
// if an explicit role is not configured, define a masters role that can
// be assumed by anyone in the account (with sts:AssumeRole permissions of
// course)
const mastersRole = props.mastersRole ?? new iam.Role(this, 'MastersRole', {
assumedBy: new iam.AccountRootPrincipal(),
});

this.awsAuth.addMastersRole(props.mastersRole);
this.awsAuth.addMastersRole(mastersRole);

if (props.outputMastersRoleArn) {
new CfnOutput(this, 'MastersRoleArn', { value: props.mastersRole.roleArn });
new CfnOutput(this, 'MastersRoleArn', { value: mastersRole.roleArn });
}

commonCommandOptions.push(`--role-arn ${props.mastersRole.roleArn}`);
commonCommandOptions.push(`--role-arn ${mastersRole.roleArn}`);
}

// allocate default capacity if non-zero (or default).
Expand Down
44 changes: 43 additions & 1 deletion packages/@aws-cdk/aws-eks/test/test.awsauth.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,20 @@ export = {
'',
[
'[{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"aws-auth","namespace":"kube-system"},"data":{"mapRoles":"[{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"username\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"groups\\":[\\"system:masters\\"]},{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'ClusterNodegroupDefaultCapacityNodeGroupRole55953B04',
Expand Down Expand Up @@ -128,13 +142,27 @@ export = {
'',
[
'[{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"aws-auth","namespace":"kube-system"},"data":{"mapRoles":"[{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"username\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"groups\\":[\\"system:masters\\"]},{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'ClusterNodegroupDefaultCapacityNodeGroupRole55953B04',
'Arn',
],
},
'\\",\\"username\\":\\"system:node:{{EC2PrivateDNSName}}\\",\\"groups\\":[\\"system:bootstrappers\\",\\"system:nodes\\"]},{\\"rolearn\\":\\"arn:aws:iam::123456789012:role/S3Access\\",\\"username\\":\\"arn:aws:iam::123456789012:role/S3Access\\",\\"groups\\":[\\"group1\\"]}]\",\"mapUsers\":\"[{\\"userarn\\":\\"arn:',
'\\",\\"username\\":\\"system:node:{{EC2PrivateDNSName}}\\",\\"groups\\":[\\"system:bootstrappers\\",\\"system:nodes\\"]},{\\"rolearn\\":\\"arn:aws:iam::123456789012:role/S3Access\\",\\"username\\":\\"arn:aws:iam::123456789012:role/S3Access\\",\\"groups\\":[\\"group1\\"]}]","mapUsers":"[{\\"userarn\\":\\"arn:',
{
Ref: 'AWS::Partition',
},
Expand Down Expand Up @@ -175,6 +203,20 @@ export = {
'',
[
'[{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"aws-auth","namespace":"kube-system"},"data":{"mapRoles":"[{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"username\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"groups\\":[\\"system:masters\\"]},{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'ClusterNodegroupDefaultCapacityNodeGroupRole55953B04',
Expand Down
43 changes: 40 additions & 3 deletions packages/@aws-cdk/aws-eks/test/test.cluster.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,20 @@ export = {
'',
[
'[{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"aws-auth","namespace":"kube-system"},"data":{"mapRoles":"[{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"username\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"groups\\":[\\"system:masters\\"]},{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'ClusterdefaultInstanceRoleF20A29CD',
Expand Down Expand Up @@ -527,7 +541,30 @@ export = {
});

// THEN
expect(stack).to(not(haveResource(eks.KubernetesResource.RESOURCE_TYPE)));
expect(stack).to(haveResource(eks.KubernetesResource.RESOURCE_TYPE, {
Manifest: {
'Fn::Join': [
'',
[
'[{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"aws-auth","namespace":"kube-system"},"data":{"mapRoles":"[{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"username\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"groups\\":[\\"system:masters\\"]}]","mapUsers":"[]","mapAccounts":"[]"}}]',
],
],
},
}));
test.done();
},

Expand Down Expand Up @@ -563,8 +600,8 @@ export = {
const assembly = app.synth();
const template = assembly.getStackByName(stack.stackName).template;
test.deepEqual(template.Outputs, {
ClusterConfigCommand43AAE40F: { Value: { 'Fn::Join': ['', ['aws eks update-kubeconfig --name ', { Ref: 'Cluster9EE0221C' }, ' --region us-east-1']] } },
ClusterGetTokenCommand06AE992E: { Value: { 'Fn::Join': ['', ['aws eks get-token --cluster-name ', { Ref: 'Cluster9EE0221C' }, ' --region us-east-1']] } },
ClusterConfigCommand43AAE40F: { Value: { 'Fn::Join': ['', ['aws eks update-kubeconfig --name ', { Ref: 'Cluster9EE0221C' }, ' --region us-east-1 --role-arn ', { 'Fn::GetAtt': [ 'ClusterMastersRole9AA35625', 'Arn' ] } ] ] } },
ClusterGetTokenCommand06AE992E: { Value: { 'Fn::Join': ['', ['aws eks get-token --cluster-name ', { Ref: 'Cluster9EE0221C' }, ' --region us-east-1 --role-arn ', { 'Fn::GetAtt': [ 'ClusterMastersRole9AA35625', 'Arn' ] } ] ] } },
});
test.done();
},
Expand Down
14 changes: 14 additions & 0 deletions packages/@aws-cdk/aws-eks/test/test.fargate.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,20 @@ export = {
'',
[
'[{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"aws-auth","namespace":"kube-system"},"data":{"mapRoles":"[{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'FargateClusterMastersRole50BAF9FD',
'Arn',
],
},
'\\",\\"username\\":\\"',
{
'Fn::GetAtt': [
'FargateClusterMastersRole50BAF9FD',
'Arn',
],
},
'\\",\\"groups\\":[\\"system:masters\\"]},{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'FargateClusterfargateprofiledefaultPodExecutionRole66F2610E',
Expand Down
14 changes: 14 additions & 0 deletions packages/@aws-cdk/aws-eks/test/test.nodegroup.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,20 @@ export = {
'',
[
'[{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"aws-auth","namespace":"kube-system"},"data":{"mapRoles":"[{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"username\\":\\"',
{
'Fn::GetAtt': [
'ClusterMastersRole9AA35625',
'Arn',
],
},
'\\",\\"groups\\":[\\"system:masters\\"]},{\\"rolearn\\":\\"',
{
'Fn::GetAtt': [
'NodegroupNodeGroupRole038A128B',
Expand Down

0 comments on commit b80c271

Please sign in to comment.