Skip to content

Commit

Permalink
Eliminated the use of temporary keys for tests with kubectl
Browse files Browse the repository at this point in the history
  • Loading branch information
vchintal committed Oct 24, 2023
1 parent 91cf9fe commit e9623ce
Show file tree
Hide file tree
Showing 5 changed files with 147 additions and 69 deletions.
136 changes: 70 additions & 66 deletions patterns/privatelink-access/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,37 @@ for further details on `AWS PrivateLink`.

## Deploy

See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/#prerequisites) for the prerequisites and steps to deploy this pattern.
See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/#prerequisites) for the prerequisites and follow the steps below to deploy this pattern.

## Validate
```sh
terraform init
terraform apply -target=module.eventbridge -target=module.nlb --auto-approve
terraform apply --auto-approve
```

### Network Connectivity
Once the pattern has successfully deployed, you will be provided with multiple
output values.

An output `ssm_test` has been provided to aid in quickly testing the
connectivity from the client EC2 instance to the private EKS cluster via AWS
PrivateLink. Copy the output value and paste it into your terminal to execute
and check the connectivity. If configured correctly, the value returned should
be `ok`.
Review the output value for `cluster_endpoint_private`, it should look similar
to snippet below:

```sh
aws eks update-cluster-config \
--region us-west-2 \
--name privatelink-access \
--resources-vpc-config endpointPublicAccess=false,endpointPrivateAccess=true
```

Copy the command and run it in a terminal session to take cluster API
endpoint private.

## Test access to EKS Kubernetes API server endpoint

Of the other output values, the value `ssm_test` is provided to aid in quickly
testing the connectivity from the client EC2 instance to the private EKS cluster
via AWS PrivateLink. Copy the output value, which looks like the snippet shown
below (as an example) and paste it into your terminal to execute and check the
connectivity. If configured correctly, the value returned should be `ok`.

```sh
COMMAND="curl -ks https://9A85B21811733524E3ABCDFEA8714642.gr7.us-west-2.eks.amazonaws.com/readyz"
Expand All @@ -36,78 +56,62 @@ aws ssm get-command-invocation --region us-west-2 \
--output text
```

### Cluster Access

To test access to the cluster, you will need to execute Kubernetes API calls
from within the private network to access the cluster. An EC2 instance has been
deployed into a "client" VPC to simulate this scenario. However, since the EKS
cluster was created with your local IAM identity, the `aws-auth` ConfigMap will
only have your local identity that is permitted to access the cluster. Since
cluster's API endpoint is private, we cannot use Terraform to reach it to
add additional entries to the ConfigMap; we can only access the cluster from
within the private network of the cluster's VPC or from the client VPC using AWS
PrivateLink access.

!!! info
The "client" EC2 instance provided and copying of AWS credentials to
that instance are merely for demonstration purposes only. Please consider
alternate methods of network access such as AWS Client VPN to provide more
secure access.
## Test access to EKS Kubernetes API with `kubectl`

Perform the following steps to access the cluster with `kubectl` from the
provided "client" EC2 instance.

1. Execute the command below on your local machine to get temporary credentials
that will be used on the "client" EC2 instance:
provided Client EC2 instance.

```sh
aws sts get-session-token --duration-seconds 3600 --output yaml
```
### Log into the Client EC2 instance
Start a new SSM session on the Client EC2 instance using the provided
`ssm_start_session` output value. It should look similar to the snippet
shown below. Copy the output value and paste it into your terminal to execute.
Your terminal will now be connected to the Client EC2 instance.

2. Start a new SSM session on the "client" EC2 instance using the provided
`ssm_start_session` output value. Copy the output value and paste it into your
terminal to execute. Your terminal will now be connected to the "client" EC2
instance.
```sh
aws ssm start-session --region us-west-2 --target i-0280cf604085f4a44
```

```sh
aws ssm start-session --region us-west-2 --target i-0280cf604085f4a44
```
### Update Kubeconfig
On the Client EC2 machine, run the following command to update the local
`~/.kube/config` file to enable access to the cluster:

3. Once logged in, export the following environment variables from the output
of step #1:
```sh
aws eks update-kubeconfig --region us-west-2 --name privatelink-access
```

!!! warning
The session credentials are only valid for 1 hour; you can
adjust the session duration in the command provided in step #1
### Test complete access with `kubectl`
Test access by listing the pods running on the cluster:

```sh
export AWS_ACCESS_KEY_ID=XXXX
export AWS_SECRET_ACCESS_KEY=YYYY
export AWS_SESSION_TOKEN=ZZZZ
```
```sh
kubectl get pods -A
```

4. Run the following command to update the local `~/.kube/config` file to enable
access to the cluster:
```text
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system aws-node-4f8g8 1/1 Running 0 1m
kube-system coredns-6ff9c46cd8-59sqp 1/1 Running 0 1m
kube-system coredns-6ff9c46cd8-svnpb 1/1 Running 0 2m
kube-system kube-proxy-mm2zc 1/1 Running 0 1m
```

```sh
aws eks update-kubeconfig --region us-west-2 --name privatelink-access
```
## Destroy

5. Test access by listing the pods running on the cluster:
Before we could destroy/teardown all the resources created, we need to ensure
that the cluster state is restored for the Terraform to do a complete cleanup.
This would mean that we make cluster API endpoint public again.

```sh
kubectl get pods -A
```
Review the output value for `cluster_endpoint_public`, it should look similar
to snippet below:

```text
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system aws-node-4f8g8 1/1 Running 0 1m
kube-system coredns-6ff9c46cd8-59sqp 1/1 Running 0 1m
kube-system coredns-6ff9c46cd8-svnpb 1/1 Running 0 2m
kube-system kube-proxy-mm2zc 1/1 Running 0 1m
```
```sh
aws eks update-cluster-config \
--region us-west-2 \
--name privatelink-access \
--resources-vpc-config endpointPublicAccess=true,endpointPrivateAccess=true
```

## Destroy
Copy the command and run it in a terminal session to take cluster API
endpoint public.

{%
include-markdown "../../docs/_partials/destroy.md"
Expand Down
32 changes: 29 additions & 3 deletions patterns/privatelink-access/client.tf
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ module "client_vpc" {
manage_default_security_group = true
default_security_group_tags = { Name = "${local.client_name}-default" }

tags = local.tags
tags = merge(local.tags, {
Name = local.client_name
})
}

################################################################################
Expand All @@ -44,6 +46,7 @@ module "client_ec2_instance" {

create_iam_instance_profile = true
iam_role_policies = {
EKSFullAccess = aws_iam_policy.eks_full_access_policy.arn
AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

Expand All @@ -64,7 +67,28 @@ module "client_ec2_instance" {
./aws/install
EOT

tags = local.tags
tags = merge(local.tags, {
Name = local.client_name
})
}

resource "aws_iam_policy" "eks_full_access_policy" {
name = "EKSFullAccess"
path = "/"
description = "EKS full-access policy"

policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = [
"eks:*"
]
Effect = "Allow"
Resource = module.eks.cluster_arn
},
]
})
}

module "client_security_group" {
Expand All @@ -84,5 +108,7 @@ module "client_security_group" {
},
]

tags = local.tags
tags = merge(local.tags, {
Name = local.client_name
})
}
21 changes: 21 additions & 0 deletions patterns/privatelink-access/eks.tf
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,34 @@
# EKS Cluster
################################################################################

provider "kubernetes" {
host = module.eks.cluster_endpoint
cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)

exec {
api_version = "client.authentication.k8s.io/v1beta1"
command = "aws"
# This requires the awscli to be installed locally where Terraform is executed
args = ["eks", "get-token", "--cluster-name", module.eks.cluster_name]
}
}

module "eks" {
source = "terraform-aws-modules/eks/aws"
version = "~> 19.16"

cluster_name = local.name
cluster_version = "1.27"

cluster_endpoint_public_access = true
manage_aws_auth_configmap = true

aws_auth_roles = [{
rolearn = module.client_ec2_instance.iam_role_arn
username = "ec2-client"
groups = ["system:masters"]
}]

cluster_addons = {
coredns = {}
kube-proxy = {}
Expand Down
23 changes: 23 additions & 0 deletions patterns/privatelink-access/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ output "ssm_start_session" {
output "ssm_test" {
description = "SSM commands to test connectivity from client EC2 instance to the private EKS cluster"
value = <<-EOT
COMMAND="curl -ks ${module.eks.cluster_endpoint}/readyz"
COMMAND_ID=$(aws ssm send-command --region ${local.region} \
Expand All @@ -22,3 +23,25 @@ output "ssm_test" {
--output text
EOT
}

output "cluster_endpoint_private" {
description = "Command to set the EKS API server endpoint access private"
value = <<-EOT
aws eks update-cluster-config \
--region ${local.region} \
--name ${module.eks.cluster_name} \
--resources-vpc-config endpointPublicAccess=false,endpointPrivateAccess=true
EOT
}

output "cluster_endpoint_public" {
description = "Command to set the EKS API server endpoint access private"
value = <<-EOT
aws eks update-cluster-config \
--region ${local.region} \
--name ${module.eks.cluster_name} \
--resources-vpc-config endpointPublicAccess=true,endpointPrivateAccess=true
EOT
}
4 changes: 4 additions & 0 deletions patterns/privatelink-access/versions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ terraform {
source = "hashicorp/dns"
version = ">= 3.0"
}
kubernetes = {
source = "hashicorp/kubernetes"
version = ">= 2.20"
}
}

# ## Used for end-to-end testing on project; update to suit your needs
Expand Down

0 comments on commit e9623ce

Please sign in to comment.