Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup heptio authenticator #5197

Merged
merged 3 commits into from
Jun 1, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions docs/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,84 @@ spec:
rbac: {}
```

## Heptio Authenticator for AWS

If you want to turn on Heptio Authenticator for AWS, you can add this block
to your cluster:

```
authentication:
heptio: {}
```

For example:

```
apiVersion: kops/v1alpha2
kind: Cluster
metadata:
name: cluster.example.com
spec:
authentication:
heptio: {}
authorization:
rbac: {}
```

Once the cluster is up you will need to create the heptio authenticator
config as a config map. (This can also be done when boostrapping a cluster using addons)
For more details on heptio authenticator please visit (heptio/authenticator)[https://github.com/heptio/authenticator]
Example config:

```
---
apiVersion: v1
Copy link
Contributor

@gambol99 gambol99 May 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happen's if the config isn't there? .. does it's just crashloop the pods? ... Just wondering in case it effects rollouts .. i'm pretty sure the validate cluster code looks for pod restarts in kube-system ..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pod will sit in a creating status. I have not tired doing a rollout while it's in this state, but will do so.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ar yes! .. it would be stuck in pending .. Still if you can quickly verify / confirm it handles on a pre-built cluster and can rollout, that would be cool

Copy link
Contributor Author

@rdrgmnzs rdrgmnzs Jun 1, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pod stays in state:
kube-system heptio-authenticator-aws-4fkzp 0/1 ContainerCreating 0 10m

with the error:
Unable to mount volumes for pod "heptio-authenticator-aws-4fkzp_kube-system(93f46fcb-654e-11e8-a014-0e21d0512306)": timeout expired waiting for volumes to attach or mount for pod "kube-system"/"heptio-authenticator-aws-4fkzp". list of unmounted volumes=[config]. list of unattached volumes=[config output state default-token-6rtmz]

However I'm still able to perform a full cluster rolling update while the cluster is in this state so we should be ok.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

much appreciated 👍

kind: ConfigMap
metadata:
namespace: kube-system
name: heptio-authenticator-aws
labels:
k8s-app: heptio-authenticator-aws
data:
config.yaml: |
# a unique-per-cluster identifier to prevent replay attacks
# (good choices are a random token or a domain name that will be unique to your cluster)
clusterID: my-dev-cluster.example.com
server:
# each mapRoles entry maps an IAM role to a username and set of groups
# Each username and group can optionally contain template parameters:
# 1) "{{AccountID}}" is the 12 digit AWS ID.
# 2) "{{SessionName}}" is the role session name.
mapRoles:
# statically map arn:aws:iam::000000000000:role/KubernetesAdmin to a cluster admin
- roleARN: arn:aws:iam::000000000000:role/KubernetesAdmin
username: kubernetes-admin
groups:
- system:masters
# map EC2 instances in my "KubernetesNode" role to users like
# "aws:000000000000:instance:i-0123456789abcdef0". Only use this if you
# trust that the role can only be assumed by EC2 instances. If an IAM user
# can assume this role directly (with sts:AssumeRole) they can control
# SessionName.
- roleARN: arn:aws:iam::000000000000:role/KubernetesNode
username: aws:{{AccountID}}:instance:{{SessionName}}
groups:
- system:bootstrappers
- aws:instances
# map federated users in my "KubernetesAdmin" role to users like
# "admin:alice-example.com". The SessionName is an arbitrary role name
# like an e-mail address passed by the identity provider. Note that if this
# role is assumed directly by an IAM User (not via federation), the user
# can control the SessionName.
- roleARN: arn:aws:iam::000000000000:role/KubernetesAdmin
username: admin:{{SessionName}}
groups:
- system:masters
# each mapUsers entry maps an IAM role to a static username and set of groups
mapUsers:
# map user IAM user Alice in 000000000000 to user "alice" in "system:masters"
- userARN: arn:aws:iam::000000000000:user/Alice
username: alice
groups:
- system:masters
```
101 changes: 100 additions & 1 deletion nodeup/pkg/model/kube_apiserver.go
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,105 @@ func (b *KubeAPIServerBuilder) writeAuthenticationConfig(c *fi.ModelBuilderConte
return nil
}

if b.Cluster.Spec.Authentication.Heptio != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps left to another PR, but should consider merging the kopeio and this together given both is essentially just a web hook with a url diff ..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will create another PR to merge / cleanup.

id := "heptio-authenticator-aws"
b.Cluster.Spec.KubeAPIServer.AuthenticationTokenWebhookConfigFile = fi.String(PathAuthnConfig)

{
caCertificate, err := b.NodeupModelContext.KeyStore.FindCert(fi.CertificateId_CA)
if err != nil {
return fmt.Errorf("error fetching Heptio Authentication CA certificate from keystore: %v", err)
}
if caCertificate == nil {
return fmt.Errorf("Heptio Authentication CA certificate %q not found", fi.CertificateId_CA)
}

cluster := kubeconfig.KubectlCluster{
Server: "https://127.0.0.1:21362/authenticate",
}
context := kubeconfig.KubectlContext{
Cluster: "heptio-authenticator-aws",
User: "kube-apiserver",
}

cluster.CertificateAuthorityData, err = caCertificate.AsBytes()
if err != nil {
return fmt.Errorf("error encoding Heptio Authentication CA certificate: %v", err)
}

config := kubeconfig.KubectlConfig{}
config.Clusters = append(config.Clusters, &kubeconfig.KubectlClusterWithName{
Name: "heptio-authenticator-aws",
Cluster: cluster,
})
config.Users = append(config.Users, &kubeconfig.KubectlUserWithName{
Name: "kube-apiserver",
})
config.CurrentContext = "webhook"
config.Contexts = append(config.Contexts, &kubeconfig.KubectlContextWithName{
Name: "webhook",
Context: context,
})

manifest, err := kops.ToRawYaml(config)
if err != nil {
return fmt.Errorf("error marshalling authentication config to yaml: %v", err)
}

c.AddTask(&nodetasks.File{
Path: PathAuthnConfig,
Contents: fi.NewBytesResource(manifest),
Type: nodetasks.FileType_File,
Mode: fi.String("600"),
})
}

{
certificate, err := b.NodeupModelContext.KeyStore.FindCert(id)
if err != nil {
return fmt.Errorf("error fetching %q certificate from keystore: %v", id, err)
}
if certificate == nil {
return fmt.Errorf("certificate %q not found", id)
}

certificateData, err := certificate.AsBytes()
if err != nil {
return fmt.Errorf("error encoding %q certificate: %v", id, err)
}

c.AddTask(&nodetasks.File{
Path: "/srv/kubernetes/heptio-authenticator-aws/cert.pem",
Contents: fi.NewBytesResource(certificateData),
Type: nodetasks.FileType_File,
Mode: fi.String("600"),
})
}

{
privateKey, err := b.NodeupModelContext.KeyStore.FindPrivateKey(id)
if err != nil {
return fmt.Errorf("error fetching %q private key from keystore: %v", id, err)
}
if privateKey == nil {
return fmt.Errorf("private key %q not found", id)
}

keyData, err := privateKey.AsBytes()
if err != nil {
return fmt.Errorf("error encoding %q private key: %v", id, err)
}

c.AddTask(&nodetasks.File{
Path: "/srv/kubernetes/heptio-authenticator-aws/key.pem",
Contents: fi.NewBytesResource(keyData),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add file perms to 0600 please Mode: fi.String("600") .. as the default is world readable

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the perms that I think are still missing @rdrgmnzs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@justinsb yeah I actually added them on diff f047677 and github says they have been pulled in to this PR but for some reason they are not showing up in the "files changed"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you added it to this task (key.pem( - I do see it for the other two file tasks though.

Type: nodetasks.FileType_File,
})
}

return nil
}

return fmt.Errorf("Unrecognized authentication config %v", b.Cluster.Spec.Authentication)
}

Expand Down Expand Up @@ -311,7 +410,7 @@ func (b *KubeAPIServerBuilder) buildPod() (*v1.Pod, error) {
}

if b.Cluster.Spec.Authentication != nil {
if b.Cluster.Spec.Authentication.Kopeio != nil {
if b.Cluster.Spec.Authentication.Kopeio != nil || b.Cluster.Spec.Authentication.Heptio != nil {
addHostPathMapping(pod, container, "authn-config", PathAuthnConfig)
}
}
Expand Down
6 changes: 5 additions & 1 deletion pkg/apis/kops/cluster.go
Original file line number Diff line number Diff line change
Expand Up @@ -230,15 +230,19 @@ type ExecContainerAction struct {

type AuthenticationSpec struct {
Kopeio *KopeioAuthenticationSpec `json:"kopeio,omitempty"`
Heptio *HeptioAuthenticationSpec `json:"heptio,omitempty"`
}

func (s *AuthenticationSpec) IsEmpty() bool {
return s.Kopeio == nil
return s.Kopeio == nil && s.Heptio == nil
}

type KopeioAuthenticationSpec struct {
}

type HeptioAuthenticationSpec struct {
}

type AuthorizationSpec struct {
AlwaysAllow *AlwaysAllowAuthorizationSpec `json:"alwaysAllow,omitempty"`
RBAC *RBACAuthorizationSpec `json:"rbac,omitempty"`
Expand Down
6 changes: 5 additions & 1 deletion pkg/apis/kops/v1alpha1/cluster.go
Original file line number Diff line number Diff line change
Expand Up @@ -229,15 +229,19 @@ type ExecContainerAction struct {

type AuthenticationSpec struct {
Kopeio *KopeioAuthenticationSpec `json:"kopeio,omitempty"`
Heptio *HeptioAuthenticationSpec `json:"heptio,omitempty"`
}

func (s *AuthenticationSpec) IsEmpty() bool {
return s.Kopeio == nil
return s.Kopeio == nil && s.Heptio == nil
}

type KopeioAuthenticationSpec struct {
}

type HeptioAuthenticationSpec struct {
}

type AuthorizationSpec struct {
AlwaysAllow *AlwaysAllowAuthorizationSpec `json:"alwaysAllow,omitempty"`
RBAC *RBACAuthorizationSpec `json:"rbac,omitempty"`
Expand Down
38 changes: 38 additions & 0 deletions pkg/apis/kops/v1alpha1/zz_generated.conversion.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

25 changes: 25 additions & 0 deletions pkg/apis/kops/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion pkg/apis/kops/v1alpha2/cluster.go
Original file line number Diff line number Diff line change
Expand Up @@ -230,15 +230,19 @@ type ExecContainerAction struct {

type AuthenticationSpec struct {
Kopeio *KopeioAuthenticationSpec `json:"kopeio,omitempty"`
Heptio *HeptioAuthenticationSpec `json:"heptio,omitempty"`
}

func (s *AuthenticationSpec) IsEmpty() bool {
return s.Kopeio == nil
return s.Kopeio == nil && s.Heptio == nil
}

type KopeioAuthenticationSpec struct {
}

type HeptioAuthenticationSpec struct {
}

type AuthorizationSpec struct {
AlwaysAllow *AlwaysAllowAuthorizationSpec `json:"alwaysAllow,omitempty"`
RBAC *RBACAuthorizationSpec `json:"rbac,omitempty"`
Expand Down
Loading