Auth token file support (kubernetes-retired#418)

Adds support for authentication via static token files, as documented [here](https://kubernetes.io/docs/admin/authentication/#static-token-file).

The token file is a csv file with a minimum of 3 columns: token, user name, user uid, followed by optional group names. Note, if you have more than one group the column must be double quoted.

Authentication via static token files, although not very flexible, is a pretty simple solution for users getting started with Kubernetes and/or want to have some mechanism to give access to different users, but do not want the complexity overhead of a more flexible solution, such as [dex](https://github.com/coreos/dex).

This change will also be the foundation which kubernetes-retired#414 will be laid on.

A more detailed change list follows:

* Renamed function

Since we'll be encrypting things that are not TLS certificates, it's
time to rename the function in order to avoid any confusion.

The function name suffix implied it created a file, which was not the
case.

* Updated test error message

* Renamed systemd unit to match name changes

* Defined a location in which to put the token auth file

The decryption of the token file should also happen provided it is put
at the correct location.

* Should only attempt to decrypt tokens if the auth token file exists

* Fixed typo

* Added tests

* Updated docs

* Fixed script

* Validate auth token file

Before generating the encrypted version of the auth token file, parse
it as a CSV and look for problems, such as comments and unmatching
number of columns.

If any problem is found, an error is raised, causing the cluster not
to be created.

Documentation/kube-aws-cluster-updates.md

@@ -18,14 +18,14 @@ git diff # view changes to rendered assets
 kube-aws update
 ```
 
-## Certificate rotation
+## Certificate and access token rotation
 
-The parameter-level update mechanism can be used to rotate in new TLS credentials.
+The parameter-level update mechanism can be used to rotate in new TLS credentials and access tokens.
 
 More concretely, steps should be taken in order to rotate your certs on nodes are:
 
 * Optionally modify the `externalDNSName` attribute in `cluster.yaml`
-* Remove all the `credentials/*.enc` which are cached encrypted certs and keys to prevent unnecessary node replacement when there's actually no update. See #107 and #237 for more context.
+* Remove all the `credentials/*.enc` which are cached encrypted certs/keys/tokens to prevent unnecessary node replacement when there's actually no update. See #107 and #237 for more context.
 * Render new credentials using kube-aws render credentials:
 
   ```sh
@@ -37,7 +37,7 @@ More concretely, steps should be taken in order to rotate your certs on nodes ar
   kube-aws update --s3-uri s3://my/own/path
   ```
 
-## the etcd caveat
+## The etcd caveat
 
 There is no solution for hosting an etcd cluster in a way that is easily updateable in this fashion- so updates are automatically masked for the etcd instances. This means that, after the cluster is created, nothing about the etcd ec2 instances is allowed to be updated.
 

Documentation/kubernetes-on-aws-render.md

@@ -102,10 +102,11 @@ There will now be a `cluster.yaml` file in the asset directory. This is the main
 * In certain cases, such as users with advanced pre-existing PKI infrastructure, the operator may wish to pre-generate all cluster TLS assets. In this case, you can run `kube-aws render stack` and copy in your TLS assets into the `credentials/` folder before running `kube-aws up`.
 
   ```sh
+
   ls -R credentials/
   credentials/:
-  admin-key.pem  apiserver-key.pem  ca.pem               etcd-client.pem  etcd.pem        worker.pem
-  admin.pem      apiserver.pem      etcd-client-key.pem  etcd-key.pem     worker-key.pem
+  admin-key.pem       apiserver-key.pem   ca-key.pem          etcd-client-key.pem etcd-key.pem        tokens.csv          worker.pem
+  admin.pem           apiserver.pem       ca.pem              etcd-client.pem     etcd.pem            worker-key.pem
   ```
 
 The next command generates the default set of cluster assets in your asset directory.
@@ -133,6 +134,7 @@ $ tree
 │   └── etcd.pem
 │   ├── etcd-client-key.pem
 │   └── etcd-client.pem
+│   └── tokens.csv
 ├── kubeconfig
 ├── stack-template.json
 └── userdata
@@ -142,7 +144,7 @@ $ tree
 
 These assets (templates and credentials) are used to create, update and interact with your Kubernetes cluster.
 
-At this point you should be ready to create your cluster. You can also now check the `my-cluster` asset directory into version control if you desire. The contents of this directory are your reproducible cluster assets. Please take care not to commit the `my-cluster/credentials` directory but rather to encrypt and/or put it to more secure storage, as it contains your TLS secrets. If you're using git, the `credentials` directory will already be ignored for you.
+At this point you should be ready to create your cluster. You can also now check the `my-cluster` asset directory into version control if you desire. The contents of this directory are your reproducible cluster assets. Please take care not to commit the `my-cluster/credentials` directory but rather to encrypt and/or put it to more secure storage, as it contains your TLS secrets and access tokens. If you're using git, the `credentials` directory will already be ignored for you.
 
 **PRODUCTION NOTE**: the TLS keys and certificates generated by `kube-aws` should *not* be used to deploy a production Kubernetes cluster.
 Each component certificate is only valid for 90 days, while the CA is valid for 365 days.
@@ -204,6 +206,8 @@ You can now customize your cluster by editing asset files. Any changes to these
 
   This directory contains both encryped and **unencrypted** TLS assets for your cluster, along with a pre-configured `kubeconfig` file which provides access to your cluster api via kubectl.
 
+  You can also specify additional access tokens in `tokens.csv` as shown in the [official docs](https://kubernetes.io/docs/admin/authentication/#static-token-file).
+
 [mount-disks]: https://coreos.com/os/docs/latest/mounting-storage.html
 [insecure-registry]: https://coreos.com/os/docs/latest/registry-authentication.html#using-a-registry-without-ssl-configured
 [update]: https://coreos.com/os/docs/latest/cloud-config.html#update

cmd/render.go

@@ -20,7 +20,7 @@ var (
 
 	cmdRenderCredentials = &cobra.Command{
 		Use:          "credentials",
-		Short:        "Render TLS credentials",
+		Short:        "Render credentials",
 		Long:         ``,
 		RunE:         runCmdRenderCredentials,
 		SilenceUsage: true,

core/controlplane/cluster/cluster_test.go

@@ -481,9 +481,9 @@ stackTags:
 			ExpectedContentLength: 2,
 		}
 
-		helper.WithDummyCredentials(func(dummyTlsAssetsDir string) {
+		helper.WithDummyCredentials(func(dummyAssetsDir string) {
 			var stackTemplateOptions = config.StackTemplateOptions{
-				TLSAssetsDir:          dummyTlsAssetsDir,
+				AssetsDir:             dummyAssetsDir,
 				ControllerTmplFile:    "../config/templates/cloud-config-controller",
 				EtcdTmplFile:          "../config/templates/cloud-config-etcd",
 				StackTemplateTmplFile: "../config/templates/stack-template.json",

core/controlplane/config/config.go

@@ -701,7 +701,7 @@ func releaseVersionIsGreaterThan(minVersion semver.Version, release string) (boo
 }
 
 type StackTemplateOptions struct {
-	TLSAssetsDir          string
+	AssetsDir             string
 	ControllerTmplFile    string
 	EtcdTmplFile          string
 	StackTemplateTmplFile string
@@ -718,24 +718,46 @@ func (c Cluster) StackConfig(opts StackTemplateOptions) (*StackConfig, error) {
 		return nil, err
 	}
 
+	// TODO: Check if new tests are needed to verify the auth token file is handled correctly
+
 	if c.ManageCertificates {
-		if c.TLSAssetsEncryptionEnabled() {
+		if c.AssetsEncryptionEnabled() {
 			var compactAssets *CompactTLSAssets
-			compactAssets, err = ReadOrCreateCompactTLSAssets(opts.TLSAssetsDir, KMSConfig{
+			var compactAuthTokens *CompactAuthTokens
+
+			compactAssets, err = ReadOrCreateCompactTLSAssets(opts.AssetsDir, KMSConfig{
 				Region:         stackConfig.Config.Region,
 				KMSKeyARN:      c.KMSKeyARN,
 				EncryptService: c.ProvidedEncryptService,
 			})
 			if err != nil {
 				return nil, err
 			}
+
+			compactAuthTokens, err = ReadOrCreateCompactAuthTokens(opts.AssetsDir, KMSConfig{
+				Region:         stackConfig.Config.Region,
+				KMSKeyARN:      c.KMSKeyARN,
+				EncryptService: c.ProvidedEncryptService,
+			})
+			if err != nil {
+				return nil, err
+			}
+
 			stackConfig.Config.TLSConfig = compactAssets
+			stackConfig.Config.AuthTokensConfig = compactAuthTokens
 		} else {
-			rawAssets, err := ReadOrCreateUnecryptedCompactTLSAssets(opts.TLSAssetsDir)
+			rawAssets, err := ReadOrCreateUnecryptedCompactTLSAssets(opts.AssetsDir)
+			if err != nil {
+				return nil, err
+			}
+
+			rawAuthTokens, err := ReadOrCreateUnecryptedCompactAuthTokens(opts.AssetsDir)
 			if err != nil {
 				return nil, err
 			}
+
 			stackConfig.Config.TLSConfig = rawAssets
+			stackConfig.Config.AuthTokensConfig = rawAuthTokens
 		}
 	}
 
@@ -764,6 +786,9 @@ type Config struct {
 
 	EtcdNodes []derived.EtcdNode
 
+	// Encoded auth tokens
+	AuthTokensConfig *CompactAuthTokens
+
 	// Encoded TLS assets
 	TLSConfig *CompactTLSAssets
 }
@@ -944,7 +969,7 @@ func (c DeploymentSettings) Valid() (*DeploymentValidationResult, error) {
 	if c.ClusterName == "" {
 		return nil, errors.New("clusterName must be set")
 	}
-	if c.KMSKeyARN == "" && c.TLSAssetsEncryptionEnabled() {
+	if c.KMSKeyARN == "" && c.AssetsEncryptionEnabled() {
 		return nil, errors.New("kmsKeyArn must be set")
 	}
 
@@ -1045,7 +1070,7 @@ func (c DeploymentSettings) Valid() (*DeploymentValidationResult, error) {
 	return &DeploymentValidationResult{vpcNet: vpcNet}, nil
 }
 
-func (c DeploymentSettings) TLSAssetsEncryptionEnabled() bool {
+func (c DeploymentSettings) AssetsEncryptionEnabled() bool {
 	return c.ManageCertificates && c.Region.SupportsKMS()
 }
 
@@ -1178,7 +1203,7 @@ func (c ControllerSettings) Valid() error {
 func (c Experimental) Valid() error {
 	for _, taint := range c.Taints {
 		if taint.Effect != "NoSchedule" && taint.Effect != "PreferNoSchedule" {
-			return fmt.Errorf("Effect must be NoSchdule or PreferNoSchedule, but was %s", taint.Effect)
+			return fmt.Errorf("Effect must be NoSchedule or PreferNoSchedule, but was %s", taint.Effect)
 		}
 	}
 

core/controlplane/config/config_test.go

@@ -160,8 +160,8 @@ func TestMinimalChinaConfig(t *testing.T) {
 		t.Error("IsChinaRegion test failed.")
 	}
 
-	if c.TLSAssetsEncryptionEnabled() {
-		t.Error("TLS Assets encryption must be disabled on China.")
+	if c.AssetsEncryptionEnabled() {
+		t.Error("Assets encryption must be disabled on China.")
 	}
 }
 

core/controlplane/config/stack_config_test.go

@@ -18,7 +18,7 @@ func TestRenderStackTemplate(t *testing.T) {
 
 	helper.WithDummyCredentials(func(dir string) {
 		var stackTemplateOptions = StackTemplateOptions{
-			TLSAssetsDir:          dir,
+			AssetsDir:             dir,
 			ControllerTmplFile:    "templates/cloud-config-controller",
 			EtcdTmplFile:          "templates/cloud-config-etcd",
 			StackTemplateTmplFile: "templates/stack-template.json",
@@ -52,7 +52,7 @@ func TestValidateUserData(t *testing.T) {
 
 	helper.WithDummyCredentials(func(dir string) {
 		var stackTemplateOptions = StackTemplateOptions{
-			TLSAssetsDir:          dir,
+			AssetsDir:             dir,
 			ControllerTmplFile:    "templates/cloud-config-controller",
 			EtcdTmplFile:          "templates/cloud-config-etcd",
 			StackTemplateTmplFile: "templates/stack-template.json",

core/controlplane/config/templates/cloud-config-controller

@@ -76,8 +76,8 @@ coreos:
             EnvironmentFile=-/run/flannel/etcd-endpoints.opts
             ExecStartPre=/usr/bin/systemctl is-active cfn-etcd-environment.service
             ExecStartPre=/bin/sh -ec "echo FLANNELD_ETCD_ENDPOINTS=${ETCD_ENDPOINTS} >/run/flannel/etcd-endpoints.opts"
-            {{- if .TLSAssetsEncryptionEnabled }}
-            ExecStartPre=/opt/bin/decrypt-tls-assets
+            {{- if .AssetsEncryptionEnabled }}
+            ExecStartPre=/opt/bin/decrypt-assets
             {{- end}}
             ExecStartPre=/usr/bin/etcdctl \
             --ca-file=/etc/kubernetes/ssl/ca.pem \
@@ -747,25 +747,25 @@ write_files:
 
 {{ end }}
 
-{{if .TLSAssetsEncryptionEnabled }}
-  - path: /opt/bin/decrypt-tls-assets
+{{if .AssetsEncryptionEnabled }}
+  - path: /opt/bin/decrypt-assets
     owner: root:root
     permissions: 0700
     content: |
       #!/bin/bash -e
 
       rkt run \
-        --volume=ssl,kind=host,source=/etc/kubernetes/ssl,readOnly=false \
-        --mount=volume=ssl,target=/etc/kubernetes/ssl \
-        --uuid-file-save=/var/run/coreos/decrypt-tls-assets.uuid \
+        --volume=kube,kind=host,source=/etc/kubernetes,readOnly=false \
+        --mount=volume=kube,target=/etc/kubernetes \
+        --uuid-file-save=/var/run/coreos/decrypt-assets.uuid \
         --volume=dns,kind=host,source=/etc/resolv.conf,readOnly=true --mount volume=dns,target=/etc/resolv.conf \
         --net=host \
         --trust-keys-from-https \
         {{.AWSCliImage.Options}}{{.AWSCliImage.RktRepo}} --exec=/bin/bash -- \
           -ec \
-          'echo decrypting tls assets
+          'echo decrypting assets
            shopt -s nullglob
-           for encKey in /etc/kubernetes/ssl/*.pem.enc; do
+           for encKey in /etc/kubernetes/{ssl,{{if .AuthTokensConfig.HasTokens}}auth{{end}}}/*.enc; do
              echo decrypting $encKey
              f=$(mktemp $encKey.XXXXXXXX)
              /usr/bin/aws \
@@ -778,7 +778,7 @@ write_files:
            done;
            echo done.'
 
-      rkt rm --uuid-file=/var/run/coreos/decrypt-tls-assets.uuid || :
+      rkt rm --uuid-file=/var/run/coreos/decrypt-assets.uuid || :
 {{ end }}
 
   - path: /opt/bin/taint-and-uncordon
@@ -929,6 +929,9 @@ write_files:
           - --service-cluster-ip-range={{.ServiceCIDR}}
           - --secure-port=443
           - --kubelet-preferred-address-types=InternalIP,Hostname,ExternalIP
+          {{ if .AuthTokensConfig.HasTokens }}
+          - --token-auth-file=/etc/kubernetes/auth/tokens.csv
+          {{ end }}
           {{if .Experimental.AuditLog.Enabled}}
           - --audit-log-maxage={{.Experimental.AuditLog.MaxAge}}
           - --audit-log-path={{.Experimental.AuditLog.LogPath}}
@@ -971,18 +974,28 @@ write_files:
           - mountPath: /etc/ssl/certs
             name: ssl-certs-host
             readOnly: true
+          {{if .AuthTokensConfig.HasTokens}}
+          - mountPath: /etc/kubernetes/auth
+            name: auth-kubernetes
+            readOnly: true
+          {{end}}
           {{if .Experimental.Authentication.Webhook.Enabled}}
           - mountPath: /etc/kubernetes/webhooks
             name: kubernetes-webhooks
             readOnly: true
-          {{ end }}
+          {{end}}
         volumes:
         - hostPath:
             path: /etc/kubernetes/ssl
           name: ssl-certs-kubernetes
         - hostPath:
             path: /usr/share/ca-certificates
           name: ssl-certs-host
+        {{if .AuthTokensConfig.HasTokens}}
+        - hostPath:
+            path: /etc/kubernetes/auth
+          name: auth-kubernetes
+        {{end}}
         {{if .Experimental.Authentication.Webhook.Enabled}}
         - hostPath:
             path: /etc/kubernetes/webhooks
@@ -1425,24 +1438,30 @@ write_files:
           - port: 80
             targetPort: 9090
 
+{{ if .AuthTokensConfig.HasTokens }}
+  - path: /etc/kubernetes/auth/tokens.csv{{if .AssetsEncryptionEnabled}}.enc{{end}}
+    encoding: gzip+base64
+    content: {{.AuthTokensConfig.Contents}}
+{{ end }}
+
 {{ if .ManageCertificates }}
-  - path: /etc/kubernetes/ssl/ca.pem{{if .TLSAssetsEncryptionEnabled}}.enc{{end}}
+  - path: /etc/kubernetes/ssl/ca.pem{{if .AssetsEncryptionEnabled}}.enc{{end}}
     encoding: gzip+base64
     content: {{.TLSConfig.CACert}}
 
-  - path: /etc/kubernetes/ssl/apiserver.pem{{if .TLSAssetsEncryptionEnabled}}.enc{{end}}
+  - path: /etc/kubernetes/ssl/apiserver.pem{{if .AssetsEncryptionEnabled}}.enc{{end}}
     encoding: gzip+base64
     content: {{.TLSConfig.APIServerCert}}
 
-  - path: /etc/kubernetes/ssl/apiserver-key.pem{{if .TLSAssetsEncryptionEnabled}}.enc{{end}}
+  - path: /etc/kubernetes/ssl/apiserver-key.pem{{if .AssetsEncryptionEnabled}}.enc{{end}}
     encoding: gzip+base64
     content: {{.TLSConfig.APIServerKey}}
 
-  - path: /etc/kubernetes/ssl/etcd-client.pem{{if .TLSAssetsEncryptionEnabled}}.enc{{end}}
+  - path: /etc/kubernetes/ssl/etcd-client.pem{{if .AssetsEncryptionEnabled}}.enc{{end}}
     encoding: gzip+base64
     content: {{.TLSConfig.EtcdClientCert}}
 
-  - path: /etc/kubernetes/ssl/etcd-client-key.pem{{if .TLSAssetsEncryptionEnabled}}.enc{{end}}
+  - path: /etc/kubernetes/ssl/etcd-client-key.pem{{if .AssetsEncryptionEnabled}}.enc{{end}}
     encoding: gzip+base64
     content: {{.TLSConfig.EtcdClientKey}}
 {{ end }}