From 0402909689c424e5a306de3410d2ae98c75497b2 Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Tue, 3 Oct 2017 13:13:36 -0400 Subject: [PATCH] First tutorial page: getting started with GCE --- docs/tutorial/gce.md | 188 +++++++++++++++++++ docs/tutorial/upgrading-kubernetes.md | 128 +++++++++++++ docs/tutorial/working-with-instancegroups.md | 186 ++++++++++++++++++ 3 files changed, 502 insertions(+) create mode 100644 docs/tutorial/gce.md create mode 100644 docs/tutorial/upgrading-kubernetes.md create mode 100644 docs/tutorial/working-with-instancegroups.md diff --git a/docs/tutorial/gce.md b/docs/tutorial/gce.md new file mode 100644 index 0000000000000..59aaac886c790 --- /dev/null +++ b/docs/tutorial/gce.md @@ -0,0 +1,188 @@ +# Getting Started with kops on GCE + +Make sure you have [installed kops](../install.md) and [installed kubectl](../install.md), and installed +the [gcloud tools](https://cloud.google.com/sdk/downloads). + +You'll need a Google Cloud account, and make sure that gcloud is logged in to your account using `gcloud init`. + +You should confirm that basic commands like `gcloud compute zones list` are working. + +You'll also need to [configure default credentials](https://developers.google.com/accounts/docs/application-default-credentials), using `gcloud auth application-default login`. + + + +# Creating a state store + +kops needs a state store, to hold the configuration for your clusters. The simplest configuration +for Google Cloud is to store it in a Google Cloud Storage bucket in the same account, so that's how we'll +start. + +So, just create an empty bucket - you can use any name: `gsutil mb s3://kubernetes-clusters/` + +# Creating our first cluster + +`kops create cluster` creates the Cluster and InstanceGroup objects you'll be working with in kops: + +``` +PROJECT=`gcloud config get-value project` +export KOPS_FEATURE_FLAGS=AlphaAllowGCE # to unlock the GCE features +kops create cluster simple.k8s.local --zones us-central1-a --state gs://kubernetes-clusters/ --project=${PROJECT} +``` + +You can now list the clusters in your kops state store (the GCS bucket we created): + +`kops get cluster --state gs://kubernetes-clusters/` + +``` +NAME CLOUD ZONES +simple.k8s.local gce us-central1-a +``` + + + +This shows that you have one Cluster configured, named `simple.k8s.local`. The Cluster holds the cluster-wide configuration for +a kubernetes cluster - things like the kubernetes version, and the authorization policy in use. + +The `kops` tool should feel a lot like `kubectl` - kops uses the same API machinery as kubernetes, +so it should behave similarly, although now you are managing kubernetes clusters, instead of managing +objects on a kubernetes cluster. + +You can see the details of your Cluster by doing: + +`> kops get cluster --state gs://kubernetes-clusters/ simple.k8s.local -oyaml` +``` +apiVersion: kops/v1alpha2 +kind: Cluster +metadata: + creationTimestamp: 2017-10-03T05:07:27Z + name: simple.k8s.local +spec: + api: + loadBalancer: + type: Public + authorization: + alwaysAllow: {} + channel: stable + cloudProvider: gce + configBase: gs://kubernetes-clusters/simple.k8s.local + etcdClusters: + - etcdMembers: + - instanceGroup: master-us-central1-a + name: a + name: main + - etcdMembers: + - instanceGroup: master-us-central1-a + name: a + name: events + iam: + legacy: false + kubernetesApiAccess: + - 0.0.0.0/0 + kubernetesVersion: 1.7.2 + masterPublicName: api.simple.k8s.local + networking: + kubenet: {} + nonMasqueradeCIDR: 100.64.0.0/10 + project: my-gce-project + sshAccess: + - 0.0.0.0/0 + subnets: + - name: us-central1 + region: us-central1 + type: Public + topology: + dns: + type: Public + masters: public + nodes: public +``` + +Similarly, you can also see your InstanceGroups using: + +`kops get instancegroup --state gs://kubernetes-clusters/ --name simple.k8s.local` +``` +NAME ROLE MACHINETYPE MIN MAX SUBNETS +master-us-central1-a Master n1-standard-1 1 1 us-central1 +nodes Node n1-standard-2 2 2 us-central1 +``` + + + +InstanceGroups are the other main kops object - an InstanceGroup manages a set of cloud instances, +which then are registered in kubernetes as Nodes. You have multiple InstanceGroups for different types +of instances / Nodes - in our simple example we have one for our master (which only has a single member), +and one for our nodes (and we have two nodes configured). + +We'll see a lot more of Clusters and InstanceGroups as we use kops to reconfigure clusters. But let's get +on with our first cluster. + +# Export KOPS_STATE_STORE + +Rather than typing the `--state` argument every time, it's much easier to export the `KOPS_STATE_STORE` +environment variable: + +``` +export KOPS_STATE_STORE=gs://kubernetes-clusters/ +``` + +You can also put this in your `~/.bashrc` or similar. + +# Creating a cluster + +`kops create cluster` created the Cluster & InstanceGroup objects in our state store, +but didn't actually create any instances or other cloud objects in GCE. To do that, we'll use +`kops update cluster`. + +`kops update cluster` without `--yes` will show you a preview of all the changes will be made; +it is very useful to see what kops is about to do, before actually making the changes. + +Run `kops update cluster simple.k8s.local` and peruse the changes. + +We're now finally ready to create the object: `kops update cluster simple.k8s.local --yes` + +(If you haven't created an SSH key, you'll have to `ssh-keygen -t rsa`) + + + +Your cluster is created in the background - kops actually creates GCE Managed Instance Groups +that run the instances; this ensures that even if instances are terminated, they will automatically +be relaunched by GCE and your cluster will self-heal. + +After a few minutes, you should be able to do `kubectl get nodes` and your first cluster should be ready! + +# Enjoy + +At this point you have a kubernetes cluster - the core commands to do so are as simple as `kops create cluster` +and `kops update cluster`. There's a lot more power in kops, and even more power in kubernetes itself, so we've +put a few jumping off places here. But when you're done, don't forget to [delete your cluster](#deleting-the-cluster). + +* [Manipulate InstanceGroups](working-with-instancegroups.md) to add more nodes, change image + +# Deleting the cluster + +When you're done using the cluster, you should delete it to release the cloud resources. `kops delete cluster` is +the command. When run without `--yes` it shows a preview of the objects it will delete: + +``` +> kops delete cluster simple.k8s.local +TYPE NAME ID +Address api-simple-k8s-local api-simple-k8s-local +Disk a-etcd-events-simple-k8s-local a-etcd-events-simple-k8s-local +Disk a-etcd-main-simple-k8s-local a-etcd-main-simple-k8s-local +ForwardingRule api-simple-k8s-local api-simple-k8s-local +Instance master-us-central1-a-9847 us-central1-a/master-us-central1-a-9847 +Instance nodes-0s0w us-central1-a/nodes-0s0w +Instance nodes-dvlq us-central1-a/nodes-dvlq +InstanceGroupManager a-master-us-central1-a-simple-k8s-local us-central1-a/a-master-us-central1-a-simple-k8s-local +InstanceGroupManager a-nodes-simple-k8s-local us-central1-a/a-nodes-simple-k8s-local +InstanceTemplate master-us-central1-a-simple-k8s-local-1507008700 master-us-central1-a-simple-k8s-local-1507008700 +InstanceTemplate nodes-simple-k8s-local-1507008700 nodes-simple-k8s-local-1507008700 +Route simple-k8s-local-715bb0c7-a7fc-11e7-93d7-42010a800002 simple-k8s-local-715bb0c7-a7fc-11e7-93d7-42010a800002 +Route simple-k8s-local-9a2a08e8-a7fc-11e7-93d7-42010a800002 simple-k8s-local-9a2a08e8-a7fc-11e7-93d7-42010a800002 +Route simple-k8s-local-9c17a4e6-a7fc-11e7-93d7-42010a800002 simple-k8s-local-9c17a4e6-a7fc-11e7-93d7-42010a800002 +TargetPool api-simple-k8s-local api-simple-k8s-local + +Must specify --yes to delete cluster +``` + +After you've double-checked you're deleting exactly what you want to delete, run `kops delete cluster simple.k8s.local --yes`. diff --git a/docs/tutorial/upgrading-kubernetes.md b/docs/tutorial/upgrading-kubernetes.md new file mode 100644 index 0000000000000..b3beaffd8d597 --- /dev/null +++ b/docs/tutorial/upgrading-kubernetes.md @@ -0,0 +1,128 @@ +# Upgrading kubernetes + +Upgrading kubernetes is very easy with kops, as long as you are using a compatible version of kops. +The kops `1.8.x` series (for example) supports the kubernetes 1.6, 1.7 and 1.8 series, +as per the kubernetes deprecation policy. Older versions of kubernetes will likely still work, but these +are on a best-effort basis and will have little if any testing. kops `1.8` will not support the kubernetes +`1.9` series, and for full support of kubernetes `1.9` it is best to wait for the kops `1.9` series release. +We aim to release the next major version of kops within a few weeks of the equivalent major release of kubernetes, +so kops `1.9.0` will be released within a few weeks of kubernetes `1.9.0`. We try to ensure that a 1.9 pre-release +(alpha or beta) is available at the kubernetes release, for early adopters. + +Upgrading kubernetes is similar to changing the image on an InstanceGroup, except that the kubernetes version is +controlled at the cluster level. So instead of `kops edit ig `, we `kops edit cluster`, and change the +`kubernetesVersion` field. `kops edit cluster` will open your editor with the cluster, similar to: + +``` +# Please edit the object below. Lines beginning with a '#' will be ignored, +# and an empty file will abort the edit. If an error occurs while saving this file will be +# reopened with the relevant failures. +# +apiVersion: kops/v1alpha2 +kind: Cluster +metadata: + creationTimestamp: 2017-10-04T03:52:25Z + name: simple.k8s.local +spec: + api: + loadBalancer: + type: Public + authorization: + alwaysAllow: {} + channel: stable + cloudProvider: gce + configBase: gs://kubernetes-clusters/simple.k8s.local + etcdClusters: + - etcdMembers: + - instanceGroup: master-us-central1-a + name: a + name: main + - etcdMembers: + - instanceGroup: master-us-central1-a + name: a + name: events + iam: + legacy: false + kubernetesApiAccess: + - 0.0.0.0/0 + kubernetesVersion: 1.7.2 + masterInternalName: api.internal.simple.k8s.local + masterPublicName: api.simple.k8s.local + networking: + kubenet: {} + nonMasqueradeCIDR: 100.64.0.0/10 + project: gce-project + sshAccess: + - 0.0.0.0/0 + subnets: + - name: us-central1 + region: us-central1 + type: Public + topology: + dns: + type: Public + masters: public + nodes: public +``` + +Edit `kubernetesVersion`, changing it to `1.7.7` for example. + + +Apply the changes to the cloud infrastructure using `kops update cluster` and `kops update cluster --yes`: + +``` +Will create resources: + InstanceTemplate/master-us-central1-a-simple-k8s-local + Network name:default id:default + Tags [simple-k8s-local-k8s-io-role-master] + Preemptible false + BootDiskImage cos-cloud/cos-stable-57-9202-64-0 + BootDiskSizeGB 64 + BootDiskType pd-standard + CanIPForward true + Scopes [compute-rw, monitoring, logging-write, storage-ro, https://www.googleapis.com/auth/ndev.clouddns.readwrite] + Metadata {cluster-name: , startup-script: } + MachineType n1-standard-1 + + InstanceTemplate/nodes-simple-k8s-local + Network name:default id:default + Tags [simple-k8s-local-k8s-io-role-node] + Preemptible false + BootDiskImage debian-cloud/debian-9-stretch-v20170918 + BootDiskSizeGB 128 + BootDiskType pd-standard + CanIPForward true + Scopes [compute-rw, monitoring, logging-write, storage-ro] + Metadata {startup-script: , cluster-name: } + MachineType n1-standard-2 + +Will modify resources: + InstanceGroupManager/us-central1-a-master-us-central1-a-simple-k8s-local + InstanceTemplate id:master-us-central1-a-simple-k8s-local-1507089163 -> name:master-us-central1-a-simple-k8s-local + + InstanceGroupManager/us-central1-a-nodes-simple-k8s-local + InstanceTemplate id:nodes-simple-k8s-local-1507089694 -> name:nodes-simple-k8s-local +``` + + +`kops rolling-update cluster` will show that all nodes need to be restarted. + +``` +NAME STATUS NEEDUPDATE READY MIN MAX NODES +master-us-central1-a NeedsUpdate 1 0 1 1 1 +nodes NeedsUpdate 3 0 3 3 3 +``` + +Restart the instances with `kops rolling-update cluster --yes`. + +``` +> kubectl get nodes -owide +NAME STATUS AGE VERSION EXTERNAL-IP OS-IMAGE KERNEL-VERSION +master-us-central1-a-8fcc Ready 26m v1.7.7 35.194.56.129 Container-Optimized OS from Google 4.4.35+ +nodes-9cml Ready 16m v1.7.7 35.193.12.73 Ubuntu 16.04.3 LTS 4.10.0-35-generic +nodes-km98 Ready 10m v1.7.7 35.194.25.144 Ubuntu 16.04.3 LTS 4.10.0-35-generic +nodes-wbb2 Ready 2m v1.7.7 35.188.177.16 Ubuntu 16.04.3 LTS 4.10.0-35-generic +``` + + + diff --git a/docs/tutorial/working-with-instancegroups.md b/docs/tutorial/working-with-instancegroups.md new file mode 100644 index 0000000000000..d02a83e9b7705 --- /dev/null +++ b/docs/tutorial/working-with-instancegroups.md @@ -0,0 +1,186 @@ +# Working with InstanceGroups + +The kops InstanceGroup is a declarative model of a group of nodes. By modifying the object, you +can change the instance type you're using, the number of nodes you have, the OS image you're running - essentially +all the per-node configuration is in the InstanceGroup. + +We'll assume you have a working cluster - if not, you probably want to read [how to get started on GCE](gce.md). + +## Changing the number of nodes + +If you `kops get ig` you should see that you have instance groups for your nodes and for your master: + +``` +> kops get ig +NAME ROLE MACHINETYPE MIN MAX SUBNETS +master-us-central1-a Master n1-standard-1 1 1 us-central1 +nodes Node n1-standard-2 2 2 us-central1 +``` + +Let's change the number of nodes to 3. We'll edit the instancegroup configuration using `kops edit` (which +should be very familiar to you if you've used `kubectl edit`). `kops edit ig nodes` will open +the instancegroup in your editor, looking a bit like this: + +``` +apiVersion: kops/v1alpha2 +kind: InstanceGroup +metadata: + creationTimestamp: 2017-10-03T15:17:31Z + labels: + kops.k8s.io/cluster: simple.k8s.local + name: nodes +spec: + image: cos-cloud/cos-stable-57-9202-64-0 + machineType: n1-standard-2 + maxSize: 2 + minSize: 2 + role: Node + subnets: + - us-central1 + zones: + - us-central1-a +``` + + + +Edit `minSize` and `maxSize`, changing both from 2 to 3, save and exit your editor. If you wanted to change +the image or the machineType, you could do that here as well. There are actually a lot more fields, +but most of them have their default values, so won't show up unless they are set. The general approach is the same though. + + + +On saving you'll note that nothing happens. Although you've changed the model, you need to tell kops to +apply your changes to the cloud. + + + +We use the same `kops update cluster` command that we used when initially creating the cluster; when +run without `--yes` it should show you a preview of the changes, and now there should be only one change: + +``` +> kops update cluster +Will modify resources: + InstanceGroupManager/us-central1-a-nodes-simple-k8s-local + TargetSize 2 -> 3 +``` + +This is saying that we will alter the `TargetSize` property of the `InstanceGroupManager` object named +`us-central1-a-nodes-simple-k8s-local`, changing it from 2 to 3. + +That's what we want, so we `kops update cluster --yes`. + + + +kops will resize the GCE managed instance group from 2 to 3, which will create a new GCE instance, +which will then boot and join the cluster. Within a minute or so you should see the new node join: + +``` +> kubectl get nodes +NAME STATUS AGE VERSION +master-us-central1-a-thjq Ready 10h v1.7.2 +nodes-g2v2 Ready 10h v1.7.2 +nodes-tmk8 Ready 10h v1.7.2 +nodes-z2cz Ready 1s v1.7.2 +``` + +`nodes-z2cz` just joined our cluster! + + +## Changing the image + +That was a fairly simple change, because we didn't have to reboot the nodes. Most changes though do +require rolling your instances - this is actually a deliberate design decision, in that we are aiming +for immutable nodes. An example is changing your image. We're using `cos-stable`, which is Google's +Container OS. Let's try Debian Stretch instead. + +If you run `gcloud compute images list` to list the images available to you in GCE, you should see +a debian-9 image: + +``` +> gcloud compute images list +... +debian-9-stretch-v20170918 debian-cloud debian-9 READY +... +``` + + + +So now we'll do the same `kops edit ig nodes`, except this time change the image to `debian-cloud/debian-9-stretch-v20170918`: + +Now `kops update cluster` will show that you're going to create a new [GCE Instance Template](https://cloud.google.com/compute/docs/reference/latest/instanceTemplates), +and that the Managed Instance Group is going to use it: + +``` +Will create resources: + InstanceTemplate/nodes-simple-k8s-local + Network name:default id:default + Tags [simple-k8s-local-k8s-io-role-node] + Preemptible false + BootDiskImage debian-cloud/debian-9-stretch-v20170918 + BootDiskSizeGB 128 + BootDiskType pd-standard + CanIPForward true + Scopes [compute-rw, monitoring, logging-write, storage-ro] + Metadata {cluster-name: , startup-script: } + MachineType n1-standard-2 + +Will modify resources: + InstanceGroupManager/us-central1-a-nodes-simple-k8s-local + InstanceTemplate id:nodes-simple-k8s-local-1507043948 -> name:nodes-simple-k8s-local +``` + +Note that the `BootDiskImage` is indeed set to the debian 9 image you requested. + +`kops update cluster --yes` will now apply the change, but if you were to run `kubectl get nodes` you would see +that the instances had not yet been reconfigured. There's a hint at the bottom: + +``` +Changes may require instances to restart: kops rolling-update cluster` +``` + +These changes require your instances to restart (we'll remove the COS images and replace them with Debian images). kops +can perform a rolling update to minimize disruption, but even so you might not want to perform the update right away; +you might want to make more changes or you might want to wait for off-peak hours. You might just want to wait for +the instances to terminate naturally - new instances will come up with the new configuration - though if you're not +using preemptible/spot instances you might be waiting for a long time. + +## Performing a rolling-update of your cluster + +When you're ready to force your instances to restart, use `kops rollling-update cluster`: + +``` +> kops rolling-update cluster +Using cluster from kubectl context: simple.k8s.local + +NAME STATUS NEEDUPDATE READY MIN MAX NODES +master-us-central1-a Ready 0 1 1 1 1 +nodes NeedsUpdate 3 0 3 3 3 + +Must specify --yes to rolling-update. +``` + +You can see that your nodes need to be restarted, and your masters do not. A `kops rolling-update cluster --yes` will perform the update. +It will only restart instances that need restarting (unless you `--force` a rolling-update). + +When you're ready, do `kops rolling-update cluster --yes`. It'll take a few minutes per node, because for each node +we cordon the node, drain the pods, shut it down and wait for the new node to join the cluster and for the cluster +to be healthy again. But this procedure minimizes disruption to your cluster - a rolling-update cluster is never +going to be something you do during your superbowl commercial, but ideally it should be minimally disruptive. + + + + + +After the rolling-update is complete, you can see that the nodes are now running a new image: + +``` +> kubectl get nodes -owide +NAME STATUS AGE VERSION EXTERNAL-IP OS-IMAGE KERNEL-VERSION +master-us-central1-a-8fcc Ready 48m v1.7.2 35.188.177.16 Container-Optimized OS from Google 4.4.35+ +nodes-9cml Ready 17m v1.7.2 35.194.25.144 Container-Optimized OS from Google 4.4.35+ +nodes-km98 Ready 11m v1.7.2 35.202.95.161 Container-Optimized OS from Google 4.4.35+ +nodes-wbb2 Ready 5m v1.7.2 35.194.56.129 Container-Optimized OS from Google 4.4.35+ +``` + + +Next steps: learn how to perform cluster-wide operations, like [upgrading kubernetes](upgrading-kubernetes.md). \ No newline at end of file