Create cluster requirements for DigitalOcean #3248

andrewsykim · 2017-08-21T22:49:32Z

Initial changes required to create a cluster state. Running kops update cluster --yes does not work yet.

Note that DO has already adopted cloud controller managers (https://github.com/digitalocean/digitalocean-cloud-controller-manager) so we set --cloud-provider=external. This will end up being the case for aws, gce and vsphere over the next couple of releases.

#2150

$ kops create cluster --cloud=digitalocean --name=dev.asykim.com --zones=tor1
I0821 18:47:06.302218   28623 create_cluster.go:845] Using SSH public key: /Users/AndrewSyKim/.ssh/id_rsa.pub
I0821 18:47:06.302293   28623 subnets.go:183] Assigned CIDR 172.20.32.0/19 to subnet tor1
Previewing changes that will be made:

I0821 18:47:11.457696   28623 executor.go:91] Tasks: 0 done / 27 total; 27 can run
I0821 18:47:12.113133   28623 executor.go:91] Tasks: 27 done / 27 total; 0 can run
Will create resources:
  Keypair/kops
  	Subject             	o=system:masters,cn=kops
  	Type                	client

  Keypair/kube-controller-manager
  	Subject             	cn=system:kube-controller-manager
  	Type                	client

  Keypair/kube-proxy
  	Subject             	cn=system:kube-proxy
  	Type                	client

  Keypair/kube-scheduler
  	Subject             	cn=system:kube-scheduler
  	Type                	client

  Keypair/kubecfg
  	Subject             	o=system:masters,cn=kubecfg
  	Type                	client

  Keypair/kubelet
  	Subject             	o=system:nodes,cn=kubelet
  	Type                	client

  Keypair/kubelet-api
  	Subject             	cn=kubelet-api
  	Type                	client

  Keypair/master
  	Subject             	cn=kubernetes-master
  	Type                	server
  	AlternateNames      	[100.64.0.1, 127.0.0.1, api.dev.asykim.com, api.internal.dev.asykim.com, kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster.local]

  ManagedFile/dev.asykim.com-addons-bootstrap
  	Location            	addons/bootstrap-channel.yaml

  ManagedFile/dev.asykim.com-addons-core.addons.k8s.io
  	Location            	addons/core.addons.k8s.io/v1.4.0.yaml

  ManagedFile/dev.asykim.com-addons-dns-controller.addons.k8s.io-k8s-1.6
  	Location            	addons/dns-controller.addons.k8s.io/k8s-1.6.yaml

  ManagedFile/dev.asykim.com-addons-dns-controller.addons.k8s.io-pre-k8s-1.6
  	Location            	addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml

  ManagedFile/dev.asykim.com-addons-kube-dns.addons.k8s.io-k8s-1.6
  	Location            	addons/kube-dns.addons.k8s.io/k8s-1.6.yaml

  ManagedFile/dev.asykim.com-addons-kube-dns.addons.k8s.io-pre-k8s-1.6
  	Location            	addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml

  ManagedFile/dev.asykim.com-addons-limit-range.addons.k8s.io
  	Location            	addons/limit-range.addons.k8s.io/v1.5.0.yaml

  ManagedFile/dev.asykim.com-addons-storage-aws.addons.k8s.io
  	Location            	addons/storage-aws.addons.k8s.io/v1.6.0.yaml

  Secret/admin

  Secret/kube

  Secret/kube-proxy

  Secret/kubelet

  Secret/system:controller_manager

  Secret/system:dns

  Secret/system:logging

  Secret/system:monitoring

  Secret/system:scheduler

Must specify --yes to apply changes

Cluster configuration has been created.

Suggestions:
 * list clusters with: kops get cluster
 * edit this cluster with: kops edit cluster dev.asykim.com
 * edit your node instance group: kops edit ig --name=dev.asykim.com nodes
 * edit your master instance group: kops edit ig --name=dev.asykim.com master-tor1

Finally configure your cluster with: kops update cluster dev.asykim.com --yes

andrewsykim · 2017-08-21T22:50:04Z

pkg/resources/do.go

+
+package resources
+
+func (c *ClusterResources) listResourcesDO() (map[string]*ResourceTracker, error) {


This is temporary, only to get cluster state creation working, going to work on a refactor to make this work better

justinsb · 2017-08-22T06:43:18Z

pkg/model/master_volumes.go

@@ -141,6 +141,8 @@ func (b *MasterVolumeBuilder) addAWSVolume(c *fi.ModelBuilderContext, name strin
 }

 func (b *MasterVolumeBuilder) addDOVolume(c *fi.ModelBuilderContext, name string, volumeSize int32, subnet *kops.ClusterSubnetSpec, etcd *kops.EtcdClusterSpec, m *kops.EtcdMemberSpec, allMembers []string) {
+	// required that names start with a lower case and only contains letters, numbers and hyphens
+	name = "kops-" + strings.Replace(name, ".", "-", -1)


Probably a good idea to check for a length limit as well - that's the other one that is pretty common. We have this code for ELBs: https://github.com/kubernetes/kops/blob/master/pkg/model/context.go#L75-L91 .

justinsb · 2017-08-22T06:46:42Z

upup/models/config/components/kube-apiserver/_do/kube-apiserver.do.options

@@ -0,0 +1,2 @@
+KubeAPIServer:
+  CloudProvider: external


We are trying to move this logic to https://github.com/kubernetes/kops/blob/master/pkg/model/components/apiserver.go
This does rely on the do tag, which I do see you define later, but soon we hope to have fully moved the tag / options logic to code (it is much easier to understand, IMO)

Noted. It was really difficult trying to figure out who to change the CloudProvider flag for kube-apiserver, I'll follow up with this in another PR

Created an issue to clean this up for all clouds #3253

justinsb · 2017-08-22T06:47:43Z

upup/pkg/fi/cloudup/apply_cluster.go

@@ -649,7 +648,7 @@ func (c *ApplyClusterCmd) Run() error {
 			BootstrapScript: bootstrapScriptBuilder,
 			Lifecycle:       clusterLifecycle,
 		})
-
+	case kops.CloudProviderDO:


Might be good to have a // Digital Ocean tasks will go here comment here, just so we don't think this is a fall-through

justinsb · 2017-08-22T06:48:51Z

upup/pkg/fi/cloudup/tagbuilder.go

@@ -66,6 +66,10 @@ func buildCloudupTags(cluster *api.Cluster) (sets.String, error) {
 		{
 			tags.Insert("_aws")
 		}
+	case "digitalocean":
+		{
+			tags.Insert("_do")


Hopefully we'll be rid of these tags soon, but I agree that for now we should add them - more logically coherent to be consistent with the other providers, doesn't create a lot more work

justinsb · 2017-08-22T06:49:28Z

/lgtm

k8s-github-robot · 2017-08-22T07:07:00Z

/test all [submit-queue is verifying that this PR is safe to merge]

k8s-github-robot · 2017-08-22T13:07:54Z

/lgtm cancel //PR changed after LGTM, removing LGTM. @andrewsykim @justinsb

andrewsykim · 2017-08-22T14:16:25Z

Comments addressed

justinsb · 2017-08-22T21:59:46Z

/lgtm

k8s-github-robot · 2017-08-22T22:00:15Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andrewsykim, justinsb

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

~~OWNERS~~ [justinsb]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

k8s-github-robot · 2017-08-22T22:00:41Z

/test all [submit-queue is verifying that this PR is safe to merge]

andrewsykim · 2017-08-22T22:59:17Z

/retest

justinsb · 2017-08-22T23:14:16Z

Flakiness is being tracked in upstream issue: kubernetes/kubernetes#51128, hopefully fixed by kubernetes/kubernetes#51144

eparis · 2017-08-23T00:03:03Z

/retest
merged the referenced fix

k8s-github-robot · 2017-08-23T01:05:56Z

/test all [submit-queue is verifying that this PR is safe to merge]

k8s-github-robot · 2017-08-23T01:41:09Z

Automatic merge from submit-queue

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 21, 2017

andrewsykim commented Aug 21, 2017

View reviewed changes

justinsb reviewed Aug 22, 2017

View reviewed changes

k8s-ci-robot assigned justinsb Aug 22, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 22, 2017

k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 22, 2017

andrewsykim mentioned this pull request Aug 22, 2017

Migrate kube-apiserver cloud provider logic out of tag builders #3253

Closed

Create cluster requirements for DigitalOcean

0411ba2

andrewsykim force-pushed the do branch from e766eab to 0411ba2 Compare August 22, 2017 13:07

k8s-github-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 22, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 22, 2017

k8s-github-robot merged commit 7a2ff89 into kubernetes:master Aug 23, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create cluster requirements for DigitalOcean #3248

Create cluster requirements for DigitalOcean #3248

andrewsykim commented Aug 21, 2017 •

edited

Loading

andrewsykim Aug 21, 2017

justinsb Aug 22, 2017

justinsb Aug 22, 2017

andrewsykim Aug 22, 2017

andrewsykim Aug 22, 2017

justinsb Aug 22, 2017

justinsb Aug 22, 2017

justinsb commented Aug 22, 2017

k8s-github-robot commented Aug 22, 2017

k8s-github-robot commented Aug 22, 2017

andrewsykim commented Aug 22, 2017 •

edited

Loading

justinsb commented Aug 22, 2017

k8s-github-robot commented Aug 22, 2017

k8s-github-robot commented Aug 22, 2017

andrewsykim commented Aug 22, 2017

justinsb commented Aug 22, 2017

eparis commented Aug 23, 2017

k8s-github-robot commented Aug 23, 2017

k8s-github-robot commented Aug 23, 2017


		package resources

		func (c ClusterResources) listResourcesDO() (map[string]ResourceTracker, error) {

Create cluster requirements for DigitalOcean #3248

Create cluster requirements for DigitalOcean #3248

Conversation

andrewsykim commented Aug 21, 2017 • edited Loading

andrewsykim Aug 21, 2017

Choose a reason for hiding this comment

justinsb Aug 22, 2017

Choose a reason for hiding this comment

justinsb Aug 22, 2017

Choose a reason for hiding this comment

andrewsykim Aug 22, 2017

Choose a reason for hiding this comment

andrewsykim Aug 22, 2017

Choose a reason for hiding this comment

justinsb Aug 22, 2017

Choose a reason for hiding this comment

justinsb Aug 22, 2017

Choose a reason for hiding this comment

justinsb commented Aug 22, 2017

k8s-github-robot commented Aug 22, 2017

k8s-github-robot commented Aug 22, 2017

andrewsykim commented Aug 22, 2017 • edited Loading

justinsb commented Aug 22, 2017

k8s-github-robot commented Aug 22, 2017

k8s-github-robot commented Aug 22, 2017

andrewsykim commented Aug 22, 2017

justinsb commented Aug 22, 2017

eparis commented Aug 23, 2017

k8s-github-robot commented Aug 23, 2017

k8s-github-robot commented Aug 23, 2017

andrewsykim commented Aug 21, 2017 •

edited

Loading

andrewsykim commented Aug 22, 2017 •

edited

Loading