KubernetesCluster tag is not applied to existing objects

[flah00]

I opted to use an existing VPC and existing subnets, because I wanted to stick with my existing public and private subnets. Unfortunately, none of the subnets or routing tables were tagged with the KubernetesCluster key and value. As best as I can tell, this prevented me from creating LoadBalancer services in my public subnet. After I tagged the configured subnets, I was able to create LoadBalancer services, with public IP addresses.

kube-aws, it seems, should be able to tag existing resources, to avoid headaches like this.

[mumoshu]

Hi @flah00,

Thanks for sharing your experience and findings!
I'd definitely like to document the fact you need to manually tag existing AWS resources if you opted to use them like described in the upstream doc.

However, I don't (currently) think that it is a good idea to make kube-aws tag them for you.
The main reason is that we can't reliably clean up those tags even if they aren't needed anymore, because we can't let CloudFormation to "tag existing resources".
Tagging existing resources outside of CloudFormation would easily end up with unnecessary tags remained. An example is that you'd initially created a kubernetes cluster via kube-aws and then destroyed it via CloudFormation console. kube-aws could tag existing resources at creation but there's no chance for kube-aws to clean up these tags in such case. Additional headaches.

Does this make sense to you?
Anyways I'd like to document it so that we can better support use-cases like yours.

Oh, and I'd definitely recommend not to reuse subnets but VPCs and route tables.
That's why we'd been not supporting reusing of existing subnets as of today.

Related historical issues i.e. recommended readings:

Ability to use existing VPC subnets by c-knowles · Pull Request #671 · coreos/coreos-kubernetes
Ability to use existing route tables for controller and workers by c-knowles · Pull Request #716 · coreos/coreos-kubernetes
Allow workers to sit inside private subnets within an existing VPC · Issue #44 · coreos/kube-aws

[flah00]

@mumoshu That definitely makes sense. As for the reuse of existing subnets, I misspoke. We are only reusing the vpc and route table.

BTW, thanks for this utility, it's been a lifesaver.

[mumoshu]

@flah00 Thanks for clarification 👍

Please let me confirm your situation a bit more. You had to manually tag your existing VPC and/or route table to successfully create type= LoadBalancer services, right?

[flah00]

I have two private subnets and three public subnets. I had to tag all of my subnets and route tables, with the name of my kubernetes cloudformation stack (key KubernetesCluster value STACKNAME), to create a LoadBalancer in my public subnet.

Without the tag, ELBs were always created in the private subnet, without a public IP.

[mumoshu]

@flah00 Thanks for the additional info!

We probably need to document that just for existing route tables then because subnets created via kube-aws are already tagged with KubernetesCluster.

[mumoshu]

I'm closing this as it is now documented in #162 🙇