Create easy way to deploy @justinsb's DNS controller for kube-aws

[whereisaaron]

@justinsb added a DNS controller to kops, it supports annotations on Ingress, Services, and Pods and automatically creates DNS entries in Route53 or Google Cloud DNS.

• dns.alpha.kubernetes.io/external will set up records for accessing the resource externally
• dns.alpha.kubernetes.io/internal will set up records for accessing the resource internally

https://github.com/kubernetes/kops/tree/master/dns-controller

https://github.com/kubernetes/kops/tree/master/images/dns-controller-builder

Is this something that could be included as an add-on for kube-aws? Or is there an alternative, better approach? I'm going to try it out on a test cluster.

dns-controller-gocode:
    go install k8s.io/kops/dns-controller/cmd/dns-controller

dns-controller-builder-image:
    docker build -f images/dns-controller-builder/Dockerfile -t dns-controller-builder .

dns-controller-build-in-docker: dns-controller-builder-image
    docker run -it -v `pwd`:/src dns-controller-builder /onbuild.sh

dns-controller-image: dns-controller-build-in-docker
    docker build -t ${DOCKER_REGISTRY}/dns-controller:1.3  -f images/dns-controller/Dockerfile .

dns-controller-push: dns-controller-image
    docker push ${DOCKER_REGISTRY}/dns-controller:1.3

[colhom]

@whereisaaron in order to even consider shipping the component with kube-aws, this controller would have to be included in the hyperkube image @aaronlevy and @pbx0 maintain.

Barriers to that happening:

• How will we deal with versioning it? AFAICT, kops does not have a release/versioning system as of now
• unit tests
• e2e/conformance test modules to cover this functionality

In addition- I worry that by adopting this component in our hyperkube image, we're committing to supporting it forever, independent of what the @justinsb and the kops project decide to do in the future.

Definitely significant work to be done- I for one think it would be worthwhile. At CoreOS, we maintain CNAME mappings to our many service lb endpoints manually 👎 This dns controller is definitely a very attractive solution to that problem. I'm just unsure of whether we (kube-aws maintainers) should be shipping this component in our hyperkube image.

\cc @mumoshu @pieterlange

[pieterlange]

Can't we run it as a separate add-on (ie kube-dns) instead of immediately including it in hyperkube?

I'm slightly worried this has the potential for creeping into a scenario where kube-aws people are maintaining/packaging a bunch of add-ons (and having to choose which implementation is best) instead of focusing on the bigger fish to fry (#340).

But this would be a cool add-on. Like @colhom i'm also still manually maintaining DNS entries. For now, i actually like it that way because some steps should be manual, and having that cutoff point at changing DNS records feels safe (but might be entirely influenced by nostalgia ;))

[colhom]

I would be open to starting an add-ons folder which contains "officially unsupported but should probably work out of the box" k8s components like this. I think the sole requirement (beyond immediate need) for this would be a versioned/maintained image repository to which the manifests can refer.

[whereisaaron]

Good points @colhom @pieterlange I'm just stoking bloat :). Maybe instead we should maintain only deployment scripts/manifests that allow us to easily build and deploy this controller to a kube-aws cluster directly from the kops repo or in a maintained fork. I can then make these automated deployments after each cluster environment comes up.

I'm also looking at making an AWS plugin for Kelsey's Let's Encrypt cert controller, or something similar if any one knows a better Let's Encrypt controller. Again, this is a little earlier and lacking in tests to be a 'core' component.
https://github.com/kelseyhightower/kube-cert-manager
https://gist.github.com/jimmycuadra/db3bb1b7c6a0d35008e0
https://github.com/lukas2511/letsencrypt.sh/wiki/Examples-for-DNS-01-hooks

[mumoshu]

I definitely find the DNS controller useful but including those add-ons into hyperkube images would be too much, as discussed before in this thread.

I agree with the add-ons folder as @colhom mentioned but what I would expect to exist in the folder will be just YAML files, each contains a set of kubernetes deployments/configmap/etc.

It would provide an unified deployment procedure/developer experience kubectl create -f addon-foo.yaml across add-ons, while preventing coreos-kubernetes repo become too big.

Also I'd like to see versioned/maintained docker images and their tests, build scripts, or etc to be reside in another git repository, something like kubernetes/contrib for kubernetes.

Anyway, I would be happy to contribute if there's place to do so :)

[whereisaaron]

@colhom having discussed it I think there is nothing to do here. I agree such add-ons should be outside the core project. Thanks @pieterlange @mumoshu @colhom for your input.

[colhom]

@colhom having discussed it I think there is nothing to do here.

Agree for now.

The general sentiment that perhaps kube-aws should in some way facilitate installing useful add-ons persists (at least in my mind). Thanks @whereisaaron for the discussion topic.