kubeadm lock down kubelet API

[joantune]

##FEATURE REQUEST

As per https://github.com/kubernetes/kubernetes.github.io/blob/master/docs/admin/kubelet-authentication-authorization.md and kubernetes/enhancements#89 kubeadm could(?)/should(?) make sure that node's kubectl 1025 port required a client certificate to connect to.

Versions

kubeadm 1.7.1

Environment:

• Kubernetes 1.7.1

What happened?

wget --no-check-certificate https://<ip>:1025 returned a 404 Not found instead of a 401 (most likely needs to be rechecked)

What you expected to happen?

A 401

How to reproduce it (as minimally and precisely as possible)?

kubeadm init

On node:
kubeadm join <etc>
From another host:

wget --no-check-certificate https://<node>:1025

returns a 401

[mikedanese]

They kubelet API is already very locked down even though we allow anonymous authentication. Authorization will still stop an anonymous user from hitting endpoints of value. I'm not sure what the value is of disabling anonymous auth. cc @liggitt

[joantune]

@mikedanese: got it!
On this topic:

how should one minimize the surface area of kubernetes to the outside world?

Kubectl's endpoint can be set to be accessible only from inside a cluster's private addresses right? one might lose access to kubectl from a local machine, but still be able to do that through a SSH tunnel (+ kubectl proxy if kubectl is only listening in another interface, right?)

I don't mean to be paranoid, it's just that the less endpoints exposed the better, I guess this was the rationale behind GCE's SSH tunnels.

[liggitt]

I guess this was the rationale behind GCE's SSH tunnels.

GCE actually added SSH tunnels in lieu of securing the endpoints. With the ability to protect the endpoints, the benefit of tunneling is marginal (not zero, but not worth the added complexity for many deployments)

[joantune]

GCE actually added SSH tunnels in lieu of securing the endpoints.

I see, I saw kubernetes/enhancements#89 original problem as a two issue thing, which was:
a) having access to kubectl's endpoint directly
b) allowing anyone to execute API endpoints

It was actually mostly b)
Although I have to say, If I want to also limit the surface area, after looking at the docs, I have no idea what is the best way to accomplish that. The concepts of Nodes/Pods/Services is well explained, but how they interact with the host's network and through which processes are not.

e.g. Do pods expose directly their container's Pod port when part of an externally accessible service [actually I think this one is explained, but when it's 'LoadBalanced' not so much] what's kubeproxy/kubectl for and what role do they play with kubernetes? can they be walled off the internet? How can that be achieved would a VPN be enough and a firewall or binding those to the VPN's interface only? would that brake things? Making this clearer would make sysadmins feel more secure on using Kubernetes inside their own clouds. A good diagram could help. Hell I'm willing to do put that effort if I understand how it goes (i.e. something like the diagrams that you already have, but detailing the Kubernete's processes and endpoints [like kubectl kubeproxy] and what kind of traffic they handle and what it's used for)

[luxas]

@joantune What happens if you do https://<ip>:10250/stats or /metrics or /containerLogs or an other valid path? I think the 404 for / is legit, since there is no associated path with / in the kubelet server.

This happens for me:

$ curl -sSLk https://localhost:10250
404 page not found
$ curl -sSLk https://localhost:10250/stats
Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=stats)
$ curl -sSLk https://localhost:10250/metrics
Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=metrics)

I do think this is working just as expected. This was locked down in v1.6.

All traffic to the kubelet's endpoint is first authenticated with the CA cert. If there is no authentication method like above, the auth module will authenticate the user as system:anonymous. The kubelet then POSTs a SubjectAccessReview for the user in question to the API server and gets a denial back, because only users in the system:masters group are allowed to access this API.

TL;DR; both authentication and authorization is enabled for this endpoint. Anonymous requests are allowed in the authentication phase, but won't survive the authorization phase unless you create some custom RBAC rules.

[luxas]

Closing this as the kubelet API endpoint is properly secured and has been since v1.6.0