Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add e2e regression tests for the kubelet being secure #838

Closed
luxas opened this issue May 21, 2018 · 3 comments · Fixed by kubernetes/kubernetes#64140
Closed

Add e2e regression tests for the kubelet being secure #838

luxas opened this issue May 21, 2018 · 3 comments · Fixed by kubernetes/kubernetes#64140
Assignees
@luxas @dixudx
Labels
area/security area/test area/upgrades lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/node Categorizes an issue or PR as relevant to SIG Node.
Milestone
v1.12

Comments

@luxas
Copy link
Member

luxas commented May 21, 2018 •
edited
Loading

As part of #732, #650 and kubernetes/kubernetes#63881 we should make sure the following things in our end-to-end testing:

  1. The kubelet cAdvisor port (4194) can't be reached, neither via the API server proxy nor directly on the public IP address
  2. The kubelet read-only port (10255) can't be reached, neither via the API server proxy nor directly on the public IP address
  3. The kubelet can delegate ServiceAccount tokens to the API server
  4. The kubelet's main port (10250) has both authentication (should fail with no credentials) and authorization (should fail with insufficient permissions) set-up

These e2e tests, which I preliminarily propose to host under [sig-cluster-lifecycle] [Feature:KubeletSecurity], would be run by any kubeadm e2e suite running against v1.11+ clusters.
These test are super important to make sure no kubeadm version regresses security-wise by accident.

@dixudx is working on creating these tests, thank you a lot!

FYI @kubernetes/sig-node-proposals @kubernetes/sig-auth-proposals
@k8s-ci-robot k8s-ci-robot added sig/node Categorizes an issue or PR as relevant to SIG Node. kind/design Categorizes issue or PR as related to design. sig/auth Categorizes an issue or PR as relevant to SIG Auth. labels May 21, 2018
@luxas luxas added priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. area/upgrades area/security area/test sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. labels May 21, 2018
@luxas luxas added this to the v1.11 milestone May 21, 2018
@luxas luxas removed the kind/design Categorizes issue or PR as related to design. label May 21, 2018
@tallclair
Copy link
Member

I think the latter 2 tests should be under sig-auth. I'd also like to make the 4th one a conformance test eventually.

@luxas
Copy link
Member Author

luxas commented May 21, 2018 •
edited
Loading

@tallclair Thanks, I'm very happy to put 3. and 4. directly under the [sig-auth] umbrella
I just proposed the sig-cluster-lifecycle thing in the meantime in case nobody wanted to own it 😉

@dixudx dixudx mentioned this issue May 22, 2018
Merged
@luxas luxas self-assigned this May 22, 2018
@luxas luxas mentioned this issue May 23, 2018
Merged
k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this issue May 24, 2018
Merge pull request #64187 from luxas/kubeadm_kubelet_improve_security
698ac55 
Automatic merge from submit-queue (batch tested with PRs 64174, 64187, 64216, 63265, 64223). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

kubeadm: Improve the kubelet default configuration security-wise

**What this PR does / why we need it**:
 - Disables the readonly port for the kubelets in the cluster
 - Enables delegated SA token authentication for the secure kubelet port (GCE also did this ref: #58178)
 - Follows up #63912 to move the last flag from the system dropin to the ComponentConfig

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes kubernetes/kubeadm#732
Fixes kubernetes/kubeadm#650
Replaces #57997

**Special notes for your reviewer**:
In order to make sure this actually works, or that clusters actually are secure, we're adding e2e tests for this: kubernetes/kubeadm#838 & #64140
Depends on #63912

**Release note**:

```release-note
[action required] kubeadm: kubelets in kubeadm clusters now disable the readonly port (10255). If you're relying on unauthenticated access to the readonly port, please switch to using the secure port (10250). Instead, you can now use ServiceAccount tokens when talking to the secure port, which will make it easier to get access to e.g. the `/metrics` endpoint of the kubelet securely.
```
@kubernetes/sig-cluster-lifecycle-pr-reviews 
@kubernetes/sig-auth-pr-reviews FYI
@dixudx dixudx mentioned this issue Jun 4, 2018
Merged
@luxas
Copy link
Member Author

luxas commented Jun 12, 2018

kubernetes/kubernetes#64140 is up there and kind of ready, but it's not gonna make v1.11.0. Maybe (hopefully) a point release. In any case, retargeting this issue to v1.12

@luxas luxas modified the milestones: v1.11, v1.12 Jun 12, 2018
k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this issue Jun 21, 2018
Merge pull request #64140 from dixudx/add_e2e_kubelet_port
0b3af19 
Automatic merge from submit-queue (batch tested with PRs 64140, 64898, 65022, 65037, 65027). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add e2e regression tests for the kubelet being secure

**What this PR does / why we need it**:
This PR does,

1. The kubelet cAdvisor port (4194) can't be reached, neither via the API server proxy nor directly on the public IP address
2. The kubelet read-only port (10255) can't be reached, neither via the API server proxy nor directly on the public IP address
3. The kubelet can delegate ServiceAccount tokens to the API server
4. The kubelet's main port (10250) has both authentication (should fail with no credentials) and authorization (should fail with insufficient permissions) set-up

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes kubernetes/kubeadm#838

**Special notes for your reviewer**:
/cc luxas tallclair 
**Release note**:

```release-note
Add e2e regression tests for the kubelet being secure
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@luxas luxas

@dixudx dixudx

Labels
area/security area/test area/upgrades lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/node Categorizes an issue or PR as relevant to SIG Node.
Projects
None yet
Milestone
v1.12
Development

Successfully merging a pull request may close this issue.

Add e2e regression tests for the kubelet being secure dixudx/kubernetes
4 participants
@luxas @dixudx @k8s-ci-robot @tallclair