Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sig-storage] Run storage e2e test_client_pod as privileged #104551

Merged

Conversation

Elbehery
Copy link
Contributor

@Elbehery Elbehery commented Aug 24, 2021

What type of PR is this?

/kind bug
/kind storage
/kind failing-test

What this PR does / why we need it:

Fix running storage e2e tests in SELinux environment.

HostPath Volume Plugin creates a directory within /tmp on host machine, to be mounted as volume. Usually, Inject-pod writes content to the volume, and a Client-pod tried the read the contents and verify.

When SELinux is enabled on the host, Client-pod can not read the content, with permission denied.

This PR run the Client-pod as privileged, so that it can access the volume content, even when SELinux is enabled on the host.

Which issue(s) this PR fixes:

Fixes #84585

Fixes hostpath storage e2e tests within SELinux enabled env

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. labels Aug 24, 2021
@k8s-ci-robot
Copy link
Contributor

@Elbehery: The label(s) kind/storage cannot be applied, because the repository doesn't have them.

In response to this:

What type of PR is this?

/kind bug
/kind storage
/kind failing-test

What this PR does / why we need it:

Fix running storage e2e tests in SELinux environment.

HostPath Volume Plugin creates a directory within /tmp on host machine, to be mounted as volume. Usually, Inject-pod writes content to the volume, and a Client-pod tried the read the contents and verify.

When SELinux is enabled on the host, Client-pod can not read the content, with permission denied.

This PR run the Client-pod as privileged, so that it can access the volume content, even when SELinux is enabled on the host.

Which issue(s) this PR fixes:

Fixes #84585

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Aug 24, 2021
@k8s-ci-robot
Copy link
Contributor

@Elbehery: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Aug 24, 2021
@k8s-ci-robot
Copy link
Contributor

Hi @Elbehery. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. area/e2e-test-framework Issues or PRs related to refactoring the kubernetes e2e test framework area/test sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Aug 24, 2021
@gnufied
Copy link
Member

gnufied commented Aug 24, 2021

IIRC privileged containers don't get to bypass SELinux policies. So if SELinux was blocking them before, it will block them after. Please test carefully before going ahead with this approach.

@Elbehery
Copy link
Contributor Author

IIRC privileged containers don't get to bypass SELinux policies. So if SELinux was blocking them before, it will block them after. Please test carefully before going ahead with this approach.

@gnufied i have tested this on a cluster with SELinux enabled and the test passed successfully
cc @vrutkovs @bertinatto

@ping035627
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 3, 2021
@@ -511,8 +511,9 @@ func TestVolumeClientSlow(f *framework.Framework, config TestConfig, fsGroup *in
}

func testVolumeClient(f *framework.Framework, config TestConfig, fsGroup *int64, fsType string, tests []Test, slow bool) {
privileged := true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Elbehery, thank you for working on this.

I have a question for you. Although this fixes the issue, could you check if you can set privileged = true only when the HostPath plugin is being used? Perhaps we could add a flag for that in the plugin definition.

The idea is to avoid always going privileged. Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bertinatto I have adjusted the fix to use privileged only for this scenario.
cc @vrutkovs

@Elbehery Elbehery force-pushed the unprivileged_storage_client branch from 4f373a7 to 775a853 Compare October 11, 2021 10:19

if (config.Prefix == "hostpathsymlink" || config.Prefix == "hostpath") && podSuffix == "client" {
privileged = true
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Elbehery could you add a comment stating why you're making it privileged on hostpath? The explanation from the PR description looks good to me 😉

Also, I think you can remove the last condition (podSuffix == "client").

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bertinatto sure I add the comment 👍🏽 ..
Just about the podSuffix == "client" condition, the test fails because the client-pod can not read the data on the mounted volume when SELinux enabled, this will minimise the privileged pods.

Shall I remove it or leave it ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we spoke offline, the pod that writes the content is already privileged (that's the reason it's able to write to the host's /tmp directory). IMO we can safely remove podSuffix == "client".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes I removed it already in the most recent commit 👍🏽

@Elbehery Elbehery force-pushed the unprivileged_storage_client branch from 775a853 to 01b6743 Compare October 18, 2021 12:02
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Oct 18, 2021
@Elbehery
Copy link
Contributor Author

@bertinatto I have added a comment and updated the commit msg

@Elbehery Elbehery force-pushed the unprivileged_storage_client branch from 01b6743 to 8a5a9bb Compare October 18, 2021 12:15
hostPath volume plugin creates a directory within /tmp on host machine, to be mounted as volume.
inject-pod writes content to the volume, and a client-pod tried the read the contents and verify.
when SELinux is enabled on the host, client-pod can not read the content, with permission denied.
running the client-pod as privileged, so that it can access the volume content, even when SEinux is enabled on the host.
@Elbehery Elbehery force-pushed the unprivileged_storage_client branch from 8a5a9bb to 04ad18c Compare October 18, 2021 13:06
@bertinatto
Copy link
Member

/lgtm
As a next step we could make this configurable so that we can add other plugins/driver, like `csi-hostpath).

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 19, 2021
@Elbehery
Copy link
Contributor Author

/assign @jsafrane

@jsafrane
Copy link
Member

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Elbehery, jsafrane

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 20, 2021
@jsafrane
Copy link
Member

/release-notes-none

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Oct 20, 2021
@k8s-ci-robot k8s-ci-robot merged commit d5de03f into kubernetes:master Oct 20, 2021
@k8s-ci-robot k8s-ci-robot added this to the v1.23 milestone Oct 20, 2021
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Oct 26, 2021
k8s-ci-robot added a commit that referenced this pull request Nov 4, 2021
…age-client-1.20

Cherry pick of #104551: Run storage hostpath e2e test client pod as privileged
k8s-ci-robot added a commit that referenced this pull request Nov 4, 2021
…age-client-1.21

Cherry pick of #104551: Run storage hostpath e2e test client pod as privileged
k8s-ci-robot added a commit that referenced this pull request Nov 4, 2021
…age-client

Cherry pick of #104551: Run storage hostpath e2e test client pod as privileged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/e2e-test-framework Issues or PRs related to refactoring the kubernetes e2e test framework area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Run unprivileged containers in storage e2e tests
6 participants