Add "--selinux-enabled" flag for Docker

[hakman]

https://success.docker.com/article/how-to-set-selinux-file-contexts-when-using-a-custom-docker-data-root

I don't thin the current solution does what is expected based on the Docker docs: https://docs.docker.com/engine/reference/commandline/dockerd/#docker-runtime-execution-options

This PR should work better for the desired use case. On my ubuntu box I see this message:

Jun 11 10:21:46 ip-10-4-211-22 dockerd[4928]: time="2020-06-11T10:21:46.664002518Z"
level=warning msg="Docker could not enable SELinux on the host system"

Ref: kubernetes/test-infra#17902

/cc @bertinatto @rifelpet

[hakman]

I also set the default log level to "info" as it was already in docs. It is not very verbose, but should help us track various issues easier in testing.

kops/pkg/apis/kops/dockerconfig.go

Lines 53 to 54 in ba1d87e

	// LogLevel is the logging level ("debug", "info", "warn", "error", "fatal") (default "info")
	LogLevel *string `json:"logLevel,omitempty" flag:"log-level"`

[bertinatto]

On my ubuntu box I see this message:

Jun 11 10:21:46 ip-10-4-211-22 dockerd[4928]: time="2020-06-11T10:21:46.664002518Z"
level=warning msg="Docker could not enable SELinux on the host system"

Just to confirm, is SELinux installed and enabled in your Ubuntu box? It's not installed by default in Ubuntu, and apparently the package migh be broken: https://wiki.ubuntu.com/SELinux

The --selinux-enabled docker flag will only work if SELinux is enabled on the host. I'm not sure about Ubuntu, but it works as expected in my Fedora box (using upstream docker):

$ sudo setenforce 1
$ mkdir /tmp/blah

$ ps aux | ag dockerd
root      189228  0.3  0.2 1608692 90776 ?       Ssl  12:59   0:00 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --selinux-enabled
fbertina  190176  0.0  0.0   3384  1844 pts/1    S+   13:02   0:00 ag dockerd

(notice the --selinux-enabled above)

$ docker run --rm -it -v /tmp/blah:/tmp/blah centos bash -c "ls -lZ /tmp/blah"
ls: cannot open directory '/tmp/blah': Permission denied

(SELinux blocked the access)

$ sudo vim /usr/lib/systemd/system/docker.service

(removed --selinux-enabled)

$ sudo systemctl daemon-reload && sudo systemctl restart docker 

$ ps aux | ag dockerd
root      190552  7.6  0.2 1460972 84044 ?       Ssl  13:07   0:00 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
fbertina  190723  0.0  0.0   3384  1956 pts/1    S+   13:07   0:00 ag dockerd

$ docker run --rm -it -v /tmp/blah:/tmp/blah centos bash -c "ls -lZ /tmp/blah"
total 0

$ docker version
Client: Docker Engine - Community
 Version:           19.03.11
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        42e35e61f3
 Built:             Mon Jun  1 09:14:45 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.11
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       42e35e61f3
  Built:            Mon Jun  1 09:12:44 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

[bertinatto]

Jun 11 10:21:46 ip-10-4-211-22 dockerd[4928]: time="2020-06-11T10:21:46.664002518Z"
level=warning msg="Docker could not enable SELinux on the host system"

Apparently this message was returned because it failed this check:

https://github.com/moby/moby/blob/3aac5f0bbb5ccaeb9081ef710aae2459cbc87fa7/daemon/daemon_unix.go#L867

Which leads to this code, that verifies that SELinux was not enabled on the host (it looks for selinuxfs mount in /proc/filesystems:

https://github.com/opencontainers/selinux/blob/b5f337853c0fbc29cf112ade767460c33a64d938/go-selinux/selinux_linux.go#L134

[hakman]

The point of the message was to show that the flag is applied. A bit harder to enable SELinux on Ubuntu, I guess.

[hakman]

Doesn't seem to work to well on a CentOS 7 box either:

Jun 11 11:58:37 ip-10-4-211-152.eu-central-1.compute.internal dockerd[897]: time="2020-06-11T11:58:37.517424146Z" level=error
msg="Handler for POST /v1.40/containers/77b8d55b2714ea9e170c61ea09ac8bdae9b892ef9048c6eabff02876a8f75d4a
start returned error: OCI runtime create failed: selinux label is specified in config, but selinux is disabled or not supported: unknown"

[hakman]

/hold

[hakman]

Tracked down the previous error to an incompatibility between Docker and containerd and will be fixed in a separate PR.
This is ready for review.
/hold cancel

[bertinatto]

Thank you @hakman!

I'm not familiar with the code base, but I reviewed the changes and they LGTM.

[rifelpet]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hakman, rifelpet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [rifelpet]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hakman]

@bertinatto you should update the test to use this flag instead.
Once #9346 or some other solution is accepted, SELinux should start working.