auto-setting 32 GB memory on Docker Driver ?

[medyagh]

on users with a huge machine, we are auto setting something as big as 32 GB of ram

minikube start -p foo    
😄  [foo] minikube v1.17.1 on Debian rodete
✨  Automatically selected the docker driver
👍  Starting control plane node foo in cluster foo
🔥  Creating docker container (CPUs=2, Memory=32100MB) ...
🤦  StartHost failed, but will try again: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname foo --name foo --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=foo --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=foo --network foo --ip 192.168.49.2 --volume foo:/var --security-opt apparmor=unconfined --memory=32100mb --memory-swap=32100mb -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 126
stdout:
4b4327555885bd42aefffa650a2ddb27087d8071f5e1a3571c8377d361985e8c
 
stderr:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: process_linux.go:422: setting cgroup config for procHooks process caused: failed to write "0" to "/sys/fs/cgroup/system.slice/docker-4b4327555885bd42aefffa650a2ddb27087d8071f5e1a3571c8377d361985e8c.scope/memory.swap.max": open /sys/fs/cgroup/system.slice/docker-4b4327555885bd42aefffa650a2ddb27087d8071f5e1a3571c8377d361985e8c.scope/memory.swap.max: permission denied: unknown.
 
🤷  docker "foo" container is missing, will recreate.
🔥  Creating docker container (CPUs=2, Memory=32100MB) ...
😿  Failed to start docker container. Running "minikube delete -p foo" may fix it: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname foo --name foo --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=foo --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=foo --network foo --ip 192.168.49.2 --volume foo:/var --security-opt apparmor=unconfined --memory=32100mb --memory-swap=32100mb -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 126
stdout:
9ca1b5bf6eb22cd0d4338525fe2984375070f70071d627f4e2080ba6894e9d1c
 
stderr:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: process_linux.go:422: setting cgroup config for procHooks process caused: failed to write "0" to "/sys/fs/cgroup/system.slice/docker-9ca1b5bf6eb22cd0d4338525fe2984375070f70071d627f4e2080ba6894e9d1c.scope/memory.swap.max": open /sys/fs/cgroup/system.slice/docker-9ca1b5bf6eb22cd0d4338525fe2984375070f70071d627f4e2080ba6894e9d1c.scope/memory.swap.max: permission denied: unknown.
 
❗  Startup with docker driver failed, trying with alternate driver ssh: Failed to start host: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname foo --name foo --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=foo --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=foo --network foo --ip 192.168.49.2 --volume foo:/var --security-opt apparmor=unconfined --memory=32100mb --memory-swap=32100mb -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 126
stdout:
9ca1b5bf6eb22cd0d4338525fe2984375070f70071d627f4e2080ba6894e9d1c
 
stderr:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: process_linux.go:422: setting cgroup config for procHooks process caused: failed to write "0" to "/sys/fs/cgroup/system.slice/docker-9ca1b5bf6eb22cd0d4338525fe2984375070f70071d627f4e2080ba6894e9d1c.scope/memory.swap.max": open /sys/fs/cgroup/system.slice/docker-9ca1b5bf6eb22cd0d4338525fe2984375070f70071d627f4e2080ba6894e9d1c.scope/memory.swap.max: permission denied: unknown.
 
🔥  Deleting "foo" in docker ...
🔥  Deleting container "foo" ...
🔥  Removing /usr/local/google/home/drewpca/.minikube/machines/foo ...
💀  Removed all traces of the "foo" cluster.
👍  Starting control plane node foo in cluster foo
🤦  StartHost failed, but will try again: config: please provide an IP address
😿  Failed to start ssh bare metal machine. Running "minikube delete -p foo" may fix it: config: please provide an IP address
 
❌  Exiting due to GUEST_PROVISION: Failed to start host: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname foo --name foo --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=foo --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=foo --network foo --ip 192.168.49.2 --volume foo:/var --security-opt apparmor=unconfined --memory=32100mb --memory-swap=32100mb -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 126
stdout:
9ca1b5bf6eb22cd0d4338525fe2984375070f70071d627f4e2080ba6894e9d1c
 
stderr:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: process_linux.go:422: setting cgroup config for procHooks process caused: failed to write "0" to "/sys/fs/cgroup/system.slice/docker-9ca1b5bf6eb22cd0d4338525fe2984375070f70071d627f4e2080ba6894e9d1c.scope/memory.swap.max": open /sys/fs/cgroup/system.slice/docker-9ca1b5bf6eb22cd0d4338525fe2984375070f70071d627f4e2080ba6894e9d1c.scope/memory.swap.max: permission denied: unknown.
 
 
😿  If the above advice does not help, please let us know: 
👉  https://github.com/kubernetes/minikube/issues/new/choose
 
minikube start -p foo  6.03s user 4.75s system 4% cpu 3:35.48 total

[afbjorklund]

The normal drivers will max out at 6000 MiB, for their suggested memory allocation...

But we only call validateRequestedMemorySize, when cmd.Flags().Changed(memory)

[afbjorklund]

This behaviour is wrong:
❗ Startup with docker driver failed, trying with alternate driver ssh
🤦 StartHost failed, but will try again: config: please provide an IP address

It's not supposed to switch to something like "ssh" or "none", just because Docker failed ?

[tstromberg]

This is intended: On Linux, Docker normally allows all system memory to be consumed. We do give the system a small buffer (48MB) that minikube refuses to allocate. This is not a memory reservation, but a memory limit. This is a bit different than our other drivers, which generally make a hard reservation.

Perhaps we should just remove the memory message for the Docker driver?

[afbjorklund]

And it seems like the actual error here was more related to trying to set swap memory with the new cgroups v2 docker ?

[tstromberg]

Closing as dupe of #10371