virt-install: do not assign vpu's when executed in Kubernetes

[darkmuggle]

No description provided.

[darkmuggle]

Example of a pod where this can be seen:

$ nproc
64
sh-4.4$ cat cgroup
11:freezer:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec33d1_1a86_11e9_b497_fa163ed2928c.slice/docker-6a9ecbba556f3c5e2cad3893da23cd675b499170ac94431843d57b5476d6fd1c.scope
10:memory:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec33d1_1a86_11e9_b497_fa163ed2928c.slice/docker-6a9ecbba556f3c5e2cad3893da23cd675b499170ac94431843d57b5476d6fd1c.scope
9:devices:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec33d1_1a86_11e9_b497_fa163ed2928c.slice/docker-6a9ecbba556f3c5e2cad3893da23cd675b499170ac94431843d57b5476d6fd1c.scope
8:net_prio,net_cls:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec33d1_1a86_11e9_b497_fa163ed2928c.slice/docker-6a9ecbba556f3c5e2cad3893da23cd675b499170ac94431843d57b5476d6fd1c.scope
7:blkio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec33d1_1a86_11e9_b497_fa163ed2928c.slice/docker-6a9ecbba556f3c5e2cad3893da23cd675b499170ac94431843d57b5476d6fd1c.scope
6:perf_event:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec33d1_1a86_11e9_b497_fa163ed2928c.slice/docker-6a9ecbba556f3c5e2cad3893da23cd675b499170ac94431843d57b5476d6fd1c.scope
5:hugetlb:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec33d1_1a86_11e9_b497_fa163ed2928c.slice/docker-6a9ecbba556f3c5e2cad3893da23cd675b499170ac94431843d57b5476d6fd1c.scope
4:pids:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec33d1_1a86_11e9_b497_fa163ed2928c.slice/docker-6a9ecbba556f3c5e2cad3893da23cd675b499170ac94431843d57b5476d6fd1c.scope
3:cpuset:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec33d1_1a86_11e9_b497_fa163ed2928c.slice/docker-6a9ecbba556f3c5e2cad3893da23cd675b499170ac94431843d57b5476d6fd1c.scope
2:cpuacct,cpu:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec33d1_1a86_11e9_b497_fa163ed2928c.slice/docker-6a9ecbba556f3c5e2cad3893da23cd675b499170ac94431843d57b5476d6fd1c.scope
1:name=systemd:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec33d1_1a86_11e9_b497_fa163ed2928c.slice/docker-6a9ecbba556f3c5e2cad3893da23cd675b499170ac94431843d57b5476d6fd1c.scope

If you assign 64 VCPU's when the resource cap is set to 1CPU, then the VM will hang during boot up and get confused about the time.

[    0.001000] kvm-clock: cpu 36, msr 0:7bed0901, secondary cpu clock
[    2.075599] KVM setup async PF for cpu 36
[    2.076000] kvm-stealtime: cpu 36, msr 78f22040
[    2.137006]  #37
[    0.001000] kvm-clock: cpu 37, msr 0:7bed0941, secondary cpu clock

[cgwalters]

Humm...I swear we tested this though. When I e.g. rsh into a MCD pod and type nproc I get 2 which matches the host.

You're saying this code is returning 64 today?

[darkmuggle]

Right, I saw this in OpenShift where I had a resource limit of 2CPU's. Looking at the logs, it was clearly getting 64.

[cgwalters]

Related: moby/moby#20770

[cgwalters]

Hm so I guess nproc isn't going to detect weighted CPU shares. OK here's a better link:
eclipse-openj9/openj9#1166

Edit: Here's what they ended up doing for the JVM: http://hg.openjdk.java.net/jdk/hs/rev/7f22774a5f42 (man, ugly C++)

[cgwalters]

What if we just capped it at 2? We don't need extreme performance but going down to 1 is a bit harsh.

[darkmuggle]

I went with 1 since AFAIK, there is no good way to tell what the vcpu allocation is. If you go with 2 and you only have a 1vcpu allocation then performance will suffer. Really, any choice is going to be harsh.

[dustymabe]

s/assing/assign/ ?

[jlebon]

Also s/scehd/sched/ :)

[darkmuggle]

Ha, re-pushed.

[dustymabe]

so we are running this in kube/openshift in centos CI. I guess we've been lucky we haven't hit a performance issue yet?

[jlebon]

so we are running this in kube/openshift in centos CI. I guess we've been lucky we haven't hit a performance issue yet?

Seems like we're assigning 20 vCPUs there: http://artifacts.ci.centos.org/fedora-coreos/prod/builds/latest/install.log. Though nproc returns 40.

[darkmuggle]

Before merging this, any thoughts on @cgwalters request for 2 over 1 vcpu? I mean, this is rock and a hard place. I don't see a good answer until we have support elsewhere.

[miabbott]

I trust @darkmuggle knows what he is doing :)

[cgwalters]

Eh it's simplest to go down to 1 for now I guess.

[jlebon]

Out of interest: the TL;DR of that OpenJDK HotSpot patch is that they're doing cpu.shares / 1024 on the basis that e.g. Kubernetes/Mesos "issues" 1024 shares per CPU. Without that assumption, there's simply no way to tell how big your piece of the pie is since you'd need to be able to sum up the total number of issued shares across all cgroups, which you can't do from within your unprivileged container.