Add vTPM support for Linux #1591

stefanberger · 2017-09-08T20:09:18Z

This patch series adds vTPM support for Linux to libcontainer.

The functionality is based on a recently added vtpm_proxy driver, which is becoming
available in Linux 4.8. The driver provides /dev/vtpmx, on which an ioctl is called
that spawns a TPM device in the host's /dev directory and returns an anonymous file-
descriptor on which a TPM emulator can listen for TPM commands. If we for example
created /dev/tpm12 on the host we make this device available as /dev/tpm0 inside the
container. We also add its major and minor numbers to the device cgroup.

We implement a VTPM class that allows us to create the device and starts a TPM
emulator 'swtpm', to which it passes the anonymous file descriptor.
Besides that, the user can choose to have the vTPM create certificates in a step
that simulates TPM manufacturing. We do this by calling the external swtpm_setup
program, which is part of the swtpm project.

VTPM support is added inside the JSON configuration as follows:

[...]
    "linux": {
        "resources": {
            "devices": [
                {
                    "allow": false,
                    "access": "rwm"
                }
            ] ,
            "vtpms": [
                {
                    "statePath": "/tmp/tpm-1",
                    "createCerts": true,
                },
            ]
        },

This JSON markup makes a single TPM available inside the created container.

o The statPath parameter indicates the directory where the TPM emulator 'swtpm'
writes the state of the TPM device to.
o The createCerts parameter indicates that certificates for the TPM are
to be created.

The swtpm project is available here : https://github.com/stefanberger/swtpm
The libtpms project is available here: https://github.com/stefanberger/libtpms

Signed-off-by: Stefan Berger [email protected]

stefanberger · 2017-09-08T20:44:31Z

This is a continuation of the PR #1082 that cannot be reopened anymore.

deitch · 2017-09-08T21:42:32Z

Thanks for reopening.

cyphar · 2017-09-09T02:42:32Z

vendor/github.com/opencontainers/runtime-spec/specs-go/config.go

@@ -320,6 +320,13 @@ type LinuxNetwork struct {
 	Priorities []LinuxInterfacePriority `json:"priorities,omitempty"`
 }

+// VTPM definition
+type VTPM struct {


@crosbymichael How do you feel about including changes to vendor/?
@stefanberger Is there a runtime-spec PR open about this?

@cyphar I opened opencontainers/runtime-spec#920 for vTPM support

What's current status of opencontainers/runtime-spec#920 ?

cc @vbatts

lukehinds · 2019-02-19T16:07:01Z

are you working on this still @stefanberger ? I don't have reviewer access, but from what I have looked at, the patch looks in good shape.

JAORMX · 2019-11-20T11:45:24Z

Seems there are merge conflicts here. Anybody working on this still?

stefanberger · 2019-12-31T02:10:34Z

I rebased this on the latest master and am now creating the vTPM devices at higher layer in utils_linux.go, which makes cleanup in the error case less invasive.

stefanberger · 2020-01-02T15:07:19Z

This now works right out of the box with a recent kernel on Fedora 31. The kernel-modules-extra package for 5.3.16-300.fc31 has the necessary kernel module.

Install the packages:
dnf -y install swtpm-tools

Create a spec:
/usr/loca/sbinl/runc spec

Add the following modifications to the spec in config.json:

                                        "allow": false,
                                        "access": "rwm"
                                }
+                       ],
+                       "vtpms": [
+                               {
+                                       "statepath": "/tmp/myvtpm1",
+                                       "vtpmversion": "2"
+                               }
                        ]
                },
                "namespaces": [

Run a container:

/usr/local/sbin/runc run test
sh-5.0# ls -l /dev/tpm*
crw-------. 1 root root 253,     1 Jan  2 14:59 /dev/tpm0
crw-------. 1 root root 253, 65537 Jan  2 14:59 /dev/tpmrm0

stefanberger · 2020-01-03T19:08:48Z

I just extended the series with patches for running the TPM emulator swtpm with an AppArmor profilevcreated on the fly. I had tried to use the existing function apparmor.ApplyProfile() that tries to apply a profile via writing to /proc/self/attr/exec, though this created 50% failures. I have to write to /proc/<tid>/attr/exec to get 100% success.

stefanberger · 2020-01-06T14:25:09Z

Running swtpm with an SELinux label has now also landed:

# ps auxZ | grep swtpm
system_u:system_r:container_t:s0:c494,c987 root 1149416 1.2  0.0 7680 3104 ?     Ss   09:20   0:00 swtpm chardev --tpmstate dir=/var/lib/runc/myvtpm1 --daemon --fd 5 --pid file=/var/lib/runc/myvtpm1/swtpm.pid --log file=/var/lib/runc/myvtpm1/swtpm.log --runas 0 --tpm2 --locality reject-locality-4,allow-set-locality --flags not-need-init
# sudo ls -lZ /var/lib/runc/myvtpm
total 20
drwxr-x---. 2 root root system_u:object_r:container_file_t:s0:c927,c931 4096 Jan  6 09:21 .
drwxr-xr-x. 3 root root unconfined_u:object_r:var_lib_t:s0              4096 Jan  4 19:46 ..
-rw-r-----. 1 root root system_u:object_r:container_file_t:s0:c927,c931    0 Jan  6 09:21 .lock
-rw-r--r--. 1 root root system_u:object_r:container_file_t:s0:c927,c931   75 Jan  4 19:46 swtpm.log
-rw-r--r--. 1 root root system_u:object_r:container_file_t:s0:c927,c931    7 Jan  6 09:21 swtpm.pid
-rw-r-----. 1 root root system_u:object_r:container_file_t:s0:c927,c931 1187 Jan  6 09:21 tpm2-00.permall

AkihiroSuda · 2020-03-27T21:04:15Z

libcontainer/vtpm/vtpm.go

+	defer vtpmx.Close()
+
+	if err := ioctl(uintptr(vtpmx.Fd()), cmd, msg); err != nil {
+		return fmt.Errorf("VTPM: vtpmx ioctl failed: %v", err)


errors.Wrap

AkihiroSuda · 2020-03-27T21:08:23Z

Seems we need to wait to see how opencontainers/runtime-spec#920 will settle

Remove whitespace at EOL Signed-off-by: Kir Kolyshkin <[email protected]>

1. When using `runc`, we should check `$status` and not `$?`. 2. Before exit code check, let's (try to) show errors from CRIU log. Signed-off-by: Kir Kolyshkin <[email protected]>

Signed-off-by: Mrunal Patel <[email protected]>

full diff: cilium/ebpf@a9f01ed...1c8d4c9 drops support for go1.12, and removes dependency on the golang.org/x/xerrors transitional package. Signed-off-by: Sebastiaan van Stijn <[email protected]>

Signed-off-by: Stefan Berger <[email protected]>

This patch adds vTPM support for Linux to libcontainer. The functionality is based on a recently added vtpm_proxy driver, which is becoming available in Linux 4.8. The driver provides /dev/vtpmx, on which an ioctl is called that spawns a TPM device in the host's /dev directory and returns an anonymous file-descriptor on which a TPM emulator can listen for TPM commands. If we for example created /dev/tpm12 on the host we make this device available as /dev/tpm0 inside the container. We also add its major and minor numbers to the device cgroup. We implement a VTPM class that allows us to create the device and starts a TPM emulator 'swtpm', to which it passes the anonymous file descriptor. Besides that, the user can choose to have the vTPM create certificates in a step that simulates TPM manufacturing. We do this by calling the external swtpm_setup program, which is part of the swtpm project. VTPM support is added inside the JSON configuration as follows: [...] "linux": { "resources": { "devices": [ { "allow": false, "access": "rwm" } ] , "vtpms": [ { "statePath": "/tmp/tpm-1", "createCerts": true }, ] }, [...] This JSON markup makes a single TPM available inside the created container. o The statPath parameter indicates the directory where the TPM emulator 'swtpm' writes the state of the TPM device to. o The createCerts parameter indicates that certificates for the TPM are to be created. The current implementation does not support checkpointing, so checkpointing of a container with an attached vTPM is prevented. The swtpm project is available here : https://github.com/stefanberger/swtpm The libtpms project is available here: https://github.com/stefanberger/libtpms Signed-off-by: Stefan Berger <[email protected]>

We need to implement ApplyProfileThread() to apply a profile via /proc/self-thread/attr/exec rather than /proc/self/attr/exec otherwise we get (~50%) failures trying to write the profile to /proc/self/attr/exec. When using self-thread we get 100% success. Signed-off-by: Stefan Berger <[email protected]>

Create an AppArmor profile and apply it so that swtpm runs with an AppArmor profile. Signed-off-by: Stefan Berger <[email protected]>

On systems supporting SELinux run swtpm with an SELinux label applied. Also label the required files in the state directory. Signed-off-by: Stefan Berger <[email protected]>

Add test cases for testing TPM 1.2 and TPM 2 by creating, stopping, restarting, and destroying it. Signed-off-by: Stefan Berger <[email protected]>

Call 'swtpm chardev --print-capabilities' to get the supported capabilites from swtpm. An JSON object is printed by swtpm that we unmarshal and we pick the 'features' part from it that is an array of strings indicating what this version of swtpm supports. This option was added in v0.2. For older versions of swtpm we return an empty array. Signed-off-by: Stefan Berger <[email protected]>

We need to startup the TPM as part of starting swtpm so that the Linux driver can successfully send its initial command to the vTPM and does not log a failure and then do the startup itself. Signed-off-by: Stefan Berger <[email protected]>

This patch adds support for encrypting the vTPM state by allowing a user to pass a password to swtpm_setup and swtpm. Signed-off-by: Stefan Berger <[email protected]>

Add /sys/devices/virtual/tpm to the mask paths to avoid isolation issues via sysfs for TPM 1.2 Signed-off-by: Stefan Berger <[email protected]>

Put vTPMs into a container's cgroup to limit their CPU usage. Signed-off-by: Stefan Berger <[email protected]>

mfranczy · 2024-03-25T14:56:36Z

@stefanberger are you still working on this PR?

kolyshkin · 2024-06-15T06:08:11Z

Since opencontainers/runtime-spec#920 is closed, we should close this one as well.

stefanberger mentioned this pull request Sep 8, 2017

Preview: Add vTPM support for Linux #1082

Closed

cyphar reviewed Sep 9, 2017

View reviewed changes

stefanberger force-pushed the vtpm branch 2 times, most recently from ca2dcb4 to 59daf73 Compare September 9, 2017 21:28

wking mentioned this pull request Sep 11, 2017

Add vTPM specification opencontainers/runtime-spec#920

Closed

coresolve mentioned this pull request Jan 13, 2018

Please add CONFIG_TCG_VTPM_PROXY=m coreos/bugs#2308

Closed

stefanberger force-pushed the vtpm branch 6 times, most recently from 64e9c4c to 23b636d Compare December 31, 2019 02:08

stefanberger force-pushed the vtpm branch 2 times, most recently from 756521d to 2a798ce Compare December 31, 2019 12:49

stefanberger force-pushed the vtpm branch 3 times, most recently from 58eae95 to a548859 Compare January 3, 2020 19:05

stefanberger force-pushed the vtpm branch 3 times, most recently from 1cb507d to 98f315b Compare January 6, 2020 14:13

stefanberger force-pushed the vtpm branch 2 times, most recently from b7e7aa7 to d8f8e71 Compare January 6, 2020 14:55

AkihiroSuda reviewed Mar 27, 2020

View reviewed changes

stefanberger force-pushed the vtpm branch 5 times, most recently from d34a3bb to 8fa2643 Compare July 6, 2020 15:50

kolyshkin and others added 5 commits July 6, 2020 15:04

tests/int/checkpoint: whitespace cleanups

09ae10b

Remove whitespace at EOL Signed-off-by: Kir Kolyshkin <[email protected]>

tests/int/checkpoint: fix checks, add logs

27475c4

1. When using `runc`, we should check `$status` and not `$?`. 2. Before exit code check, let's (try to) show errors from CRIU log. Signed-off-by: Kir Kolyshkin <[email protected]>

VERSION: release 1.0.0-rc91

d88bdd3

Signed-off-by: Mrunal Patel <[email protected]>

VERSION: back to development

3097e53

Signed-off-by: Mrunal Patel <[email protected]>

vendor: update cilium/ebpf v0.0.0-20200702112145-1c8d4c9ef775

5a05fe7

full diff: cilium/ebpf@a9f01ed...1c8d4c9 drops support for go1.12, and removes dependency on the golang.org/x/xerrors transitional package. Signed-off-by: Sebastiaan van Stijn <[email protected]>

stefanberger force-pushed the vtpm branch 2 times, most recently from ab0db75 to b8c570c Compare July 6, 2020 19:10

stefanberger added 11 commits July 6, 2020 15:24

TEMP: specs: Extend specs with vTPM support

475c88f

Signed-off-by: Stefan Berger <[email protected]>

vtpm: Run swtpm with an AppArmor profile

463e2f5

Create an AppArmor profile and apply it so that swtpm runs with an AppArmor profile. Signed-off-by: Stefan Berger <[email protected]>

vtpm: Run swtpm with an SELinux label

03af7cc

On systems supporting SELinux run swtpm with an SELinux label applied. Also label the required files in the state directory. Signed-off-by: Stefan Berger <[email protected]>

vtpm: Add test cases

ffbcce6

Add test cases for testing TPM 1.2 and TPM 2 by creating, stopping, restarting, and destroying it. Signed-off-by: Stefan Berger <[email protected]>

vtpm: Add support for encrypted vTPM state

0a352ff

This patch adds support for encrypting the vTPM state by allowing a user to pass a password to swtpm_setup and swtpm. Signed-off-by: Stefan Berger <[email protected]>

maskedPaths: Add /sys/devices/virtual/tpm

d65b66a

Add /sys/devices/virtual/tpm to the mask paths to avoid isolation issues via sysfs for TPM 1.2 Signed-off-by: Stefan Berger <[email protected]>

vtpm: Put vTPMs into container's cgroup

83ac63c

Put vTPMs into a container's cgroup to limit their CPU usage. Signed-off-by: Stefan Berger <[email protected]>

stefanberger force-pushed the vtpm branch from b8c570c to 83ac63c Compare July 6, 2020 19:29

ayankuma mentioned this pull request Apr 1, 2022

Build issues seen on Alpine linux aarch64 stefanberger/swtpm#687

Closed

kolyshkin closed this Jun 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add vTPM support for Linux #1591

Add vTPM support for Linux #1591

stefanberger commented Sep 8, 2017 •

edited

Loading

stefanberger commented Sep 8, 2017 •

edited

Loading

deitch commented Sep 8, 2017

cyphar Sep 9, 2017

stefanberger Sep 9, 2017

AkihiroSuda Mar 27, 2020

lukehinds commented Feb 19, 2019

JAORMX commented Nov 20, 2019

stefanberger commented Dec 31, 2019

stefanberger commented Jan 2, 2020

stefanberger commented Jan 3, 2020

stefanberger commented Jan 6, 2020 •

edited

Loading

AkihiroSuda Mar 27, 2020

AkihiroSuda commented Mar 27, 2020

mfranczy commented Mar 25, 2024

kolyshkin commented Jun 15, 2024

Add vTPM support for Linux #1591

Add vTPM support for Linux #1591

Conversation

stefanberger commented Sep 8, 2017 • edited Loading

stefanberger commented Sep 8, 2017 • edited Loading

deitch commented Sep 8, 2017

cyphar Sep 9, 2017

Choose a reason for hiding this comment

stefanberger Sep 9, 2017

Choose a reason for hiding this comment

AkihiroSuda Mar 27, 2020

Choose a reason for hiding this comment

lukehinds commented Feb 19, 2019

JAORMX commented Nov 20, 2019

stefanberger commented Dec 31, 2019

stefanberger commented Jan 2, 2020

stefanberger commented Jan 3, 2020

stefanberger commented Jan 6, 2020 • edited Loading

AkihiroSuda Mar 27, 2020

Choose a reason for hiding this comment

AkihiroSuda commented Mar 27, 2020

mfranczy commented Mar 25, 2024

kolyshkin commented Jun 15, 2024

stefanberger commented Sep 8, 2017 •

edited

Loading

stefanberger commented Sep 8, 2017 •

edited

Loading

stefanberger commented Jan 6, 2020 •

edited

Loading