Guest FW Attestation (#16)

.gitignore

@@ -1,4 +1,5 @@
 # Installed binaries
+cosign
 crictl
 kubeadm
 kubectl

README.md

@@ -43,7 +43,8 @@ inv kubeadm.create
 ```
 
 Second, install both the operator and the CC runtime from the upstream tag.
-We currently pin to version `v0.7.0` (see the [`COCO_RELEASE_VERSION` variable](https://github.com/csegarragonz/coco-serverless/tree/main/tasks/util/env.py)).
+We currently pin to version `v0.7.0` (see the [`COCO_RELEASE_VERSION` variable](
+https://github.com/csegarragonz/coco-serverless/tree/main/tasks/util/env.py)).
 
 ```bash
 inv operator.install
@@ -62,6 +63,7 @@ if it is the first time, you will have to manually build the agent following
 Then, you are ready to run one of the supported apps:
 * [Hello World! (Py)](./docs/helloworld_py.md) - simple HTTP server running in Python to test CoCo and Kata.
 * [Hello World! (Knative)](./docs/helloworld_knative.md) - same app as before, but invoked over Knatvie.
+* [Hello Attested World! (Knative + Attestation)](./docs/helloworld_knative_attestation.md) -
 
 If your app uses Knative, you will have to install it first:
 

docs/helloworld_knative.md

@@ -7,7 +7,7 @@ This sample application does not use any attestation or image encryption, so
 you should disable it by running:
 
 ```bash
-inv coco.disable-attestation
+inv coco.guest-attestation --mode off
 ```
 
 To deploy it, you may run:

docs/helloworld_knative_attestation.md

@@ -0,0 +1,83 @@
+# Knative + CoCo + Attestation
+
+In this demo, we set up CoCo to attest the guest VM that is bootstrapped.
+In order to run a functional attested system we need a service that, given
+a launch digest, returns a secret (i.e. a relying party). We use CoCo's
+[simple KBS](https://github.com/confidential-containers/simple-kbs).
+
+For demonstration purposes, the simple KBS runs as a process in the same
+node (outside Knative, and Kubernetes for that same reason). To start it,
+just run:
+
+```bash
+inv kbs.start
+```
+
+Depending on what features do you want to measure/attest, browse through the
+following subsections (in increasing order of security):
+* [Firmware Digest](#firmware-digest) - Attest that the VM runs in an SEV node,
+with the right firmware, kernel, initrd, and Kata Agent configuration.
+
+After that, you may jump to [running the application](#run-the-application).
+
+## Firmware Digest
+
+Before running the application, we need to generate the expected launch digest
+for our confidential VM. This launch digest will be generated by reading some
+of the fields in the Kata config file, so we must update it before generating
+the digest.
+
+First, we need to set `guest_pre_attestation` so that the Kata Shim opens a
+secure channel between the PSP and the KBS at boot time. We will later use this
+channel to validate firmware and software measurements from inside the guest:
+
+```bash
+inv coco.guest-attestation --mode on
+```
+
+note that this method also sets the right KBS URI, as the default one
+(`localhost`) is not reachable from inside the guest.
+
+Now, we must also enable signature verification. Signature verification is the
+only method available in `v0.7.0` to check the launch digest against a user-
+provided digest:
+
+```bash
+inv coco.signature-verification --mode on
+```
+
+In order to validate the launch digest, we need to pre-provision the launch
+digest. To do so you may run:
+
+```bash
+inv kbs.provision-launch-digest --signature-policy none
+```
+
+this method will generate launch digests from the data in the Kata configuration
+files (i.e. `/opt/confidential-containers/share/defaults/kata-containers/configuration-*.toml`
+if installed by the operator), and include them in the KBS.
+
+If you don't want to sign and/or encrypt your container images, you may jump
+straight to [running the application](#run-the-application).
+
+## Signed Container Images
+
+If we enable signature verification, we then need to sign our container image
+too. As [recommended](https://github.com/sigstore/cosign#sign-a-container-and-store-the-signature-in-the-registry),
+we sign images based on their digest:
+
+```bash
+inv cosign.sign-container-image "docker.io/csegarragonz/coco-helloworld-py@sha256:af0fec55e9aed9a259e8da9dcaa28ab3fc1277dc8db4b8883265f98272cef11d"
+inv cosign.sign-container-image "gcr.io/knative-releases/knative.dev/serving/cmd/queue@sha256:987f53e3ead58627e3022c8ccbb199ed71b965f10c59485bab8015ecf18b44af"
+```
+
+## Encrypted Container Images
+
+## Run the application
+
+Once the KBS has been populated with the right measurements and secrets, we can
+deploy the workload just like with the `Hello world! (Knative)` app:
+
+```bash
+kubectl apply -f ./apps/helloworld-knative
+```

requirements.txt

@@ -1,5 +1,6 @@
 ansible>=8.4.0
 black>=23.9.1
 invoke>=2.1.0
+pymysql>=1.1.0
 python-language-server[all]
 sev-snp-measure>=0.0.7

tasks/__init__.py

@@ -3,6 +3,7 @@
 from . import apps
 from . import coco
 from . import containerd
+from . import cosign
 from . import format_code
 from . import k8s
 from . import k9s
@@ -17,6 +18,7 @@
     apps,
     coco,
     containerd,
+    cosign,
     format_code,
     k8s,
     k9s,

tasks/coco.py

@@ -1,17 +1,67 @@
 from invoke import task
 from os.path import join
 from tasks.util.env import KATA_CONFIG_DIR
-from tasks.util.toml import update_toml
+from tasks.util.kbs import KBS_PORT, get_kbs_url
+from tasks.util.toml import read_value_from_toml, update_toml
 
 
 @task
-def disable_attestation(ctx):
+def guest_attestation(ctx, mode="off"):
     """
-    Disable attestation for CoCo
+    Toggle guest attestation for CoCo: -attestation --mode=[on,off]
     """
     conf_file_path = join(KATA_CONFIG_DIR, "configuration-qemu-sev.toml")
+
+    # Update the pre_attestation flag
+    att_val = str(mode == "on").lower()
     updated_toml_str = """
     [hypervisor.qemu]
-    guest_pre_attestation = false
+    guest_pre_attestation = {att_val}
+    """.format(
+        att_val=att_val
+    )
+    update_toml(conf_file_path, updated_toml_str)
+
+    # We also update the KBS URI if pre_attestation is enabled
+    if mode == "on":
+        # We need to set the KBS URL to something that is reachable both from
+        # the host _and_ the guest
+        updated_toml_str = """
+        [hypervisor.qemu]
+        guest_pre_attestation_kbs_uri = "{kbs_url}:{kbs_port}"
+        """.format(
+            kbs_url=get_kbs_url(), kbs_port=KBS_PORT
+        )
+        update_toml(conf_file_path, updated_toml_str)
+
+
+@task
+def signature_verification(ctx, mode="off"):
     """
+    Toggle signature verification for CoCo's agent: --mode=[on,off]
+    """
+    conf_file_path = join(KATA_CONFIG_DIR, "configuration-qemu-sev.toml")
+    att_val = str(mode == "on").lower()
+
+    # We need to update the kernel parameters, which is a string, so we are
+    # particularly careful
+    original_kernel_params = read_value_from_toml(
+        conf_file_path, "hypervisor.qemu.kernel_params"
+    )
+    # Whenever I learn regex, this will be less hacky
+    pattern = "enable_signature_verification="
+    value_beg = original_kernel_params.find(pattern) + len(pattern)
+    value_end = original_kernel_params.find(" ", value_beg)
+    updated_kernel_params = (
+        original_kernel_params[:value_beg]
+        + att_val
+        + original_kernel_params[value_end:]
+    )
+
+    updated_toml_str = """
+    [hypervisor.qemu]
+    kernel_params = "{updated_kernel_params}"
+    """.format(
+        updated_kernel_params=updated_kernel_params
+    )
     update_toml(conf_file_path, updated_toml_str)

tasks/containerd.py

@@ -158,7 +158,7 @@ def set_log_level(ctx, log_level):
 
     updated_toml_str = """
     [debug]
-    level = {log_level}
+    level = "{log_level}"
     """.format(
         log_level=log_level
     )

tasks/cosign.py

@@ -0,0 +1,29 @@
+from invoke import task
+from os.path import join
+from subprocess import run
+from tasks.util.cosign import (
+    COSIGN_BINARY,
+    COSIGN_VERSION,
+    sign_container_image as do_sign_container_image,
+)
+from tasks.util.env import BIN_DIR
+
+
+@task
+def install(ctx):
+    """
+    Install the cosign tool to sign container images
+    """
+    cosign_url = "https://github.com/sigstore/cosign/releases/download/"
+    cosign_url += "v{}/cosign-linux-amd64".format(COSIGN_VERSION)
+    cosign_path = join(BIN_DIR, COSIGN_BINARY)
+    run("wget {} -O {}".format(cosign_url, cosign_path), shell=True, check=True)
+    run("chmod +x {}".format(cosign_path), shell=True, check=True)
+
+
+@task
+def sign_container_image(ctx, image_tag):
+    """
+    Sign a container image
+    """
+    do_sign_container_image(image_tag)

tasks/kbs.py

@@ -1,18 +1,139 @@
+from base64 import b64encode
 from invoke import task
-from os.path import exists, join
+from os.path import exists
 from subprocess import run
-from tasks.util.env import PROJ_ROOT
+from tasks.util.kbs import SIMPLE_KBS_DIR, create_kbs_resource, connect_to_kbs_db
+from tasks.util.sev import get_launch_digest
 
-SIMPLE_KBS_DIR = join(PROJ_ROOT, "..", "simple-kbs")
+SIMPLE_KBS_DEFAULT_POLICY = "/usr/local/bin/default_policy.json"
+
+SIGNATURE_POLICY_STRING_ID = "default/security-policy/test"
+SIGNATURE_POLICY_NONE = "none"
+ALLOWED_SIGNATURE_POLICIES = [SIGNATURE_POLICY_NONE]
+SIGNATURE_POLICY_NONE_JSON = """{
+    "default": [{"type": "insecureAcceptAnything"}],
+    "transports": {}
+}
+"""
+
+
+def check_kbs_dir():
+    if not exists(SIMPLE_KBS_DIR):
+        print("Error: could not find local KBS checkout at {}".format(SIMPLE_KBS_DIR))
+        raise RuntimeError("Simple KBS local checkout not found!")
+
+
+@task
+def cli(ctx):
+    """
+    Get a development CLI in the simple KBS server
+    """
+    # Make sure the KBS is running
+    check_kbs_dir()
+    run(
+        "docker compose up -d --no-recreate cli",
+        shell=True,
+        check=True,
+        cwd=SIMPLE_KBS_DIR,
+    )
+    run("docker compose exec -it cli bash", shell=True, check=True, cwd=SIMPLE_KBS_DIR)
 
 
 @task
 def start(ctx):
     """
     Start the simple KBS service
     """
-    if not exists(SIMPLE_KBS_DIR):
-        print("Error: could not find local KBS checkout at {}".format(SIMPLE_KBS_DIR))
-        raise RuntimeError("Simple KBS local checkout not found!")
+    check_kbs_dir()
+    run("docker compose up -d server", shell=True, check=True, cwd=SIMPLE_KBS_DIR)
+
+
+@task
+def stop(ctx):
+    """
+    Stop the simple KBS service
+    """
+    check_kbs_dir()
+    run("docker compose down", shell=True, check=True, cwd=SIMPLE_KBS_DIR)
+
+
+@task
+def clear_db(ctx):
+    """
+    Clear the contents of the KBS DB
+    """
+    connection = connect_to_kbs_db()
+    with connection:
+        with connection.cursor() as cursor:
+            cursor.execute("DELETE from policy")
+            cursor.execute("DELETE from resources")
+            cursor.execute("DELETE from secrets")
+
+        connection.commit()
+
+
+@task
+def provision_launch_digest(ctx, signature_policy):
+    """
+    Provision the KBS with the launch digest for the current node
+
+    In order to make the Kata Agent validate the FW launch digest measurement
+    we need to enable signature verification. Signature verification has an
+    associated resource that contains the verification policy. By associating
+    this resource to a launch digest policy (beware of the `policy` term
+    overloading, but these are KBS terms), we force the Kata Agent to also
+    enforce the launch digest policy.
+
+    We support different kinds of signature verification policies, and only
+    one kind of launch digest policy.
+
+    For signature verification, we have the NONE policy, that accepts all
+    images.
+
+    For launch digest, we manually generate the measure digest, and include it
+    in the policy. If the FW digest is not exactly the one in the policy, boot
+    fails.
+    """
+    if signature_policy not in ALLOWED_SIGNATURE_POLICIES:
+        print(
+            "--signature-policy must be one in: {}".format(ALLOWED_SIGNATURE_POLICIES)
+        )
+        raise RuntimeError("Disallowed signature policy: {}".format(signature_policy))
+
+    # First, add our launch digest to the KBS policy
+    ld = get_launch_digest("sev")
+    ld_b64 = b64encode(ld).decode()
+
+    # Create a new record
+    connection = connect_to_kbs_db()
+    with connection:
+        with connection.cursor() as cursor:
+            policy_id = 10
+
+            # When enabling signature verification, we need to provide a
+            # signature policy. This policy has a constant string identifier
+            # that the kata agent will ask for (default/security-policy/test),
+            # which points to a config file that specifies how to validate
+            # signatures
+            if signature_policy == SIGNATURE_POLICY_NONE:
+                # If we set a `none` signature policy, it means that we don't
+                # check any signatures on the pulled container images
+                resource_path = "signature_policy_{}.json".format(signature_policy)
+                create_kbs_resource(resource_path, SIGNATURE_POLICY_NONE_JSON)
+
+            # Create the resource (containing the signature policy) in the KBS
+            sql = "INSERT INTO resources VALUES(NULL, NULL, "
+            sql += "'{}', '{}', {})".format(
+                SIGNATURE_POLICY_STRING_ID, resource_path, policy_id
+            )
+            cursor.execute(sql)
+
+            # We associate the signature policy to a digest policy, meaning
+            # that irrespective of what signature policy are we using (even if
+            # we are not checking any signatures) we will always check the FW
+            # digest against the measured one
+            sql = "INSERT INTO policy VALUES ({}, ".format(policy_id)
+            sql += "'[\"{}\"]', '[]', 0, 0, '[]', now(), NULL, 1)".format(ld_b64)
+            cursor.execute(sql)
 
-    run("docker compose up -d", shell=True, check=True, cwd=SIMPLE_KBS_DIR)
+        connection.commit()