From 42790102eca325b67f1836f97f2d527602734d24 Mon Sep 17 00:00:00 2001 From: timflannagan1 Date: Thu, 7 Jan 2021 18:27:42 -0500 Subject: [PATCH] docs/design: Minor documentation fixes --- docs/design/operator-bundle.md | 96 +++++++++++++++++++++------------- docs/design/opm-tooling.md | 42 ++++++++++----- 2 files changed, 90 insertions(+), 48 deletions(-) diff --git a/docs/design/operator-bundle.md b/docs/design/operator-bundle.md index d6ca1ae23..7575cbda8 100644 --- a/docs/design/operator-bundle.md +++ b/docs/design/operator-bundle.md @@ -1,25 +1,27 @@ # Operator Bundle -An `Operator Bundle` is a container image that stores Kubernetes manifests and metadata associated with an operator. A bundle is meant to present a *specific* version of an operator. +An `Operator Bundle` is a container image that stores the Kubernetes manifests and metadata associated with an operator. A bundle is meant to represent a *specific* version of an operator. ## Operator Bundle Overview -The operator manifests refers to a set of Kubernetes manifest(s) the defines the deployment and RBAC model of the operator. The operator metadata on the other hand are, but not limited to: +The operator manifests refer to a set of Kubernetes manifest(s) that defines the deployment and RBAC model of the operator. The operator metadata on the other hand are, but not limited to the following properties: + * Information that identifies the operator, its name, version etc. * Additional information that drives the UI: - * Icon - * Example CR(s) + * Icon + * Example CR(s) * Channel(s) * API(s) provided and required. * Related images. -An `Operator Bundle` is built as a scratch (non-runnable) container image that contains operator manifests and specific metadata in designated directories inside the image. Then, it can be pushed and pulled from an OCI-compliant container registry. Ultimately, an operator bundle will be used by [Operator Registry](https://github.com/operator-framework/operator-registry) and [Operator-Lifecycle-Manager (OLM)](https://github.com/operator-framework/operator-lifecycle-manager) to install an operator in OLM-enabled clusters. +An `Operator Bundle` is built as a scratch (i.e. non-runnable) container image that contains operator manifests and specific metadata in designated directories inside the image. Then, it can be pushed and pulled from an OCI-compliant container registry. Ultimately, an operator bundle will be used by [Operator Registry](https://github.com/operator-framework/operator-registry) and [Operator-Lifecycle-Manager (OLM)](https://github.com/operator-framework/operator-lifecycle-manager) to install an operator in OLM-enabled clusters. ### Bundle Manifest Format -The standard bundle format requires two directories named `manifests` and `metadata`. The `manifests` directory is where all operator manifests are resided including ClusterServiceVersion (CSV), CustomResourceDefinition (CRD) and other supported Kubernetes types. The `metadata` directory is where operator metadata is located including `annotations.yaml` which contains additional information such as package name, channels and media type. Also, `dependencies.yaml` which contains the operator dependency information can be included in `metadata` directory. +The standard bundle format requires two directories named `manifests` and `metadata`. The `manifests` directory is where all operator manifests are resided including the `ClusterServiceVersion` (CSV), `CustomResourceDefinition` (CRD) and other supported Kubernetes types. The `metadata` directory is where operator metadata is located including `annotations.yaml` which contains additional information such as the package name, channels and media type. Also, `dependencies.yaml`, which contains the operator dependency information can be included in `metadata` directory. + +Below is the directory layout of an example operator bundle inside a bundle image: -Below is the directory layout of an operator bundle inside a bundle image: ```bash $ tree / @@ -32,24 +34,25 @@ $ tree ``` *Notes:* -* The names of manifests and metadata directories must match the bundle annotations that are provided in `annotations.yaml` file. Currently, those names are set to `manifests` and `metadata`. + +* The names of manifests and metadata directories must match the bundle annotations that are specified in `annotations.yaml` file. Currently, those names are set to `manifests` and `metadata`. ### Bundle Annotations -We use the following labels to annotate the operator bundle image. -* The label `operators.operatorframework.io.bundle.mediatype.v1` reflects the media type or format of the operator bundle. It could be helm charts, plain Kubernetes manifests etc. -* The label `operators.operatorframework.io.bundle.manifests.v1 `reflects the path in the image to the directory that contains the operator manifests. This label is reserved for the future use and is set to `manifests/` for the time being. +We use the following labels to annotate the operator bundle image: + +* The label `operators.operatorframework.io.bundle.mediatype.v1` reflects the media type or format of the operator bundle. It could be helm charts, plain Kubernetes manifests, etc. +* The label `operators.operatorframework.io.bundle.manifests.v1` reflects the path in the image to the directory that contains the operator manifests. This label is reserved for the future use and is set to `manifests/` for the time being. * The label `operators.operatorframework.io.bundle.metadata.v1` reflects the path in the image to the directory that contains metadata files about the bundle. This label is reserved for the future use and is set to `metadata/` for the time being. * The `manifests.v1` and `metadata.v1` labels imply the bundle type: - * The value `manifests.v1` implies that this bundle contains operator manifests. - * The value `metadata.v1` implies that this bundle has operator metadata. + * The value `manifests.v1` implies that this bundle contains operator manifests. + * The value `metadata.v1` implies that this bundle has operator metadata. * The label `operators.operatorframework.io.bundle.package.v1` reflects the package name of the bundle. -* The label `operators.operatorframework.io.bundle.channels.v1` reflects the list of channels the bundle is subscribing to when added into an operator registry +* The label `operators.operatorframework.io.bundle.channels.v1` reflects the list of channels the bundle is subscribing to when added into an operator registry. * The label `operators.operatorframework.io.bundle.channel.default.v1` reflects the default channel an operator should be subscribed to when installed from a registry. This label is optional if the default channel has been set by previous bundles and the default channel is unchanged for this bundle. -The labels will also be put inside a YAML file, as shown below. +The labels will also be put inside a YAML file, `annotations.yaml`, as shown below: -*annotations.yaml* ```yaml annotations: operators.operatorframework.io.bundle.mediatype.v1: "registry+v1" @@ -61,6 +64,7 @@ annotations: ``` *Notes:* + * In case of a mismatch, the `annotations.yaml` file is authoritative because the on-cluster operator-registry that relies on these annotations has access to the yaml file only. * The potential use case for the `LABELS` is - an external off-cluster tool can inspect the image to check the type of a given bundle image without downloading the content. * The annotations for bundle manifests and metadata are reserved for future use. They are set to be `manifests/` and `metadata/` for the time being. @@ -73,7 +77,7 @@ The dependency list will contain a `type` field for each item to specify what ki An example of a `dependencies.yaml` that specifies Prometheus operator and etcd CRD dependencies: -``` +```yaml dependencies: - type: olm.package value: @@ -89,7 +93,8 @@ dependencies: ### Bundle Dockerfile This is an example of a `Dockerfile` for operator bundle: -``` + +```dockerfile FROM scratch # We are pushing an operator-registry bundle @@ -116,22 +121,25 @@ In order to use `opm` CLI, follow the `opm` build instruction: 1. Clone the operator registry repository: ```bash -$ git clone https://github.com/operator-framework/operator-registry +git clone https://github.com/operator-framework/operator-registry ``` 2. Build `opm` binary using this command: ```bash -$ make build +make build ``` Now, a binary named `opm` is now built in current directory and ready to be used. -### Generate Bundle Annotations and DockerFile +### Generating Bundle Annotations and DockerFile + *Notes:* + * If there are `annotations.yaml` and `Dockerfile` existing in the directory, they will be overwritten. Using `opm` CLI, bundle annotations can be generated from provided operator manifests. The overall `bundle generate` command usage is: + ```bash Usage: opm alpha bundle generate [flags] @@ -151,12 +159,14 @@ Flags: The `--directory/-d`, `--channels/-c`, `--package/-p` are required flags while `--default/-e` and `--output-dir/-u` are optional. The command for `generate` task is: + ```bash $ ./opm alpha bundle generate --directory /test --package test-operator \ --channels stable,beta --default stable ``` The `--directory` or `-d` specifies the directory where the operator manifests, including CSVs and CRDs, are located. For example: + ```bash $ tree test test @@ -164,11 +174,14 @@ test └── etcdoperator.clusterserviceversion.yaml ``` -The `--package` or `-p` is the name of package fo the operator such as `etcd` which which map `channels` to a particular application definition. `channels` allow package authors to write different upgrade paths for different users (e.g. `beta` vs. `stable`). The `channels` list is provided via `--channels` or `-c` flag. Multiple `channels` are separated by a comma (`,`). The default channel is provided optionally via `--default` or `-e` flag. If the default channel is not provided, the first channel in channel list is selected as default. +The `--package` or `-p` is the name of package of the operator such as `etcd`, which maps `channels` to a particular application definition. `channels` allow package authors to write different upgrade paths for different users (e.g. `beta` vs. `stable`). The `channels` list is provided via the `--channels` or `-c` flag. Multiple `channels` are separated by a comma (`,`). The default channel is provided optionally via `--default` or `-e` flag. If the default channel is not provided, the first channel in the list of channels is selected as the default. + +**Note**: All information specified in `annotations.yaml` must also exist in the `LABEL` section of a `Dockerfile`. + +After the generate command is executed, the `Dockerfile` is generated in the same directory where the command was run. By default, the `annotations.yaml` file is located in a folder named `metadata` in the same root directory as the input directory containing manifests. -All information in `annotations.yaml` is also existed in `LABEL` section of `Dockerfile`. +For example: -After the generate command is executed, the `Dockerfile` is generated in the directory where command is run. By default, the `annotations.yaml` file is located in a folder named `metadata` in the same root directory as the input directory containing manifests. For example: ```bash $ tree test test @@ -180,7 +193,7 @@ test └── Dockerfile ``` -If the `--output-dir` parameter is specified, that directory becomes the parent for a new pair of folders `manifests/` and `metadata/`, where `manifests/` is a copy of the passed in directory of manifests and `metadata/` is the folder containing annotations.yaml: +If the `--output-dir` parameter is specified, that directory becomes the parent for a new pair of folders `manifests/` and `metadata/`, where `manifests/` is a copy of the passed in directory of manifests and `metadata/` is the folder containing the `annotations.yaml` file: ```bash $ tree test @@ -197,15 +210,20 @@ test └── Dockerfile ``` -The `Dockerfile` can be used manually to build the bundle image using container image tools such as Docker, Podman or Buildah. For example, the Docker build command would be: +The `Dockerfile` can be used manually to build the bundle image using various container image tooling such as Docker, Podman or Buildah. + +The following example uses `docker` to build the bundle image: ```bash -$ docker build -f /path/to/Dockerfile -t quay.io/test/test-operator:latest /path/to/manifests/ +docker build -f /path/to/Dockerfile -t quay.io/test/test-operator:latest /path/to/manifests/ ``` ### Build Bundle Image -Operator bundle image can be built from provided operator manifests using `build` command (see *Notes* below). The overall `bundle build` command usage is: +An operator bundle image can be built from the provided operator manifests using the `build` command (see *Notes* below). + +The overall `bundle build` command usage is: + ```bash Usage: opm alpha bundle build [flags] @@ -226,29 +244,35 @@ Flags: * All manifests yaml must be in the same directory. ``` -The command for `build` task is: +The command for the `build` task is: + ```bash $ ./opm alpha bundle build --directory /test --tag quay.io/coreos/test-operator.v0.1.0:latest \ --package test-operator --channels stable,beta --default stable ``` -The `--directory` or `-d` specifies the directory where the operator manifests for a specific version are located. The `--tag` or `-t` specifies the image tag that you want the operator bundle image to have. By using `build` command, the `annotations.yaml` and `Dockerfile` are automatically generated in the background. +The `--directory` or `-d` specifies the directory location that contains the manifests for a specific version of an operator. The `--tag` or `-t` specifies the image tag that you want the operator bundle image to have. By using `build` command, the `annotations.yaml` and `Dockerfile` are automatically generated in the background. + +The default image builder is `Docker`. However, `Buildah` and `Podman` are also supported. An image builder can be specified via `--image-builder` or the optional `-b` tag in the `build` command. For example: -The default image builder is `Docker`. However, ` Buildah` and `Podman` are also supported. An image builder can be specified via `--image-builder` or `-b` optional tag in `build` command. For example: ```bash -$ ./opm alpha bundle build --directory /test/0.1.0/ --tag quay.io/coreos/test-operator.v0.1.0:latest \ ---image-builder podman --package test-operator --channels stable,beta --default stable +$ ./opm alpha bundle build --directory /test/0.1.0/ \ + --tag quay.io/coreos/test-operator.v0.1.0:latest \ + --image-builder podman --package test-operator \ + --channels stable,beta --default stable ``` -The `--package` or `-p` is the name of package for the operator such as `etcd` which maps `channels` to a particular application definition. `channels` allow package authors to write different upgrade paths for different users (e.g. `beta` vs. `stable`). The `channels` list is provided via `--channels` or `-c` flag. Multiple `channels` are separated by a comma (`,`). The default channel is provided optionally via `--default` or `-e` flag. +The `--package` or `-p` is the name of the package for the operator such as `etcd`, which maps `channels` to a particular application definition. Here, `channels` allow package authors to write different upgrade paths for different users (e.g. `beta` vs. `stable`). The `channels` list is provided via `--channels` or `-c` flag. Multiple `channels` are separated by a comma (`,`). The default channel is provided optionally via `--default` or `-e` flag. *Notes:* + * If there is `Dockerfile` existing in the directory, it will be overwritten. * If there is an existing `annotations.yaml` in `/metadata` directory, the cli will attempt to validate it and returns any found errors. If the ``annotations.yaml`` is valid, it will be used as a part of build process. The optional boolean `--overwrite/-o` flag can be enabled (false by default) to allow cli to overwrite the `annotations.yaml` if existed. ### Validate Bundle Image Operator bundle image can validate bundle image that is publicly available in an image registry using `validate` command (see *Notes* below). The overall `bundle validate` command usage is: + ```bash Usage: opm alpha bundle validate [flags] @@ -260,8 +284,9 @@ Flags: ``` The command for `validate` task is: + ```bash -$ ./opm alpha bundle validate --tag quay.io/coreos/test-operator.v0.1.0:latest --image-builder docker +./opm alpha bundle validate --tag quay.io/coreos/test-operator.v0.1.0:latest --image-builder docker ``` The `validate` command will first extract the content of the bundle image into a temporary directory after it pulls the image from its image registry. Then, it will validate the format of bundle image to ensure manifests and metadata are located in their appropriate directories (`/manifests/` for bundle manifests files such as CSV and `/metadata/` for metadata files such as `annotations.yaml`). Also, it will validate the information in `annotations.yaml` to confirm that metadata is matching the provided data. For example, the provided media type in annotations.yaml just matches the actual media type is provided in the bundle image. @@ -269,4 +294,5 @@ The `validate` command will first extract the content of the bundle image into a After the bundle image format is confirmed, the command will validate the bundle contents such as manifests and metadata files if the bundle format is `RegistryV1` or "Plain" type. "RegistryV1" format means it contains `ClusterServiceVersion` and its associated Kubernetes objects while `PlainType` means it contains all Kubernetes objects. The content validation process will ensure the individual file in the bundle image is valid and can be applied to an OLM-enabled cluster provided all necessary permissions and configurations are met. *Notes:* + * The bundle content validation is best effort which means it will not guarantee 100% accuracy due to nature of Kubernetes objects may need certain permissions and configurations, which users may not have, in order to be applied successfully in a cluster. diff --git a/docs/design/opm-tooling.md b/docs/design/opm-tooling.md index b2faf7e5d..76f6ad673 100644 --- a/docs/design/opm-tooling.md +++ b/docs/design/opm-tooling.md @@ -1,6 +1,6 @@ # Operator Registry Tooling -When compiled, the `operator-registry` project results in a collection of tools that in aggregate define a way of packaging and delivering operator manifests to Kubernetes clusters. Historically, this is done with multiple tools. For example, you can use `initializer` to generate an immutable database and then use `registry-serve` to serve the database via an API. We have added the `opm` tool that aggregates these functions togeother and allows a user to interact with container images and tooling directly to generate and update registry databases in a mutable way. +When compiled, the `operator-registry` project results in a collection of tools that in aggregate define a way of packaging and delivering operator manifests to Kubernetes clusters. Historically, this is done with multiple tools. For example, you can use `initializer` to generate an immutable database and then use `registry-serve` to serve the database via an API. We have added the `opm` tool that aggregates these functions together and allows a user to interact with container images and tooling directly to generate and update registry databases in a mutable way. The following document describes the tooling that `opm` provides along with descriptions of how to use them including each command's purpose, their inputs and outputs, and some examples. @@ -14,7 +14,9 @@ The following document describes the tooling that `opm` provides along with desc #### add -First, let's look at adding a version of an operator bundle to a registry database. For example: +First, let's look at adding a version of an operator bundle to a registry database. + +For example: `opm registry add -b "quay.io/operator-framework/operator-bundle-prometheus:0.14.0" -d "test-registry.db"` @@ -28,7 +30,9 @@ Great! The existing `test-registry.db` file is updated. Now we have a registry t #### rm -`opm` also currently supports removing entire packages from a registry. For example: +`opm` also currently supports removing entire packages from a registry. + +For example: `opm registry rm -o "prometheus" -d "test-registry.db"` @@ -36,7 +40,9 @@ Calling this on our existing test registry removes all versions of the prometheu #### prune -`opm` supports specifying which packages should be kept in an operator database. For example: +`opm` supports specifying which packages should be kept in an operator database. + +For example: `opm registry prune -p "prometheus" -d "test-registry.db"` @@ -58,7 +64,7 @@ Index add works much the same way as registry add. For example: `opm index add --bundles quay.io/operator-framework/operator-bundle-prometheus:0.14.0 --tag quay.io/operator-framework/monitoring-index:1.0.0` -Just like `opm registry add`, this command pulls a given container bundle and attempts to put it into a registry. The real difference is that the result is more than just a database file. By default, this command actually builds a container image and, looking at the `--tag` flag, will tag the output image as `quay.io/operator-framework/monitoring-index:1.0.0`. The resulting image has the database and the opm binary in it and, when run, calls the `registry serve` command on the database that was generated. +Similar to `opm registry add`, this command will pull the specified container bundle and insert it into a registry. The real difference is that the result is more than just a database file. By default, this command will also attempt to build a container image and depending on the value of the `--tag` flag, will tag the output image as `quay.io/operator-framework/monitoring-index:1.0.0`. The resulting image has the database and the opm binary in it and, when run, calls the `registry serve` command on the database that was generated. Just like registry add command, the updates are cumulative. In this case, rather than pointing at a database file, we can use the `--from-index` flag to specify a previous index to build off of a previous registry: @@ -66,7 +72,9 @@ Just like registry add command, the updates are cumulative. In this case, rather This results in a fresh image that includes the updated prometheus operator in the prometheus package's update graph. -At a high level, this command operates by wrapping `registry add` around some additional interaction with pulling and building container images. To that end, the last thing it does is actually shell out to a container CLI tool to build the resulting container (by default, `podman build`). It does this by generating a dockerfile and then passing that file to the shell command. For example: +At a high level, this command operates by wrapping `registry add` around some additional interaction with pulling and building container images. To that end, the last thing it does is actually shell out to a container CLI tool to build the resulting container (by default, `podman build`). It does this by generating a dockerfile and then passing that file to the shell command. + +For example: ```dockerfile FROM quay.io/operator-framework/upstream-registry-builder AS builder @@ -81,13 +89,15 @@ ENTRYPOINT ["/opm"] CMD ["registry", "serve", "--database", "index.db"] ``` -Of note here is that we use a builder image to get the latest upstream released version of opm in order to call `opm registry serve` to host the gRPC API. If a developer or CI system would prefer to point to a different version of `opm` to serve their operator (perhaps one in a private release or a fork) then they just need to deliver their own version in a container and then use the `--binary-image` command. ex: +In the above example, it's important to note that we use a builder image to get the latest upstream released version of opm in order to call `opm registry serve` to host the gRPC API. If a developer or CI system would prefer to point to a different version of `opm` to serve their operator (perhaps one in a private release or a fork) then they just need to deliver their own version in a container and then use the `--binary-image` command. + +For example: `opm index add --bundles quay.io/operator-framework/operator-bundle-prometheus:0.14.0 --tag quay.io/operator-framework/monitoring-index:1.0.0 --binary-image quay.io/$user/my-opm-source` This will update the above dockerfile and replace the builder image with the image specified in the `--binary-image` flag. -We are aware of the fact that, in many cases, users will want to make other changes to this dockerfile (adding additional labels, adding other binaries for metrics, using a different port, etc.). For these more complex use cases, we have added the `--generate` and `--out-dockerfile` flags. Adding `--generate` will skip the container build command entirely and instead write a Dockerfile to the local filesystem. By default this file is called `index.Dockerfile` and is put in the directory you run `opm` from. If you want to rename this generated dockerfile and write it somewhere else, just specify the `--out-dockerfile` flag: +We are aware of the fact that, in many cases, users will want to make other changes to this dockerfile (adding additional labels, adding other binaries for metrics, using a different port, etc.). For these more complex use cases, we have added the `--generate` and `--out-dockerfile` flags. Adding `--generate` will skip the container build command entirely and instead write a Dockerfile to the local filesystem. By default, this file is called `index.Dockerfile` and is put in the directory you run `opm` from. If you want to rename this generated dockerfile and write it somewhere else, just specify the `--out-dockerfile` flag: `opm index add --bundles quay.io/operator-framework/operator-bundle-prometheus:0.14.0 --generate --out-dockerfile "my.Dockerfile"` @@ -95,7 +105,9 @@ Running this command will still generate the updated registry database, but it w #### rm -Like `opm registry rm`, this command will remove all versions an entire operator package from the index and results in a container image that does not include that package. It supports virtually all of the same options and flags as `opm index add` with the exception of replacing `--bundles` with `--operators`. Ex: +Like `opm registry rm`, this command will remove all versions an entire operator package from the index and results in a container image that does not include that package. It supports virtually all of the same options and flags as `opm index add` with the exception of replacing `--bundles` with `--operators`. + +For example: `opm index rm --operators prometheus --tag quay.io/operator-framework/monitoring-index:1.0.2 --binary-image quay.io/$user/my-opm-source` @@ -103,15 +115,19 @@ This will result in the tagged container image `quay.io/operator-framework/monit #### prune -`opm index prune` allows the user to specify which operator packages should be maintained in an index. For example: +`opm index prune` allows the user to specify which operator packages should be maintained in an index. + +For example: `opm index prune -p "prometheus" --from-index quay.io/operator-framework/example-index:1.0.0 --tag quay.io/operator-framework/example-index:1.0.1` -Would remove all but the `prometheus` package from the index. +This would remove all but the `prometheus` package from the index. #### export -`opm index export` will export a package from an index image into a directory. The format of this directory will match the appregistry manifest format: containing all versions of the package in the index along with a `package.yaml` file. This command takes an `--index` flag that points to an index image, a `--package` flag that states a package name, an optional `--download-folder` as the export location (default is `./downloaded`), and just as the other index commands it takes a `--container-tool` flag. Ex: +`opm index export` will export a package from an index image into a directory. The format of this directory will match the appregistry manifest format: containing all versions of the package in the index along with a `package.yaml` file. This command takes an `--index` flag that points to an index image, a `--package` flag that states a package name, an optional `--download-folder` as the export location (default is `./downloaded`), and just as the other index commands it takes a `--container-tool` flag. + +For example: `opm index export --index="quay.io/operator-framework/monitoring:1.0.0" --package="prometheus" -c="podman"` @@ -148,7 +164,7 @@ which can be pushed to appregistry. Of note, many of these commands require some form of shelling to common container tooling. By default, the container tool that `opm` shells to is [podman](https://podman.io/). However, we also support overriding this via the `--container-tool`. -_Ex._ +For example: `opm index add --bundles quay.io/operator-framework/operator-bundle-prometheus:0.14.0 --tag quay.io/operator-framework/monitoring-index:1.0.0 --container-tool docker`