feat: add persistentvolumeclaim module type and volumes for containers

This makes it possible to mount persistent volumes in container modules, including in tasks and tests. With the right storage classes in place, you can even share volumes between container services, tasks and tests. These volumes can of course also be referenced in `kubernetes` and `helm` modules. See the added section to the container modules guide for details on how to use the new features. As a small bonus, in order to be able to use the upstream PersistentVolumeClaim schema, I've added a `joi.customObject().jsonSchema()` type, which allows us weave JSON schemas into our existing Joi schemas. My hope is that we can gradually get rid of most usages of Joi schemas, at least in our plugin APIs.
garden-io · Feb 19, 2020 · 4d6bfee · 4d6bfee
1 parent 86ee925
commit 4d6bfee
Show file tree

Hide file tree

Showing 55 changed files with 3,339 additions and 734 deletions.
diff --git a/docs/README.md b/docs/README.md
@@ -46,6 +46,7 @@
   * [`kubernetes`](./module-types/kubernetes.md)
   * [`maven-container`](./module-types/maven-container.md)
   * [`openfaas`](./module-types/openfaas.md)
+  * [`persistentvolumeclaim`](./module-types/persistentvolumeclaim.md)
   * [`terraform`](./module-types/terraform.md)
 * [Reference](./reference/README.md)
   * [Glossary](./reference/glossary.md)

diff --git a/docs/guides/container-modules.md b/docs/guides/container-modules.md
@@ -216,3 +216,92 @@ values:
 Here, we declare `my-image` as a dependency for the `my-service` Helm chart. In order for the Helm chart to be able to reference the built container image, we must provide the correct image name and version.
 
 For a full list of keys that are available for the `container` module type, take a look at the [outputs reference](../module-types/container.md#outputs).
+
+## Mounting volumes
+
+`container` services, tasks and tests can all mount volumes, using _volume modules_. One such is the [`persistentvolumeclaim` module type](../module-types/persistentvolumeclaim.md), supported by the `kubernetes` provider. To mount a volume, you need to define a volume module, and reference it using the `volumes` key on your services, tasks and/or tests.
+
+Example:
+
+```yaml
+kind: Module
+name: my-volume
+type: persistentvolumeclaim
+spec:
+  accessModes: [ReadWriteOnce]
+  resources:
+    requests:
+      storage: 1Gi
+---
+kind: Module
+name: my-module
+type: container
+services:
+  - name: my-service
+    replicas: 1  # <- Important! Unless your volume supports ReadWriteMany, you can't run multiple replicas with it
+    volumes:
+      - name: my-volume
+        module: my-volume
+        containerPath: /volume
+    ...
+```
+
+This will mount the `my-volume` PVC at `/volume` in the `my-service` service when it is run. The `my-volume` module creates a `PersistentVolumeClaim` resource in your project namespace, and the `spec` field is passed directly to the same field on the PVC resource.
+
+{% hint style="warning" %}
+Notice the `accessModes` field in the volume module above. The default storage classes in Kubernetes generally don't support being mounted by multiple Pods at the same time. If your volume module doesn't support the `ReadWriteMany` access mode, you must take care not to use the same volume in multiple services, tasks or tests, or multiple replicas. See [Shared volumes](#shared-volumes) below for how to share a single volume with multiple Pods.
+{% endhint %}
+
+You can do the same for tests and tasks using the [`tests.volumes`](../module-types/container.md#testsvolumes) and [`tasks.volumes`](../module-types/container.md#tasksvolumes) fields. `persistentvolumeclaim` volumes can of course also be referenced in `kubernetes` and
+`helm` modules, since they are deployed as standard PersistentVolumeClaim resources.
+
+Take a look at the [`persistentvolumeclaim` module type](../module-types/persistentvolumeclaim.md) and [`container` module](../module-types/container.md#servicesvolumes) docs for more details.
+
+### Shared volumes
+
+For a volume to be shared between multiple replicas, or multiple services, tasks and/or tests, it needs to be configured with a storage class (using the `storageClassName` field) that supports the `ReadWriteMany` (RWX) access mode. The available storage classes that support RWX vary by cloud providers and cluster setups, and in many cases you need to define a `StorageClass` or deploy a _storage class provisioner_ to your cluster.
+
+You can find a list of storage options and their supported access modes [here](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes). Here are a few commonly used RWX provisioners and storage classes:
+
+* [NFS Server Provisioner](https://github.com/helm/charts/tree/master/stable/nfs-server-provisioner)
+* [Azure File](https://docs.microsoft.com/en-us/azure/aks/azure-files-dynamic-pv)
+* [AWS EFS Provisioner](https://github.com/helm/charts/tree/master/stable/efs-provisioner)
+* [Ceph (via Rook)](https://rook.io/docs/rook/v1.2/ceph-filesystem.html)
+
+Once any of those is set up you can create a `persistentvolumeclaim` module that uses the configured storage class. Here, for example, is how you might use a shared volume with a configured `azurefile` storage class:
+
+```yaml
+kind: Module
+name: shared-volume
+type: persistentvolumeclaim
+spec:
+  accessModes: [ReadWriteMany]
+  resources:
+    requests:
+      storage: 1Gi
+  storageClassName: azurefile
+---
+kind: Module
+name: my-module
+type: container
+services:
+  - name: my-service
+    volumes:
+      - &volume   # <- using a YAML anchor to re-use the volume spec in tasks and tests
+        name: shared-volume
+        module: shared-volume
+        containerPath: /volume
+    ...
+tasks:
+  - name: my-task
+    volumes:
+      - *volume
+    ...
+tests:
+  - name: my-test
+    volumes:
+      - *volume
+    ...
+```
+
+Here the same volume is used across a service, task and a test in the same module. You could similarly use the same volume across multiple container modules.
diff --git a/docs/module-types/README.md b/docs/module-types/README.md
@@ -13,4 +13,5 @@ title: Module Types
 * [`kubernetes`](./kubernetes.md)
 * [`maven-container`](./maven-container.md)
 * [`openfaas`](./openfaas.md)
+* [`persistentvolumeclaim`](./persistentvolumeclaim.md)
 * [`terraform`](./terraform.md)
diff --git a/docs/module-types/container.md b/docs/module-types/container.md
@@ -292,7 +292,10 @@ services:
     # with hot-reloading enabled, or if the provider doesn't support multiple replicas.
     replicas:
 
-    # List of volumes that should be mounted when deploying the container.
+    # List of volumes that should be mounted when deploying the service.
+    #
+    # Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when
+    # deploying the container.
     volumes:
       - # The name of the allocated volume.
         name:
@@ -307,6 +310,18 @@ services:
         # module source path (or absolute).
         hostPath:
 
+        # The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will
+        # depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim
+        # module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.
+        #
+        # When a `module` is specified, the referenced module/volume will be automatically configured as a runtime
+        # dependency of this service, as well as a build dependency of this module.
+        #
+        # Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports
+        # the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple
+        # services at the same time. Refer to the documentation of the module type in question to learn more.
+        module:
+
 # A list of tests to run in the module.
 tests:
   - # The name of the test.
@@ -345,6 +360,36 @@ tests:
     # `GARDEN`) and values must be primitives or references to secrets.
     env: {}
 
+    # List of volumes that should be mounted when deploying the test.
+    #
+    # Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when
+    # deploying the container.
+    volumes:
+      - # The name of the allocated volume.
+        name:
+
+        # The path where the volume should be mounted in the container.
+        containerPath:
+
+        # _NOTE: Usage of hostPath is generally discouraged, since it doesn't work reliably across different platforms
+        # and providers. Some providers may not support it at all._
+        #
+        # A local path or path on the node that's running the container, to mount in the container, relative to the
+        # module source path (or absolute).
+        hostPath:
+
+        # The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will
+        # depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim
+        # module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.
+        #
+        # When a `module` is specified, the referenced module/volume will be automatically configured as a runtime
+        # dependency of this service, as well as a build dependency of this module.
+        #
+        # Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports
+        # the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple
+        # services at the same time. Refer to the documentation of the module type in question to learn more.
+        module:
+
 # A list of tasks that can be run from this container module. These can be used as dependencies for services (executed
 # before the service is deployed) or for other tasks.
 tasks:
@@ -398,6 +443,36 @@ tasks:
     # Key/value map of environment variables. Keys must be valid POSIX environment variable names (must not start with
     # `GARDEN`) and values must be primitives or references to secrets.
     env: {}
+
+    # List of volumes that should be mounted when deploying the task.
+    #
+    # Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when
+    # deploying the container.
+    volumes:
+      - # The name of the allocated volume.
+        name:
+
+        # The path where the volume should be mounted in the container.
+        containerPath:
+
+        # _NOTE: Usage of hostPath is generally discouraged, since it doesn't work reliably across different platforms
+        # and providers. Some providers may not support it at all._
+        #
+        # A local path or path on the node that's running the container, to mount in the container, relative to the
+        # module source path (or absolute).
+        hostPath:
+
+        # The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will
+        # depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim
+        # module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.
+        #
+        # When a `module` is specified, the referenced module/volume will be automatically configured as a runtime
+        # dependency of this service, as well as a build dependency of this module.
+        #
+        # Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports
+        # the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple
+        # services at the same time. Refer to the documentation of the module type in question to learn more.
+        module:
 ```
 
 ## Configuration Keys
@@ -1191,7 +1266,9 @@ Note: This setting may be overridden or ignored in some cases. For example, when
 
 [services](#services) > volumes
 
-List of volumes that should be mounted when deploying the container.
+List of volumes that should be mounted when deploying the service.
+
+Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when deploying the container.
 
 | Type            | Default | Required |
 | --------------- | ------- | -------- |
@@ -1239,6 +1316,20 @@ services:
       - hostPath: "/some/dir"
 ```
 
+### `services[].volumes[].module`
+
+[services](#services) > [volumes](#servicesvolumes) > module
+
+The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.
+
+When a `module` is specified, the referenced module/volume will be automatically configured as a runtime dependency of this service, as well as a build dependency of this module.
+
+Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple services at the same time. Refer to the documentation of the module type in question to learn more.
+
+| Type     | Required |
+| -------- | -------- |
+| `string` | No       |
+
 ### `tests[]`
 
 A list of tests to run in the module.
@@ -1408,6 +1499,74 @@ tests:
         - {}
 ```
 
+### `tests[].volumes[]`
+
+[tests](#tests) > volumes
+
+List of volumes that should be mounted when deploying the test.
+
+Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when deploying the container.
+
+| Type            | Default | Required |
+| --------------- | ------- | -------- |
+| `array[object]` | `[]`    | No       |
+
+### `tests[].volumes[].name`
+
+[tests](#tests) > [volumes](#testsvolumes) > name
+
+The name of the allocated volume.
+
+| Type     | Required |
+| -------- | -------- |
+| `string` | Yes      |
+
+### `tests[].volumes[].containerPath`
+
+[tests](#tests) > [volumes](#testsvolumes) > containerPath
+
+The path where the volume should be mounted in the container.
+
+| Type        | Required |
+| ----------- | -------- |
+| `posixPath` | Yes      |
+
+### `tests[].volumes[].hostPath`
+
+[tests](#tests) > [volumes](#testsvolumes) > hostPath
+
+_NOTE: Usage of hostPath is generally discouraged, since it doesn't work reliably across different platforms
+and providers. Some providers may not support it at all._
+
+A local path or path on the node that's running the container, to mount in the container, relative to the
+module source path (or absolute).
+
+| Type        | Required |
+| ----------- | -------- |
+| `posixPath` | No       |
+
+Example:
+
+```yaml
+tests:
+  - volumes:
+      - hostPath: "/some/dir"
+```
+
+### `tests[].volumes[].module`
+
+[tests](#tests) > [volumes](#testsvolumes) > module
+
+The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.
+
+When a `module` is specified, the referenced module/volume will be automatically configured as a runtime dependency of this service, as well as a build dependency of this module.
+
+Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple services at the same time. Refer to the documentation of the module type in question to learn more.
+
+| Type     | Required |
+| -------- | -------- |
+| `string` | No       |
+
 ### `tasks[]`
 
 A list of tasks that can be run from this container module. These can be used as dependencies for services (executed before the service is deployed) or for other tasks.
@@ -1598,6 +1757,74 @@ tasks:
         - {}
 ```
 
+### `tasks[].volumes[]`
+
+[tasks](#tasks) > volumes
+
+List of volumes that should be mounted when deploying the task.
+
+Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when deploying the container.
+
+| Type            | Default | Required |
+| --------------- | ------- | -------- |
+| `array[object]` | `[]`    | No       |
+
+### `tasks[].volumes[].name`
+
+[tasks](#tasks) > [volumes](#tasksvolumes) > name
+
+The name of the allocated volume.
+
+| Type     | Required |
+| -------- | -------- |
+| `string` | Yes      |
+
+### `tasks[].volumes[].containerPath`
+
+[tasks](#tasks) > [volumes](#tasksvolumes) > containerPath
+
+The path where the volume should be mounted in the container.
+
+| Type        | Required |
+| ----------- | -------- |
+| `posixPath` | Yes      |
+
+### `tasks[].volumes[].hostPath`
+
+[tasks](#tasks) > [volumes](#tasksvolumes) > hostPath
+
+_NOTE: Usage of hostPath is generally discouraged, since it doesn't work reliably across different platforms
+and providers. Some providers may not support it at all._
+
+A local path or path on the node that's running the container, to mount in the container, relative to the
+module source path (or absolute).
+
+| Type        | Required |
+| ----------- | -------- |
+| `posixPath` | No       |
+
+Example:
+
+```yaml
+tasks:
+  - volumes:
+      - hostPath: "/some/dir"
+```
+
+### `tasks[].volumes[].module`
+
+[tasks](#tasks) > [volumes](#tasksvolumes) > module
+
+The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.
+
+When a `module` is specified, the referenced module/volume will be automatically configured as a runtime dependency of this service, as well as a build dependency of this module.
+
+Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple services at the same time. Refer to the documentation of the module type in question to learn more.
+
+| Type     | Required |
+| -------- | -------- |
+| `string` | No       |
+
 
 ## Outputs