diff --git a/docs/reference/config.md b/docs/reference/config.md index c851240dd6..dd4fee0601 100644 --- a/docs/reference/config.md +++ b/docs/reference/config.md @@ -50,7 +50,7 @@ The default environment to use when calling commands without the `--env` paramet | `string` | No ### `environmentDefaults` -Default environment settings. These are inherited (but can be overridden) by each configured environment. +DEPRECATED - Please use the `providers` field instead, and omit the environments key in the configured provider to use it for all environments, and use the `variables` field to configure variables across all environments. | Type | Required | | ---- | -------- | @@ -65,7 +65,7 @@ environmentDefaults: ### `environmentDefaults.providers[]` [environmentDefaults](#environmentdefaults) > providers -A list of providers that should be used for this environment, and their configuration. Please refer to individual plugins/providers for details on how to configure them. +DEPRECATED - Please use the top-level `providers` field instead, and if needed use the `environments` key on the provider configurations to limit them to specific environments. | Type | Required | | ---- | -------- | @@ -88,10 +88,30 @@ environmentDefaults: providers: - name: "local-kubernetes" ``` +### `environmentDefaults.providers[].environments[]` +[environmentDefaults](#environmentdefaults) > [providers](#environmentdefaults.providers[]) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +environmentDefaults: + providers: [] + variables: {} + ... + providers: + - environments: + - dev + - stage +``` ### `environmentDefaults.variables` [environmentDefaults](#environmentdefaults) > variables -A key/value map of variables that modules can reference when using this environment. +A key/value map of variables that modules can reference when using this environment. These take precedence over variables defined in the top-level `variables` field. | Type | Required | | ---- | -------- | @@ -102,26 +122,21 @@ A list of environments to configure for the project. | Type | Required | | ---- | -------- | -| `array[object]` | No +| `alternatives` | No Example: ```yaml -environments: - - name: local - providers: - - name: local-kubernetes - variables: {} +environments: [{"name":"local","providers":[{"name":"local-kubernetes","environments":[]}],"variables":{}}] ``` -### `environments[].providers[]` -[environments](#environments) > providers +### `providers` -A list of providers that should be used for this environment, and their configuration. Please refer to individual plugins/providers for details on how to configure them. +A list of providers that should be used for this project, and their configuration. Please refer to individual plugins/providers for details on how to configure them. | Type | Required | | ---- | -------- | | `array[object]` | No -### `environments[].providers[].name` -[environments](#environments) > [providers](#environments[].providers[]) > name +### `providers[].name` +[providers](#providers) > name The name of the provider plugin to use. @@ -131,30 +146,25 @@ The name of the provider plugin to use. Example: ```yaml -environments: - - name: local - providers: - - name: local-kubernetes - variables: {} - - providers: - - name: "local-kubernetes" +providers: + - name: "local-kubernetes" ``` -### `environments[].variables` -[environments](#environments) > variables +### `providers[].environments[]` +[providers](#providers) > environments -A key/value map of variables that modules can reference when using this environment. +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. | Type | Required | | ---- | -------- | -| `object` | No -### `environments[].name` -[environments](#environments) > name - -Valid RFC1035/RFC1123 (DNS) label (may contain lowercase letters, numbers and dashes, must start with a letter, and cannot end with a dash), cannot contain consecutive dashes or start with `garden`, or be longer than 63 characters. +| `array[string]` | No -| Type | Required | -| ---- | -------- | -| `string` | Yes +Example: +```yaml +providers: + - environments: + - dev + - stage +``` ### `sources` A list of remote sources to import into project. @@ -184,6 +194,13 @@ Example: sources: - repositoryUrl: "git+https://github.com/org/repo.git#v2.0" ``` +### `variables` + +Variables to configure for all environments. + +| Type | Required | +| ---- | -------- | +| `object` | No ## Project YAML schema @@ -195,15 +212,16 @@ defaultEnvironment: '' environmentDefaults: providers: - name: + environments: variables: {} environments: - - providers: - - name: - variables: {} - name: +providers: + - name: + environments: sources: - name: repositoryUrl: +variables: {} ``` ## Module configuration keys diff --git a/docs/reference/module-types/openfaas.md b/docs/reference/module-types/openfaas.md index 1ce4f6a9d5..f7dfe46ed1 100644 --- a/docs/reference/module-types/openfaas.md +++ b/docs/reference/module-types/openfaas.md @@ -1,7 +1,6 @@ # `openfaas` reference -Deploy [OpenFaaS](https://www.openfaas.com/) functions using Garden. Requires either the `kubernetes` or -`local-kubernetes` provider to be configured. Everything else is installed automatically. + Below is the schema reference. For an introduction to configuring Garden modules, please look at our [Configuration guide](../../using-garden/configuration-files.md). @@ -153,153 +152,6 @@ POSIX-style path or filename to copy the directory or file(s) to (defaults to sa | Type | Required | | ---- | -------- | | `string` | No -### `build.command[]` -[build](#build) > command - -The command to run inside the module's directory to perform the build. - -| Type | Required | -| ---- | -------- | -| `array[string]` | No - -Example: -```yaml -build: - ... - command: - - npm - - run - - build -``` -### `env` - -Key/value map of environment variables. Keys must be valid POSIX environment variable names (must not start with `GARDEN`) and values must be primitives. - -| Type | Required | -| ---- | -------- | -| `object` | No -### `tasks` - -A list of tasks that can be run in this module. - -| Type | Required | -| ---- | -------- | -| `array[object]` | No -### `tasks[].name` -[tasks](#tasks) > name - -The name of the task. - -| Type | Required | -| ---- | -------- | -| `string` | Yes -### `tasks[].description` -[tasks](#tasks) > description - -A description of the task. - -| Type | Required | -| ---- | -------- | -| `string` | No -### `tasks[].dependencies[]` -[tasks](#tasks) > dependencies - -The names of any tasks that must be executed, and the names of any services that must be running, before this task is executed. - -| Type | Required | -| ---- | -------- | -| `array[string]` | No -### `tasks[].timeout` -[tasks](#tasks) > timeout - -Maximum duration (in seconds) of the task's execution. - -| Type | Required | -| ---- | -------- | -| `number` | No -### `tasks[].command[]` -[tasks](#tasks) > command - -The command to run in the module build context. - -| Type | Required | -| ---- | -------- | -| `array[string]` | No -### `tests` - -A list of tests to run in the module. - -| Type | Required | -| ---- | -------- | -| `array[object]` | No -### `tests[].name` -[tests](#tests) > name - -The name of the test. - -| Type | Required | -| ---- | -------- | -| `string` | Yes -### `tests[].dependencies[]` -[tests](#tests) > dependencies - -The names of any services that must be running, and the names of any tasks that must be executed, before the test is run. - -| Type | Required | -| ---- | -------- | -| `array[string]` | No -### `tests[].timeout` -[tests](#tests) > timeout - -Maximum duration (in seconds) of the test run. - -| Type | Required | -| ---- | -------- | -| `number` | No -### `tests[].command[]` -[tests](#tests) > command - -The command to run in the module build context in order to test it. - -| Type | Required | -| ---- | -------- | -| `array[string]` | No -### `tests[].env` -[tests](#tests) > env - -Key/value map of environment variables. Keys must be valid POSIX environment variable names (must not start with `GARDEN`) and values must be primitives. - -| Type | Required | -| ---- | -------- | -| `object` | No -### `dependencies` - -The names of services/functions that this function depends on at runtime. - -| Type | Required | -| ---- | -------- | -| `array[string]` | No -### `handler` - -Specify which directory under the module contains the handler file/function. - -| Type | Required | -| ---- | -------- | -| `string` | No -### `image` - -The image name to use for the built OpenFaaS container (defaults to the module name) - -| Type | Required | -| ---- | -------- | -| `string` | No -### `lang` - -The OpenFaaS language template to use to build this function. - -| Type | Required | -| ---- | -------- | -| `string` | Yes ## Complete YAML schema @@ -318,23 +170,4 @@ build: copy: - source: target: '' - command: - [] -env: {} -tasks: - - name: - description: - dependencies: [] - timeout: null - command: -tests: - - name: - dependencies: [] - timeout: null - command: - env: {} -dependencies: [] -handler: . -image: -lang: ``` \ No newline at end of file diff --git a/docs/reference/providers/kubernetes.md b/docs/reference/providers/kubernetes.md index 635267501b..49791fa053 100644 --- a/docs/reference/providers/kubernetes.md +++ b/docs/reference/providers/kubernetes.md @@ -41,7 +41,7 @@ The default environment to use when calling commands without the `--env` paramet | `string` | No ### `environmentDefaults` -Default environment settings. These are inherited (but can be overridden) by each configured environment. +DEPRECATED - Please use the `providers` field instead, and omit the environments key in the configured provider to use it for all environments, and use the `variables` field to configure variables across all environments. | Type | Required | | ---- | -------- | @@ -56,7 +56,7 @@ environmentDefaults: ### `environmentDefaults.providers[]` [environmentDefaults](#environmentdefaults) > providers -A list of providers that should be used for this environment, and their configuration. Please refer to individual plugins/providers for details on how to configure them. +DEPRECATED - Please use the top-level `providers` field instead, and if needed use the `environments` key on the provider configurations to limit them to specific environments. | Type | Required | | ---- | -------- | @@ -79,14 +79,71 @@ environmentDefaults: providers: - name: "local-kubernetes" ``` +### `environmentDefaults.providers[].environments[]` +[environmentDefaults](#environmentdefaults) > [providers](#environmentdefaults.providers[]) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +environmentDefaults: + providers: [] + variables: {} + ... + providers: + - environments: + - dev + - stage +``` ### `environmentDefaults.variables` [environmentDefaults](#environmentdefaults) > variables -A key/value map of variables that modules can reference when using this environment. +A key/value map of variables that modules can reference when using this environment. These take precedence over variables defined in the top-level `variables` field. | Type | Required | | ---- | -------- | | `object` | No +### `providers` + +A list of providers that should be used for this project, and their configuration. Please refer to individual plugins/providers for details on how to configure them. + +| Type | Required | +| ---- | -------- | +| `array[object]` | No +### `providers[].name` +[providers](#providers) > name + +The name of the provider plugin to use. + +| Type | Required | +| ---- | -------- | +| `string` | Yes + +Example: +```yaml +providers: + - name: "local-kubernetes" +``` +### `providers[].environments[]` +[providers](#providers) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +providers: + - environments: + - dev + - stage +``` ### `sources` A list of remote sources to import into project. @@ -116,6 +173,13 @@ Example: sources: - repositoryUrl: "git+https://github.com/org/repo.git#v2.0" ``` +### `variables` + +Variables to configure for all environments. + +| Type | Required | +| ---- | -------- | +| `object` | No ### `environments` @@ -131,6 +195,23 @@ sources: | Type | Required | | ---- | -------- | | `array[object]` | No +### `environments[].providers[].environments[]` +[environments](#environments) > [providers](#environments[].providers[]) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +environments: + - providers: + - environments: + - dev + - stage +``` ### `environments[].providers[].buildMode` [environments](#environments) > [providers](#environments[].providers[]) > buildMode @@ -600,13 +681,19 @@ defaultEnvironment: '' environmentDefaults: providers: - name: + environments: variables: {} +providers: + - name: + environments: sources: - name: repositoryUrl: +variables: {} environments: - providers: - - buildMode: local + - environments: + buildMode: local defaultHostname: defaultUsername: forceSsl: false diff --git a/docs/reference/providers/local-kubernetes.md b/docs/reference/providers/local-kubernetes.md index 4f4125dbc5..5c1fb97caf 100644 --- a/docs/reference/providers/local-kubernetes.md +++ b/docs/reference/providers/local-kubernetes.md @@ -41,7 +41,7 @@ The default environment to use when calling commands without the `--env` paramet | `string` | No ### `environmentDefaults` -Default environment settings. These are inherited (but can be overridden) by each configured environment. +DEPRECATED - Please use the `providers` field instead, and omit the environments key in the configured provider to use it for all environments, and use the `variables` field to configure variables across all environments. | Type | Required | | ---- | -------- | @@ -56,7 +56,7 @@ environmentDefaults: ### `environmentDefaults.providers[]` [environmentDefaults](#environmentdefaults) > providers -A list of providers that should be used for this environment, and their configuration. Please refer to individual plugins/providers for details on how to configure them. +DEPRECATED - Please use the top-level `providers` field instead, and if needed use the `environments` key on the provider configurations to limit them to specific environments. | Type | Required | | ---- | -------- | @@ -79,14 +79,71 @@ environmentDefaults: providers: - name: "local-kubernetes" ``` +### `environmentDefaults.providers[].environments[]` +[environmentDefaults](#environmentdefaults) > [providers](#environmentdefaults.providers[]) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +environmentDefaults: + providers: [] + variables: {} + ... + providers: + - environments: + - dev + - stage +``` ### `environmentDefaults.variables` [environmentDefaults](#environmentdefaults) > variables -A key/value map of variables that modules can reference when using this environment. +A key/value map of variables that modules can reference when using this environment. These take precedence over variables defined in the top-level `variables` field. | Type | Required | | ---- | -------- | | `object` | No +### `providers` + +A list of providers that should be used for this project, and their configuration. Please refer to individual plugins/providers for details on how to configure them. + +| Type | Required | +| ---- | -------- | +| `array[object]` | No +### `providers[].name` +[providers](#providers) > name + +The name of the provider plugin to use. + +| Type | Required | +| ---- | -------- | +| `string` | Yes + +Example: +```yaml +providers: + - name: "local-kubernetes" +``` +### `providers[].environments[]` +[providers](#providers) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +providers: + - environments: + - dev + - stage +``` ### `sources` A list of remote sources to import into project. @@ -116,6 +173,13 @@ Example: sources: - repositoryUrl: "git+https://github.com/org/repo.git#v2.0" ``` +### `variables` + +Variables to configure for all environments. + +| Type | Required | +| ---- | -------- | +| `object` | No ### `environments` @@ -131,6 +195,23 @@ sources: | Type | Required | | ---- | -------- | | `array[object]` | No +### `environments[].providers[].environments[]` +[environments](#environments) > [providers](#environments[].providers[]) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +environments: + - providers: + - environments: + - dev + - stage +``` ### `environments[].providers[].buildMode` [environments](#environments) > [providers](#environments[].providers[]) > buildMode @@ -525,13 +606,19 @@ defaultEnvironment: '' environmentDefaults: providers: - name: + environments: variables: {} +providers: + - name: + environments: sources: - name: repositoryUrl: +variables: {} environments: - providers: - - buildMode: local + - environments: + buildMode: local defaultHostname: defaultUsername: forceSsl: false diff --git a/docs/reference/providers/local-openfaas.md b/docs/reference/providers/local-openfaas.md new file mode 100644 index 0000000000..c4ddd0f255 --- /dev/null +++ b/docs/reference/providers/local-openfaas.md @@ -0,0 +1,274 @@ +# `local-openfaas` reference + +Below is the schema reference for the `local-openfaas` provider. For an introduction to configuring a Garden project with providers, please look at our [configuration guide](../../using-garden/configuration-files.md). + +The reference is divided into two sections. The [first section](#configuration-keys) lists and describes the available schema keys. The [second section](#complete-yaml-schema) contains the complete YAML schema. + +## Configuration keys + +### `apiVersion` + +The schema version of this project's config (currently not used). + +| Type | Required | Allowed Values | +| ---- | -------- | -------------- | +| `string` | Yes | "garden.io/v0" +### `kind` + + + +| Type | Required | Allowed Values | +| ---- | -------- | -------------- | +| `string` | Yes | "Project" +### `name` + +The name of the project. + +| Type | Required | +| ---- | -------- | +| `string` | Yes + +Example: +```yaml +name: "my-sweet-project" +``` +### `defaultEnvironment` + +The default environment to use when calling commands without the `--env` parameter. + +| Type | Required | +| ---- | -------- | +| `string` | No +### `environmentDefaults` + +DEPRECATED - Please use the `providers` field instead, and omit the environments key in the configured provider to use it for all environments, and use the `variables` field to configure variables across all environments. + +| Type | Required | +| ---- | -------- | +| `object` | No + +Example: +```yaml +environmentDefaults: + providers: [] + variables: {} +``` +### `environmentDefaults.providers[]` +[environmentDefaults](#environmentdefaults) > providers + +DEPRECATED - Please use the top-level `providers` field instead, and if needed use the `environments` key on the provider configurations to limit them to specific environments. + +| Type | Required | +| ---- | -------- | +| `array[object]` | No +### `environmentDefaults.providers[].name` +[environmentDefaults](#environmentdefaults) > [providers](#environmentdefaults.providers[]) > name + +The name of the provider plugin to use. + +| Type | Required | +| ---- | -------- | +| `string` | Yes + +Example: +```yaml +environmentDefaults: + providers: [] + variables: {} + ... + providers: + - name: "local-kubernetes" +``` +### `environmentDefaults.providers[].environments[]` +[environmentDefaults](#environmentdefaults) > [providers](#environmentdefaults.providers[]) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +environmentDefaults: + providers: [] + variables: {} + ... + providers: + - environments: + - dev + - stage +``` +### `environmentDefaults.variables` +[environmentDefaults](#environmentdefaults) > variables + +A key/value map of variables that modules can reference when using this environment. These take precedence over variables defined in the top-level `variables` field. + +| Type | Required | +| ---- | -------- | +| `object` | No +### `providers` + +A list of providers that should be used for this project, and their configuration. Please refer to individual plugins/providers for details on how to configure them. + +| Type | Required | +| ---- | -------- | +| `array[object]` | No +### `providers[].name` +[providers](#providers) > name + +The name of the provider plugin to use. + +| Type | Required | +| ---- | -------- | +| `string` | Yes + +Example: +```yaml +providers: + - name: "local-kubernetes" +``` +### `providers[].environments[]` +[providers](#providers) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +providers: + - environments: + - dev + - stage +``` +### `sources` + +A list of remote sources to import into project. + +| Type | Required | +| ---- | -------- | +| `array[object]` | No +### `sources[].name` +[sources](#sources) > name + +The name of the source to import + +| Type | Required | +| ---- | -------- | +| `string` | Yes +### `sources[].repositoryUrl` +[sources](#sources) > repositoryUrl + +A remote repository URL. Currently only supports git servers. Must contain a hash suffix pointing to a specific branch or tag, with the format: # + +| Type | Required | +| ---- | -------- | +| `string` | Yes + +Example: +```yaml +sources: + - repositoryUrl: "git+https://github.com/org/repo.git#v2.0" +``` +### `variables` + +Variables to configure for all environments. + +| Type | Required | +| ---- | -------- | +| `object` | No +### `environments` + + + +| Type | Required | +| ---- | -------- | +| `array[object]` | No +### `environments[].providers[]` +[environments](#environments) > providers + + + +| Type | Required | +| ---- | -------- | +| `array[object]` | No +### `environments[].providers[].environments[]` +[environments](#environments) > [providers](#environments[].providers[]) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +environments: + - providers: + - environments: + - dev + - stage +``` +### `environments[].providers[].name` +[environments](#environments) > [providers](#environments[].providers[]) > name + +The name of the provider plugin to use. + +| Type | Required | +| ---- | -------- | +| `string` | Yes + +Example: +```yaml +environments: + - providers: + - name: "openfaas" +``` +### `environments[].providers[].hostname` +[environments](#environments) > [providers](#environments[].providers[]) > hostname + +The hostname to configure for the function gateway. +Defaults to the default hostname of the configured Kubernetes provider. + +Important: If you have other types of services, this should be different from their ingress hostnames, +or the other services should not expose paths under /function and /system to avoid routing conflicts. + +| Type | Required | +| ---- | -------- | +| `string` | No + +Example: +```yaml +environments: + - providers: + - hostname: "functions.mydomain.com" +``` + + +## Complete YAML schema +```yaml +apiVersion: garden.io/v0 +kind: Project +name: +defaultEnvironment: '' +environmentDefaults: + providers: + - name: + environments: + variables: {} +providers: + - name: + environments: +sources: + - name: + repositoryUrl: +variables: {} +environments: + - providers: + - environments: + name: openfaas + hostname: +``` diff --git a/docs/reference/providers/maven-container.md b/docs/reference/providers/maven-container.md index ed037b254a..2c6fac58d8 100644 --- a/docs/reference/providers/maven-container.md +++ b/docs/reference/providers/maven-container.md @@ -41,7 +41,7 @@ The default environment to use when calling commands without the `--env` paramet | `string` | No ### `environmentDefaults` -Default environment settings. These are inherited (but can be overridden) by each configured environment. +DEPRECATED - Please use the `providers` field instead, and omit the environments key in the configured provider to use it for all environments, and use the `variables` field to configure variables across all environments. | Type | Required | | ---- | -------- | @@ -56,7 +56,7 @@ environmentDefaults: ### `environmentDefaults.providers[]` [environmentDefaults](#environmentdefaults) > providers -A list of providers that should be used for this environment, and their configuration. Please refer to individual plugins/providers for details on how to configure them. +DEPRECATED - Please use the top-level `providers` field instead, and if needed use the `environments` key on the provider configurations to limit them to specific environments. | Type | Required | | ---- | -------- | @@ -79,14 +79,71 @@ environmentDefaults: providers: - name: "local-kubernetes" ``` +### `environmentDefaults.providers[].environments[]` +[environmentDefaults](#environmentdefaults) > [providers](#environmentdefaults.providers[]) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +environmentDefaults: + providers: [] + variables: {} + ... + providers: + - environments: + - dev + - stage +``` ### `environmentDefaults.variables` [environmentDefaults](#environmentdefaults) > variables -A key/value map of variables that modules can reference when using this environment. +A key/value map of variables that modules can reference when using this environment. These take precedence over variables defined in the top-level `variables` field. | Type | Required | | ---- | -------- | | `object` | No +### `providers` + +A list of providers that should be used for this project, and their configuration. Please refer to individual plugins/providers for details on how to configure them. + +| Type | Required | +| ---- | -------- | +| `array[object]` | No +### `providers[].name` +[providers](#providers) > name + +The name of the provider plugin to use. + +| Type | Required | +| ---- | -------- | +| `string` | Yes + +Example: +```yaml +providers: + - name: "local-kubernetes" +``` +### `providers[].environments[]` +[providers](#providers) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +providers: + - environments: + - dev + - stage +``` ### `sources` A list of remote sources to import into project. @@ -116,6 +173,13 @@ Example: sources: - repositoryUrl: "git+https://github.com/org/repo.git#v2.0" ``` +### `variables` + +Variables to configure for all environments. + +| Type | Required | +| ---- | -------- | +| `object` | No ### `environments` @@ -131,6 +195,23 @@ sources: | Type | Required | | ---- | -------- | | `array[object]` | No +### `environments[].providers[].environments[]` +[environments](#environments) > [providers](#environments[].providers[]) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +environments: + - providers: + - environments: + - dev + - stage +``` ### `environments[].providers[].name` [environments](#environments) > [providers](#environments[].providers[]) > name @@ -157,11 +238,17 @@ defaultEnvironment: '' environmentDefaults: providers: - name: + environments: variables: {} +providers: + - name: + environments: sources: - name: repositoryUrl: +variables: {} environments: - providers: - - name: maven-container + - environments: + name: maven-container ``` diff --git a/docs/reference/providers/openfaas.md b/docs/reference/providers/openfaas.md index e6152e3a9f..30d59d455c 100644 --- a/docs/reference/providers/openfaas.md +++ b/docs/reference/providers/openfaas.md @@ -41,7 +41,7 @@ The default environment to use when calling commands without the `--env` paramet | `string` | No ### `environmentDefaults` -Default environment settings. These are inherited (but can be overridden) by each configured environment. +DEPRECATED - Please use the `providers` field instead, and omit the environments key in the configured provider to use it for all environments, and use the `variables` field to configure variables across all environments. | Type | Required | | ---- | -------- | @@ -56,7 +56,7 @@ environmentDefaults: ### `environmentDefaults.providers[]` [environmentDefaults](#environmentdefaults) > providers -A list of providers that should be used for this environment, and their configuration. Please refer to individual plugins/providers for details on how to configure them. +DEPRECATED - Please use the top-level `providers` field instead, and if needed use the `environments` key on the provider configurations to limit them to specific environments. | Type | Required | | ---- | -------- | @@ -79,14 +79,71 @@ environmentDefaults: providers: - name: "local-kubernetes" ``` +### `environmentDefaults.providers[].environments[]` +[environmentDefaults](#environmentdefaults) > [providers](#environmentdefaults.providers[]) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +environmentDefaults: + providers: [] + variables: {} + ... + providers: + - environments: + - dev + - stage +``` ### `environmentDefaults.variables` [environmentDefaults](#environmentdefaults) > variables -A key/value map of variables that modules can reference when using this environment. +A key/value map of variables that modules can reference when using this environment. These take precedence over variables defined in the top-level `variables` field. | Type | Required | | ---- | -------- | | `object` | No +### `providers` + +A list of providers that should be used for this project, and their configuration. Please refer to individual plugins/providers for details on how to configure them. + +| Type | Required | +| ---- | -------- | +| `array[object]` | No +### `providers[].name` +[providers](#providers) > name + +The name of the provider plugin to use. + +| Type | Required | +| ---- | -------- | +| `string` | Yes + +Example: +```yaml +providers: + - name: "local-kubernetes" +``` +### `providers[].environments[]` +[providers](#providers) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +providers: + - environments: + - dev + - stage +``` ### `sources` A list of remote sources to import into project. @@ -116,6 +173,13 @@ Example: sources: - repositoryUrl: "git+https://github.com/org/repo.git#v2.0" ``` +### `variables` + +Variables to configure for all environments. + +| Type | Required | +| ---- | -------- | +| `object` | No ### `environments` @@ -131,6 +195,23 @@ sources: | Type | Required | | ---- | -------- | | `array[object]` | No +### `environments[].providers[].environments[]` +[environments](#environments) > [providers](#environments[].providers[]) > environments + +If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field. + +| Type | Required | +| ---- | -------- | +| `array[string]` | No + +Example: +```yaml +environments: + - providers: + - environments: + - dev + - stage +``` ### `environments[].providers[].name` [environments](#environments) > [providers](#environments[].providers[]) > name @@ -176,12 +257,18 @@ defaultEnvironment: '' environmentDefaults: providers: - name: + environments: variables: {} +providers: + - name: + environments: sources: - name: repositoryUrl: +variables: {} environments: - providers: - - name: openfaas + - environments: + name: openfaas hostname: ``` diff --git a/docs/reference/template-strings.md b/docs/reference/template-strings.md index b64bca4410..f1a2898a40 100644 --- a/docs/reference/template-strings.md +++ b/docs/reference/template-strings.md @@ -107,27 +107,27 @@ environment: # name: -# Retrieve information about modules that are defined in the project. +# Retrieve information about providers that are defined in the project. # # Type: object # # Example: -# my-module: -# path: /home/me/code/my-project/my-module -# version: v-v17ad4cb3fd +# kubernetes: +# config: +# clusterHostname: my-cluster.example.com # -modules: {} +providers: {} -# A map of all configured plugins/providers for this environment and their configuration. +# Retrieve information about modules that are defined in the project. # # Type: object # # Example: -# kubernetes: -# name: local-kubernetes -# context: my-kube-context +# my-module: +# path: /home/me/code/my-project/my-module +# version: v-17ad4cb3fd # -providers: {} +modules: {} # A map of all variables defined in the project configuration. # diff --git a/examples/hello-world/garden.yml b/examples/hello-world/garden.yml index e090b1db79..d3018a965a 100644 --- a/examples/hello-world/garden.yml +++ b/examples/hello-world/garden.yml @@ -9,5 +9,4 @@ project: - name: local providers: - name: local-kubernetes - - name: openfaas - + - name: local-openfaas diff --git a/garden-service/src/actions.ts b/garden-service/src/actions.ts index 5c6646dd29..38d02cb0c2 100644 --- a/garden-service/src/actions.ts +++ b/garden-service/src/actions.ts @@ -15,11 +15,10 @@ import { fromPairs, keyBy, mapValues, omit, pickBy, values } from "lodash" import { PublishModuleParams, PublishResult } from "./types/plugin/module/publishModule" import { SetSecretParams, SetSecretResult } from "./types/plugin/provider/setSecret" import { validate } from "./config/common" -import { defaultProvider } from "./config/project" +import { defaultProvider, Provider } from "./config/provider" import { ConfigurationError, ParameterError, PluginError } from "./exceptions" import { ActionHandlerMap, Garden, ModuleActionHandlerMap, ModuleActionMap, PluginActionMap } from "./garden" import { LogEntry } from "./logger/log-entry" -import { createPluginContext } from "./plugin-context" import { ProcessResults, processServices } from "./process" import { getDependantTasksForModule } from "./tasks/helpers" import { Module } from "./types/module" @@ -110,9 +109,31 @@ export class ActionHelper implements TypeGuard { private readonly actionHandlers: PluginActionMap private readonly moduleActionHandlers: ModuleActionMap - constructor(private garden: Garden) { + constructor( + private garden: Garden, + providers: Provider[], + ) { this.actionHandlers = fromPairs(pluginActionNames.map(n => [n, {}])) this.moduleActionHandlers = fromPairs(moduleActionNames.map(n => [n, {}])) + + for (const provider of providers) { + const plugin = garden.getPlugin(provider.name) + const actions = plugin.actions || {} + + for (const actionType of pluginActionNames) { + const handler = actions[actionType] + handler && this.addActionHandler(provider.name, actionType, handler) + } + + const moduleActions = plugin.moduleActions || {} + + for (const moduleType of Object.keys(moduleActions)) { + for (const actionType of moduleActionNames) { + const handler = moduleActions[moduleType][actionType] + handler && this.addModuleActionHandler(provider.name, actionType, moduleType, handler) + } + } + } } //=========================================================================== @@ -122,13 +143,13 @@ export class ActionHelper implements TypeGuard { async getEnvironmentStatus( { pluginName, log }: ActionHelperParams, ): Promise { - const handlers = this.getActionHandlers("getEnvironmentStatus", pluginName) + const handlers = this.getActionHelpers("getEnvironmentStatus", pluginName) const logEntry = log.debug({ msg: "Getting status...", status: "active", - section: `${this.garden.environment.name} environment`, + section: `${this.garden.environmentName} environment`, }) - const res = await Bluebird.props(mapValues(handlers, h => h({ ...this.commonParams(h, logEntry) }))) + const res = await Bluebird.props(mapValues(handlers, async (h) => h({ ...await this.commonParams(h, logEntry) }))) logEntry.setSuccess("Ready") return res } @@ -143,7 +164,7 @@ export class ActionHelper implements TypeGuard { { force = false, pluginName, log, allowUserInput = false }: { force?: boolean, pluginName?: string, log: LogEntry, allowUserInput?: boolean }, ) { - const handlers = this.getActionHandlers("prepareEnvironment", pluginName) + const handlers = this.getActionHelpers("prepareEnvironment", pluginName) // FIXME: We're calling getEnvironmentStatus before preparing the environment. // Results in 404 errors for unprepared/missing services. // See: https://github.com/garden-io/garden/issues/353 @@ -195,7 +216,7 @@ export class ActionHelper implements TypeGuard { }) await handler({ - ...this.commonParams(handler, log), + ...await this.commonParams(handler, log), force: forcePrep, status, log: envLogEntry, @@ -214,8 +235,8 @@ export class ActionHelper implements TypeGuard { async cleanupEnvironment( { pluginName, log }: ActionHelperParams, ): Promise { - const handlers = this.getActionHandlers("cleanupEnvironment", pluginName) - await Bluebird.each(values(handlers), h => h({ ...this.commonParams(h, log) })) + const handlers = this.getActionHelpers("cleanupEnvironment", pluginName) + await Bluebird.each(values(handlers), async (h) => h({ ...await this.commonParams(h, log) })) return this.getEnvironmentStatus({ pluginName, log }) } @@ -415,16 +436,16 @@ export class ActionHelper implements TypeGuard { } async getDebugInfo({ log }: { log: LogEntry }): Promise { - const handlers = this.getActionHandlers("getDebugInfo") - return await Bluebird.props(mapValues(handlers, h => h({ ...this.commonParams(h, log) }))) + const handlers = this.getActionHelpers("getDebugInfo") + return Bluebird.props(mapValues(handlers, async (h) => h({ ...await this.commonParams(h, log) }))) } //endregion // TODO: find a nicer way to do this (like a type-safe wrapper function) - private commonParams(handler, log: LogEntry): PluginActionParamsBase { + private async commonParams(handler, log: LogEntry): Promise { return { - ctx: createPluginContext(this.garden, handler["pluginName"]), + ctx: await this.garden.getPluginContext(handler["pluginName"]), // TODO: find a better way for handlers to log during execution log, } @@ -439,13 +460,13 @@ export class ActionHelper implements TypeGuard { defaultHandler?: PluginActions[T], }, ): Promise { - const handler = this.getActionHandler({ + const handler = this.getActionHelper({ actionType, pluginName, defaultHandler, }) const handlerParams: PluginActionParams[T] = { - ...this.commonParams(handler, (params).log), + ...await this.commonParams(handler, (params).log), ...params, } return (handler)(handlerParams) @@ -468,7 +489,7 @@ export class ActionHelper implements TypeGuard { }) const handlerParams: any = { - ...this.commonParams(handler, (params).log), + ...await this.commonParams(handler, (params).log), ...params, module: omit(module, ["_ConfigType"]), } @@ -496,7 +517,7 @@ export class ActionHelper implements TypeGuard { }) const handlerParams: any = { - ...this.commonParams(handler, log), + ...await this.commonParams(handler, log), ...params, module, runtimeContext, @@ -528,7 +549,7 @@ export class ActionHelper implements TypeGuard { }) const handlerParams: any = { - ...this.commonParams(handler, (params).log), + ...await this.commonParams(handler, (params).log), ...params, module, task, @@ -539,7 +560,7 @@ export class ActionHelper implements TypeGuard { return (handler)(handlerParams) } - public addActionHandler( + private addActionHandler( pluginName: string, actionType: T, handler: PluginActions[T], ) { const plugin = this.garden.getPlugin(pluginName) @@ -555,7 +576,7 @@ export class ActionHelper implements TypeGuard { this.actionHandlers[actionType][pluginName] = wrapped } - public addModuleActionHandler( + private addModuleActionHandler( pluginName: string, actionType: T, moduleType: string, handler: ModuleActions[T], ) { const plugin = this.garden.getPlugin(pluginName) @@ -583,7 +604,7 @@ export class ActionHelper implements TypeGuard { /** * Get a handler for the specified action. */ - public getActionHandlers(actionType: T, pluginName?: string): ActionHandlerMap { + public getActionHelpers(actionType: T, pluginName?: string): ActionHandlerMap { return this.filterActionHandlers(this.actionHandlers[actionType], pluginName) } @@ -613,12 +634,12 @@ export class ActionHelper implements TypeGuard { /** * Get the last configured handler for the specified action (and optionally module type). */ - public getActionHandler( + public getActionHelper( { actionType, pluginName, defaultHandler }: { actionType: T, pluginName?: string, defaultHandler?: PluginActions[T] }, ): PluginActions[T] { - const handlers = Object.values(this.getActionHandlers(actionType, pluginName)) + const handlers = Object.values(this.getActionHelpers(actionType, pluginName)) if (handlers.length) { return handlers[handlers.length - 1] @@ -629,7 +650,7 @@ export class ActionHelper implements TypeGuard { const errorDetails = { requestedHandlerType: actionType, - environment: this.garden.environment.name, + environment: this.garden.environmentName, pluginName, } @@ -637,7 +658,7 @@ export class ActionHelper implements TypeGuard { throw new PluginError(`Plugin '${pluginName}' does not have a '${actionType}' handler.`, errorDetails) } else { throw new ParameterError( - `No '${actionType}' handler configured in environment '${this.garden.environment.name}'. ` + + `No '${actionType}' handler configured in environment '${this.garden.environmentName}'. ` + `Are you missing a provider configuration?`, errorDetails, ) @@ -664,7 +685,7 @@ export class ActionHelper implements TypeGuard { const errorDetails = { requestedHandlerType: actionType, requestedModuleType: moduleType, - environment: this.garden.environment.name, + environment: this.garden.environmentName, pluginName, } @@ -676,7 +697,7 @@ export class ActionHelper implements TypeGuard { } else { throw new ParameterError( `No '${actionType}' handler configured for module type '${moduleType}' in environment ` + - `'${this.garden.environment.name}'. Are you missing a provider configuration?`, + `'${this.garden.environmentName}'. Are you missing a provider configuration?`, errorDetails, ) } diff --git a/garden-service/src/cli/cli.ts b/garden-service/src/cli/cli.ts index 003dd0d09a..360ef1963d 100644 --- a/garden-service/src/cli/cli.ts +++ b/garden-service/src/cli/cli.ts @@ -47,9 +47,8 @@ import { getLogLevelChoices, parseLogLevel, } from "./helpers" -import { GardenConfig } from "../config/base" -import { defaultEnvironments } from "../config/project" -import { ERROR_LOG_FILENAME } from "../constants" +import { defaultEnvironments, ProjectConfig } from "../config/project" +import { ERROR_LOG_FILENAME, DEFAULT_API_VERSION } from "../constants" import stringify = require("json-stringify-safe") import { generateBasicDebugInfoReport } from "../commands/get/get-debug-info" @@ -78,23 +77,19 @@ const GLOBAL_OPTIONS_GROUP_NAME = "Global options" const DEFAULT_CLI_LOGGER_TYPE = "fancy" // For initializing garden without a project config -export const MOCK_CONFIG: GardenConfig = { - dirname: "/", +export const MOCK_CONFIG: ProjectConfig = { path: process.cwd(), - project: { - apiVersion: "garden.io/v0", - name: "mock-project", - defaultEnvironment: "local", - environments: defaultEnvironments, - environmentDefaults: { - providers: [ - { - name: "local-kubernetes", - }, - ], - variables: {}, + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "mock-project", + defaultEnvironment: "local", + environments: defaultEnvironments, + providers: [ + { + name: "local-kubernetes", }, - }, + ], + variables: {}, } export const GLOBAL_OPTIONS = { diff --git a/garden-service/src/commands/call.ts b/garden-service/src/commands/call.ts index f5bd2a704b..a29c13cb21 100644 --- a/garden-service/src/commands/call.ts +++ b/garden-service/src/commands/call.ts @@ -56,7 +56,8 @@ export class CallCommand extends Command { const graph = await garden.getConfigGraph() const service = await graph.getService(serviceName) const runtimeContext = await getServiceRuntimeContext(garden, graph, service) - const status = await garden.actions.getServiceStatus({ service, log, hotReload: false, runtimeContext }) + const actions = await garden.getActionHelper() + const status = await actions.getServiceStatus({ service, log, hotReload: false, runtimeContext }) if (!includes(["ready", "outdated"], status.state)) { throw new RuntimeError(`Service ${service.name} is not running`, { diff --git a/garden-service/src/commands/delete.ts b/garden-service/src/commands/delete.ts index 4184b5e5fd..fc4da95857 100644 --- a/garden-service/src/commands/delete.ts +++ b/garden-service/src/commands/delete.ts @@ -67,7 +67,8 @@ export class DeleteSecretCommand extends Command { { garden, log, args }: CommandParams, ): Promise> { const key = args.key! - const result = await garden.actions.deleteSecret({ log, pluginName: args.provider!, key }) + const actions = await garden.getActionHelper() + const result = await actions.deleteSecret({ log, pluginName: args.provider!, key }) if (result.found) { log.info(`Deleted config key ${args.key}`) @@ -93,10 +94,10 @@ export class DeleteEnvironmentCommand extends Command { ` async action({ garden, log }: CommandParams): Promise> { - const { name } = garden.environment - logHeader({ log, emoji: "skull_and_crossbones", command: `Deleting ${name} environment` }) + logHeader({ log, emoji: "skull_and_crossbones", command: `Deleting ${garden.environmentName} environment` }) - const result = await garden.actions.cleanupEnvironment({ log }) + const actions = await garden.getActionHelper() + const result = await actions.cleanupEnvironment({ log }) return { result } } @@ -139,9 +140,11 @@ export class DeleteServiceCommand extends Command { const result: { [key: string]: ServiceStatus } = {} + const actions = await garden.getActionHelper() + await Bluebird.map(services, async service => { const runtimeContext = await getServiceRuntimeContext(garden, graph, service) - result[service.name] = await garden.actions.deleteService({ log, service, runtimeContext }) + result[service.name] = await actions.deleteService({ log, service, runtimeContext }) }) return { result } diff --git a/garden-service/src/commands/deploy.ts b/garden-service/src/commands/deploy.ts index e78525632e..c3fe9470e1 100644 --- a/garden-service/src/commands/deploy.ts +++ b/garden-service/src/commands/deploy.ts @@ -121,7 +121,8 @@ export class DeployCommand extends Command { } // TODO: make this a task - await garden.actions.prepareEnvironment({ log }) + const actions = await garden.getActionHelper() + await actions.prepareEnvironment({ log }) const results = await processServices({ garden, diff --git a/garden-service/src/commands/dev.ts b/garden-service/src/commands/dev.ts index 19ff51175f..78bd01ed1f 100644 --- a/garden-service/src/commands/dev.ts +++ b/garden-service/src/commands/dev.ts @@ -90,6 +90,9 @@ export class DevCommand extends Command { async action({ garden, log, logFooter, opts }: CommandParams): Promise { this.server.setGarden(garden) + const actions = await garden.getActionHelper() + await actions.prepareEnvironment({ log }) + const graph = await garden.getConfigGraph() const modules = await graph.getModules() @@ -109,7 +112,7 @@ export class DevCommand extends Command { } } - await garden.actions.prepareEnvironment({ log }) + await actions.prepareEnvironment({ log }) const tasksForModule = (watch: boolean) => { return async (updatedGraph: ConfigGraph, module: Module) => { diff --git a/garden-service/src/commands/exec.ts b/garden-service/src/commands/exec.ts index 2630cdb732..e1f72dc6a8 100644 --- a/garden-service/src/commands/exec.ts +++ b/garden-service/src/commands/exec.ts @@ -76,7 +76,8 @@ export class ExecCommand extends Command { const graph = await garden.getConfigGraph() const service = await graph.getService(serviceName) const runtimeContext = await getServiceRuntimeContext(garden, graph, service) - const result = await garden.actions.execInService({ + const actions = await garden.getActionHelper() + const result = await actions.execInService({ log, service, command, diff --git a/garden-service/src/commands/get/get-debug-info.ts b/garden-service/src/commands/get/get-debug-info.ts index fafad78895..e2e4cc8461 100644 --- a/garden-service/src/commands/get/get-debug-info.ts +++ b/garden-service/src/commands/get/get-debug-info.ts @@ -127,7 +127,8 @@ export async function collectProviderDebugInfo(garden: Garden, log: LogEntry, fo const tempPath = join(garden.projectRoot, GARDEN_DIR_NAME, TEMP_DEBUG_ROOT) await ensureDir(tempPath) // Collect debug info from providers - const providersDebugInfo = await garden.actions.getDebugInfo({ log }) + const actions = await garden.getActionHelper() + const providersDebugInfo = await actions.getDebugInfo({ log }) // Create a provider folder and report for each provider. for (const [providerName, info] of Object.entries(providersDebugInfo)) { diff --git a/garden-service/src/commands/get/get-secret.ts b/garden-service/src/commands/get/get-secret.ts index f13ec0d18e..64a4dcf07f 100644 --- a/garden-service/src/commands/get/get-secret.ts +++ b/garden-service/src/commands/get/get-secret.ts @@ -47,7 +47,8 @@ export class GetSecretCommand extends Command { async action({ garden, log, args }: CommandParams): Promise { const key = args.key - const { value } = await garden.actions.getSecret({ + const actions = await garden.getActionHelper() + const { value } = await actions.getSecret({ pluginName: args.provider, key, log, diff --git a/garden-service/src/commands/get/get-status.ts b/garden-service/src/commands/get/get-status.ts index 3d89fea4f9..277f95ffbe 100644 --- a/garden-service/src/commands/get/get-status.ts +++ b/garden-service/src/commands/get/get-status.ts @@ -40,9 +40,11 @@ export class GetStatusCommand extends Command { help = "Outputs the status of your environment." async action({ garden, log, opts }: CommandParams): Promise> { - const status = await garden.actions.getStatus({ log }) + const actions = await garden.getActionHelper() + const status = await actions.getStatus({ log }) + + let result: AllEnvironmentStatus - let result if (opts.output) { const graph = await garden.getConfigGraph() result = await Bluebird.props({ @@ -66,10 +68,12 @@ export class GetStatusCommand extends Command { async function getTestStatuses(garden: Garden, configGraph: ConfigGraph, log: LogEntry) { const modules = await configGraph.getModules() + const actions = await garden.getActionHelper() + return fromPairs(flatten(await Bluebird.map(modules, async (module) => { return Bluebird.map(module.testConfigs, async (testConfig) => { const testVersion = await getTestVersion(garden, configGraph, module, testConfig) - const result = await garden.actions.getTestResult({ + const result = await actions.getTestResult({ module, log, testVersion, testName: testConfig.name, }) return [`${module.name}.${testConfig.name}`, runStatus(result)] @@ -79,9 +83,11 @@ async function getTestStatuses(garden: Garden, configGraph: ConfigGraph, log: Lo async function getTaskStatuses(garden: Garden, configGraph: ConfigGraph, log: LogEntry): Promise { const tasks = await configGraph.getTasks() + const actions = await garden.getActionHelper() + return fromPairs(await Bluebird.map(tasks, async (task) => { const taskVersion = await getTaskVersion(garden, configGraph, task) - const result = await garden.actions.getTaskResult({ task, taskVersion, log }) + const result = await actions.getTaskResult({ task, taskVersion, log }) return [task.name, runStatus(result)] })) } diff --git a/garden-service/src/commands/get/get-task-result.ts b/garden-service/src/commands/get/get-task-result.ts index 984c27d588..8062033c6b 100644 --- a/garden-service/src/commands/get/get-task-result.ts +++ b/garden-service/src/commands/get/get-task-result.ts @@ -48,9 +48,13 @@ export class GetTaskResultCommand extends Command { args, }: CommandParams): Promise> { const taskName = args.name + const graph: ConfigGraph = await garden.getConfigGraph() const task = await graph.getTask(taskName) - const taskResult: RunTaskResult | null = await garden.actions.getTaskResult( + + const actions = await garden.getActionHelper() + + const taskResult: RunTaskResult | null = await actions.getTaskResult( { log, task, diff --git a/garden-service/src/commands/get/get-test-result.ts b/garden-service/src/commands/get/get-test-result.ts index dfeadd99a9..3b6da624a1 100644 --- a/garden-service/src/commands/get/get-test-result.ts +++ b/garden-service/src/commands/get/get-test-result.ts @@ -64,6 +64,8 @@ export class GetTestResultCommand extends Command { }) const graph = await garden.getConfigGraph() + const actions = await garden.getActionHelper() + const module = await graph.getModule(moduleName) const testConfig = findByName(module.testConfigs, testName) @@ -81,7 +83,7 @@ export class GetTestResultCommand extends Command { const testVersion = await getTestVersion(garden, graph, module, testConfig) - const testResult: TestResult | null = await garden.actions.getTestResult({ + const testResult: TestResult | null = await actions.getTestResult({ log, testName, module, diff --git a/garden-service/src/commands/init.ts b/garden-service/src/commands/init.ts index fcefd1c5ec..1ae60ce294 100644 --- a/garden-service/src/commands/init.ts +++ b/garden-service/src/commands/init.ts @@ -41,10 +41,11 @@ export class InitCommand extends Command { options = initOpts async action({ garden, log, opts }: CommandParams<{}, Opts>): Promise> { - const { name } = garden.environment + const name = garden.environmentName logHeader({ log, emoji: "gear", command: `Initializing ${name} environment` }) - await garden.actions.prepareEnvironment({ log, force: opts.force, allowUserInput: true }) + const actions = await garden.getActionHelper() + await actions.prepareEnvironment({ log, force: opts.force, allowUserInput: true }) log.info("") logFooter({ log, emoji: "heavy_check_mark", command: `Done!` }) diff --git a/garden-service/src/commands/logs.ts b/garden-service/src/commands/logs.ts index 28eff25a88..f0164eb802 100644 --- a/garden-service/src/commands/logs.ts +++ b/garden-service/src/commands/logs.ts @@ -95,13 +95,15 @@ export class LogsCommand extends Command { } }) + const actions = await garden.getActionHelper() + await Bluebird.map(services, async (service: Service) => { const voidLog = log.placeholder(LogLevel.silly, { childEntriesInheritLevel: true }) const runtimeContext = await getServiceRuntimeContext(garden, graph, service) - const status = await garden.actions.getServiceStatus({ log: voidLog, service, hotReload: false, runtimeContext }) + const status = await actions.getServiceStatus({ log: voidLog, service, hotReload: false, runtimeContext }) if (status.state === "ready" || status.state === "outdated") { - await garden.actions.getServiceLogs({ log, service, stream, follow, tail, runtimeContext }) + await actions.getServiceLogs({ log, service, stream, follow, tail, runtimeContext }) } else { await stream.write({ serviceName: service.name, diff --git a/garden-service/src/commands/run/module.ts b/garden-service/src/commands/run/module.ts index cd20126a2d..a5205f76bf 100644 --- a/garden-service/src/commands/run/module.ts +++ b/garden-service/src/commands/run/module.ts @@ -83,7 +83,8 @@ export class RunModuleCommand extends Command { command: msg, }) - await garden.actions.prepareEnvironment({ log }) + const actions = await garden.getActionHelper() + await actions.prepareEnvironment({ log }) const buildTask = new BuildTask({ garden, log, module, force: opts["force-build"] }) await garden.processTasks([buildTask]) @@ -96,7 +97,7 @@ export class RunModuleCommand extends Command { log.info("") - const result = await garden.actions.runModule({ + const result = await actions.runModule({ log, module, command, diff --git a/garden-service/src/commands/run/service.ts b/garden-service/src/commands/run/service.ts index ce5ea894ef..439010758b 100644 --- a/garden-service/src/commands/run/service.ts +++ b/garden-service/src/commands/run/service.ts @@ -64,7 +64,8 @@ export class RunServiceCommand extends Command { command: `Running service ${chalk.cyan(serviceName)} in module ${chalk.cyan(module.name)}`, }) - await garden.actions.prepareEnvironment({ log }) + const actions = await garden.getActionHelper() + await actions.prepareEnvironment({ log }) const buildTask = new BuildTask({ garden, log, module, force: opts["force-build"] }) await garden.processTasks([buildTask]) @@ -72,7 +73,7 @@ export class RunServiceCommand extends Command { const runtimeContext = await runtimeContextForServiceDeps(garden, graph, module) printRuntimeContext(log, runtimeContext) - const result = await garden.actions.runService({ + const result = await actions.runService({ log, service, runtimeContext, diff --git a/garden-service/src/commands/run/task.ts b/garden-service/src/commands/run/task.ts index fbf6cc3bda..5023214c60 100644 --- a/garden-service/src/commands/run/task.ts +++ b/garden-service/src/commands/run/task.ts @@ -57,7 +57,8 @@ export class RunTaskCommand extends Command { logHeader({ log, emoji: "runner", command: msg }) - await garden.actions.prepareEnvironment({ log }) + const actions = await garden.getActionHelper() + await actions.prepareEnvironment({ log }) const taskTask = await TaskTask.factory({ garden, graph, task, log, force: true, forceBuild: opts["force-build"] }) const result = (await garden.processTasks([taskTask]))[taskTask.getKey()] diff --git a/garden-service/src/commands/run/test.ts b/garden-service/src/commands/run/test.ts index 007fec09f8..114436a897 100644 --- a/garden-service/src/commands/run/test.ts +++ b/garden-service/src/commands/run/test.ts @@ -90,7 +90,8 @@ export class RunTestCommand extends Command { command: `Running test ${chalk.cyan(testName)} in module ${chalk.cyan(moduleName)}`, }) - await garden.actions.prepareEnvironment({ log }) + const actions = await garden.getActionHelper() + await actions.prepareEnvironment({ log }) const buildTask = new BuildTask({ garden, log, module, force: opts["force-build"] }) await garden.processTasks([buildTask]) @@ -103,7 +104,7 @@ export class RunTestCommand extends Command { const testVersion = await getTestVersion(garden, graph, module, testConfig) - const result = await garden.actions.testModule({ + const result = await actions.testModule({ log, module, interactive, diff --git a/garden-service/src/commands/set.ts b/garden-service/src/commands/set.ts index cb772f5c1f..2b39753589 100644 --- a/garden-service/src/commands/set.ts +++ b/garden-service/src/commands/set.ts @@ -66,7 +66,8 @@ export class SetSecretCommand extends Command { async action({ garden, log, args }: CommandParams): Promise> { const key = args.key - const result = await garden.actions.setSecret({ + const actions = await garden.getActionHelper() + const result = await actions.setSecret({ pluginName: args.provider, key, value: args.value, diff --git a/garden-service/src/commands/test.ts b/garden-service/src/commands/test.ts index 3442ee213c..830222494a 100644 --- a/garden-service/src/commands/test.ts +++ b/garden-service/src/commands/test.ts @@ -103,7 +103,8 @@ export class TestCommand extends Command { modules = await graph.getModules() } - await garden.actions.prepareEnvironment({ log }) + const actions = await garden.getActionHelper() + await actions.prepareEnvironment({ log }) const name = opts.name const force = opts.force diff --git a/garden-service/src/commands/unlink/module.ts b/garden-service/src/commands/unlink/module.ts index 984da69c5a..8f1d9100c7 100644 --- a/garden-service/src/commands/unlink/module.ts +++ b/garden-service/src/commands/unlink/module.ts @@ -62,7 +62,7 @@ export class UnlinkModuleCommand extends Command { const { modules = [] } = args if (opts.all) { - await garden.localConfigStore.set([localConfigKeys.linkedModuleSources], []) + await garden.configStore.set([localConfigKeys.linkedModuleSources], []) log.info("Unlinked all modules") return { result: [] } } diff --git a/garden-service/src/commands/unlink/source.ts b/garden-service/src/commands/unlink/source.ts index dc7406d66b..45541640ab 100644 --- a/garden-service/src/commands/unlink/source.ts +++ b/garden-service/src/commands/unlink/source.ts @@ -62,7 +62,7 @@ export class UnlinkSourceCommand extends Command { const { sources = [] } = args if (opts.all) { - await garden.localConfigStore.set([localConfigKeys.linkedProjectSources], []) + await garden.configStore.set([localConfigKeys.linkedProjectSources], []) log.info("Unlinked all sources") return { result: [] } } diff --git a/garden-service/src/config-store.ts b/garden-service/src/config-store.ts index 98ff107bbf..c4a3b61628 100644 --- a/garden-service/src/config-store.ts +++ b/garden-service/src/config-store.ts @@ -202,6 +202,10 @@ const localConfigSchema = Joi.object() .keys(localConfigSchemaKeys) .meta({ internal: true }) +// TODO: we should not be passing this to provider actions +export const configStoreSchema = Joi.object() + .description("Helper class for managing local configuration for plugins.") + export class LocalConfigStore extends ConfigStore { getConfigPath(projectPath): string { diff --git a/garden-service/src/config/base.ts b/garden-service/src/config/base.ts index e4e903184f..49bf9ac4fd 100644 --- a/garden-service/src/config/base.ts +++ b/garden-service/src/config/base.ts @@ -6,96 +6,55 @@ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ -import { join, basename, sep, resolve, relative } from "path" -import { findByName, getNames } from "../util/util" -import * as Joi from "joi" +import { join, sep, resolve, relative } from "path" import * as yaml from "js-yaml" import { readFile } from "fs-extra" -import { omit } from "lodash" -import { baseModuleSpecSchema, ModuleConfig } from "./module" -import { validateWithPath } from "./common" +import { omit, flatten, isPlainObject, find } from "lodash" +import { baseModuleSpecSchema, ModuleResource } from "./module" import { ConfigurationError } from "../exceptions" -import { defaultEnvironments, ProjectConfig, projectSchema } from "../config/project" -import { CONFIG_FILENAME } from "../constants" +import { CONFIG_FILENAME, DEFAULT_API_VERSION } from "../constants" +import { ProjectResource } from "../config/project" -export interface GardenConfig { - dirname: string +export interface GardenResource { + apiVersion: string + kind: string + name: string path: string - modules?: ModuleConfig[] - project?: ProjectConfig } -export const configSchema = Joi.object() - .keys({ - dirname: Joi.string().meta({ internal: true }), - path: Joi.string().meta({ internal: true }), - module: baseModuleSpecSchema, - project: projectSchema, - }) - .optionalKeys(["module", "project"]) - .required() - .description("The garden.yml config file.") +const baseModuleSchemaKeys = Object.keys(baseModuleSpecSchema.describe().children).concat(["kind"]) -const baseModuleSchemaKeys = Object.keys(baseModuleSpecSchema.describe().children) - -export async function loadConfig(projectRoot: string, path: string): Promise { +export async function loadConfig(projectRoot: string, path: string): Promise { // TODO: nicer error messages when load/validation fails const absPath = join(path, CONFIG_FILENAME) - let fileData + let fileData: Buffer let rawSpecs: any[] // loadConfig returns undefined if config file is not found in the given directory try { fileData = await readFile(absPath) } catch (err) { - return undefined + return [] } try { - rawSpecs = yaml.safeLoadAll(fileData) || [] + rawSpecs = yaml.safeLoadAll(fileData.toString()) || [] } catch (err) { throw new ConfigurationError(`Could not parse ${CONFIG_FILENAME} in directory ${path} as valid YAML`, err) } - const specs: ConfigDoc[] = rawSpecs.map(s => prepareConfigDoc(s, path, projectRoot)) + const resources: GardenResource[] = flatten(rawSpecs.map(s => prepareResources(s, path, projectRoot))) - const projectSpecs = specs.filter(s => s.project) + const projectSpecs = resources.filter(s => s.kind === "Project") if (projectSpecs.length > 1) { throw new ConfigurationError(`Multiple project declarations in ${path}`, { projectSpecs }) } - const project = projectSpecs[0] ? projectSpecs[0].project : undefined - const modules: ModuleConfig[] = specs.filter(s => s.module).map(s => s.module!) - - const dirname = basename(path) - - return { - dirname, - path, - modules: modules.length > 0 ? modules : undefined, - project, - } -} - -type ConfigDoc = { - module?: ModuleConfig, - project?: ProjectConfig, + return resources } export type ConfigKind = "Module" | "Project" -export const configKinds = new Set(["Module", "Project"]) - -const configKindSettings = { - Module: { - specKey: "module", - validationSchema: baseModuleSpecSchema, - }, - Project: { - specKey: "project", - validationSchema: projectSchema, - }, -} /** * Each YAML document in a garden.yml file consists of a project definition and/or a module definition. @@ -109,38 +68,16 @@ const configKindSettings = { * definitions). The kind key is removed before validation, so that specs following both styles can be validated * with the same schema. */ -function prepareConfigDoc(spec: any, path: string, projectRoot: string): ConfigDoc { - - const kind = spec.kind - - if (!spec.kind) { - const preparedSpec = prepareScopedConfigDoc(spec, path) - // validate with scoped config schema - return validateWithPath({ - config: preparedSpec, - schema: configSchema, - configType: "config", - path, - projectRoot, - }) +function prepareResources(spec: any, path: string, projectRoot: string): GardenResource[] { + if (!isPlainObject(spec)) { + throw new ConfigurationError(`Invalid configuration found in ${path}`, { spec, path }) } - if (configKinds.has(kind)) { - const { specKey, validationSchema } = configKindSettings[kind] - const preparedSpec = prepareFlatConfigDoc(spec, path) - const validated = validateWithPath({ - config: preparedSpec, - schema: validationSchema, - configType: specKey, - path, - projectRoot, - }) - return { [specKey]: validated } + if (spec.kind) { + return [prepareFlatConfigDoc(spec, path, projectRoot)] } else { - const relPath = `${relative(projectRoot, path)}/garden.yml` - throw new ConfigurationError(`Unknown config kind ${kind} in ${relPath}`, { kind, path: relPath }) + return prepareScopedConfigDoc(spec, path) } - } /** @@ -148,20 +85,17 @@ function prepareConfigDoc(spec: any, path: string, projectRoot: string): ConfigD * * The spec defines either a project or a module (determined by its "kind" field). */ -function prepareFlatConfigDoc(spec: any, path: string): ConfigDoc { - +function prepareFlatConfigDoc(spec: any, path: string, projectRoot: string): GardenResource { const kind = spec.kind - delete spec.kind if (kind === "Project") { - spec = prepareProjectConfig(spec, path) - } - - if (kind === "Module") { - spec = prepareModuleConfig(spec, path) + return prepareProjectConfig(spec, path) + } else if (kind === "Module") { + return prepareModuleResource(spec, path) + } else { + const relPath = `${relative(projectRoot, path)}/garden.yml` + throw new ConfigurationError(`Unknown config kind ${kind} in ${relPath}`, { kind, path: relPath }) } - - return spec } /** @@ -170,113 +104,84 @@ function prepareFlatConfigDoc(spec: any, path: string): ConfigDoc { * The spec defines a project and/or a module, with the config for each nested under the "project" / "module" field, * respectively. */ -function prepareScopedConfigDoc(spec: any, path: string): ConfigDoc { +function prepareScopedConfigDoc(spec: any, path: string): GardenResource[] { + const resources: GardenResource[] = [] + if (spec.project) { - spec.project = prepareProjectConfig(spec.project, path) + resources.push(prepareProjectConfig(spec.project, path)) } if (spec.module) { - spec.module = prepareModuleConfig(spec.module, path) + resources.push(prepareModuleResource(spec.module, path)) } - return spec + return resources } -function prepareProjectConfig(projectSpec: any, path: string): ProjectConfig { - - const validatedSpec = validateWithPath({ - config: projectSpec, - schema: projectSchema, - configType: "project", - path, - projectRoot: path, // If there's a project spec, we can assume path === projectRoot. - }) - - if (!validatedSpec.environments) { - validatedSpec.environments = defaultEnvironments - } - - // we include the default local environment unless explicitly overridden - for (const env of defaultEnvironments) { - if (!findByName(validatedSpec.environments, env.name)) { - validatedSpec.environments.push(env) - } +function prepareProjectConfig(spec: any, path: string): ProjectResource { + if (!spec.apiVersion) { + spec.apiVersion = DEFAULT_API_VERSION } - // the default environment is the first specified environment in the config, unless specified - const defaultEnvironment = validatedSpec.defaultEnvironment + spec.kind = "Project" + spec.path = path - if (defaultEnvironment === "") { - validatedSpec.defaultEnvironment = validatedSpec.environments[0].name - } else { - if (!findByName(validatedSpec.environments, defaultEnvironment)) { - throw new ConfigurationError(`The specified default environment ${defaultEnvironment} is not defined`, { - defaultEnvironment, - availableEnvironments: getNames(validatedSpec.environments), - }) - } - } - - return validatedSpec + return spec } -function prepareModuleConfig(moduleSpec: any, path: string): ModuleConfig { +function prepareModuleResource(spec: any, path: string): ModuleResource { /** * We allow specifying modules by name only as a shorthand: * dependencies: * - foo-module * - name: foo-module // same as the above */ - const dependencies = moduleSpec.build && moduleSpec.build.dependencies - ? moduleSpec.build.dependencies.map(dep => typeof dep === "string" ? { name: dep, copy: [] } : dep) + const dependencies = spec.build && spec.build.dependencies + ? spec.build.dependencies.map(dep => typeof dep === "string" ? { name: dep, copy: [] } : dep) : [] // Built-in keys are validated here and the rest are put into the `spec` field - const module = { - apiVersion: moduleSpec.apiVersion, - allowPublish: moduleSpec.allowPublish, + return { + apiVersion: spec.apiVersion || DEFAULT_API_VERSION, + kind: "Module", + allowPublish: spec.allowPublish, build: { dependencies, }, - description: moduleSpec.description, - include: moduleSpec.include, - name: moduleSpec.name, + description: spec.description, + include: spec.include, + name: spec.name, outputs: {}, path, - repositoryUrl: moduleSpec.repositoryUrl, + repositoryUrl: spec.repositoryUrl, serviceConfigs: [], spec: { - ...omit(moduleSpec, baseModuleSchemaKeys), - build: { ...moduleSpec.build, dependencies }, + ...omit(spec, baseModuleSchemaKeys), + build: { ...spec.build, dependencies }, }, testConfigs: [], - type: moduleSpec.type, + type: spec.type, taskConfigs: [], } - - return module } -export async function findProjectConfig(path: string, allowInvalid = false): Promise { - let config: GardenConfig | undefined - +export async function findProjectConfig(path: string, allowInvalid = false): Promise { let sepCount = path.split(sep).length - 1 for (let i = 0; i < sepCount; i++) { try { - config = await loadConfig(path, path) + const resources = await loadConfig(path, path) + const projectResource = find(resources, r => r.kind === "Project") + if (projectResource) { + return projectResource + } } catch (err) { if (!allowInvalid) { throw err } else { - continue + path = resolve(path, "..") } } - if (!config || !config.project) { - path = resolve(path, "..") - } else if (config.project) { - return config - } } - return config + return } diff --git a/garden-service/src/config/common.ts b/garden-service/src/config/common.ts index b9fa03eb97..aea8a55157 100644 --- a/garden-service/src/config/common.ts +++ b/garden-service/src/config/common.ts @@ -16,7 +16,7 @@ import { relative } from "path" export type Primitive = string | number | boolean | null export interface PrimitiveMap { [key: string]: Primitive } -export interface DeepPrimitiveMap { [key: string]: Primitive | DeepPrimitiveMap } +export interface DeepPrimitiveMap { [key: string]: Primitive | DeepPrimitiveMap | Primitive[] | DeepPrimitiveMap[] } // export type ConfigWithSpec = { // spec: Omit & Partial diff --git a/garden-service/src/config/config-context.ts b/garden-service/src/config/config-context.ts index e987a2aee5..8597a47f75 100644 --- a/garden-service/src/config/config-context.ts +++ b/garden-service/src/config/config-context.ts @@ -8,7 +8,7 @@ import { isString } from "lodash" import { PrimitiveMap, isPrimitive, Primitive, joiIdentifierMap, joiStringMap, joiPrimitive } from "./common" -import { Provider, Environment, providerConfigBaseSchema } from "./project" +import { Provider, ProviderConfig } from "./provider" import { ModuleConfig } from "./module" import { ConfigurationError } from "../exceptions" import { resolveTemplateString } from "../template-string" @@ -207,8 +207,58 @@ class EnvironmentContext extends ConfigContext { } } +const providersExample = { kubernetes: { config: { clusterHostname: "my-cluster.example.com" } } } + +class ProviderContext extends ConfigContext { + @schema( + Joi.object() + .description("The resolved configuration for the provider.") + .example(providersExample.kubernetes), + ) + public config: ProviderConfig + + // TODO: Need further steps to be able to reference runtime outputs for providers. + // @schema( + // joiIdentifierMap(joiPrimitive()) + // .description("The outputs defined by the provider (see individual plugin docs for details).") + // .example({ "cluster-ip": "1.2.3.4" }), + // ) + // public outputs: PrimitiveMap + + constructor(root: ConfigContext, config: ProviderConfig) { + super(root) + this.config = config + } +} + +export class ProviderConfigContext extends ProjectConfigContext { + @schema( + EnvironmentContext.getSchema() + .description("Information about the environment that Garden is running against."), + ) + public environment: EnvironmentContext + + @schema( + joiIdentifierMap(ProviderContext.getSchema()) + .description("Retrieve information about providers that are defined in the project.") + .example(providersExample), + ) + public providers: Map + + constructor(environmentName: string, resolvedProviders: Provider[]) { + super() + const _this = this + + this.environment = new EnvironmentContext(this, environmentName) + + this.providers = new Map(resolvedProviders.map(p => + <[string, ProviderContext]>[p.name, new ProviderContext(_this, p)], + )) + } +} + const exampleOutputs = { endpoint: "http://my-service/path/to/endpoint" } -const exampleVersion = "v-v17ad4cb3fd" +const exampleVersion = "v-17ad4cb3fd" class ModuleContext extends ConfigContext { @schema( @@ -244,13 +294,7 @@ class ModuleContext extends ConfigContext { * This context is available for template strings under the `module` key in configuration files. * It is a superset of the context available under the `project` key. */ -export class ModuleConfigContext extends ProjectConfigContext { - @schema( - EnvironmentContext.getSchema() - .description("Information about the environment that Garden is running against."), - ) - public environment: EnvironmentContext - +export class ModuleConfigContext extends ProviderConfigContext { @schema( joiIdentifierMap(ModuleContext.getSchema()) .description("Retrieve information about modules that are defined in the project.") @@ -258,13 +302,6 @@ export class ModuleConfigContext extends ProjectConfigContext { ) public modules: Map Promise> - @schema( - joiIdentifierMap(providerConfigBaseSchema) - .description("A map of all configured plugins/providers for this environment and their configuration.") - .example({ kubernetes: { name: "local-kubernetes", context: "my-kube-context" } }), - ) - public providers: Map - @schema( joiIdentifierMap(joiPrimitive()) .description("A map of all variables defined in the project configuration.") @@ -280,15 +317,15 @@ export class ModuleConfigContext extends ProjectConfigContext { constructor( garden: Garden, - environment: Environment, + environmentName: string, + resolvedProviders: Provider[], + variables: PrimitiveMap, moduleConfigs: ModuleConfig[], ) { - super() + super(environmentName, resolvedProviders) const _this = this - this.environment = new EnvironmentContext(_this, environment.name) - this.modules = new Map(moduleConfigs.map((config) => <[string, () => Promise]>[config.name, async (opts: ContextResolveOpts) => { // NOTE: This is a temporary hacky solution until we implement module resolution as a TaskGraph task @@ -305,8 +342,6 @@ export class ModuleConfigContext extends ProjectConfigContext { }], )) - this.providers = new Map(environment.providers.map(p => <[string, Provider]>[p.name, p])) - - this.var = this.variables = environment.variables + this.var = this.variables = variables } } diff --git a/garden-service/src/config/module.ts b/garden-service/src/config/module.ts index 50808b4874..e3a3eb3987 100644 --- a/garden-service/src/config/module.ts +++ b/garden-service/src/config/module.ts @@ -21,6 +21,7 @@ import { } from "./common" import { TestConfig, TestSpec, testConfigSchema } from "./test" import { TaskConfig, TaskSpec, taskConfigSchema } from "./task" +import { DEFAULT_API_VERSION } from "../constants" export interface BuildCopySpec { source: string @@ -66,11 +67,12 @@ export interface ModuleSpec { } export interface BaseModuleSpec { apiVersion: string + name: string + path: string allowPublish: boolean build: BaseBuildSpec description?: string include?: string[] - name: string type: string repositoryUrl?: string } @@ -90,8 +92,8 @@ export const baseBuildSpecSchema = Joi.object() export const baseModuleSpecSchema = Joi.object() .keys({ apiVersion: Joi.string() - .default("garden.io/v0") - .only("garden.io/v0") + .default(DEFAULT_API_VERSION) + .only(DEFAULT_API_VERSION) .description("The schema version of this module's config (currently not used)."), kind: Joi.string().default("Module").only("Module"), type: joiIdentifier() @@ -179,3 +181,7 @@ export const moduleConfigSchema = baseModuleSpecSchema export function serializeConfig(moduleConfig: ModuleConfig) { return stableStringify(moduleConfig) } + +export interface ModuleResource extends ModuleConfig { + kind: "Module" +} diff --git a/garden-service/src/config/project.ts b/garden-service/src/config/project.ts index 039d0b39c4..642c9d0c2e 100644 --- a/garden-service/src/config/project.ts +++ b/garden-service/src/config/project.ts @@ -8,6 +8,8 @@ import * as Joi from "joi" import { safeDump } from "js-yaml" +import { apply, merge } from "json-merge-patch" +import { deline } from "../util/string" import { joiArray, joiIdentifier, @@ -15,29 +17,20 @@ import { Primitive, joiRepositoryUrl, joiUserIdentifier, + validateWithPath, } from "./common" - -export interface ProviderConfig { - name: string - [key: string]: any -} - -export const providerConfigBaseSchema = Joi.object() - .keys({ - name: joiIdentifier().required() - .description("The name of the provider plugin to use.") - .example("local-kubernetes"), - }) - .unknown(true) - .meta({ extendable: true }) - -export interface Provider { - name: string - config: T -} +import { resolveTemplateStrings } from "../template-string" +import { ProjectConfigContext } from "./config-context" +import { findByName, getNames } from "../util/util" +import { ConfigurationError, ParameterError } from "../exceptions" +import { PrimitiveMap } from "./common" +import { fixedPlugins } from "../plugins/plugins" +import { cloneDeep, omit } from "lodash" +import { providerConfigBaseSchema, Provider, ProviderConfig } from "./provider" +import { DEFAULT_API_VERSION } from "../constants" export interface CommonEnvironmentConfig { - providers: ProviderConfig[] // further validated by each plugin + providers?: ProviderConfig[] // further validated by each plugin variables: { [key: string]: Primitive } } @@ -45,12 +38,16 @@ export const environmentConfigSchema = Joi.object() .keys({ providers: joiArray(providerConfigBaseSchema) .unique("name") - .description( - "A list of providers that should be used for this environment, and their configuration. " + - "Please refer to individual plugins/providers for details on how to configure them.", - ), + .meta({ deprecated: true }) + .description(deline` + DEPRECATED - Please use the top-level \`providers\` field instead, and if needed use the \`environments\` key + on the provider configurations to limit them to specific environments. + `), variables: joiVariables() - .description("A key/value map of variables that modules can reference when using this environment."), + .description(deline` + A key/value map of variables that modules can reference when using this environment. These take precedence + over variables defined in the top-level \`variables\` field. + `), }) export interface EnvironmentConfig extends CommonEnvironmentConfig { @@ -61,13 +58,21 @@ export interface Environment extends EnvironmentConfig { providers: Provider[] } -export const environmentSchema = environmentConfigSchema +export const environmentNameSchema = joiUserIdentifier() + .required() + .description("The name of the environment.") + +const environmentSchema = environmentConfigSchema .keys({ - name: Joi.string() - .required() - .description("The name of the current environment."), + name: environmentNameSchema, }) +const environmentsSchema = Joi.alternatives( + Joi.array().items(environmentSchema).unique("name"), + // Allow a string as a shorthand for { name: foo } + Joi.array().items(joiUserIdentifier()), +) + export interface SourceConfig { name: string repositoryUrl: string @@ -87,17 +92,21 @@ export const projectSourcesSchema = joiArray(projectSourceSchema) .description("A list of remote sources to import into project.") export interface ProjectConfig { - apiVersion: string, + apiVersion: string + kind: "Project", name: string + path: string defaultEnvironment: string - environmentDefaults: CommonEnvironmentConfig + environmentDefaults?: CommonEnvironmentConfig environments: EnvironmentConfig[] + providers: ProviderConfig[] sources?: SourceConfig[] + variables: PrimitiveMap } -export const defaultProviders = [ - { name: "container" }, -] +export interface ProjectResource extends ProjectConfig { + kind: "Project" +} export const defaultEnvironments: EnvironmentConfig[] = [ { @@ -105,13 +114,14 @@ export const defaultEnvironments: EnvironmentConfig[] = [ providers: [ { name: "local-kubernetes", + environments: [], }, ], variables: {}, }, ] -const environmentDefaults = { +const emptyEnvironmentDefaults = { providers: [], variables: {}, } @@ -124,33 +134,185 @@ export const projectNameSchema = joiIdentifier() export const projectSchema = Joi.object() .keys({ apiVersion: Joi.string() - .default("garden.io/v0") - .only("garden.io/v0") + .default(DEFAULT_API_VERSION) + .only(DEFAULT_API_VERSION) .description("The schema version of this project's config (currently not used)."), kind: Joi.string().default("Project").only("Project"), + path: Joi.string().meta({ internal: true }), name: projectNameSchema, defaultEnvironment: Joi.string() + .allow("") .default("", "") .description("The default environment to use when calling commands without the `--env` parameter."), environmentDefaults: environmentConfigSchema - .default(() => environmentDefaults, safeDump(environmentDefaults)) - .example([environmentDefaults, {}]) - .description( - "Default environment settings. These are inherited (but can be overridden) by each configured environment.", - ), - environments: joiArray(environmentConfigSchema.keys({ name: joiUserIdentifier().required() })) - .unique("name") + .default(() => emptyEnvironmentDefaults, safeDump(emptyEnvironmentDefaults)) + .example([emptyEnvironmentDefaults, {}]) + .meta({ deprecated: true }) + .description(deline` + DEPRECATED - Please use the \`providers\` field instead, and omit the environments key in the configured + provider to use it for all environments, and use the \`variables\` field to configure variables across all + environments. + `), + environments: environmentsSchema .description("A list of environments to configure for the project.") .example([defaultEnvironments, {}]), + providers: joiArray(providerConfigBaseSchema) + .description( + "A list of providers that should be used for this project, and their configuration. " + + "Please refer to individual plugins/providers for details on how to configure them.", + ), sources: projectSourcesSchema, + variables: joiVariables() + .description("Variables to configure for all environments."), }) .required() .description( "Configuration for a Garden project. This should be specified in the garden.yml file in your project root.", ) -// this is used for default handlers in the action handler -export const defaultProvider: Provider = { - name: "_default", - config: { name: "_default" }, +/** + * Resolves and validates the given raw project configuration, and returns it in a canonical form. + * + * Note: Does _not_ resolve template strings on providers (this needs to happen later in the process). + * + * @param config raw project configuration + */ +export async function resolveProjectConfig(config: ProjectConfig): Promise { + // Resolve template strings for non-environment-specific fields + let { environmentDefaults, environments = [] } = config + + const globalConfig = await resolveTemplateStrings( + { + apiVersion: config.apiVersion, + defaultEnvironment: config.defaultEnvironment, + environmentDefaults: { variables: {}, ...environmentDefaults || {}, providers: [] }, + name: config.name, + sources: config.sources, + variables: config.variables, + environments: environments.map(e => omit(e, ["providers"])), + }, + new ProjectConfigContext(), + ) + + // Validate after resolving global fields + config = validateWithPath({ + config: { ...config, ...globalConfig }, + schema: projectSchema, + configType: "project", + path: config.path, + projectRoot: config.path, + }) + + // Convert deprecated fields + if (!environmentDefaults) { + environmentDefaults = config.environmentDefaults + } + + const { defaultEnvironment } = config + + // Note: The ordering here is important + const providers = [ + ...environmentDefaults!.providers || [], + ...config.providers, + ] + + for (const environment of environments || []) { + for (const provider of environment.providers || []) { + providers.push({ + ...provider, + environments: [environment.name], + }) + } + environment.providers = [] + } + + const variables = { ...config.environmentDefaults!.variables, ...config.variables } + + config = { + ...config, + environmentDefaults: { + providers: [], + variables: {}, + }, + environments: config.environments || [], + providers, + variables, + } + + // TODO: get rid of the default environment config + if (config.environments.length === 0) { + config.environments = cloneDeep(defaultEnvironments) + } + + // the default environment is the first specified environment in the config, unless specified + if (defaultEnvironment === "") { + config.defaultEnvironment = config.environments[0].name + } else { + if (!findByName(config.environments, defaultEnvironment)) { + throw new ConfigurationError(`The specified default environment ${defaultEnvironment} is not defined`, { + defaultEnvironment, + availableEnvironments: getNames(config.environments), + }) + } + } + + return config +} + +/** + * Given an environment name, pulls the relevant environment-specific configuration from the specified project + * config, and merges values appropriately. + * + * For project variables, we apply the variables specified to the selected environment on the global variables + * specified on the top-level `variables` key using a JSON Merge Patch (https://tools.ietf.org/html/rfc7396). + * + * For provider configuration, we filter down to the providers that are enabled for all environments (no `environments` + * key specified) and those that explicitly list the specified environments. Then we merge any provider configs with + * the same provider name, again using JSON Merge Patching, with later entries in the list taking precedence over + * prior ones. + * + * Because we use JSON Merge Patch, be aware that specifying a _null_ value means that it will be omitted in the + * resulting config. + * + * Note: This assumes that deprecated fields have been converted, e.g. by the resolveProjectConfig() function. + * + * @param config a resolved project config (as returned by `resolveProjectConfig()`) + * @param environmentName the name of the environment to use + */ +export function pickEnvironment(config: ProjectConfig, environmentName: string) { + const { environments, name: projectName } = config + + const environmentConfig = findByName(environments, environmentName) + + if (!environmentConfig) { + throw new ParameterError(`Project ${projectName} does not specify environment ${environmentName}`, { + projectName, + environmentName, + definedEnvironments: getNames(environments), + }) + } + + const fixedProviders = fixedPlugins.map(name => ({ name })) + const allProviders = [ + ...fixedProviders, + ...config.providers.filter(p => !p.environments || p.environments.includes(environmentName)), + ] + + const mergedProviders: { [name: string]: ProviderConfig } = {} + + for (const provider of allProviders) { + if (!!mergedProviders[provider.name]) { + // Merge using a JSON Merge Patch (see https://tools.ietf.org/html/rfc7396) + apply(mergedProviders[provider.name], provider) + } else { + mergedProviders[provider.name] = cloneDeep(provider) + } + } + + const variables: PrimitiveMap = merge(config.variables, environmentConfig.variables) + + return { + providers: Object.values(mergedProviders), + variables, + } } diff --git a/garden-service/src/config/provider.ts b/garden-service/src/config/provider.ts new file mode 100644 index 0000000000..9b089665b8 --- /dev/null +++ b/garden-service/src/config/provider.ts @@ -0,0 +1,109 @@ +/* + * Copyright (C) 2018 Garden Technologies, Inc. + * + * This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. + */ + +import * as Joi from "joi" +import { deline } from "../util/string" +import { joiIdentifier, joiUserIdentifier, joiArray } from "./common" +import { collectTemplateReferences } from "../template-string" +import { ConfigurationError } from "../exceptions" +import { ModuleConfig, moduleConfigSchema } from "./module" +import { uniq } from "lodash" +import { GardenPlugin } from "../types/plugin/plugin" + +export interface ProviderConfig { + name: string + environments?: string[] + [key: string]: any +} + +const providerFixedFieldsSchema = Joi.object() + .keys({ + name: joiIdentifier() + .required() + .description("The name of the provider plugin to use.") + .example("local-kubernetes"), + environments: Joi.array().items(joiUserIdentifier()) + .optional() + .description(deline` + If specified, this provider will only be used in the listed environments. Note that an empty array effectively + disables the provider. To use a provider in all environments, omit this field. + `) + .example([["dev", "stage"], {}]), + }) + +export const providerConfigBaseSchema = providerFixedFieldsSchema + .unknown(true) + .meta({ extendable: true }) + +export interface Provider { + name: string + dependencies: Provider[] + environments?: string[] + moduleConfigs: ModuleConfig[] + config: T +} + +export const providerSchema = providerFixedFieldsSchema + .keys({ + dependencies: Joi.lazy(() => providersSchema) + .required(), + config: Joi.lazy(() => providerConfigBaseSchema) + .required(), + moduleConfigs: joiArray(moduleConfigSchema.optional()), + }) + +export const providersSchema = joiArray(providerSchema) + .description("List of all the providers that this provider depdends on.") + +export interface ProviderMap { [name: string]: Provider } + +export const defaultProviders = [ + { name: "container" }, +] + +// this is used for default handlers in the action handler +export const defaultProvider: Provider = { + name: "_default", + dependencies: [], + moduleConfigs: [], + config: { name: "_default" }, +} + +export function providerFromConfig( + config: ProviderConfig, dependencies: Provider[], moduleConfigs: ModuleConfig[], +): Provider { + return { + name: config.name, + dependencies, + moduleConfigs, + config, + } +} + +export async function getProviderDependencies(plugin: GardenPlugin, config: ProviderConfig) { + const deps: string[] = [...plugin.dependencies || []] + + // Implicit dependencies from template strings + const references = await collectTemplateReferences(config) + + for (const key of references) { + if (key[0] === "provider") { + const providerName = key[1] + if (!providerName) { + throw new ConfigurationError(deline` + Invalid template key '${key.join(".")}' in configuration for provider '${config.name}'. You must + specify a provider name as well (e.g. \${provider.my-provider}). + `, { config, key: key.join(".") }, + ) + } + deps.push(providerName) + } + } + + return uniq(deps).sort() +} diff --git a/garden-service/src/constants.ts b/garden-service/src/constants.ts index fec6d5c04a..c6e2e03f00 100644 --- a/garden-service/src/constants.ts +++ b/garden-service/src/constants.ts @@ -24,9 +24,9 @@ export const PROJECT_SOURCES_DIR_NAME = join(GARDEN_DIR_NAME, "sources", "projec export const MODULE_SOURCES_DIR_NAME = join(GARDEN_DIR_NAME, "sources", "module") export const GARDEN_BUILD_VERSION_FILENAME = "garden-build-version" export const GARDEN_VERSIONFILE_NAME = ".garden-version" -export const DEFAULT_NAMESPACE = "default" export const DEFAULT_PORT_PROTOCOL = "TCP" +export const DEFAULT_API_VERSION = "garden.io/v0" export const GARDEN_ANNOTATION_PREFIX = "garden.io/" export const GARDEN_ANNOTATION_KEYS_SERVICE = GARDEN_ANNOTATION_PREFIX + "service" export const GARDEN_ANNOTATION_KEYS_VERSION = GARDEN_ANNOTATION_PREFIX + "version" diff --git a/garden-service/src/docs/config.ts b/garden-service/src/docs/config.ts index 5f7ca6a1c3..0129ad8ce1 100644 --- a/garden-service/src/docs/config.ts +++ b/garden-service/src/docs/config.ts @@ -51,12 +51,13 @@ const moduleTypes = [ { name: "helm", pluginName: "local-kubernetes" }, { name: "kubernetes", pluginName: "local-kubernetes" }, { name: "maven-container" }, - { name: "openfaas" }, + { name: "openfaas", pluginName: "local-kubernetes" }, ] const providers = [ { name: "local-kubernetes", schema: populateProviderSchema(localK8sConfigSchema) }, { name: "kubernetes", schema: populateProviderSchema(k8sConfigSchema) }, + { name: "local-openfaas", schema: populateProviderSchema(openfaasConfigSchema) }, { name: "maven-container", schema: populateProviderSchema(mavenContainerConfigSchema) }, { name: "openfaas", schema: populateProviderSchema(openfaasConfigSchema) }, ] @@ -414,24 +415,20 @@ export async function writeConfigReferenceDocs(docsRoot: string) { const moduleProviders = uniq(moduleTypes.map(m => m.pluginName || m.name)).map(name => ({ name })) const garden = await Garden.factory(__dirname, { config: { - dirname: "generate-docs", path: __dirname, - project: { - apiVersion: "garden.io/v0", - name: "generate-docs", - defaultEnvironment: "default", - environmentDefaults: { + apiVersion: "garden.io/v0", + kind: "Project", + name: "generate-docs", + defaultEnvironment: "default", + providers: moduleProviders, + variables: {}, + environments: [ + { + name: "default", providers: [], variables: {}, }, - environments: [ - { - name: "default", - providers: moduleProviders, - variables: {}, - }, - ], - }, + ], }, }) @@ -448,7 +445,8 @@ export async function writeConfigReferenceDocs(docsRoot: string) { const readme = ["# Module Types", ""] for (const { name } of moduleTypes) { const path = resolve(moduleTypeDir, `${name}.md`) - const { docs, schema, title } = await garden.actions.describeType(name) + const actions = await garden.getActionHelper() + const { docs, schema, title } = await actions.describeType(name) console.log("->", path) writeFileSync(path, renderModuleTypeReference(populateModuleSchema(schema), name, docs)) diff --git a/garden-service/src/garden.ts b/garden-service/src/garden.ts index d8b0c27295..2191f27ed0 100644 --- a/garden-service/src/garden.ts +++ b/garden-service/src/garden.ts @@ -7,67 +7,44 @@ */ import Bluebird = require("bluebird") -import { - parse, - relative, - resolve, - sep, - join, -} from "path" -import { - extend, - flatten, - isString, - merge, - keyBy, - cloneDeep, - sortBy, - findIndex, -} from "lodash" +import { parse, relative, resolve, sep, join } from "path" +import { flatten, isString, cloneDeep, sortBy, set, zip } from "lodash" const AsyncLock = require("async-lock") import { TreeCache } from "./cache" -import { builtinPlugins, fixedPlugins } from "./plugins/plugins" +import { builtinPlugins } from "./plugins/plugins" import { Module, getModuleCacheContext, getModuleKey, ModuleConfigMap } from "./types/module" -import { moduleActionNames, pluginModuleSchema, pluginSchema } from "./types/plugin/plugin" -import { Environment, SourceConfig, ProviderConfig, Provider } from "./config/project" -import { findByName, getNames, pickKeys } from "./util/util" -import { DEFAULT_NAMESPACE, CONFIG_FILENAME } from "./constants" -import { - ConfigurationError, - ParameterError, - PluginError, - RuntimeError, -} from "./exceptions" +import { pluginModuleSchema, pluginSchema } from "./types/plugin/plugin" +import { SourceConfig, ProjectConfig, resolveProjectConfig, pickEnvironment } from "./config/project" +import { findByName, pickKeys, getPackageVersion } from "./util/util" +import { ConfigurationError, PluginError, RuntimeError } from "./exceptions" import { VcsHandler, ModuleVersion } from "./vcs/vcs" import { GitHandler } from "./vcs/git" import { BuildDir } from "./build-dir" import { ConfigGraph } from "./config-graph" import { TaskGraph, TaskResults } from "./task-graph" import { getLogger } from "./logger/logger" -import { pluginActionNames, PluginActions, PluginFactory, GardenPlugin } from "./types/plugin/plugin" +import { PluginActions, PluginFactory, GardenPlugin } from "./types/plugin/plugin" import { joiIdentifier, validate, PrimitiveMap, validateWithPath } from "./config/common" import { resolveTemplateStrings } from "./template-string" -import { - configSchema, - GardenConfig, - loadConfig, - findProjectConfig, -} from "./config/base" +import { loadConfig, findProjectConfig } from "./config/base" import { BaseTask } from "./tasks/base" -import { LocalConfigStore } from "./config-store" +import { LocalConfigStore, ConfigStore } from "./config-store" import { getLinkedSources, ExternalSourceType } from "./util/ext-source-util" -import { BuildDependencyConfig, ModuleConfig } from "./config/module" -import { ProjectConfigContext, ModuleConfigContext, ContextResolveOpts } from "./config/config-context" -import { ActionHelper } from "./actions" +import { BuildDependencyConfig, ModuleConfig, baseModuleSpecSchema, ModuleResource } from "./config/module" +import { ModuleConfigContext, ContextResolveOpts } from "./config/config-context" import { createPluginContext } from "./plugin-context" import { ModuleAndRuntimeActions, Plugins, RegisterPluginParam } from "./types/plugin/plugin" -import { SUPPORTED_PLATFORMS, SupportedPlatform } from "./constants" +import { SUPPORTED_PLATFORMS, SupportedPlatform, CONFIG_FILENAME } from "./constants" import { platform, arch } from "os" import { LogEntry } from "./logger/log-entry" import { EventBus } from "./events" import { Watcher } from "./watch" import { getIgnorer, Ignorer, getModulesPathsFromPath } from "./util/fs" +import { Provider, ProviderConfig, getProviderDependencies } from "./config/provider" +import { ResolveProviderTask } from "./tasks/resolve-provider" +import { ActionHelper } from "./actions" +import { DependencyGraph, detectCycles, cyclesToString } from "./util/validate-dependencies" export interface ActionHandlerMap { [actionName: string]: PluginActions[T] @@ -92,7 +69,7 @@ export type ModuleActionMap = { } export interface GardenOpts { - config?: GardenConfig, + config?: ProjectConfig, environmentName?: string, log?: LogEntry, plugins?: Plugins, @@ -102,34 +79,36 @@ interface ModuleConfigResolveOpts extends ContextResolveOpts { configContext?: ModuleConfigContext } -const scanLock = new AsyncLock() +const asyncLock = new AsyncLock() export class Garden { public readonly log: LogEntry private readonly loadedPlugins: { [key: string]: GardenPlugin } private moduleConfigs: ModuleConfigMap private pluginModuleConfigs: ModuleConfig[] + private resolvedProviders: Provider[] private modulesScanned: boolean private readonly registeredPlugins: { [key: string]: PluginFactory } private readonly taskGraph: TaskGraph private readonly watcher: Watcher - public readonly environment: Environment - public readonly localConfigStore: LocalConfigStore + public readonly configStore: ConfigStore public readonly vcs: VcsHandler public readonly cache: TreeCache - public readonly actions: ActionHelper + private actionHelper: ActionHelper public readonly events: EventBus constructor( public readonly projectRoot: string, public readonly projectName: string, - environmentName: string, - variables: PrimitiveMap, + public readonly environmentName: string, + public readonly variables: PrimitiveMap, public readonly projectSources: SourceConfig[] = [], public readonly buildDir: BuildDir, public readonly ignorer: Ignorer, public readonly opts: GardenOpts, + plugins: Plugins, + private readonly providerConfigs: ProviderConfig[], ) { // make sure we're on a supported platform const currentPlatform = platform() @@ -147,107 +126,55 @@ export class Garden { this.log = opts.log || getLogger().placeholder() // TODO: Support other VCS options. this.vcs = new GitHandler(this.projectRoot) - this.localConfigStore = new LocalConfigStore(this.projectRoot) + this.configStore = new LocalConfigStore(this.projectRoot) this.cache = new TreeCache() - this.environment = { - name: environmentName, - // The providers are populated when adding plugins in the factory. - providers: [], - variables, - } - this.moduleConfigs = {} this.loadedPlugins = {} this.pluginModuleConfigs = [] this.registeredPlugins = {} this.taskGraph = new TaskGraph(this, this.log) - this.actions = new ActionHelper(this) this.events = new EventBus(this.log) this.watcher = new Watcher(this, this.log) + + // Register plugins + for (const [name, pluginFactory] of Object.entries({ ...builtinPlugins, ...plugins })) { + // This cast is required for the linter to accept the instance type hackery. + this.registerPlugin(name, pluginFactory) + } } static async factory( this: T, currentDirectory: string, opts: GardenOpts = {}, ): Promise> { - let parsedConfig: GardenConfig let { environmentName, config, plugins = {} } = opts - if (config) { - parsedConfig = validate(config, configSchema, { context: "root configuration" }) - - if (!parsedConfig.project) { - throw new ConfigurationError(`Supplied config does not contain a project configuration`, { - currentDirectory, - config, - }) - } - } else { + if (!config) { config = await findProjectConfig(currentDirectory) - if (!config || !config.project) { + if (!config) { throw new ConfigurationError( `Not a project directory (or any of the parent directories): ${currentDirectory}`, { currentDirectory }, ) } - - parsedConfig = await resolveTemplateStrings(config!, new ProjectConfigContext()) } - const projectRoot = parsedConfig.path + config = await resolveProjectConfig(config) const { defaultEnvironment, - environments, name: projectName, - environmentDefaults, sources: projectSources, - } = parsedConfig.project! + path: projectRoot, + } = config if (!environmentName) { environmentName = defaultEnvironment } - const parts = environmentName.split(".") - environmentName = parts[0] - const namespace = parts.slice(1).join(".") || DEFAULT_NAMESPACE - - const environmentConfig = findByName(environments, environmentName) - - if (!environmentConfig) { - throw new ParameterError(`Project ${projectName} does not specify environment ${environmentName}`, { - projectName, - environmentName, - definedEnvironments: getNames(environments), - }) - } - - if (!environmentConfig.providers || environmentConfig.providers.length === 0) { - throw new ConfigurationError(`Environment '${environmentName}' does not specify any providers`, { - projectName, - environmentName, - environmentConfig, - }) - } - - if (namespace.startsWith("garden-")) { - throw new ParameterError(`Namespace cannot start with "garden-"`, { - environmentConfig, - namespace, - }) - } - - const fixedProviders = fixedPlugins.map(name => ({ name })) - - const mergedProviderConfigs = merge( - fixedProviders, - keyBy(environmentDefaults.providers, "name"), - keyBy(environmentConfig.providers, "name"), - ) - - const variables = merge({}, environmentDefaults.variables, environmentConfig.variables) + const { providers, variables } = pickEnvironment(config, environmentName) const buildDir = await BuildDir.factory(projectRoot) const ignorer = await getIgnorer(projectRoot) @@ -261,20 +188,10 @@ export class Garden { buildDir, ignorer, opts, + plugins, + providers, ) as InstanceType - // Register plugins - for (const [name, pluginFactory] of Object.entries({ ...builtinPlugins, ...plugins })) { - // This cast is required for the linter to accept the instance type hackery. - (garden).registerPlugin(name, pluginFactory) - } - - // Load configured plugins - // Validate configuration - for (const provider of Object.values(mergedProviderConfigs)) { - await (garden).loadPlugin(provider.name, provider) - } - return garden } @@ -321,7 +238,7 @@ export class Garden { moduleNameOrLocation = resolve(this.projectRoot, moduleNameOrLocation) } - let pluginModule + let pluginModule: any try { pluginModule = require(moduleNameOrLocation) @@ -368,7 +285,11 @@ export class Garden { this.registeredPlugins[name] = factory } - private async loadPlugin(pluginName: string, config: ProviderConfig) { + private async loadPlugin(pluginName: string) { + if (this.loadedPlugins[pluginName]) { + return this.loadedPlugins[pluginName] + } + this.log.silly(`Loading plugin ${pluginName}`) const factory = this.registeredPlugins[pluginName] @@ -397,83 +318,110 @@ export class Garden { this.loadedPlugins[pluginName] = plugin - for (const modulePath of plugin.modules || []) { - let moduleConfigs = await this.loadModuleConfigs(modulePath) - if (!moduleConfigs) { - throw new PluginError(`Could not load module(s) at "${modulePath}" specified in plugin "${pluginName}"`, { - pluginName, - modulePath, - }) - } + this.log.silly(`Done loading plugin ${pluginName}`) - for (const moduleConfig of moduleConfigs) { - moduleConfig.plugin = pluginName - this.pluginModuleConfigs.push(moduleConfig) - } - } + return plugin + } - const actions = plugin.actions || {} + getPlugin(pluginName: string) { + const plugin = this.loadedPlugins[pluginName] - for (const actionType of pluginActionNames) { - const handler = actions[actionType] - handler && this.actions.addActionHandler(pluginName, actionType, handler) + if (!plugin) { + throw new PluginError(`Could not find plugin '${pluginName}'. Are you missing a provider configuration?`, { + pluginName, + availablePlugins: Object.keys(this.loadedPlugins), + }) } - const moduleActions = plugin.moduleActions || {} + return plugin + } - for (const moduleType of Object.keys(moduleActions)) { - for (const actionType of moduleActionNames) { - const handler = moduleActions[moduleType][actionType] - handler && this.actions.addModuleActionHandler(pluginName, actionType, moduleType, handler) + getRawProviderConfigs() { + return this.providerConfigs + } + + async resolveProviders(): Promise { + await asyncLock.acquire("resolve-providers", async () => { + if (this.resolvedProviders) { + return } - } - // allow plugins to be configured more than once - // (to support extending config for fixed plugins and environment defaults) - let providerIndex = findIndex(this.environment.providers, ["name", pluginName]) - let providerConfig: ProviderConfig = providerIndex === -1 - ? config - : this.environment.providers[providerIndex].config + const rawConfigs = this.getRawProviderConfigs() + const plugins = await Bluebird.map(rawConfigs, async (config) => this.loadPlugin(config.name)) - extend(providerConfig, config) + // Detect circular deps here + const pluginGraph: DependencyGraph = {} - // call configureProvider action if provided - const configureHandler = actions.configureProvider + await Bluebird.map(zip(plugins, rawConfigs), async ([plugin, config]) => { + for (const dep of await getProviderDependencies(plugin!, config!)) { + set(pluginGraph, [config!.name, dep], { distance: 1, next: dep }) + } + }) - if (plugin.configSchema) { - providerConfig = validate(providerConfig, plugin.configSchema, { context: `${pluginName} configuration` }) - } + const cycles = detectCycles(pluginGraph) - if (configureHandler) { - this.log.silly(`Calling configureProvider on ${pluginName}`) - const configureOutput = await configureHandler({ - config: providerConfig, - projectName: this.projectName, - log: this.log, - }) - providerConfig = configureOutput.config - } + if (cycles.length > 0) { + const cyclesStr = cyclesToString(cycles) - if (providerIndex === -1) { - this.environment.providers.push({ name: pluginName, config: providerConfig }) - } else { - this.environment.providers[providerIndex].config = providerConfig - } + throw new PluginError( + "One or more circular dependencies found between providers or their configurations: " + cyclesStr, + { cycles }, + ) + } - this.log.silly(`Done loading plugin ${pluginName}`) - } + const tasks = rawConfigs.map((config, i) => { + // TODO: actually resolve version, based on the VCS version of the plugin and its dependencies + const version = { + versionString: getPackageVersion(), + dirtyTimestamp: null, + commitHash: getPackageVersion(), + dependencyVersions: {}, + files: [], + } - getPlugin(pluginName: string) { - const plugin = this.loadedPlugins[pluginName] + const plugin = plugins[i] - if (!plugin) { - throw new PluginError(`Could not find plugin ${pluginName}. Are you missing a provider configuration?`, { - pluginName, - availablePlugins: Object.keys(this.loadedPlugins), + return new ResolveProviderTask({ + garden: this, + log: this.log, + plugin, + config, + version, + }) }) + const taskResults = await this.processTasks(tasks) + + const failed = Object.values(taskResults).filter(r => !!r.error) + + if (failed.length) { + const messages = failed.map(r => `- ${r.name}: ${r.error!.message}`) + throw new PluginError( + `Failed resolving one or more provider configurations:\n${messages.join("\n")}`, + { rawConfigs, taskResults, messages }, + ) + } + + this.resolvedProviders = Object.values(taskResults).map(result => result.output) + + for (const provider of this.resolvedProviders) { + for (const moduleConfig of provider.moduleConfigs) { + // Make sure module and all nested entities are scoped to the plugin + moduleConfig.plugin = provider.name + this.addModule(moduleConfig) + } + } + }) + + return this.resolvedProviders + } + + async getActionHelper() { + if (!this.actionHelper) { + const providers = await this.resolveProviders() + this.actionHelper = new ActionHelper(this, providers) } - return plugin + return this.actionHelper } /** @@ -496,15 +444,22 @@ export class Garden { * Scans for modules in the project root and remote/linked sources if it hasn't already been done. */ async resolveModuleConfigs(keys?: string[], opts: ModuleConfigResolveOpts = {}): Promise { + const actions = await this.getActionHelper() const configs = await this.getRawModuleConfigs(keys) if (!opts.configContext) { - opts.configContext = new ModuleConfigContext(this, this.environment, Object.values(this.moduleConfigs)) + opts.configContext = new ModuleConfigContext( + this, + this.environmentName, + await this.resolveProviders(), + this.variables, + Object.values(this.moduleConfigs), + ) } return Bluebird.map(configs, async (config) => { config = await resolveTemplateStrings(cloneDeep(config), opts.configContext!, opts) - const description = await this.actions.describeType(config.type) + const description = await actions.describeType(config.type) config.spec = validateWithPath({ config: config.spec, @@ -514,14 +469,58 @@ export class Garden { projectRoot: this.projectRoot, }) - const configureHandler = await this.actions.getModuleActionHandler({ + /* + We allow specifying modules by name only as a shorthand: + + dependencies: + - foo-module + - name: foo-module // same as the above + */ + if (config.build && config.build.dependencies) { + config.build.dependencies = config.build.dependencies + .map(dep => typeof dep === "string" ? { name: dep, copy: [] } : dep) + } + + config = validateWithPath({ + config, + schema: baseModuleSpecSchema, + configType: "module", + name: config.name, + path: config.path, + projectRoot: this.projectRoot, + }) + + if (config.repositoryUrl) { + config.path = await this.loadExtSourcePath({ + name: config.name, + repositoryUrl: config.repositoryUrl, + sourceType: "module", + }) + } + + const configureHandler = await actions.getModuleActionHandler({ actionType: "configure", moduleType: config.type, }) - const ctx = this.getPluginContext(configureHandler["pluginName"]) + const ctx = await this.getPluginContext(configureHandler["pluginName"]) config = await configureHandler({ ctx, moduleConfig: config, log: this.log }) + if (config.plugin) { + // Make sure nested entities in plugin modules are scoped by name + for (const serviceConfig of config.serviceConfigs) { + serviceConfig.name = `${config.plugin}--${serviceConfig.name}` + } + + for (const taskConfig of config.taskConfigs) { + taskConfig.name = `${config.plugin}--${taskConfig.name}` + } + + for (const testConfig of config.testConfigs) { + testConfig.name = `${config.plugin}--${testConfig.name}` + } + } + // FIXME: We should be able to avoid this config.name = getModuleKey(config.name, config.plugin) @@ -591,7 +590,7 @@ export class Garden { Scans the project root for modules and adds them to the context. */ async scanModules(force = false) { - return scanLock.acquire("scan-modules", async () => { + return asyncLock.acquire("scan-modules", async () => { if (this.modulesScanned && !force) { return } @@ -661,23 +660,9 @@ export class Garden { * * @param path Directory containing the module */ - private async loadModuleConfigs(path: string): Promise { - const config = await loadConfig(this.projectRoot, resolve(this.projectRoot, path)) - - if (!config || !config.modules) { - return null - } - - return Bluebird.map(cloneDeep(config.modules), async (moduleConfig) => { - if (moduleConfig.repositoryUrl) { - moduleConfig.path = await this.loadExtSourcePath({ - name: moduleConfig.name, - repositoryUrl: moduleConfig.repositoryUrl, - sourceType: "module", - }) - } - return moduleConfig - }) + private async loadModuleConfigs(path: string): Promise { + const resources = await loadConfig(this.projectRoot, resolve(this.projectRoot, path)) + return resources.filter(r => r.kind === "Module") } //=========================================================================== @@ -711,9 +696,9 @@ export class Garden { */ public async dumpConfig(): Promise { return { - environmentName: this.environment.name, - providers: this.environment.providers, - variables: this.environment.variables, + environmentName: this.environmentName, + providers: await this.resolveProviders(), + variables: this.variables, moduleConfigs: sortBy(await this.resolveModuleConfigs(), "name"), } } diff --git a/garden-service/src/plugin-context.ts b/garden-service/src/plugin-context.ts index 03deb1a4ec..165255a237 100644 --- a/garden-service/src/plugin-context.ts +++ b/garden-service/src/plugin-context.ts @@ -9,38 +9,22 @@ import { Garden } from "./garden" import { keyBy, cloneDeep } from "lodash" import * as Joi from "joi" -import { - Provider, - projectNameSchema, - projectSourcesSchema, - environmentSchema, - providerConfigBaseSchema, - ProviderConfig, -} from "./config/project" -import { joiIdentifier, joiIdentifierMap } from "./config/common" +import { projectNameSchema, projectSourcesSchema, environmentNameSchema } from "./config/project" import { PluginError } from "./exceptions" -import { defaultProvider } from "./config/project" +import { defaultProvider, Provider, providerSchema, ProviderConfig } from "./config/provider" +import { configStoreSchema } from "./config-store" type WrappedFromGarden = Pick -const providerSchema = Joi.object() - .options({ presence: "required" }) - .keys({ - name: joiIdentifier() - .description("The name of the provider (plugin)."), - config: providerConfigBaseSchema, - }) - export interface PluginContext extends WrappedFromGarden { provider: Provider - providers: { [name: string]: Provider } } // NOTE: this is used more for documentation than validation, outside of internal testing @@ -53,18 +37,14 @@ export const pluginContextSchema = Joi.object() .uri({ relativeOnly: true }) .description("The absolute path of the project root."), projectSources: projectSourcesSchema, - localConfigStore: Joi.object() - .description("Helper class for managing local configuration for plugins."), - environment: environmentSchema, + configStore: configStoreSchema, + environmentName: environmentNameSchema, provider: providerSchema .description("The provider being used for this context."), - providers: joiIdentifierMap(providerSchema) - .description("Map of all configured providers for the current environment and project."), }) -export function createPluginContext(garden: Garden, providerName: string): PluginContext { - const projectConfig = cloneDeep(garden.environment) - const providers = keyBy(projectConfig.providers, "name") +export async function createPluginContext(garden: Garden, providerName: string): Promise { + const providers = keyBy(await garden.resolveProviders(), "name") let provider = providers[providerName] if (providerName === "_default") { @@ -76,12 +56,11 @@ export function createPluginContext(garden: Garden, providerName: string): Plugi } return { + environmentName: garden.environmentName, projectName: garden.projectName, projectRoot: garden.projectRoot, projectSources: cloneDeep(garden.projectSources), - environment: projectConfig, - localConfigStore: garden.localConfigStore, + configStore: garden.configStore, provider, - providers, } } diff --git a/garden-service/src/plugins/container/config.ts b/garden-service/src/plugins/container/config.ts index f4358dbd57..a32f2fda80 100644 --- a/garden-service/src/plugins/container/config.ts +++ b/garden-service/src/plugins/container/config.ts @@ -322,7 +322,7 @@ export interface ContainerModuleSpec extends ModuleSpec { tasks: ContainerTaskSpec[], } -export type ContainerModuleConfig = ModuleConfig +export interface ContainerModuleConfig extends ModuleConfig { } export const defaultNamespace = "_" export const defaultTag = "latest" diff --git a/garden-service/src/plugins/exec.ts b/garden-service/src/plugins/exec.ts index 98a424f5f9..29d21cfbf1 100644 --- a/garden-service/src/plugins/exec.ts +++ b/garden-service/src/plugins/exec.ts @@ -16,7 +16,7 @@ import { CommonServiceSpec } from "../config/service" import { BaseTestSpec, baseTestSpecSchema } from "../config/test" import { readModuleVersionFile, writeModuleVersionFile, ModuleVersion } from "../vcs/vcs" import { GARDEN_BUILD_VERSION_FILENAME } from "../constants" -import { ModuleSpec, BaseBuildSpec, baseBuildSpecSchema } from "../config/module" +import { ModuleSpec, BaseBuildSpec, baseBuildSpecSchema, ModuleConfig } from "../config/module" import execa = require("execa") import { BaseTaskSpec, baseTaskSpecSchema } from "../config/task" import { dedent } from "../util/string" @@ -64,6 +64,8 @@ export interface ExecModuleSpec extends ModuleSpec { tests: ExecTestSpec[], } +export type ExecModuleConfig = ModuleConfig + export const execBuildSpecSchema = baseBuildSpecSchema .keys({ command: joiArray(Joi.string()) diff --git a/garden-service/src/plugins/google/google-app-engine.ts b/garden-service/src/plugins/google/google-app-engine.ts index 85edaaeecb..41d30b5db9 100644 --- a/garden-service/src/plugins/google/google-app-engine.ts +++ b/garden-service/src/plugins/google/google-app-engine.ts @@ -18,7 +18,7 @@ import { dumpYaml } from "../../util/util" import { GardenPlugin } from "../../types/plugin/plugin" import { configureContainerModule } from "../container/container" import { ContainerModule } from "../container/config" -import { providerConfigBaseSchema } from "../../config/project" +import { providerConfigBaseSchema } from "../../config/provider" import * as Joi from "joi" import { ConfigureModuleParams } from "../../types/plugin/module/configure" import { DeployServiceParams } from "../../types/plugin/service/deployService" diff --git a/garden-service/src/plugins/google/google-cloud-functions.ts b/garden-service/src/plugins/google/google-cloud-functions.ts index 1ec741a587..cf68e3a6fa 100644 --- a/garden-service/src/plugins/google/google-cloud-functions.ts +++ b/garden-service/src/plugins/google/google-cloud-functions.ts @@ -21,7 +21,7 @@ import { } from "./common" import { GardenPlugin } from "../../types/plugin/plugin" import { baseServiceSpecSchema, CommonServiceSpec } from "../../config/service" -import { Provider, providerConfigBaseSchema } from "../../config/project" +import { Provider, providerConfigBaseSchema } from "../../config/provider" import { ConfigureModuleParams, ConfigureModuleResult } from "../../types/plugin/module/configure" import { DeployServiceParams } from "../../types/plugin/service/deployService" import { GetServiceStatusParams } from "../../types/plugin/service/getServiceStatus" diff --git a/garden-service/src/plugins/kubernetes/config.ts b/garden-service/src/plugins/kubernetes/config.ts index eba89e9ce0..dd9027532f 100644 --- a/garden-service/src/plugins/kubernetes/config.ts +++ b/garden-service/src/plugins/kubernetes/config.ts @@ -10,7 +10,7 @@ import * as Joi from "joi" import dedent = require("dedent") import { joiArray, joiIdentifier, joiProviderName } from "../../config/common" -import { Provider, providerConfigBaseSchema, ProviderConfig } from "../../config/project" +import { Provider, providerConfigBaseSchema, ProviderConfig } from "../../config/provider" import { containerRegistryConfigSchema, ContainerRegistryConfig } from "../container/config" import { PluginContext } from "../../plugin-context" import { deline } from "../../util/string" diff --git a/garden-service/src/plugins/kubernetes/helm/build.ts b/garden-service/src/plugins/kubernetes/helm/build.ts index e1f43e5968..e84650e493 100644 --- a/garden-service/src/plugins/kubernetes/helm/build.ts +++ b/garden-service/src/plugins/kubernetes/helm/build.ts @@ -19,7 +19,13 @@ import { BuildModuleParams, BuildResult } from "../../../types/plugin/module/bui export async function buildHelmModule({ ctx, module, log }: BuildModuleParams): Promise { const k8sCtx = ctx - const namespace = await getNamespace({ ctx: k8sCtx, log, provider: k8sCtx.provider, skipCreate: true }) + const namespace = await getNamespace({ + configStore: ctx.configStore, + log, + projectName: k8sCtx.projectName, + provider: k8sCtx.provider, + skipCreate: true, + }) const context = ctx.provider.config.context const baseModule = getBaseModule(module) diff --git a/garden-service/src/plugins/kubernetes/helm/common.ts b/garden-service/src/plugins/kubernetes/helm/common.ts index 56292cc1b1..bf1ea02603 100644 --- a/garden-service/src/plugins/kubernetes/helm/common.ts +++ b/garden-service/src/plugins/kubernetes/helm/common.ts @@ -39,10 +39,16 @@ export async function containsSource(config: HelmModuleConfig) { * Render the template in the specified Helm module (locally), and return all the resources in the chart. */ export async function getChartResources(ctx: PluginContext, module: Module, log: LogEntry) { - const k8sCtx = ctx const chartPath = await getChartPath(module) const valuesPath = getValuesPath(chartPath) - const namespace = await getNamespace({ ctx: k8sCtx, log, provider: k8sCtx.provider, skipCreate: true }) + const k8sCtx = ctx + const namespace = await getNamespace({ + configStore: k8sCtx.configStore, + log, + projectName: k8sCtx.projectName, + provider: k8sCtx.provider, + skipCreate: true, + }) const context = ctx.provider.config.context const releaseName = getReleaseName(module) @@ -269,7 +275,13 @@ async function renderHelmTemplateString( const tempFilePath = join(chartPath, "templates", cryptoRandomString({ length: 16 })) const valuesPath = getValuesPath(chartPath) const k8sCtx = ctx - const namespace = await getNamespace({ ctx: k8sCtx, log, provider: k8sCtx.provider, skipCreate: true }) + const namespace = await getNamespace({ + configStore: k8sCtx.configStore, + log, + projectName: k8sCtx.projectName, + provider: k8sCtx.provider, + skipCreate: true, + }) const releaseName = getReleaseName(module) const context = ctx.provider.config.context diff --git a/garden-service/src/plugins/kubernetes/helm/config.ts b/garden-service/src/plugins/kubernetes/helm/config.ts index eee6f6fb1c..49c9f1f243 100644 --- a/garden-service/src/plugins/kubernetes/helm/config.ts +++ b/garden-service/src/plugins/kubernetes/helm/config.ts @@ -11,12 +11,12 @@ import { find } from "lodash" import { ServiceSpec } from "../../../config/service" import { - Primitive, joiPrimitive, joiArray, joiIdentifier, joiEnvVars, joiUserIdentifier, + DeepPrimitiveMap, } from "../../../config/common" import { Module, FileCopySpec } from "../../../types/module" import { containsSource, getReleaseName } from "./common" @@ -135,7 +135,7 @@ export interface HelmServiceSpec extends ServiceSpec { tasks: HelmTaskSpec[] tests: HelmTestSpec[] version?: string - values: { [key: string]: Primitive } + values: DeepPrimitiveMap } export type HelmService = Service diff --git a/garden-service/src/plugins/kubernetes/init.ts b/garden-service/src/plugins/kubernetes/init.ts index 5a8191e830..6974fdfed1 100644 --- a/garden-service/src/plugins/kubernetes/init.ts +++ b/garden-service/src/plugins/kubernetes/init.ts @@ -132,7 +132,7 @@ export async function prepareEnvironment({ ctx, log, force, status }: PrepareEnv if (systemServiceNames.length > 0 && !systemReady) { // Install Tiller to system namespace const sysGarden = await getSystemGarden(k8sCtx.provider, variables || {}) - const sysCtx = sysGarden.getPluginContext(k8sCtx.provider.name) + const sysCtx = await sysGarden.getPluginContext(k8sCtx.provider.name) await installTiller({ ctx: sysCtx, provider: sysCtx.provider, log, force }) const namespace = await getAppNamespace(k8sCtx, log, k8sCtx.provider) diff --git a/garden-service/src/plugins/kubernetes/kubernetes-module/handlers.ts b/garden-service/src/plugins/kubernetes/kubernetes-module/handlers.ts index 92210e1270..4e443d2624 100644 --- a/garden-service/src/plugins/kubernetes/kubernetes-module/handlers.ts +++ b/garden-service/src/plugins/kubernetes/kubernetes-module/handlers.ts @@ -49,7 +49,13 @@ async function getServiceStatus( { ctx, module, log }: GetServiceStatusParams, ): Promise { const k8sCtx = ctx - const namespace = await getNamespace({ ctx: k8sCtx, log, provider: k8sCtx.provider, skipCreate: true }) + const namespace = await getNamespace({ + configStore: ctx.configStore, + log, + projectName: k8sCtx.projectName, + provider: k8sCtx.provider, + skipCreate: true, + }) const context = ctx.provider.config.context const api = await KubeApi.factory(log, context) const manifests = await getManifests(module) @@ -69,7 +75,13 @@ async function deployService( const { ctx, force, module, service, log } = params const k8sCtx = ctx - const namespace = await getNamespace({ ctx: k8sCtx, log, provider: k8sCtx.provider, skipCreate: true }) + const namespace = await getNamespace({ + configStore: ctx.configStore, + log, + projectName: k8sCtx.projectName, + provider: k8sCtx.provider, + skipCreate: true, + }) const context = ctx.provider.config.context const manifests = await getManifests(module) diff --git a/garden-service/src/plugins/kubernetes/local/config.ts b/garden-service/src/plugins/kubernetes/local/config.ts index 435b2dce38..d579d0645f 100644 --- a/garden-service/src/plugins/kubernetes/local/config.ts +++ b/garden-service/src/plugins/kubernetes/local/config.ts @@ -129,6 +129,7 @@ export async function configureProvider({ config, log, projectName }: ConfigureP const ingressClass = config.ingressClass || config.setupIngressController || undefined config = { + // Setting the name to kubernetes, so that plugins that depend on kubernetes can reference it. name: config.name, buildMode: config.buildMode, context, @@ -148,5 +149,5 @@ export async function configureProvider({ config, log, projectName }: ConfigureP _systemServices, } - return { name: config.name, config } + return { config } } diff --git a/garden-service/src/plugins/kubernetes/namespace.ts b/garden-service/src/plugins/kubernetes/namespace.ts index 55fc6b2935..219c4eb6d2 100644 --- a/garden-service/src/plugins/kubernetes/namespace.ts +++ b/garden-service/src/plugins/kubernetes/namespace.ts @@ -18,6 +18,7 @@ import { getPackageVersion, sleep } from "../../util/util" import { GetEnvironmentStatusParams } from "../../types/plugin/provider/getEnvironmentStatus" import { kubectl, KUBECTL_DEFAULT_TIMEOUT } from "./kubectl" import { LogEntry } from "../../logger/log-entry" +import { ConfigStore } from "../../config-store" const GARDEN_VERSION = getPackageVersion() type CreateNamespaceStatus = "pending" | "created" @@ -58,9 +59,17 @@ export async function createNamespace(api: KubeApi, namespace: string) { }) } +interface GetNamespaceParams { + configStore: ConfigStore, + log: LogEntry, + projectName: string, + provider: KubernetesProvider, + suffix?: string, + skipCreate?: boolean, +} + export async function getNamespace( - { ctx, log, provider, suffix, skipCreate }: - { ctx: PluginContext, log: LogEntry, provider: KubernetesProvider, suffix?: string, skipCreate?: boolean }, + { projectName, configStore: localConfigStore, log, provider, suffix, skipCreate }: GetNamespaceParams, ): Promise { let namespace @@ -69,7 +78,7 @@ export async function getNamespace( } else { // Note: The local-kubernetes always defines a namespace name, so this logic only applies to the kubernetes provider // TODO: Move this logic out to the kubernetes plugin init/validation - const localConfig = await ctx.localConfigStore.get() + const localConfig = await localConfigStore.get() const k8sConfig = localConfig.kubernetes || {} let { username, ["previous-usernames"]: previousUsernames } = k8sConfig @@ -85,7 +94,7 @@ export async function getNamespace( ) } - namespace = `${username}--${ctx.projectName}` + namespace = `${username}--${projectName}` } if (suffix) { @@ -101,11 +110,22 @@ export async function getNamespace( } export async function getAppNamespace(ctx: PluginContext, log: LogEntry, provider: KubernetesProvider) { - return getNamespace({ ctx, log, provider }) + return getNamespace({ + configStore: ctx.configStore, + log, + projectName: ctx.projectName, + provider, + }) } export function getMetadataNamespace(ctx: PluginContext, log: LogEntry, provider: KubernetesProvider) { - return getNamespace({ ctx, log, provider, suffix: "metadata" }) + return getNamespace({ + configStore: ctx.configStore, + log, + projectName: ctx.projectName, + provider, + suffix: "metadata", + }) } export async function getAllNamespaces(api: KubeApi): Promise { diff --git a/garden-service/src/plugins/kubernetes/system.ts b/garden-service/src/plugins/kubernetes/system.ts index 84a1b0def4..c5b6ed2e3a 100644 --- a/garden-service/src/plugins/kubernetes/system.ts +++ b/garden-service/src/plugins/kubernetes/system.ts @@ -11,7 +11,7 @@ import { values, find } from "lodash" import { V1Namespace } from "@kubernetes/client-node" import * as semver from "semver" -import { STATIC_DIR } from "../../constants" +import { STATIC_DIR, DEFAULT_API_VERSION } from "../../constants" import { Garden } from "../../garden" import { KubernetesProvider, KubernetesPluginContext } from "./config" import { LogEntry } from "../../logger/log-entry" @@ -41,34 +41,24 @@ export async function getSystemGarden(provider: KubernetesProvider, variables: P return Garden.factory(systemProjectPath, { environmentName: "default", config: { - dirname: "system", path: systemProjectPath, - project: { - apiVersion: "garden.io/v0", - name: systemNamespace, - environmentDefaults: { - providers: [], - variables: {}, + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: systemNamespace, + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { + name: "local-kubernetes", + context: provider.config.context, + namespace: systemNamespace, + _system: systemSymbol, + _systemServices: [], }, - defaultEnvironment: "default", - environments: [ - { - name: "default", - providers: [ - { - name: provider.name, - ...provider.config, - // Note: this means we can't build images as part of system services - deploymentRegistry: undefined, - namespace: systemNamespace, - _system: systemSymbol, - _systemServices: [], - }, - ], - variables, - }, - ], - }, + ], + variables, }, }) } @@ -139,8 +129,9 @@ export async function getSystemServiceStatus( let dashboardPages: DashboardPage[] = [] const sysGarden = await getSystemGarden(ctx.provider, variables) + const actions = await sysGarden.getActionHelper() - const serviceStatuses = await sysGarden.actions.getServiceStatuses({ log, serviceNames }) + const serviceStatuses = await actions.getServiceStatuses({ log, serviceNames }) const state = combineStates(values(serviceStatuses).map(s => s.state || "unknown")) // Add the Kubernetes dashboard to the Garden dashboard @@ -195,7 +186,8 @@ export async function prepareSystemServices( // Deploy enabled system services if (serviceNames.length > 0) { - const results = await sysGarden.actions.deployServices({ + const actions = await sysGarden.getActionHelper() + const results = await actions.deployServices({ log, serviceNames, force, diff --git a/garden-service/src/plugins/local/local-docker-swarm.ts b/garden-service/src/plugins/local/local-docker-swarm.ts index 4bd44cd84e..6cd995df0b 100644 --- a/garden-service/src/plugins/local/local-docker-swarm.ts +++ b/garden-service/src/plugins/local/local-docker-swarm.ts @@ -75,7 +75,7 @@ export const gardenPlugin = (): GardenPlugin => ({ const opts: any = { Name: getSwarmServiceName(ctx, service.name), Labels: { - environment: ctx.environment.name, + environment: ctx.environmentName, provider: pluginName, }, TaskTemplate: { diff --git a/garden-service/src/plugins/local/local-google-cloud-functions.ts b/garden-service/src/plugins/local/local-google-cloud-functions.ts index 7d8bd03d61..640ae9a785 100644 --- a/garden-service/src/plugins/local/local-google-cloud-functions.ts +++ b/garden-service/src/plugins/local/local-google-cloud-functions.ts @@ -6,17 +6,13 @@ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ -import { ConfigureModuleParams } from "../../types/plugin/module/configure" +import { ConfigureModuleParams, ConfigureProviderParams } from "../../types/plugin/params" import { join } from "path" -import { - GcfModule, - configureGcfModule, -} from "../google/google-cloud-functions" -import { - GardenPlugin, -} from "../../types/plugin/plugin" -import { STATIC_DIR } from "../../constants" +import { GcfModule, configureGcfModule } from "../google/google-cloud-functions" +import { GardenPlugin } from "../../types/plugin/plugin" +import { STATIC_DIR, DEFAULT_API_VERSION } from "../../constants" import { ServiceConfig } from "../../config/service" +import { ContainerModuleConfig } from "../container/config" import { ContainerServiceSpec, ServicePortProtocol, @@ -29,7 +25,39 @@ const emulatorBaseModulePath = join(STATIC_DIR, emulatorModuleName) const emulatorPort = 8010 export const gardenPlugin = (): GardenPlugin => ({ - modules: [emulatorBaseModulePath], + actions: { + async configureProvider({ config }: ConfigureProviderParams) { + const emulatorConfig: ContainerModuleConfig = { + allowPublish: false, + apiVersion: DEFAULT_API_VERSION, + build: { + dependencies: [], + }, + description: "Base container for running Google Cloud Functions emulator", + name: "local-gcf-container", + path: emulatorBaseModulePath, + outputs: {}, + serviceConfigs: [], + spec: { + build: { + dependencies: [], + }, + buildArgs: {}, + services: [], + tasks: [], + tests: [], + }, + taskConfigs: [], + testConfigs: [], + type: "container", + } + + return { + config, + moduleConfigs: [emulatorConfig], + } + }, + }, moduleActions: { "google-cloud-function": { @@ -80,7 +108,7 @@ export const gardenPlugin = (): GardenPlugin => ({ }) return { - apiVersion: "garden.io/v0", + apiVersion: DEFAULT_API_VERSION, allowPublish: true, build: { command: [], diff --git a/garden-service/src/plugins/maven-container/maven-container.ts b/garden-service/src/plugins/maven-container/maven-container.ts index 2c379322bb..3a8940656d 100644 --- a/garden-service/src/plugins/maven-container/maven-container.ts +++ b/garden-service/src/plugins/maven-container/maven-container.ts @@ -27,7 +27,7 @@ import { containerHelpers } from "../container/helpers" import { STATIC_DIR } from "../../constants" import { xml2json } from "xml-js" import { containerModuleSpecSchema } from "../container/config" -import { providerConfigBaseSchema } from "../../config/project" +import { providerConfigBaseSchema } from "../../config/provider" import { openJdks } from "./openjdk" import { maven } from "./maven" import { LogEntry } from "../../logger/log-entry" @@ -116,7 +116,7 @@ async function describeType() { export async function configureMavenContainerModule(params: ConfigureModuleParams) { const { moduleConfig } = params - let containerConfig: ContainerModuleConfig = { ...moduleConfig } + let containerConfig: ContainerModuleConfig = { ...moduleConfig, type: "container" } containerConfig.spec = omit(moduleConfig.spec, Object.keys(mavenKeys)) const jdkVersion = moduleConfig.spec.jdkVersion! diff --git a/garden-service/src/plugins/openfaas/local.ts b/garden-service/src/plugins/openfaas/local.ts new file mode 100644 index 0000000000..24bbd4d5cb --- /dev/null +++ b/garden-service/src/plugins/openfaas/local.ts @@ -0,0 +1,19 @@ +/* + * Copyright (C) 2018 Garden Technologies, Inc. + * + * This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. + */ + +import { GardenPlugin } from "../../types/plugin/plugin" +import { gardenPlugin as o6sPlugin } from "./openfaas" + +export const name = "local-openfaas" + +// TODO: avoid having to configure separate plugins, by allowing for this scenario in the plugin mechanism +export function gardenPlugin(): GardenPlugin { + const plugin = o6sPlugin() + plugin.dependencies = ["local-kubernetes"] + return plugin +} diff --git a/garden-service/src/plugins/openfaas/openfaas.ts b/garden-service/src/plugins/openfaas/openfaas.ts index 85e6ef4b57..897da2296f 100644 --- a/garden-service/src/plugins/openfaas/openfaas.ts +++ b/garden-service/src/plugins/openfaas/openfaas.ts @@ -6,15 +6,15 @@ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ +import dedent = require("dedent") import * as Joi from "joi" import { join } from "path" import { resolve as urlResolve } from "url" -import { STATIC_DIR } from "../../constants" -import { PluginError, ConfigurationError } from "../../exceptions" -import { Garden } from "../../garden" +import { ConfigurationError } from "../../exceptions" import { PluginContext } from "../../plugin-context" import { joiArray, PrimitiveMap, joiProviderName } from "../../config/common" import { Module } from "../../types/module" +import { ConfigureProviderResult } from "../../types/plugin/outputs" import { ServiceStatus, ServiceIngress, Service } from "../../types/service" import { ExecModuleSpec, @@ -25,30 +25,27 @@ import { } from "../exec" import { KubernetesProvider } from "../kubernetes/config" import { getNamespace, getAppNamespace } from "../kubernetes/namespace" -import { every, values } from "lodash" import { dumpYaml, findByName } from "../../util/util" import { KubeApi } from "../kubernetes/api" import { waitForResources, checkWorkloadStatus } from "../kubernetes/status" -import { systemSymbol } from "../kubernetes/system" import { CommonServiceSpec } from "../../config/service" import { GardenPlugin } from "../../types/plugin/plugin" -import { Provider, providerConfigBaseSchema } from "../../config/project" +import { Provider, providerConfigBaseSchema, ProviderConfig } from "../../config/provider" import { faasCli } from "./faas-cli" -import { CleanupEnvironmentParams } from "../../types/plugin/provider/cleanupEnvironment" -import dedent = require("dedent") import { getAllLogs } from "../kubernetes/logs" -import { installTiller, checkTillerStatus } from "../kubernetes/helm/tiller" import { LogEntry } from "../../logger/log-entry" import { BuildModuleParams } from "../../types/plugin/module/build" import { DeployServiceParams } from "../../types/plugin/service/deployService" import { GetServiceStatusParams } from "../../types/plugin/service/getServiceStatus" -import { GetEnvironmentStatusParams } from "../../types/plugin/provider/getEnvironmentStatus" -import { PrepareEnvironmentParams } from "../../types/plugin/provider/prepareEnvironment" import { ConfigureModuleParams, ConfigureModuleResult } from "../../types/plugin/module/configure" import { GetServiceLogsParams } from "../../types/plugin/service/getServiceLogs" import { DeleteServiceParams } from "../../types/plugin/service/deleteService" +import { HelmModuleConfig } from "../kubernetes/helm/config" +import { keyBy, union } from "lodash" +import { DEFAULT_API_VERSION } from "../../constants" +import { ExecModuleConfig } from "../exec" +import { ConfigureProviderParams } from "../../types/plugin/provider/configureProvider" -const systemProjectPath = join(STATIC_DIR, "openfaas", "system") export const stackFilename = "stack.yml" export interface OpenFaasModuleSpec extends ExecModuleSpec { @@ -78,7 +75,7 @@ export interface OpenFaasModule extends Module { } -export interface OpenFaasConfig extends Provider { +export interface OpenFaasConfig extends ProviderConfig { hostname: string } @@ -113,203 +110,267 @@ async function describeType() { export function gardenPlugin(): GardenPlugin { return { configSchema, - modules: [join(STATIC_DIR, "openfaas", "templates")], + dependencies: ["kubernetes"], actions: { - async getEnvironmentStatus({ ctx, log }: GetEnvironmentStatusParams) { - const openFaasCtx = ctx - const ofGarden = await getOpenFaasGarden(openFaasCtx, log) - const status = await ofGarden.actions.getStatus({ log }) - const envReady = every(values(status.providers).map(s => s.ready)) - const servicesReady = every(values(status.services).map(s => s.state === "ready")) - - // TODO: get rid of this convoluted nested Garden setup - const k8sProviderName = getK8sProvider(openFaasCtx).name - const ofCtx = (await ofGarden.getPluginContext(k8sProviderName)) - const ofK8sProvider = getK8sProvider(ofCtx) - const tillerState = await checkTillerStatus(ofCtx, ofK8sProvider, log) - - return { - ready: envReady && servicesReady && tillerState === "ready", - detail: status.services, - } + configureProvider, + }, + moduleActions: { + openfaas: { + describeType, + configure: configureModule, + getBuildStatus: getExecModuleBuildStatus, + build: buildModule, + // TODO: design and implement a proper test flow for openfaas functions + testModule: testExecModule, + getServiceStatus, + getServiceLogs, + deployService, + deleteService, }, + }, + } +} - async prepareEnvironment({ ctx, force, log }: PrepareEnvironmentParams) { - const openFaasCtx = ctx - // TODO: refactor to dedupe similar code in local-kubernetes - const ofGarden = await getOpenFaasGarden(openFaasCtx, log) +const templateModuleConfig: ExecModuleConfig = { + allowPublish: false, + apiVersion: DEFAULT_API_VERSION, + build: { + dependencies: [], + }, + description: "OpenFaaS templates for building functions", + name: "templates", + path: __dirname, + repositoryUrl: "https://github.com/openfaas/templates.git#master", + outputs: {}, + serviceConfigs: [], + spec: { + build: { + command: [], + dependencies: [], + }, + env: {}, + tasks: [], + tests: [], + }, + taskConfigs: [], + testConfigs: [], + type: "exec", +} - await ofGarden.actions.prepareEnvironment({ force, log }) +async function configureProvider( + { log, config, projectName, dependencies, configStore }: ConfigureProviderParams, +): Promise { + const k8sProvider = getK8sProvider(dependencies) - // TODO: avoid this coupling (requires work on plugin dependencies) - const k8sProviderName = getK8sProvider(openFaasCtx).name - const ofCtx = (await ofGarden.getPluginContext(k8sProviderName)) - const ofK8sProvider = getK8sProvider(ofCtx) - await installTiller({ ctx, provider: ofK8sProvider, log, force }) + if (!config.hostname) { + if (!k8sProvider.config.defaultHostname) { + throw new ConfigurationError( + `openfaas: Must configure hostname if no default hostname is configured on Kubernetes provider.`, + { config }, + ) + } - const results = await ofGarden.actions.deployServices({ log, force }) - const failed = values(results.taskResults).filter(r => !!r.error).length + config.hostname = k8sProvider.config.defaultHostname + } - if (failed) { - throw new PluginError(`openfaas: ${failed} errors occurred when configuring environment`, { - results, - }) - } + const namespace = await getNamespace({ + configStore, + log, + provider: k8sProvider, + projectName, + skipCreate: true, + }) - return {} - }, + // Need to scope the release name, because the OpenFaaS Helm chart installs some cluster-wide resources + // that could conflict across projects/users. + const releaseName = `${namespace}--openfaas` - async cleanupEnvironment({ ctx, log }: CleanupEnvironmentParams) { - const ofGarden = await getOpenFaasGarden(ctx, log) - await ofGarden.actions.cleanupEnvironment({ log }) - return {} + const systemModule: HelmModuleConfig = { + allowPublish: false, + apiVersion: DEFAULT_API_VERSION, + build: { + dependencies: [], + }, + description: "OpenFaaS runtime", + name: "system", + outputs: {}, + path: __dirname, + serviceConfigs: [], + taskConfigs: [], + testConfigs: [], + type: "helm", + spec: { + repo: "https://openfaas.github.io/faas-netes/", + chart: "openfaas", + chartPath: ".", + dependencies: [], + skipDeploy: false, + tasks: [], + tests: [], + version: "1.7.0", + releaseName, + values: { + exposeServices: false, + functionNamespace: namespace, + ingress: { + enabled: true, + hosts: [ + { + host: config.hostname, + serviceName: "gateway", + servicePort: 8080, + path: "/function/", + }, + { + host: config.hostname, + serviceName: "gateway", + servicePort: 8080, + path: "/system/", + }, + ], + }, + faasnetesd: { + imagePullPolicy: "IfNotPresent", + }, + securityContext: false, }, }, - moduleActions: { - openfaas: { - describeType, + } - async configure( - { ctx, log, moduleConfig }: ConfigureModuleParams, - ): Promise { - moduleConfig.build.dependencies.push({ - name: "templates", - plugin: "openfaas", - copy: [{ - source: "template", - target: ".", - }], - }) - - moduleConfig.serviceConfigs = [{ - dependencies: [], - hotReloadable: false, - name: moduleConfig.name, - spec: { - name: moduleConfig.name, - dependencies: [], - }, - }] + const moduleConfigs = [systemModule, templateModuleConfig] - moduleConfig.testConfigs = moduleConfig.spec.tests.map(t => ({ - name: t.name, - dependencies: t.dependencies, - spec: t, - timeout: t.timeout, - })) + return { config, moduleConfigs } +} - moduleConfig.outputs = { - endpoint: await getInternalServiceUrl(ctx, log, moduleConfig), - } +async function configureModule( + { ctx, log, moduleConfig }: ConfigureModuleParams, +): Promise { + moduleConfig.build.dependencies.push({ + name: "templates", + plugin: ctx.provider.name, + copy: [{ + source: "template", + target: ".", + }], + }) - return moduleConfig - }, + const dependencies = [`${ctx.provider.name}--system`] - getBuildStatus: getExecModuleBuildStatus, + moduleConfig.serviceConfigs = [{ + dependencies, + hotReloadable: false, + name: moduleConfig.name, + spec: { + name: moduleConfig.name, + dependencies, + }, + }] - async build({ ctx, log, module }: BuildModuleParams) { - await writeStackFile(ctx, module, {}) + moduleConfig.testConfigs = moduleConfig.spec.tests.map(t => ({ + name: t.name, + dependencies: union(t.dependencies, dependencies), + spec: t, + timeout: t.timeout, + })) - const buildLog = await faasCli.stdout({ - log, - cwd: module.buildPath, - args: ["build", "-f", stackFilename], - }) + moduleConfig.outputs = { + endpoint: await getInternalServiceUrl(ctx, log, moduleConfig), + } - return { fresh: true, buildLog } - }, + return moduleConfig +} - // TODO: design and implement a proper test flow for openfaas functions - testModule: testExecModule, +async function buildModule({ ctx, log, module }: BuildModuleParams) { + await writeStackFile(ctx.provider, module, {}) - getServiceStatus, + const buildLog = await faasCli.stdout({ + log, + cwd: module.buildPath, + args: ["build", "-f", stackFilename], + }) + + return { fresh: true, buildLog } +} - async getServiceLogs(params: GetServiceLogsParams) { - const { ctx, log, service } = params - const k8sProvider = getK8sProvider(ctx) - const context = k8sProvider.config.context - const namespace = await getAppNamespace(ctx, log, k8sProvider) +async function getServiceLogs(params: GetServiceLogsParams) { + const { ctx, log, service } = params + const k8sProvider = getK8sProvider(ctx.provider.dependencies) + const context = k8sProvider.config.context + const namespace = await getAppNamespace(ctx, log, k8sProvider) - const api = await KubeApi.factory(log, k8sProvider.config.context) - const resources = await getResources(api, service, namespace) + const api = await KubeApi.factory(log, k8sProvider.config.context) + const resources = await getResources(api, service, namespace) - return getAllLogs({ ...params, context, namespace, resources }) - }, + return getAllLogs({ ...params, context, namespace, resources }) +} - async deployService(params: DeployServiceParams): Promise { - const { ctx, module, service, log, runtimeContext } = params - - const openFaasCtx = ctx - - // write the stack file again with environment variables - await writeStackFile(openFaasCtx, module, runtimeContext.envVars) - - // use faas-cli to do the deployment - await faasCli.stdout({ - log, - cwd: module.buildPath, - args: ["deploy", "-f", stackFilename], - }) - - // wait until deployment is ready - const k8sProvider = getK8sProvider(openFaasCtx) - const namespace = await getAppNamespace(openFaasCtx, log, k8sProvider) - const api = await KubeApi.factory(log, k8sProvider.config.context) - const resources = await getResources(api, service, namespace) - - await waitForResources({ - ctx: openFaasCtx, - provider: k8sProvider, - serviceName: service.name, - log, - resources, - }) - - // TODO: avoid duplicate work here - return getServiceStatus(params) - }, +async function deployService(params: DeployServiceParams): Promise { + const { ctx, module, service, log, runtimeContext } = params - async deleteService(params: DeleteServiceParams): Promise { - const { ctx, log, service, runtimeContext } = params - const openFaasCtx = ctx - let status - let found = true - - try { - status = await getServiceStatus({ - ctx: openFaasCtx, - log, - service, - runtimeContext, - module: service.module, - hotReload: false, - }) - - found = !!status.state - - await faasCli.stdout({ - log, - cwd: service.module.buildPath, - args: ["remove", "-f", stackFilename], - }) - - } catch (err) { - found = false - } - - if (log) { - found ? log.setSuccess("Service deleted") : log.setWarn("Service not deployed") - } - - return status - }, - }, - }, + // write the stack file again with environment variables + await writeStackFile(ctx.provider, module, runtimeContext.envVars) + + // use faas-cli to do the deployment + await faasCli.stdout({ + log, + cwd: module.buildPath, + args: ["deploy", "-f", stackFilename], + }) + + // wait until deployment is ready + const k8sProvider = getK8sProvider(ctx.provider.dependencies) + const namespace = await getAppNamespace(ctx, log, k8sProvider) + const api = await KubeApi.factory(log, k8sProvider.config.context) + const resources = await getResources(api, service, namespace) + + await waitForResources({ + ctx, + provider: k8sProvider, + serviceName: service.name, + log, + resources, + }) + + // TODO: avoid duplicate work here + return getServiceStatus(params) +} + +async function deleteService(params: DeleteServiceParams): Promise { + const { ctx, log, service, runtimeContext } = params + let status + let found = true + + try { + status = await getServiceStatus({ + ctx, + log, + service, + runtimeContext, + module: service.module, + hotReload: false, + }) + + found = !!status.state + + await faasCli.stdout({ + log, + cwd: service.module.buildPath, + args: ["remove", "-f", stackFilename], + }) + + } catch (err) { + found = false + } + + if (log) { + found ? log.setSuccess("Service deleted") : log.setWarn("Service not deployed") } + + return status } async function writeStackFile( - ctx: PluginContext, module: OpenFaasModule, envVars: PrimitiveMap, + provider: OpenFaasProvider, module: OpenFaasModule, envVars: PrimitiveMap, ) { const image = getImageName(module) @@ -318,7 +379,7 @@ async function writeStackFile( return dumpYaml(stackPath, { provider: { name: "faas", - gateway: getExternalGatewayUrl(ctx), + gateway: getExternalGatewayUrl(provider), }, functions: { [module.name]: { @@ -338,10 +399,10 @@ async function getResources(api: KubeApi, service: OpenFaasService, namespace: s async function getServiceStatus({ ctx, module, service, log }: GetServiceStatusParams) { const openFaasCtx = ctx - const k8sProvider = getK8sProvider(openFaasCtx) + const k8sProvider = getK8sProvider(ctx.provider.dependencies) const ingresses: ServiceIngress[] = [{ - hostname: getExternalGatewayHostname(openFaasCtx.provider, k8sProvider), + hostname: ctx.provider.config.hostname, path: getServicePath(module), port: k8sProvider.config.ingressHttpPort, protocol: "http", @@ -378,46 +439,13 @@ function getImageName(module: OpenFaasModule) { return `${module.name || module.spec.image}:${module.version.versionString}` } -// NOTE: we're currently not using the CRD/operator, but might change that in the future -// -// async function createFunctionObject(service: OpenFaasService, namespace: string): Promise { -// const image = await getImageName(service.module) - -// return { -// apiVersion: "openfaas.com/v1alpha2", -// kind: "Function", -// metadata: { -// name: service.name, -// namespace, -// }, -// spec: { -// name: service.name, -// image, -// labels: { -// "com.openfaas.scale.min": "1", -// "com.openfaas.scale.max": "5", -// }, -// environment: { -// write_debug: "true", -// }, -// limits: { -// cpu: DEFAULT_CPU_LIMIT, -// memory: DEFAULT_MEMORY_LIMIT, -// }, -// requests: { -// cpu: DEFAULT_CPU_REQUEST, -// memory: DEFAULT_MEMORY_REQUEST, -// }, -// }, -// } -// } - -function getK8sProvider(ctx: PluginContext): KubernetesProvider { - const provider = (ctx.providers["local-kubernetes"] || ctx.providers.kubernetes) +function getK8sProvider(providers: Provider[]): KubernetesProvider { + const providerMap = keyBy(providers, "name") + const provider = (providerMap["local-kubernetes"] || providerMap.kubernetes) if (!provider) { throw new ConfigurationError(`openfaas requires a kubernetes (or local-kubernetes) provider to be configured`, { - configuredProviders: Object.keys(ctx.providers), + configuredProviders: Object.keys(providers), }) } @@ -429,29 +457,20 @@ function getServicePath(config: OpenFaasModuleConfig) { } async function getInternalGatewayUrl(ctx: PluginContext, log: LogEntry) { - const k8sProvider = getK8sProvider(ctx) - const namespace = await getOpenfaasNamespace(ctx, log, k8sProvider, true) + const k8sProvider = getK8sProvider(ctx.provider.dependencies) + const namespace = await getNamespace({ + configStore: ctx.configStore, + log, + projectName: ctx.projectName, + provider: k8sProvider, + skipCreate: true, + }) return `http://gateway.${namespace}.svc.cluster.local:8080` } -function getExternalGatewayHostname(provider: OpenFaasProvider, k8sProvider: KubernetesProvider) { - const hostname = provider.config.hostname || k8sProvider.config.defaultHostname - - if (!hostname) { - throw new ConfigurationError( - `openfaas: Must configure hostname if no default hostname is configured on Kubernetes provider.`, - { - config: provider, - }, - ) - } - - return hostname -} - -function getExternalGatewayUrl(ctx: PluginContext) { - const k8sProvider = getK8sProvider(ctx) - const hostname = getExternalGatewayHostname(ctx.provider, k8sProvider) +function getExternalGatewayUrl(provider: OpenFaasProvider) { + const k8sProvider = getK8sProvider(provider.dependencies) + const hostname = provider.config.hostname const ingressPort = k8sProvider.config.ingressHttpPort return `http://${hostname}:${ingressPort}` } @@ -459,58 +478,3 @@ function getExternalGatewayUrl(ctx: PluginContext) { async function getInternalServiceUrl(ctx: PluginContext, log: LogEntry, config: OpenFaasModuleConfig) { return urlResolve(await getInternalGatewayUrl(ctx, log), getServicePath(config)) } - -async function getOpenfaasNamespace( - ctx: PluginContext, log: LogEntry, k8sProvider: KubernetesProvider, skipCreate?: boolean, -) { - return getNamespace({ ctx, log, provider: k8sProvider, skipCreate, suffix: "openfaas" }) -} - -export async function getOpenFaasGarden(ctx: PluginContext, log: LogEntry): Promise { - // TODO: figure out good way to retrieve namespace from kubernetes plugin through an exposed interface - // (maybe allow plugins to expose arbitrary data on the Provider object?) - const k8sProvider = getK8sProvider(ctx) - const namespace = await getOpenfaasNamespace(ctx, log, k8sProvider, true) - const functionNamespace = await getAppNamespace(ctx, log, k8sProvider) - - const hostname = getExternalGatewayHostname(ctx.provider, k8sProvider) - - // TODO: allow passing variables/parameters here to be parsed as part of the garden.yml project config - // (this would allow us to use a garden.yml for the project config, instead of speccing it here) - return Garden.factory(systemProjectPath, { - environmentName: "default", - config: { - dirname: "system", - path: systemProjectPath, - project: { - apiVersion: "garden.io/v0", - name: `${ctx.projectName}-openfaas`, - environmentDefaults: { - providers: [], - variables: {}, - }, - defaultEnvironment: "default", - environments: [ - { - name: "default", - providers: [ - { - ...k8sProvider.config, - namespace, - // TODO: this is clumsy, we should find a better way to configure this - _system: systemSymbol, - }, - ], - variables: { - "function-namespace": functionNamespace, - "gateway-hostname": hostname, - // Need to scope the release name, because the OpenFaaS Helm chart installs some cluster-wide resources - // that could conflict across projects/users. - "release-name": `${functionNamespace}--openfaas`, - }, - }, - ], - }, - }, - }) -} diff --git a/garden-service/src/plugins/plugins.ts b/garden-service/src/plugins/plugins.ts index ba1b9d4bde..af648aecd6 100644 --- a/garden-service/src/plugins/plugins.ts +++ b/garden-service/src/plugins/plugins.ts @@ -16,6 +16,7 @@ const kubernetes = require("./kubernetes/kubernetes") const localKubernetes = require("./kubernetes/local/local") const npmPackage = require("./npm-package") const gae = require("./google/google-app-engine") +const localOpenfaas = require("./openfaas/local") const openfaas = require("./openfaas/openfaas") const mavenContainer = require("./maven-container/maven-container") @@ -29,6 +30,7 @@ export const builtinPlugins = mapValues({ "local-kubernetes": localKubernetes, "npm-package": npmPackage, "google-app-engine": gae, + "local-openfaas": localOpenfaas, openfaas, "maven-container": mavenContainer, }, (m => m.gardenPlugin)) diff --git a/garden-service/src/task-graph.ts b/garden-service/src/task-graph.ts index debc42a02d..c353bd0acc 100644 --- a/garden-service/src/task-graph.ts +++ b/garden-service/src/task-graph.ts @@ -24,6 +24,7 @@ export interface TaskResult { type: string description: string key: string + name: string output?: any dependencyResults?: TaskResults error?: Error @@ -165,7 +166,7 @@ export class TaskGraph { const key = node.getKey() const description = node.getDescription() - let result: TaskResult = { type, description, key: task.getKey() } + let result: TaskResult = { type, description, key: task.getKey(), name: task.getName() } try { this.logTask(node) @@ -449,12 +450,13 @@ class TaskNode { } } - async process(dependencyResults: TaskResults) { + async process(dependencyResults: TaskResults): Promise { const output = await this.task.process(dependencyResults) return { type: this.getType(), key: this.getKey(), + name: this.task.getName(), description: this.getDescription(), output, dependencyResults, diff --git a/garden-service/src/tasks/base.ts b/garden-service/src/tasks/base.ts index 97112078cf..7508af611c 100644 --- a/garden-service/src/tasks/base.ts +++ b/garden-service/src/tasks/base.ts @@ -10,10 +10,9 @@ import { TaskResults } from "../task-graph" import { ModuleVersion } from "../vcs/vcs" import { v1 as uuidv1 } from "uuid" import { Garden } from "../garden" -import { DependencyGraphNodeType } from "../config-graph" import { LogEntry } from "../logger/log-entry" -export type TaskType = "build" | "deploy" | "publish" | "hot-reload" | "task" | "test" +export type TaskType = "build" | "deploy" | "publish" | "hot-reload" | "resolve-provider" | "task" | "test" export class TaskDefinitionError extends Error { } @@ -30,7 +29,6 @@ export interface TaskParams { export abstract class BaseTask { abstract type: TaskType - abstract depType: DependencyGraphNodeType garden: Garden log: LogEntry uid: string @@ -52,7 +50,7 @@ export abstract class BaseTask { return this.dependencies } - protected abstract getName(): string + abstract getName(): string getKey(): string { return makeBaseKey(this.type, this.getName()) diff --git a/garden-service/src/tasks/build.ts b/garden-service/src/tasks/build.ts index b4de426b54..a2af1ccaaa 100644 --- a/garden-service/src/tasks/build.ts +++ b/garden-service/src/tasks/build.ts @@ -12,7 +12,6 @@ import { Module, getModuleKey } from "../types/module" import { BuildResult } from "../types/plugin/module/build" import { BaseTask, TaskType } from "../tasks/base" import { Garden } from "../garden" -import { DependencyGraphNodeType } from "../config-graph" import { LogEntry } from "../logger/log-entry" export interface BuildTaskParams { @@ -26,7 +25,6 @@ export interface BuildTaskParams { export class BuildTask extends BaseTask { type: TaskType = "build" - depType: DependencyGraphNodeType = "build" private module: Module private fromWatch: boolean @@ -41,7 +39,7 @@ export class BuildTask extends BaseTask { async getDependencies() { const dg = await this.garden.getConfigGraph() - const deps = (await dg.getDependencies(this.depType, this.getName(), false)).build + const deps = (await dg.getDependencies("build", this.getName(), false)).build return Bluebird.map(deps, async (m: Module) => { return new BuildTask({ @@ -55,7 +53,7 @@ export class BuildTask extends BaseTask { }) } - protected getName() { + getName() { return getModuleKey(this.module.name, this.module.plugin) } @@ -65,6 +63,7 @@ export class BuildTask extends BaseTask { async process(): Promise { const module = this.module + const actions = await this.garden.getActionHelper() const log = this.log.info({ section: this.getName(), @@ -81,7 +80,7 @@ export class BuildTask extends BaseTask { if (!this.force) { log.setState({ msg: `Getting build status...` }) - const status = await this.garden.actions.getBuildStatus({ log: this.log, module }) + const status = await actions.getBuildStatus({ log: this.log, module }) if (status.ready) { logSuccess() @@ -93,7 +92,7 @@ export class BuildTask extends BaseTask { let result: BuildResult try { - result = await this.garden.actions.build({ + result = await actions.build({ module, log, }) diff --git a/garden-service/src/tasks/deploy.ts b/garden-service/src/tasks/deploy.ts index 08538b8871..1605428555 100644 --- a/garden-service/src/tasks/deploy.ts +++ b/garden-service/src/tasks/deploy.ts @@ -14,8 +14,8 @@ import { BaseTask, TaskType } from "./base" import { Service, ServiceStatus, getServiceRuntimeContext, getIngressUrl } from "../types/service" import { Garden } from "../garden" import { TaskTask } from "./task" -import { DependencyGraphNodeType, ConfigGraph } from "../config-graph" import { BuildTask } from "./build" +import { ConfigGraph } from "../config-graph" export interface DeployTaskParams { garden: Garden @@ -30,7 +30,6 @@ export interface DeployTaskParams { export class DeployTask extends BaseTask { type: TaskType = "deploy" - depType: DependencyGraphNodeType = "service" private graph: ConfigGraph private service: Service @@ -53,8 +52,8 @@ export class DeployTask extends BaseTask { const dg = this.graph // We filter out service dependencies on services configured for hot reloading (if any) - const deps = await dg.getDependencies(this.depType, this.getName(), false, - (depNode) => !(depNode.type === this.depType && includes(this.hotReloadServiceNames, depNode.name))) + const deps = await dg.getDependencies("service", this.getName(), false, + (depNode) => !(depNode.type === "service" && includes(this.hotReloadServiceNames, depNode.name))) const deployTasks = await Bluebird.map(deps.service, async (service) => { return new DeployTask({ @@ -96,7 +95,7 @@ export class DeployTask extends BaseTask { } } - protected getName() { + getName() { return this.service.name } @@ -116,8 +115,9 @@ export class DeployTask extends BaseTask { const hotReload = includes(this.hotReloadServiceNames, this.service.name) const runtimeContext = await getServiceRuntimeContext(this.garden, this.graph, this.service) + const actions = await this.garden.getActionHelper() - let status = await this.garden.actions.getServiceStatus({ + let status = await actions.getServiceStatus({ service: this.service, log, hotReload, @@ -140,7 +140,7 @@ export class DeployTask extends BaseTask { log.setState(`Deploying version ${versionString}...`) try { - status = await this.garden.actions.deployService({ + status = await actions.deployService({ service: this.service, runtimeContext, log, diff --git a/garden-service/src/tasks/hot-reload.ts b/garden-service/src/tasks/hot-reload.ts index 10f017698f..69202fbe8b 100644 --- a/garden-service/src/tasks/hot-reload.ts +++ b/garden-service/src/tasks/hot-reload.ts @@ -11,7 +11,7 @@ import { LogEntry } from "../logger/log-entry" import { BaseTask, TaskType } from "./base" import { Service, getServiceRuntimeContext } from "../types/service" import { Garden } from "../garden" -import { DependencyGraphNodeType, ConfigGraph } from "../config-graph" +import { ConfigGraph } from "../config-graph" interface Params { garden: Garden @@ -23,7 +23,6 @@ interface Params { export class HotReloadTask extends BaseTask { type: TaskType = "hot-reload" - depType: DependencyGraphNodeType = "service" private graph: ConfigGraph private service: Service @@ -36,7 +35,7 @@ export class HotReloadTask extends BaseTask { this.service = service } - protected getName() { + getName() { return this.service.name } @@ -52,9 +51,10 @@ export class HotReloadTask extends BaseTask { }) const runtimeContext = await getServiceRuntimeContext(this.garden, this.graph, this.service) + const actions = await this.garden.getActionHelper() try { - await this.garden.actions.hotReloadService({ log, service: this.service, runtimeContext }) + await actions.hotReloadService({ log, service: this.service, runtimeContext }) } catch (err) { log.setError() throw err diff --git a/garden-service/src/tasks/publish.ts b/garden-service/src/tasks/publish.ts index 20fde58f53..2af9b39dd0 100644 --- a/garden-service/src/tasks/publish.ts +++ b/garden-service/src/tasks/publish.ts @@ -12,7 +12,6 @@ import { Module } from "../types/module" import { PublishResult } from "../types/plugin/module/publishModule" import { BaseTask, TaskType } from "../tasks/base" import { Garden } from "../garden" -import { DependencyGraphNodeType } from "../config-graph" import { LogEntry } from "../logger/log-entry" export interface PublishTaskParams { @@ -24,7 +23,6 @@ export interface PublishTaskParams { export class PublishTask extends BaseTask { type: TaskType = "publish" - depType: DependencyGraphNodeType = "publish" private module: Module private forceBuild: boolean @@ -71,9 +69,11 @@ export class PublishTask extends BaseTask { status: "active", }) + const actions = await this.garden.getActionHelper() + let result: PublishResult try { - result = await this.garden.actions.publishModule({ module: this.module, log }) + result = await actions.publishModule({ module: this.module, log }) } catch (err) { log.setError() throw err diff --git a/garden-service/src/tasks/resolve-provider.ts b/garden-service/src/tasks/resolve-provider.ts new file mode 100644 index 0000000000..a209356150 --- /dev/null +++ b/garden-service/src/tasks/resolve-provider.ts @@ -0,0 +1,120 @@ +/* + * Copyright (C) 2018 Garden Technologies, Inc. + * + * This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. + */ + +import { BaseTask, TaskParams, TaskType } from "./base" +import { ProviderConfig, Provider, getProviderDependencies, providerFromConfig } from "../config/provider" +import { resolveTemplateStrings } from "../template-string" +import { ConfigurationError } from "../exceptions" +import { keyBy } from "lodash" +import { TaskResults } from "../task-graph" +import { ProviderConfigContext } from "../config/config-context" +import { ModuleConfig } from "../config/module" +import { GardenPlugin } from "../types/plugin/plugin" +import { validateWithPath } from "../config/common" + +interface Params extends TaskParams { + plugin: GardenPlugin + config: ProviderConfig +} + +/** + * Resolves the configuration for the specified provider. + */ +export class ResolveProviderTask extends BaseTask { + type: TaskType = "resolve-provider" + + private config: ProviderConfig + private plugin: GardenPlugin + + constructor(params: Params) { + super(params) + this.config = params.config + this.plugin = params.plugin + } + + getName() { + return this.config.name + } + + getDescription() { + return `resolving provider ${this.getName()}` + } + + async getDependencies() { + const deps = await getProviderDependencies(this.plugin, this.config) + + const rawProviderConfigs = keyBy(this.garden.getRawProviderConfigs(), "name") + + return deps.map(providerName => { + const config = rawProviderConfigs[providerName] + + if (!config) { + throw new ConfigurationError( + `Missing provider dependency '${providerName}' in configuration for provider '${this.config.name}'. ` + + `Are you missing a provider configuration?`, + { config: this.config, missingProviderName: providerName }, + ) + } + + const plugin = this.garden.getPlugin(providerName) + + return new ResolveProviderTask({ + garden: this.garden, + plugin, + config, + log: this.log, + version: this.version, + }) + }) + } + + async process(dependencyResults: TaskResults) { + const resolvedProviders: Provider[] = Object.values(dependencyResults).map(result => result.output) + + const context = new ProviderConfigContext(this.garden.environmentName, resolvedProviders) + let resolvedConfig = await resolveTemplateStrings(this.config, context) + + resolvedConfig.path = this.garden.projectRoot + const providerName = resolvedConfig.name + + if (this.plugin.configSchema) { + resolvedConfig = validateWithPath({ + config: resolvedConfig, + schema: this.plugin.configSchema, + path: resolvedConfig.path, + projectRoot: this.garden.projectRoot, + configType: "provider", + ErrorClass: ConfigurationError, + }) + } + + const configureHandler = (this.plugin.actions || {}).configureProvider + + let moduleConfigs: ModuleConfig[] = [] + + if (configureHandler) { + this.log.silly(`Calling configureProvider on ${providerName}`) + + const configureOutput = await configureHandler({ + log: this.log, + config: resolvedConfig, + configStore: this.garden.configStore, + projectName: this.garden.projectName, + dependencies: resolvedProviders, + }) + + resolvedConfig = configureOutput.config + + if (configureOutput.moduleConfigs) { + moduleConfigs = configureOutput.moduleConfigs + } + } + + return providerFromConfig(resolvedConfig, resolvedProviders, moduleConfigs) + } +} diff --git a/garden-service/src/tasks/task.ts b/garden-service/src/tasks/task.ts index 80bc080091..afe07b4f85 100644 --- a/garden-service/src/tasks/task.ts +++ b/garden-service/src/tasks/task.ts @@ -14,7 +14,7 @@ import { Task } from "../types/task" import { DeployTask } from "./deploy" import { LogEntry } from "../logger/log-entry" import { prepareRuntimeContext } from "../types/service" -import { DependencyGraphNodeType, ConfigGraph } from "../config-graph" +import { ConfigGraph } from "../config-graph" import { ModuleVersion } from "../vcs/vcs" import { BuildTask } from "./build" import { RunTaskResult } from "../types/plugin/task/runTask" @@ -30,7 +30,6 @@ export interface TaskTaskParams { export class TaskTask extends BaseTask { // ... to be renamed soon. type: TaskType = "task" - depType: DependencyGraphNodeType = "task" private graph: ConfigGraph private task: Task @@ -59,7 +58,7 @@ export class TaskTask extends BaseTask { // ... to be renamed soon. }) const dg = await this.garden.getConfigGraph() - const deps = await dg.getDependencies(this.depType, this.getName(), false) + const deps = await dg.getDependencies("task", this.getName(), false) const deployTasks = deps.service.map(service => { return new DeployTask({ @@ -87,7 +86,7 @@ export class TaskTask extends BaseTask { // ... to be renamed soon. } - protected getName() { + getName() { return this.task.name } @@ -118,12 +117,13 @@ export class TaskTask extends BaseTask { // ... to be renamed soon. }) // combine all dependencies for all services in the module, to be sure we have all the context we need - const serviceDeps = (await this.graph.getDependencies(this.depType, this.getName(), false)).service + const serviceDeps = (await this.graph.getDependencies("task", this.getName(), false)).service const runtimeContext = await prepareRuntimeContext(this.garden, this.graph, module, serviceDeps) + const actions = await this.garden.getActionHelper() let result: RunTaskResult try { - result = await this.garden.actions.runTask({ + result = await actions.runTask({ task, log, runtimeContext, diff --git a/garden-service/src/tasks/test.ts b/garden-service/src/tasks/test.ts index 385b171e35..12dc316f91 100644 --- a/garden-service/src/tasks/test.ts +++ b/garden-service/src/tasks/test.ts @@ -17,7 +17,7 @@ import { BaseTask, TaskParams, TaskType } from "../tasks/base" import { prepareRuntimeContext } from "../types/service" import { Garden } from "../garden" import { LogEntry } from "../logger/log-entry" -import { DependencyGraphNodeType, ConfigGraph } from "../config-graph" +import { ConfigGraph } from "../config-graph" import { makeTestTaskName } from "./helpers" import { BuildTask } from "./build" @@ -39,7 +39,6 @@ export interface TestTaskParams { export class TestTask extends BaseTask { type: TaskType = "test" - depType: DependencyGraphNodeType = "test" private module: Module private graph: ConfigGraph @@ -69,7 +68,7 @@ export class TestTask extends BaseTask { } const dg = this.graph - const services = (await dg.getDependencies(this.depType, this.getName(), false)).service + const services = (await dg.getDependencies("test", this.getName(), false)).service const deps: BaseTask[] = [new BuildTask({ garden: this.garden, @@ -121,10 +120,11 @@ export class TestTask extends BaseTask { const dependencies = await getTestDependencies(this.graph, this.testConfig) const runtimeContext = await prepareRuntimeContext(this.garden, this.graph, this.module, dependencies) + const actions = await this.garden.getActionHelper() let result: TestResult try { - result = await this.garden.actions.testModule({ + result = await actions.testModule({ log, interactive: false, module: this.module, @@ -152,7 +152,9 @@ export class TestTask extends BaseTask { return null } - return this.garden.actions.getTestResult({ + const actions = await this.garden.getActionHelper() + + return actions.getTestResult({ log: this.log, module: this.module, testName: this.testConfig.name, diff --git a/garden-service/src/template-string.ts b/garden-service/src/template-string.ts index 3337117e95..ff013faa1e 100644 --- a/garden-service/src/template-string.ts +++ b/garden-service/src/template-string.ts @@ -9,7 +9,9 @@ import Bluebird = require("bluebird") import { asyncDeepMap } from "./util/util" import { GardenBaseError } from "./exceptions" -import { ConfigContext, ContextResolveOpts } from "./config/config-context" +import { ConfigContext, ContextResolveOpts, ContextResolveParams } from "./config/config-context" +import { KeyedSet } from "./util/keyed-set" +import { uniq } from "lodash" export type StringOrStringPromise = Promise | string @@ -66,3 +68,26 @@ export async function resolveTemplateStrings( { concurrency: 1 }, ) } + +/** + * Scans for all template strings in the given object and lists the referenced keys. + */ +export async function collectTemplateReferences(obj: T): Promise { + const context = new ScanContext() + await resolveTemplateStrings(obj, context) + return uniq(context.foundKeys.entries()).sort() +} + +class ScanContext extends ConfigContext { + foundKeys: KeyedSet + + constructor() { + super() + this.foundKeys = new KeyedSet(v => v.join(".")) + } + + async resolve({ key }: ContextResolveParams) { + this.foundKeys.add(key) + return key.join(".") + } +} diff --git a/garden-service/src/types/plugin/outputs.ts b/garden-service/src/types/plugin/outputs.ts new file mode 100644 index 0000000000..cb972a1ce1 --- /dev/null +++ b/garden-service/src/types/plugin/outputs.ts @@ -0,0 +1,350 @@ +/* + * Copyright (C) 2018 Garden Technologies, Inc. + * + * This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. + */ + +import * as Joi from "joi" +import { ModuleVersion, moduleVersionSchema } from "../../vcs/vcs" +import { Module } from "../module" +import { ServiceStatus } from "../service" +import { moduleConfigSchema, ModuleConfig } from "../../config/module" +import { DashboardPage, dashboardPagesSchema } from "../../config/dashboard" +import { ProviderConfig, providerConfigBaseSchema } from "../../config/provider" +import { joiArray } from "../../config/common" +import { deline } from "../../util/string" + +export interface ConfigureProviderResult { + config: T + moduleConfigs?: ModuleConfig[] +} +export const configureProviderResultSchema = Joi.object() + .keys({ + config: providerConfigBaseSchema, + moduleConfigs: joiArray(moduleConfigSchema) + .description(deline` + Providers may return one or more module configs, that are included with the provider. This can be used for + modules that should always be built, or deployed as part of bootstrapping the provider. + + They become part of the project graph like other modules, but need to be referenced with the provider name + as a prefix and a double dash, e.g. \`provider-name--module-name\`. + `), + }) + +export interface EnvironmentStatus { + ready: boolean + needUserInput?: boolean + dashboardPages?: DashboardPage[] + detail?: any +} + +export const environmentStatusSchema = Joi.object() + .keys({ + ready: Joi.boolean() + .required() + .description("Set to true if the environment is fully configured for a provider."), + needUserInput: Joi.boolean() + .description( + "Set to true if the environment needs user input to be initialized, " + + "and thus needs to be initialized via `garden init`.", + ), + dashboardPages: dashboardPagesSchema, + detail: Joi.object() + .meta({ extendable: true }) + .description("Use this to include additional information that is specific to the provider."), + }) + .description("Description of an environment's status for a provider.") + +export type EnvironmentStatusMap = { + [key: string]: EnvironmentStatus, +} + +export interface PrepareEnvironmentResult { } + +export const prepareEnvironmentResultSchema = Joi.object().keys({}) + +export interface CleanupEnvironmentResult { } + +export const cleanupEnvironmentResultSchema = Joi.object().keys({}) + +export interface GetSecretResult { + value: string | null +} + +export const getSecretResultSchema = Joi.object() + .keys({ + value: Joi.string() + .allow(null) + .required() + .description("The config value found for the specified key (as string), or null if not found."), + }) + +export interface SetSecretResult { } + +export const setSecretResultSchema = Joi.object().keys({}) + +export interface DeleteSecretResult { + found: boolean +} + +export const deleteSecretResultSchema = Joi.object() + .keys({ + found: Joi.boolean() + .required() + .description("Set to true if the key was deleted, false if it was not found."), + }) + +export interface ExecInServiceResult { + code: number + output: string + stdout?: string + stderr?: string +} + +export const execInServiceResultSchema = Joi.object() + .keys({ + code: Joi.number() + .required() + .description("The exit code of the command executed in the service container."), + output: Joi.string() + .allow("") + .required() + .description("The output of the executed command."), + stdout: Joi.string() + .allow("") + .description("The stdout output of the executed command (if available)."), + stderr: Joi.string() + .allow("") + .description("The stderr output of the executed command (if available)."), + }) + +export interface ServiceLogEntry { + serviceName: string + timestamp?: Date + msg: string +} + +export const serviceLogEntrySchema = Joi.object() + .keys({ + serviceName: Joi.string() + .required() + .description("The name of the service the log entry originated from."), + timestamp: Joi.date() + .required() + .description("The time when the log entry was generated by the service."), + msg: Joi.string() + .required() + .description("The content of the log entry."), + }) + .description("A log entry returned by a getServiceLogs action handler.") + +export interface GetServiceLogsResult { } + +export const getServiceLogsResultSchema = Joi.object().keys({}) + +export interface ModuleTypeDescription { + docs: string + // TODO: specify the schema using primitives and not Joi objects + schema: Joi.ObjectSchema + title?: string +} + +export const moduleTypeDescriptionSchema = Joi.object() + .keys({ + docs: Joi.string() + .required() + .description("Documentation for the module type, in markdown format."), + schema: Joi.object() + .required() + .description( + "A valid Joi schema describing the configuration keys for the `module` " + + "field in the module's `garden.yml`.", + ), + title: Joi.string() + .description( + "Readable title for the module type. Defaults to the title-cased type name, with dashes replaced by spaces.", + ), + }) + +export type ConfigureModuleResult = + ModuleConfig< + T["spec"], + T["serviceConfigs"][0]["spec"], + T["testConfigs"][0]["spec"], + T["taskConfigs"][0]["spec"] + > + +export const configureModuleResultSchema = moduleConfigSchema + +export interface BuildResult { + buildLog?: string + fetched?: boolean + fresh?: boolean + version?: string + details?: any +} +export const buildModuleResultSchema = Joi.object() + .keys({ + buildLog: Joi.string() + .allow("") + .description("The full log from the build."), + fetched: Joi.boolean() + .description("Set to true if the build was fetched from a remote registry."), + fresh: Joi.boolean() + .description("Set to true if the build was performed, false if it was already built, or fetched from a registry"), + version: Joi.string() + .description("The version that was built."), + details: Joi.object() + .description("Additional information, specific to the provider."), + }) + +export interface HotReloadServiceResult { } +export const hotReloadServiceResultSchema = Joi.object() + +export interface PublishResult { + published: boolean + message?: string +} +export const publishModuleResultSchema = Joi.object() + .keys({ + published: Joi.boolean() + .required() + .description("Set to true if the module was published."), + message: Joi.string() + .description("Optional result message."), + }) + +export interface RunResult { + moduleName: string + command: string[] + version: ModuleVersion + success: boolean + startedAt: Date + completedAt: Date + output: string +} + +export const runResultSchema = Joi.object() + .keys({ + moduleName: Joi.string() + .description("The name of the module that was run."), + command: Joi.array().items(Joi.string()) + .required() + .description("The command that was run in the module."), + version: moduleVersionSchema, + success: Joi.boolean() + .required() + .description("Whether the module was successfully run."), + startedAt: Joi.date() + .required() + .description("When the module run was started."), + completedAt: Joi.date() + .required() + .description("When the module run was completed."), + output: Joi.string() + .required() + .allow("") + .description("The output log from the run."), + }) + +export interface TestResult extends RunResult { + testName: string +} + +export const testResultSchema = runResultSchema + .keys({ + testName: Joi.string() + .required() + .description("The name of the test that was run."), + }) + +export const getTestResultSchema = testResultSchema.allow(null) + +export interface BuildStatus { + ready: boolean +} + +export const buildStatusSchema = Joi.object() + .keys({ + ready: Joi.boolean() + .required() + .description("Whether an up-to-date build is ready for the module."), + }) + +export interface RunTaskResult extends RunResult { + moduleName: string + taskName: string + command: string[] + version: ModuleVersion + success: boolean + startedAt: Date + completedAt: Date + output: string +} + +export const runTaskResultSchema = Joi.object() + .keys({ + moduleName: Joi.string() + .description("The name of the module that the task belongs to."), + taskName: Joi.string() + .description("The name of the task that was run."), + command: Joi.array().items(Joi.string()) + .required() + .description("The command that the task ran in the module."), + version: moduleVersionSchema, + success: Joi.boolean() + .required() + .description("Whether the task was successfully run."), + startedAt: Joi.date() + .required() + .description("When the task run was started."), + completedAt: Joi.date() + .required() + .description("When the task run was completed."), + output: Joi.string() + .required() + .allow("") + .description("The output log from the run."), + }) + +export const getTaskResultSchema = runTaskResultSchema.allow(null) + +export interface PluginActionOutputs { + configureProvider: Promise + + getEnvironmentStatus: Promise + prepareEnvironment: Promise + cleanupEnvironment: Promise + + getSecret: Promise + setSecret: Promise + deleteSecret: Promise +} + +export interface ServiceActionOutputs { + getServiceStatus: Promise + deployService: Promise + hotReloadService: Promise + deleteService: Promise + execInService: Promise + getServiceLogs: Promise<{}> + runService: Promise +} + +export interface TaskActionOutputs { + runTask: Promise + getTaskResult: Promise +} + +export interface ModuleActionOutputs extends ServiceActionOutputs { + describeType: Promise + configure: Promise + getBuildStatus: Promise + build: Promise + publishModule: Promise + runModule: Promise + testModule: Promise + getTestResult: Promise +} diff --git a/garden-service/src/types/plugin/params.ts b/garden-service/src/types/plugin/params.ts new file mode 100644 index 0000000000..761eaa2cd0 --- /dev/null +++ b/garden-service/src/types/plugin/params.ts @@ -0,0 +1,377 @@ +/* + * Copyright (C) 2018 Garden Technologies, Inc. + * + * This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. + */ + +import * as Joi from "joi" +import Stream from "ts-stream" +import { LogEntry } from "../../logger/log-entry" +import { PluginContext, pluginContextSchema } from "../../plugin-context" +import { ModuleVersion, moduleVersionSchema } from "../../vcs/vcs" +import { Primitive, joiPrimitive, joiArray } from "../../config/common" +import { Module, moduleSchema } from "../module" +import { RuntimeContext, Service, serviceSchema, runtimeContextSchema } from "../service" +import { Task } from "../task" +import { EnvironmentStatus, ServiceLogEntry, environmentStatusSchema } from "./outputs" +import { baseModuleSpecSchema } from "../../config/module" +import { testConfigSchema } from "../../config/test" +import { taskSchema } from "../../config/task" +import { deline } from "../../util/string" +import { ProviderConfig, providerConfigBaseSchema, Provider, providersSchema } from "../../config/provider" +import { ConfigStore, configStoreSchema } from "../../config-store" +import { projectNameSchema } from "../../config/project" + +export interface PluginActionContextParams { + ctx: PluginContext +} + +export interface PluginActionParamsBase extends PluginActionContextParams { + log: LogEntry +} + +// Note: not specifying this further because we will later remove it from the API +const logEntrySchema = Joi.object() + .description("Logging context handler that the handler can use to log messages and progress.") + .required() + +const actionParamsSchema = Joi.object() + .keys({ + ctx: pluginContextSchema + .required(), + log: logEntrySchema + .required(), + }) + +export interface PluginModuleActionParamsBase extends PluginActionParamsBase { + module: T +} +const moduleActionParamsSchema = actionParamsSchema + .keys({ + module: moduleSchema, + }) + +export interface PluginServiceActionParamsBase + extends PluginModuleActionParamsBase { + runtimeContext?: RuntimeContext + service: Service +} +const serviceActionParamsSchema = moduleActionParamsSchema + .keys({ + runtimeContext: runtimeContextSchema + .optional(), + service: serviceSchema, + }) + +export interface PluginTaskActionParamsBase extends PluginModuleActionParamsBase { + task: Task +} +const taskActionParamsSchema = moduleActionParamsSchema + .keys({ + task: taskSchema, + }) + +/** + * Plugin actions + */ +export interface ConfigureProviderParams { + config: T + log: LogEntry + projectName: string + dependencies: Provider[] + configStore: ConfigStore +} +export const configureProviderParamsSchema = Joi.object() + .keys({ + config: providerConfigBaseSchema.required(), + log: logEntrySchema, + projectName: projectNameSchema, + dependencies: providersSchema, + configStore: configStoreSchema, + }) + +export interface GetEnvironmentStatusParams extends PluginActionParamsBase { } +export const getEnvironmentStatusParamsSchema = actionParamsSchema + +export interface PrepareEnvironmentParams extends PluginActionParamsBase { + status: EnvironmentStatus + force: boolean +} +export const prepareEnvironmentParamsSchema = actionParamsSchema + .keys({ + status: environmentStatusSchema, + force: Joi.boolean() + .description("Force re-configuration of the environment."), + }) + +export interface CleanupEnvironmentParams extends PluginActionParamsBase { +} +export const cleanupEnvironmentParamsSchema = actionParamsSchema + +export interface GetSecretParams extends PluginActionParamsBase { + key: string +} +export const getSecretParamsSchema = actionParamsSchema + .keys({ + key: Joi.string() + .description("A unique identifier for the secret."), + }) + +export interface SetSecretParams extends PluginActionParamsBase { + key: string + value: Primitive +} +export const setSecretParamsSchema = getSecretParamsSchema + .keys({ + value: joiPrimitive() + .description("The value of the secret."), + }) + +export interface DeleteSecretParams extends PluginActionParamsBase { + key: string +} +export const deleteSecretParamsSchema = getSecretParamsSchema + +export interface PluginActionParams { + configureProvider: ConfigureProviderParams + + getEnvironmentStatus: GetEnvironmentStatusParams + prepareEnvironment: PrepareEnvironmentParams + cleanupEnvironment: CleanupEnvironmentParams + + getSecret: GetSecretParams + setSecret: SetSecretParams + deleteSecret: DeleteSecretParams +} + +/** + * Module actions + */ +export interface DescribeModuleTypeParams { } +export const describeModuleTypeParamsSchema = Joi.object() + .keys({}) + +export interface ConfigureModuleParams { + ctx: PluginContext + log: LogEntry + moduleConfig: T["_ConfigType"] +} +export const configureModuleParamsSchema = Joi.object() + .keys({ + ctx: pluginContextSchema + .required(), + log: logEntrySchema, + moduleConfig: baseModuleSpecSchema + .required(), + }) + +export interface GetBuildStatusParams extends PluginModuleActionParamsBase { } +export const getBuildStatusParamsSchema = moduleActionParamsSchema + +export interface BuildModuleParams extends PluginModuleActionParamsBase { } +export const buildModuleParamsSchema = moduleActionParamsSchema + +export interface PublishModuleParams extends PluginModuleActionParamsBase { } +export const publishModuleParamsSchema = moduleActionParamsSchema + +export interface RunModuleParams extends PluginModuleActionParamsBase { + command: string[] + interactive: boolean + runtimeContext: RuntimeContext + ignoreError?: boolean + timeout?: number +} +const runBaseParams = { + interactive: Joi.boolean() + .description("Whether to run the module interactively (i.e. attach to the terminal)."), + runtimeContext: runtimeContextSchema, + silent: Joi.boolean() + .description("Set to false if the output should not be logged to the console."), + timeout: Joi.number() + .optional() + .description("If set, how long to run the command before timing out."), +} +const runModuleBaseSchema = moduleActionParamsSchema + .keys(runBaseParams) +export const runModuleParamsSchema = runModuleBaseSchema + .keys({ + command: joiArray(Joi.string()) + .description("The command to run in the module."), + }) + +export const testVersionSchema = moduleVersionSchema + .description(deline` + The test run's version. In addition to the parent module's version, this also + factors in the module versions of the test's runtime dependencies (if any).`) + +export interface TestModuleParams extends PluginModuleActionParamsBase { + interactive: boolean + runtimeContext: RuntimeContext + silent: boolean + testConfig: T["testConfigs"][0] + testVersion: ModuleVersion +} +export const testModuleParamsSchema = runModuleBaseSchema + .keys({ testConfig: testConfigSchema, testVersion: testVersionSchema }) + +export interface GetTestResultParams extends PluginModuleActionParamsBase { + testName: string + testVersion: ModuleVersion +} +export const getTestResultParamsSchema = moduleActionParamsSchema + .keys({ + testName: Joi.string() + .description("A unique name to identify the test run."), + testVersion: testVersionSchema, + }) + +/** + * Service actions + */ + +export type hotReloadStatus = "enabled" | "disabled" + +export interface GetServiceStatusParams + extends PluginServiceActionParamsBase { + hotReload: boolean, + runtimeContext: RuntimeContext +} + +export const getServiceStatusParamsSchema = serviceActionParamsSchema + .keys({ + runtimeContext: runtimeContextSchema, + hotReload: Joi.boolean() + .default(false) + .description("Whether the service should be configured for hot-reloading."), + }) + +export interface DeployServiceParams + extends PluginServiceActionParamsBase { + force: boolean, + hotReload: boolean, + runtimeContext: RuntimeContext +} +export const deployServiceParamsSchema = serviceActionParamsSchema + .keys({ + force: Joi.boolean() + .description("Whether to force a re-deploy, even if the service is already deployed."), + runtimeContext: runtimeContextSchema, + hotReload: Joi.boolean() + .default(false) + .description("Whether to configure the service for hot-reloading."), + }) + +export interface HotReloadServiceParams + extends PluginServiceActionParamsBase { + runtimeContext: RuntimeContext +} +export const hotReloadServiceParamsSchema = serviceActionParamsSchema + .keys({ runtimeContext: runtimeContextSchema }) + +export interface DeleteServiceParams + extends PluginServiceActionParamsBase { + runtimeContext: RuntimeContext +} +export const deleteServiceParamsSchema = serviceActionParamsSchema + .keys({ + runtimeContext: runtimeContextSchema, + }) + +export interface ExecInServiceParams + extends PluginServiceActionParamsBase { + command: string[] + runtimeContext: RuntimeContext + interactive: boolean +} +export const execInServiceParamsSchema = serviceActionParamsSchema + .keys({ + command: joiArray(Joi.string()) + .description("The command to run alongside the service."), + runtimeContext: runtimeContextSchema, + interactive: Joi.boolean(), + }) + +export interface GetServiceLogsParams + extends PluginServiceActionParamsBase { + runtimeContext: RuntimeContext + stream: Stream + follow: boolean + tail: number + startTime?: Date +} +export const getServiceLogsParamsSchema = serviceActionParamsSchema + .keys({ + runtimeContext: runtimeContextSchema, + stream: Joi.object() + .description("A Stream object, to write the logs to."), + follow: Joi.boolean() + .description("Whether to keep listening for logs until aborted."), + tail: Joi.number() + .description("Number of lines to get from end of log. Defaults to -1, showing all log lines.") + .default(-1), + startTime: Joi.date() + .optional() + .description("If set, only return logs that are as new or newer than this date."), + }) + +export interface RunServiceParams + extends PluginServiceActionParamsBase { + interactive: boolean + runtimeContext: RuntimeContext + timeout?: number +} +export const runServiceParamsSchema = serviceActionParamsSchema + .keys(runBaseParams) + +/** + * Task actions + */ + +export const taskVersionSchema = moduleVersionSchema + .description(deline` + The task run's version. In addition to the parent module's version, this also + factors in the module versions of the tasks's runtime dependencies (if any).`) + +export interface GetTaskResultParams extends PluginTaskActionParamsBase { + taskVersion: ModuleVersion +} +export const getTaskResultParamsSchema = taskActionParamsSchema + .keys({ taskVersion: taskVersionSchema }) + +export interface RunTaskParams extends PluginTaskActionParamsBase { + interactive: boolean + runtimeContext: RuntimeContext + taskVersion: ModuleVersion + timeout?: number +} +export const runTaskParamsSchema = taskActionParamsSchema + .keys(runBaseParams) + .keys({ taskVersion: taskVersionSchema }) + +export interface ServiceActionParams { + getServiceStatus: GetServiceStatusParams + deployService: DeployServiceParams + hotReloadService: HotReloadServiceParams + deleteService: DeleteServiceParams + execInService: ExecInServiceParams + getServiceLogs: GetServiceLogsParams + runService: RunServiceParams +} + +export interface TaskActionParams { + getTaskResult: GetTaskResultParams + runTask: RunTaskParams +} + +export interface ModuleActionParams { + describeType: DescribeModuleTypeParams, + configure: ConfigureModuleParams + getBuildStatus: GetBuildStatusParams + build: BuildModuleParams + publishModule: PublishModuleParams + runModule: RunModuleParams + testModule: TestModuleParams + getTestResult: GetTestResultParams +} diff --git a/garden-service/src/types/plugin/plugin.ts b/garden-service/src/types/plugin/plugin.ts index b38f24e091..f3b49b5e1c 100644 --- a/garden-service/src/types/plugin/plugin.ts +++ b/garden-service/src/types/plugin/plugin.ts @@ -40,6 +40,7 @@ import { RunResult } from "./base" import { ServiceStatus } from "../service" import { mapValues } from "lodash" import { getDebugInfo, DebugInfo, GetDebugInfoParams } from "./provider/getDebugInfo" +import { deline } from "../../util/string" export type ServiceActions = { [P in keyof ServiceActionParams]: (params: ServiceActionParams[P]) => ServiceActionOutputs[P] @@ -202,7 +203,7 @@ export interface GardenPlugin { configSchema?: Joi.Schema, configKeys?: string[] - modules?: string[] + dependencies?: string[] actions?: Partial moduleActions?: { [moduleType: string]: Partial } @@ -225,12 +226,12 @@ export const pluginSchema = Joi.object() .keys({ // TODO: make this an OpenAPI schema for portability configSchema: Joi.object({ isJoi: Joi.boolean().only(true).required() }).unknown(true), - modules: joiArray(Joi.string()) - .description( - "Plugins may optionally provide paths to Garden modules that are loaded as part of the plugin. " + - "This is useful, for example, to provide build dependencies for other modules " + - "or as part of the plugin operation.", - ), + dependencies: joiArray(Joi.string()) + .description(deline` + Names of plugins that need to be configured prior to this plugin. This plugin will be able to reference the + configuration from the listed plugins. Note that the dependencies will not be implicitly configured—the user + will need to explicitly configure them in their project configuration. + `), // TODO: document plugin actions further actions: Joi.object().keys(mapValues(pluginActionDescriptions, () => Joi.func())) .description("A map of plugin action handlers provided by the plugin."), diff --git a/garden-service/src/types/plugin/provider/configureProvider.ts b/garden-service/src/types/plugin/provider/configureProvider.ts index 635146de81..711b55dce9 100644 --- a/garden-service/src/types/plugin/provider/configureProvider.ts +++ b/garden-service/src/types/plugin/provider/configureProvider.ts @@ -8,17 +8,27 @@ import * as Joi from "joi" import dedent = require("dedent") -import { ProviderConfig, Provider, providerConfigBaseSchema, projectNameSchema } from "../../../config/project" +import { projectNameSchema } from "../../../config/project" +import { ProviderConfig, Provider, providerConfigBaseSchema, providersSchema } from "../../../config/provider" import { LogEntry } from "../../../logger/log-entry" import { logEntrySchema } from "../base" +import { configStoreSchema, ConfigStore } from "../../../config-store" +import { joiArray } from "../../../config/common" +import { moduleConfigSchema, ModuleConfig } from "../../../config/module" +import { deline } from "../../../util/string" export interface ConfigureProviderParams { config: T log: LogEntry projectName: string + dependencies: Provider[] + configStore: ConfigStore } -export interface ConfigureProviderResult extends Provider { } +export interface ConfigureProviderResult { + config: T + moduleConfigs?: ModuleConfig[] +} export const configureProvider = { description: dedent` @@ -37,9 +47,19 @@ export const configureProvider = { config: providerConfigBaseSchema.required(), log: logEntrySchema, projectName: projectNameSchema, + dependencies: providersSchema, + configStore: configStoreSchema, }), resultSchema: Joi.object() .keys({ config: providerConfigBaseSchema, + moduleConfigs: joiArray(moduleConfigSchema) + .description(deline` + Providers may return one or more module configs, that are included with the provider. This can be used for + modules that should always be built, or deployed as part of bootstrapping the provider. + + They become part of the project graph like other modules, but need to be referenced with the provider name + as a prefix and a double dash, e.g. \`provider-name--module-name\`. + `), }), } diff --git a/garden-service/src/types/service.ts b/garden-service/src/types/service.ts index 8dfb7958e9..240946be19 100644 --- a/garden-service/src/types/service.ts +++ b/garden-service/src/types/service.ts @@ -223,7 +223,7 @@ export async function prepareRuntimeContext( GARDEN_VERSION: versionString, } - for (const [key, value] of Object.entries(garden.environment.variables)) { + for (const [key, value] of Object.entries(garden.variables)) { const envVarName = `GARDEN_VARIABLES_${key.replace(/-/g, "_").toUpperCase()}` envVars[envVarName] = value } diff --git a/garden-service/src/util/ext-source-util.ts b/garden-service/src/util/ext-source-util.ts index 3a056f628e..521079faa3 100644 --- a/garden-service/src/util/ext-source-util.ts +++ b/garden-service/src/util/ext-source-util.ts @@ -64,7 +64,7 @@ export async function getLinkedSources( garden: Garden, type: ExternalSourceType, ): Promise { - const localConfig = await garden.localConfigStore.get() + const localConfig = await garden.configStore.get() return (type === "project" ? localConfig.linkedProjectSources : localConfig.linkedModuleSources) || [] @@ -76,7 +76,7 @@ export async function addLinkedSources({ garden, sourceType, sources }: { sources: LinkedSource[], }): Promise { const linked = uniqBy([...await getLinkedSources(garden, sourceType), ...sources], "name") - await garden.localConfigStore.set([getConfigKey(sourceType)], linked) + await garden.configStore.set([getConfigKey(sourceType)], linked) return linked } @@ -100,6 +100,6 @@ export async function removeLinkedSources({ garden, sourceType, names }: { } const linked = currentlyLinked.filter(({ name }) => !names.includes(name)) - await garden.localConfigStore.set([getConfigKey(sourceType)], linked) + await garden.configStore.set([getConfigKey(sourceType)], linked) return linked } diff --git a/garden-service/src/util/util.ts b/garden-service/src/util/util.ts index 99ebfbb33c..32befd5b76 100644 --- a/garden-service/src/util/util.ts +++ b/garden-service/src/util/util.ts @@ -60,7 +60,7 @@ export function registerCleanupFunction(name: string, func: HookCallback) { exitHook(func) } -export function getPackageVersion(): String { +export function getPackageVersion(): string { const version = require("../../../package.json").version return version } diff --git a/garden-service/src/util/validate-dependencies.ts b/garden-service/src/util/validate-dependencies.ts index 93e403e61e..285cbb9460 100644 --- a/garden-service/src/util/validate-dependencies.ts +++ b/garden-service/src/util/validate-dependencies.ts @@ -7,7 +7,7 @@ */ import dedent = require("dedent") -import { merge } from "lodash" +import { merge, flatten, uniq } from "lodash" import * as indentString from "indent-string" import { get, isEqual, join, set, uniqWith } from "lodash" import { getModuleKey } from "../types/module" @@ -22,7 +22,7 @@ export function validateDependencies( ): void { const missingDepsError = detectMissingDependencies(moduleConfigs, serviceNames, taskNames) - const circularDepsError = detectCircularDependencies(moduleConfigs) + const circularDepsError = detectCircularModuleDependencies(moduleConfigs) let errMsg = "" let detail = {} @@ -102,22 +102,18 @@ export function detectMissingDependencies( export type Cycle = string[] /** - * Implements a variation on the Floyd-Warshall algorithm to compute minimal cycles. - * - * This is approximately O(m^3) + O(s^3), where m is the number of modules and s is the number of services. - * - * Returns an error if cycles were found. + * Computes build and runtime dependency graphs for the given modules, and returns an error if cycles were found. */ -export function detectCircularDependencies(moduleConfigs: ModuleConfig[]): ConfigurationError | null { +export function detectCircularModuleDependencies(moduleConfigs: ModuleConfig[]): ConfigurationError | null { // Sparse matrices - const buildGraph = {} - const runtimeGraph = {} + const buildGraph: DependencyGraph = {} + const runtimeGraph: DependencyGraph = {} const services: ServiceConfig[] = [] const tasks: TaskConfig[] = [] /** * Since dependencies listed in test configs cannot introduce circularities (because - * builds/deployments/tasks/tests cannot currently depend on tests), we don't need to + * builds/deployments/tasks/tests cannot currently depend on tesxts), we don't need to * account for test dependencies here. */ for (const module of moduleConfigs) { @@ -143,10 +139,8 @@ export function detectCircularDependencies(moduleConfigs: ModuleConfig[]): Confi } } - const serviceNames = services.map(s => s.name) - const taskNames = tasks.map(w => w.name) - const buildCycles = detectCycles(buildGraph, moduleConfigs.map(m => m.name)) - const runtimeCycles = detectCycles(runtimeGraph, serviceNames.concat(taskNames)) + const buildCycles = detectCycles(buildGraph) + const runtimeCycles = detectCycles(runtimeGraph) if (buildCycles.length > 0 || runtimeCycles.length > 0) { const detail = {} @@ -175,7 +169,32 @@ export function detectCircularDependencies(moduleConfigs: ModuleConfig[]): Confi return null } -export function detectCycles(graph, vertices: string[]): Cycle[] { +export interface DependencyGraph { + [key: string]: { + [target: string]: { + distance: number, + next: string, + }, + } +} + +/** + * Implements a variation on the Floyd-Warshall algorithm to compute minimal cycles. + * + * This is approximately O(m^3) + O(s^3), where m is the number of modules and s is the number of services. + * + * Returns a list of cycles found. + */ +export function detectCycles(graph: DependencyGraph): Cycle[] { + // Collect all the vertices + const vertices = uniq( + Object.keys(graph).concat( + flatten( + Object.values(graph).map(v => Object.keys(v)), + ), + ), + ) + // Compute shortest paths for (const k of vertices) { for (const i of vertices) { @@ -206,15 +225,15 @@ export function detectCycles(graph, vertices: string[]): Cycle[] { (c1, c2) => isEqual(c1.concat().sort(), c2.concat().sort())) } -function distance(graph, source, destination): number { +function distance(graph: DependencyGraph, source: string, destination: string): number { return get(graph, [source, destination, "distance"], Infinity) } -function next(graph, source, destination): string | undefined { +function next(graph: DependencyGraph, source: string, destination: string): string | undefined { return get(graph, [source, destination, "next"]) } -function cyclesToString(cycles: Cycle[]) { +export function cyclesToString(cycles: Cycle[]) { const cycleDescriptions = cycles.map(c => join(c.concat([c[0]]), " <- ")) return cycleDescriptions.length === 1 ? cycleDescriptions[0] : cycleDescriptions } diff --git a/garden-service/static/openfaas/system/openfaas-system/garden.yml b/garden-service/static/openfaas/system/openfaas-system/garden.yml deleted file mode 100644 index abdf254d95..0000000000 --- a/garden-service/static/openfaas/system/openfaas-system/garden.yml +++ /dev/null @@ -1,25 +0,0 @@ -module: - description: OpenFaaS runtime - name: system - type: helm - repo: https://openfaas.github.io/faas-netes/ - chart: openfaas - version: 1.7.0 - releaseName: ${var.release-name} - values: - exposeServices: false - functionNamespace: ${var.function-namespace} - ingress: - enabled: true - hosts: - - host: ${var.gateway-hostname} - serviceName: gateway - servicePort: 8080 - path: /function/ - - host: ${var.gateway-hostname} - serviceName: gateway - servicePort: 8080 - path: /system/ - faasnetesd: - imagePullPolicy: IfNotPresent - securityContext: false diff --git a/garden-service/static/openfaas/templates/garden.yml b/garden-service/static/openfaas/templates/garden.yml deleted file mode 100644 index c1235be9e2..0000000000 --- a/garden-service/static/openfaas/templates/garden.yml +++ /dev/null @@ -1,5 +0,0 @@ -module: - description: OpenFaaS templates for building functions - name: templates - type: exec - repositoryUrl: https://github.com/openfaas/templates.git#master diff --git a/garden-service/test/helpers.ts b/garden-service/test/helpers.ts index e4109f3aa3..9d288cdb9a 100644 --- a/garden-service/test/helpers.ts +++ b/garden-service/test/helpers.ts @@ -22,7 +22,7 @@ import { ModuleActions, Plugins, } from "../src/types/plugin/plugin" -import { Garden } from "../src/garden" +import { Garden, GardenOpts } from "../src/garden" import { ModuleConfig } from "../src/config/module" import { mapValues, fromPairs } from "lodash" import { ModuleVersion } from "../src/vcs/vcs" @@ -33,6 +33,7 @@ import { Ignorer } from "../src/util/fs" import { SourceConfig } from "../src/config/project" import { BuildDir } from "../src/build-dir" import { LogEntry } from "../src/logger/log-entry" +import { ProviderConfig } from "../src/config/provider" import timekeeper = require("timekeeper") import { GLOBAL_OPTIONS } from "../src/cli/cli" import { RunModuleParams } from "../src/types/plugin/module/runModule" @@ -292,14 +293,19 @@ export class TestGarden extends Garden { constructor( public readonly projectRoot: string, public readonly projectName: string, - environmentName: string, - variables: PrimitiveMap, + public readonly environmentName: string, + public readonly variables: PrimitiveMap, public readonly projectSources: SourceConfig[] = [], public readonly buildDir: BuildDir, public readonly ignorer: Ignorer, - log?, + public readonly opts: GardenOpts, + plugins: Plugins, + providerConfigs: ProviderConfig[], ) { - super(projectRoot, projectName, environmentName, variables, projectSources, buildDir, ignorer, log) + super( + projectRoot, projectName, environmentName, variables, projectSources, + buildDir, ignorer, opts, plugins, providerConfigs, + ) this.events = new TestEventBus(this.log) } } diff --git a/garden-service/test/src/config/project.ts b/garden-service/test/src/config/project.ts new file mode 100644 index 0000000000..121e0ee7c9 --- /dev/null +++ b/garden-service/test/src/config/project.ts @@ -0,0 +1,492 @@ +import { platform } from "os" +import { expect } from "chai" +import { ProjectConfig, resolveProjectConfig, defaultEnvironments, pickEnvironment } from "../../../src/config/project" +import { DEFAULT_API_VERSION } from "../../../src/constants" +import { expectError } from "../../helpers" + +describe("resolveProjectConfig", () => { + it("should pass through a canonical project config", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "some-provider" }, + ], + variables: {}, + } + + expect(await resolveProjectConfig(config)).to.eql({ + ...config, + environmentDefaults: { + providers: [], + variables: {}, + }, + environments: [ + { name: "default", providers: [], variables: {} }, + ], + sources: [], + }) + }) + + it("should inject a default environment if none is specified", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "local", + environments: [], + providers: [ + { name: "some-provider" }, + ], + variables: {}, + } + + expect(await resolveProjectConfig(config)).to.eql({ + ...config, + environmentDefaults: { + providers: [], + variables: {}, + }, + environments: defaultEnvironments, + sources: [], + }) + }) + + it("should resolve template strings on fields other than provider configs", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "default", + environmentDefaults: { + variables: { + defaultEnvVar: "\${local.env.TEST_ENV_VAR}", + }, + }, + environments: [ + { + name: "default", + variables: { + envVar: "\${local.env.TEST_ENV_VAR}", + }, + }, + ], + providers: [ + { name: "some-provider" }, + ], + sources: [ + { + name: "\${local.env.TEST_ENV_VAR}", + repositoryUrl: "git://\${local.env.TEST_ENV_VAR}", + }, + ], + variables: { + platform: "\${local.platform}", + }, + } + + process.env.TEST_ENV_VAR = "foo" + + expect(await resolveProjectConfig(config)).to.eql({ + ...config, + environmentDefaults: { + providers: [], + variables: {}, + }, + environments: [ + { + name: "default", + providers: [], + variables: { + envVar: "foo", + }, + }, + ], + sources: [ + { + name: "foo", + repositoryUrl: "git://foo", + }, + ], + variables: { + defaultEnvVar: "foo", + platform: platform(), + }, + }) + + delete process.env.TEST_ENV_VAR + }) + + it("should pass through templated fields on provider configs", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "default", + environmentDefaults: { + providers: [ + { + name: "provider-a", + someKey: "\${local.env.TEST_ENV_VAR_A}", + }, + ], + variables: {}, + }, + environments: [ + { + name: "default", + providers: [ + { + name: "provider-b", + someKey: "\${local.env.TEST_ENV_VAR_B}", + }, + ], + variables: { + envVar: "foo", + }, + }, + ], + providers: [ + { + name: "provider-c", + someKey: "\${local.env.TEST_ENV_VAR_C}", + }, + ], + variables: {}, + } + + process.env.TEST_ENV_VAR_A = "foo" + process.env.TEST_ENV_VAR_B = "boo" + process.env.TEST_ENV_VAR_C = "moo" + + expect(await resolveProjectConfig(config)).to.eql({ + ...config, + environmentDefaults: { + providers: [], + variables: {}, + }, + environments: [ + { + name: "default", + providers: [], + variables: { + envVar: "foo", + }, + }, + ], + providers: [ + { + name: "provider-a", + someKey: "\${local.env.TEST_ENV_VAR_A}", + }, + { + name: "provider-c", + someKey: "\${local.env.TEST_ENV_VAR_C}", + }, + { + name: "provider-b", + environments: ["default"], + someKey: "\${local.env.TEST_ENV_VAR_B}", + }, + ], + sources: [], + }) + + delete process.env.TEST_ENV_VAR_A + delete process.env.TEST_ENV_VAR_B + delete process.env.TEST_ENV_VAR_C + }) + + it("should set defaultEnvironment to first environment if not configured", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "", + environments: [], + providers: [ + { name: "some-provider" }, + ], + variables: {}, + } + + expect(await resolveProjectConfig(config)).to.eql({ + ...config, + defaultEnvironment: "local", + environmentDefaults: { + providers: [], + variables: {}, + }, + environments: defaultEnvironments, + sources: [], + }) + }) + + it("should populate default values in the schema", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "", + environments: [], + providers: [ + { name: "some-provider" }, + ], + variables: {}, + } + + expect(await resolveProjectConfig(config)).to.eql({ + ...config, + defaultEnvironment: "local", + environmentDefaults: { + providers: [], + variables: {}, + }, + environments: defaultEnvironments, + sources: [], + }) + }) + + it("should include providers in correct precedency order from all possible config keys", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "default", + environmentDefaults: { + providers: [ + { + name: "provider-a", + }, + ], + variables: {}, + }, + environments: [ + { + name: "default", + providers: [ + { + name: "provider-b", + }, + ], + variables: { + envVar: "foo", + }, + }, + ], + providers: [ + { + name: "provider-c", + }, + ], + variables: {}, + } + + expect(await resolveProjectConfig(config)).to.eql({ + ...config, + environmentDefaults: { + providers: [], + variables: {}, + }, + environments: [ + { + name: "default", + providers: [], + variables: { + envVar: "foo", + }, + }, + ], + providers: [ + { + name: "provider-a", + }, + { + name: "provider-c", + }, + { + name: "provider-b", + environments: ["default"], + }, + ], + sources: [], + }) + }) + + it("should convert old-style environment/provider config to the new canonical form", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "default", + environmentDefaults: { + providers: [ + { + name: "provider-a", + }, + ], + variables: { + defaultVar: "foo", + }, + }, + environments: [ + { + name: "default", + providers: [ + { + name: "provider-b", + }, + ], + variables: { + envVar: "bar", + }, + }, + ], + providers: [], + variables: {}, + } + + expect(await resolveProjectConfig(config)).to.eql({ + ...config, + environmentDefaults: { + providers: [], + variables: {}, + }, + environments: [ + { + name: "default", + providers: [], + variables: { + envVar: "bar", + }, + }, + ], + providers: [ + { + name: "provider-a", + }, + { + name: "provider-b", + environments: ["default"], + }, + ], + sources: [], + variables: { + defaultVar: "foo", + }, + }) + }) +}) + +describe("pickEnvironment", () => { + it("should throw if selected environment isn't configured", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [], + variables: {}, + } + + await expectError(() => pickEnvironment(config, "foo"), "parameter") + }) + + it("should include fixed providers in output", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [], + variables: {}, + } + + expect(await pickEnvironment(config, "default")).to.eql({ + providers: [ + { name: "exec" }, + { name: "container" }, + ], + variables: {}, + }) + }) + + it("should correctly merge provider configurations using JSON Merge Patch", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "container", newKey: "foo" }, + { name: "my-provider", a: "a" }, + { name: "my-provider", b: "b" }, + { name: "my-provider", a: "c" }, + ], + variables: {}, + } + + expect(await pickEnvironment(config, "default")).to.eql({ + providers: [ + { name: "exec" }, + { name: "container", newKey: "foo" }, + { name: "my-provider", a: "c", b: "b" }, + ], + variables: {}, + }) + }) + + it("should remove null values in provider configs (as per the JSON Merge Patch spec)", async () => { + const config: ProjectConfig = { + apiVersion: DEFAULT_API_VERSION, + kind: "Project", + name: "my-project", + path: "/tmp/foo", + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "container", newKey: "foo" }, + { name: "my-provider", a: "a" }, + { name: "my-provider", b: "b" }, + { name: "my-provider", a: null }, + ], + variables: {}, + } + + expect(await pickEnvironment(config, "default")).to.eql({ + providers: [ + { name: "exec" }, + { name: "container", newKey: "foo" }, + { name: "my-provider", b: "b" }, + ], + variables: {}, + }) + }) +}) diff --git a/garden-service/test/src/config/provider.ts b/garden-service/test/src/config/provider.ts new file mode 100644 index 0000000000..29dedae83e --- /dev/null +++ b/garden-service/test/src/config/provider.ts @@ -0,0 +1,45 @@ +import { expect } from "chai" +import { ProviderConfig, getProviderDependencies } from "../../../src/config/provider" +import { expectError } from "../../helpers" + +describe("getProviderDependencies", () => { + it("should extract implicit provider dependencies from template strings", async () => { + const config: ProviderConfig = { + name: "my-provider", + someKey: "\${provider.other-provider.foo}", + anotherKey: "foo-\${provider.another-provider.bar}", + } + expect(await getProviderDependencies(config)).to.eql([ + "another-provider", + "other-provider", + ]) + }) + + it("should ignore template strings that don't reference providers", async () => { + const config: ProviderConfig = { + name: "my-provider", + someKey: "\${provider.other-provider.foo}", + anotherKey: "foo-\${some.other.ref}", + } + expect(await getProviderDependencies(config)).to.eql([ + "other-provider", + ]) + }) + + it("should throw on provider-scoped template strings without a provider name", async () => { + const config: ProviderConfig = { + name: "my-provider", + someKey: "\${provider}", + } + + await expectError( + () => getProviderDependencies(config), + (err) => { + expect(err.message).to.equal( + "Invalid template key 'provider' in configuration for provider 'my-provider'. " + + "You must specify a provider name as well (e.g. \\\${provider.my-provider}).", + ) + }, + ) + }) +}) diff --git a/garden-service/test/unit/data/test-projects/new-provider-spec/garden.yml b/garden-service/test/unit/data/test-projects/new-provider-spec/garden.yml new file mode 100644 index 0000000000..05048dda74 --- /dev/null +++ b/garden-service/test/unit/data/test-projects/new-provider-spec/garden.yml @@ -0,0 +1,12 @@ +kind: Project +name: test-project-a +environmentDefaults: + variables: + some: variable +environments: + - name: local + - name: other +providers: + - name: test-plugin + environments: [local] + - name: test-plugin-b diff --git a/garden-service/test/unit/src/actions.ts b/garden-service/test/unit/src/actions.ts index ab4b686036..2bb8e2dedb 100644 --- a/garden-service/test/unit/src/actions.ts +++ b/garden-service/test/unit/src/actions.ts @@ -36,7 +36,7 @@ describe("ActionHelper", () => { const plugins = { "test-plugin": testPlugin, "test-plugin-b": testPluginB } garden = await makeTestGardenA(plugins) log = garden.log - actions = garden.actions + actions = await garden.getActionHelper() const graph = await garden.getConfigGraph() module = await graph.getModule("module-a") service = await graph.getService("service-a") @@ -321,9 +321,9 @@ describe("ActionHelper", () => { }) }) - describe("getActionHandlers", () => { + describe("getActionHelpers", () => { it("should return all handlers for a type", async () => { - const handlers = actions.getActionHandlers("prepareEnvironment") + const handlers = actions.getActionHelpers("prepareEnvironment") expect(Object.keys(handlers)).to.eql([ "test-plugin", @@ -342,10 +342,11 @@ describe("ActionHelper", () => { }) }) - describe("getActionHandler", () => { + describe("getActionHelper", () => { it("should return last configured handler for specified action type", async () => { const gardenA = await makeTestGardenA() - const handler = gardenA.actions.getActionHandler({ actionType: "prepareEnvironment" }) + const actionsA = await gardenA.getActionHelper() + const handler = actionsA.getActionHelper({ actionType: "prepareEnvironment" }) expect(handler["actionType"]).to.equal("prepareEnvironment") expect(handler["pluginName"]).to.equal("test-plugin-b") @@ -353,7 +354,8 @@ describe("ActionHelper", () => { it("should optionally filter to only handlers for the specified module type", async () => { const gardenA = await makeTestGardenA() - const handler = gardenA.actions.getActionHandler({ actionType: "prepareEnvironment" }) + const actionsA = await gardenA.getActionHelper() + const handler = actionsA.getActionHelper({ actionType: "prepareEnvironment" }) expect(handler["actionType"]).to.equal("prepareEnvironment") expect(handler["pluginName"]).to.equal("test-plugin-b") @@ -361,14 +363,16 @@ describe("ActionHelper", () => { it("should throw if no handler is available", async () => { const gardenA = await makeTestGardenA() - await expectError(() => gardenA.actions.getActionHandler({ actionType: "cleanupEnvironment" }), "parameter") + const actionsA = await gardenA.getActionHelper() + await expectError(() => actionsA.getActionHelper({ actionType: "cleanupEnvironment" }), "parameter") }) }) describe("getModuleActionHandler", () => { it("should return last configured handler for specified module action type", async () => { const gardenA = await makeTestGardenA() - const handler = gardenA.actions.getModuleActionHandler({ actionType: "deployService", moduleType: "test" }) + const actionsA = await gardenA.getActionHelper() + const handler = actionsA.getModuleActionHandler({ actionType: "deployService", moduleType: "test" }) expect(handler["actionType"]).to.equal("deployService") expect(handler["pluginName"]).to.equal("test-plugin-b") @@ -376,8 +380,9 @@ describe("ActionHelper", () => { it("should throw if no handler is available", async () => { const gardenA = await makeTestGardenA() + const actionsA = await gardenA.getActionHelper() await expectError( - () => gardenA.actions.getModuleActionHandler({ actionType: "execInService", moduleType: "container" }), + () => actionsA.getModuleActionHandler({ actionType: "execInService", moduleType: "container" }), "parameter", ) }) diff --git a/garden-service/test/unit/src/commands/delete.ts b/garden-service/test/unit/src/commands/delete.ts index 6210402494..3469d5eb2d 100644 --- a/garden-service/test/unit/src/commands/delete.ts +++ b/garden-service/test/unit/src/commands/delete.ts @@ -23,7 +23,8 @@ describe("DeleteSecretCommand", () => { const key = "mykey" const value = "myvalue" - await garden.actions.setSecret({ log, key, value, pluginName }) + const actions = await garden.getActionHelper() + await actions.setSecret({ log, key, value, pluginName }) await command.action({ garden, @@ -33,7 +34,7 @@ describe("DeleteSecretCommand", () => { opts: withDefaultGlobalOpts({}), }) - expect(await garden.actions.getSecret({ log, pluginName, key })).to.eql({ value: null }) + expect(await actions.getSecret({ log, pluginName, key })).to.eql({ value: null }) }) it("should throw on missing key", async () => { diff --git a/garden-service/test/unit/src/commands/get/get-config.ts b/garden-service/test/unit/src/commands/get/get-config.ts index 564dc2d06b..e79ef21f00 100644 --- a/garden-service/test/unit/src/commands/get/get-config.ts +++ b/garden-service/test/unit/src/commands/get/get-config.ts @@ -20,10 +20,12 @@ describe("GetConfigCommand", () => { opts: withDefaultGlobalOpts({}), }) + const providers = await garden.resolveProviders() + const config = { - environmentName: garden.environment.name, - providers: garden.environment.providers, - variables: garden.environment.variables, + environmentName: garden.environmentName, + providers, + variables: garden.variables, moduleConfigs: sortBy(await garden.resolveModuleConfigs(), "name"), } diff --git a/garden-service/test/unit/src/commands/get/get-secret.ts b/garden-service/test/unit/src/commands/get/get-secret.ts index 5a18d8e2c8..02fad6bf6f 100644 --- a/garden-service/test/unit/src/commands/get/get-secret.ts +++ b/garden-service/test/unit/src/commands/get/get-secret.ts @@ -11,7 +11,8 @@ describe("GetSecretCommand", () => { const log = garden.log const command = new GetSecretCommand() - await garden.actions.setSecret({ + const actions = await garden.getActionHelper() + await actions.setSecret({ log, pluginName, key: "project.mykey", diff --git a/garden-service/test/unit/src/commands/link.ts b/garden-service/test/unit/src/commands/link.ts index e5bd774e93..583564d17d 100644 --- a/garden-service/test/unit/src/commands/link.ts +++ b/garden-service/test/unit/src/commands/link.ts @@ -44,7 +44,7 @@ describe("LinkCommand", () => { opts: withDefaultGlobalOpts({}), }) - const { linkedModuleSources } = await garden.localConfigStore.get() + const { linkedModuleSources } = await garden.configStore.get() expect(linkedModuleSources).to.eql([ { name: "module-a", path: join(projectRoot, "mock-local-path", "module-a") }, @@ -63,7 +63,7 @@ describe("LinkCommand", () => { opts: withDefaultGlobalOpts({}), }) - const { linkedModuleSources } = await garden.localConfigStore.get() + const { linkedModuleSources } = await garden.configStore.get() expect(linkedModuleSources).to.eql([ { name: "module-a", path: join(projectRoot, "mock-local-path", "module-a") }, @@ -113,7 +113,7 @@ describe("LinkCommand", () => { opts: withDefaultGlobalOpts({}), }) - const { linkedProjectSources } = await garden.localConfigStore.get() + const { linkedProjectSources } = await garden.configStore.get() expect(linkedProjectSources).to.eql([ { name: "source-a", path: join(projectRoot, "mock-local-path", "source-a") }, @@ -132,7 +132,7 @@ describe("LinkCommand", () => { opts: withDefaultGlobalOpts({}), }) - const { linkedProjectSources } = await garden.localConfigStore.get() + const { linkedProjectSources } = await garden.configStore.get() expect(linkedProjectSources).to.eql([ { name: "source-a", path: join(projectRoot, "mock-local-path", "source-a") }, diff --git a/garden-service/test/unit/src/commands/set.ts b/garden-service/test/unit/src/commands/set.ts index 54d1ead922..7abfd3db22 100644 --- a/garden-service/test/unit/src/commands/set.ts +++ b/garden-service/test/unit/src/commands/set.ts @@ -19,6 +19,7 @@ describe("SetSecretCommand", () => { opts: withDefaultGlobalOpts({}), }) - expect(await garden.actions.getSecret({ log, pluginName, key: "mykey" })).to.eql({ value: "myvalue" }) + const actions = await garden.getActionHelper() + expect(await actions.getSecret({ log, pluginName, key: "mykey" })).to.eql({ value: "myvalue" }) }) }) diff --git a/garden-service/test/unit/src/commands/unlink.ts b/garden-service/test/unit/src/commands/unlink.ts index 1cb7322ee0..2c0c0a8d3f 100644 --- a/garden-service/test/unit/src/commands/unlink.ts +++ b/garden-service/test/unit/src/commands/unlink.ts @@ -73,7 +73,7 @@ describe("UnlinkCommand", () => { args: { modules: ["module-a", "module-b"] }, opts: withDefaultGlobalOpts({ all: false }), }) - const { linkedModuleSources } = await garden.localConfigStore.get() + const { linkedModuleSources } = await garden.configStore.get() expect(linkedModuleSources).to.eql([ { name: "module-c", path: join(projectRoot, "mock-local-path", "module-c") }, ]) @@ -87,7 +87,7 @@ describe("UnlinkCommand", () => { args: { modules: undefined }, opts: withDefaultGlobalOpts({ all: true }), }) - const { linkedModuleSources } = await garden.localConfigStore.get() + const { linkedModuleSources } = await garden.configStore.get() expect(linkedModuleSources).to.eql([]) }) }) @@ -147,7 +147,7 @@ describe("UnlinkCommand", () => { args: { sources: ["source-a", "source-b"] }, opts: withDefaultGlobalOpts({ all: false }), }) - const { linkedProjectSources } = await garden.localConfigStore.get() + const { linkedProjectSources } = await garden.configStore.get() expect(linkedProjectSources).to.eql([ { name: "source-c", path: join(projectRoot, "mock-local-path", "source-c") }, ]) @@ -161,7 +161,7 @@ describe("UnlinkCommand", () => { args: { sources: undefined }, opts: withDefaultGlobalOpts({ all: true }), }) - const { linkedProjectSources } = await garden.localConfigStore.get() + const { linkedProjectSources } = await garden.configStore.get() expect(linkedProjectSources).to.eql([]) }) }) diff --git a/garden-service/test/unit/src/config/base.ts b/garden-service/test/unit/src/config/base.ts index caa7beba73..6b55f18341 100644 --- a/garden-service/test/unit/src/config/base.ts +++ b/garden-service/test/unit/src/config/base.ts @@ -1,7 +1,7 @@ import { expect } from "chai" import { loadConfig } from "../../../../src/config/base" import { resolve } from "path" -import { dataDir, expectError } from "../../../helpers" +import { dataDir, expectError, getDataDir } from "../../../helpers" const projectPathA = resolve(dataDir, "test-project-a") const modulePathA = resolve(projectPathA, "module-a") @@ -15,11 +15,10 @@ const projectPathFlat = resolve(dataDir, "test-project-flat-config") const modulePathFlatInvalid = resolve(projectPathFlat, "invalid-config-kind") describe("loadConfig", () => { - it("should not throw an error if no file was found", async () => { const parsed = await loadConfig(projectPathA, resolve(projectPathA, "non-existent-module")) - expect(parsed).to.eql(undefined) + expect(parsed).to.eql([]) }) it("should throw a config error if the file couldn't be parsed", async () => { @@ -31,51 +30,39 @@ describe("loadConfig", () => { }) }) - it("should include the module's relative path in the error message for invalid config", async () => { - const projectPath = resolve(dataDir, "test-project-invalid-config") - await expectError( - async () => await loadConfig(projectPath, resolve(projectPath, "invalid-config-module")), - (err) => { - expect(err.message).to.match(/invalid-config-module\/garden.yml/) - }) - }) - // TODO: test more cases it("should load and parse a project config", async () => { const parsed = await loadConfig(projectPathA, projectPathA) - expect(parsed!.project).to.eql({ - apiVersion: "garden.io/v0", - kind: "Project", - name: "test-project-a", - defaultEnvironment: "local", - sources: [], - environmentDefaults: { - providers: [], - variables: { some: "variable" }, - }, - environments: [ - { - name: "local", - providers: [ - { name: "test-plugin" }, - { name: "test-plugin-b" }, - ], - variables: {}, - }, - { - name: "other", - providers: [], - variables: {}, + expect(parsed).to.eql([ + { + apiVersion: "garden.io/v0", + kind: "Project", + path: projectPathA, + name: "test-project-a", + environmentDefaults: { + variables: { some: "variable" }, }, - ], - }) + environments: [ + { + name: "local", + providers: [ + { name: "test-plugin" }, + { name: "test-plugin-b" }, + ], + }, + { + name: "other", + }, + ], + }, + ]) }) it("should load and parse a module config", async () => { const parsed = await loadConfig(projectPathA, modulePathA) - expect(parsed!.modules).to.eql([ + expect(parsed).to.eql([ { apiVersion: "garden.io/v0", kind: "Module", @@ -84,7 +71,7 @@ describe("loadConfig", () => { description: undefined, include: undefined, repositoryUrl: undefined, - allowPublish: true, + allowPublish: undefined, build: { dependencies: [] }, outputs: {}, path: modulePathA, @@ -115,69 +102,65 @@ describe("loadConfig", () => { it("should load and parse a config file defining a project and a module", async () => { const parsed = await loadConfig(projectPathMultipleModules, projectPathMultipleModules) - expect(parsed!.project).to.eql({ - apiVersion: "garden.io/v0", - kind: "Project", - defaultEnvironment: "local", - environmentDefaults: { - providers: [], - variables: { - some: "variable", + expect(parsed).to.eql([ + { + apiVersion: "garden.io/v0", + kind: "Project", + path: projectPathMultipleModules, + environmentDefaults: { + variables: { + some: "variable", + }, }, + environments: [ + { + name: "local", + providers: [ + { name: "test-plugin" }, + { name: "test-plugin-b" }, + ], + }, + { + name: "other", + }, + ], + name: "test-project-multiple-modules", }, - environments: [ - { - name: "local", - providers: [ - { name: "test-plugin" }, - { name: "test-plugin-b" }, - ], - variables: {}, - }, - { - name: "other", - providers: [], - variables: {}, - }, - ], - name: "test-project-multiple-modules", - sources: [], - }) - - expect(parsed!.modules).to.eql([{ - apiVersion: "garden.io/v0", - kind: "Module", - name: "module-from-project-config", - type: "test", - description: undefined, - repositoryUrl: undefined, - allowPublish: true, - build: { dependencies: [] }, - include: undefined, - outputs: {}, - path: projectPathMultipleModules, - serviceConfigs: [], - spec: { - build: { - command: ["echo", "project"], - dependencies: [], + { + apiVersion: "garden.io/v0", + kind: "Module", + name: "module-from-project-config", + type: "test", + description: undefined, + include: undefined, + repositoryUrl: undefined, + allowPublish: undefined, + build: { dependencies: [] }, + outputs: {}, + path: projectPathMultipleModules, + serviceConfigs: [], + spec: { + build: { + command: ["echo", "project"], + dependencies: [], + }, }, + testConfigs: [], + taskConfigs: [], }, - testConfigs: [], - taskConfigs: [], - }]) + ]) }) it("should load and parse a config file defining multiple modules", async () => { const parsed = await loadConfig(projectPathMultipleModules, modulePathAMultiple) - expect(parsed!.modules).to.eql([ + expect(parsed).to.eql([ { apiVersion: "garden.io/v0", kind: "Module", name: "module-a1", type: "test", - allowPublish: true, + allowPublish: undefined, description: undefined, include: undefined, repositoryUrl: undefined, @@ -208,7 +191,7 @@ describe("loadConfig", () => { kind: "Module", name: "module-a2", type: "test", - allowPublish: true, + allowPublish: undefined, description: undefined, include: undefined, repositoryUrl: undefined, @@ -234,57 +217,78 @@ describe("loadConfig", () => { it("should parse a config file using the flat config style", async () => { const parsed = await loadConfig(projectPathFlat, projectPathFlat) - expect(parsed!.project).to.eql({ - apiVersion: "garden.io/v0", - kind: "Project", - defaultEnvironment: "local", - environmentDefaults: { - providers: [], - variables: { some: "variable" }, - }, - environments: [ - { - name: "local", - providers: [ - { name: "test-plugin" }, - { name: "test-plugin-b" }, - ], - variables: {}, - }, - { - name: "other", - providers: [], - variables: {}, + expect(parsed).to.eql([ + { + apiVersion: "garden.io/v0", + kind: "Project", + path: projectPathFlat, + environmentDefaults: { + variables: { some: "variable" }, }, - ], - name: "test-project-flat-config", - sources: [], - }) - - expect(parsed!.modules).to.eql([{ - apiVersion: "garden.io/v0", - kind: "Module", - name: "module-from-project-config", - type: "test", - description: undefined, - build: { - dependencies: [], + environments: [ + { + name: "local", + providers: [ + { name: "test-plugin" }, + { name: "test-plugin-b" }, + ], + }, + { + name: "other", + }, + ], + name: "test-project-flat-config", }, - allowPublish: true, - include: undefined, - outputs: {}, - path: projectPathFlat, - repositoryUrl: undefined, - serviceConfigs: [], - spec: { + { + apiVersion: "garden.io/v0", + kind: "Module", + name: "module-from-project-config", + type: "test", + description: undefined, build: { - command: ["echo", "project"], dependencies: [], }, + allowPublish: undefined, + include: undefined, + outputs: {}, + path: projectPathFlat, + repositoryUrl: undefined, + serviceConfigs: [], + spec: { + build: { + command: ["echo", "project"], + dependencies: [], + }, + }, + taskConfigs: [], + testConfigs: [], + }, + ]) + }) + + it("should load a project config with a top-level provider field", async () => { + const projectPath = getDataDir("test-projects", "new-provider-spec") + const parsed = await loadConfig(projectPath, projectPath) + + expect(parsed).to.eql([ + { + apiVersion: "garden.io/v0", + kind: "Project", + path: projectPath, + name: "test-project-a", + environmentDefaults: { + variables: { some: "variable" }, + }, + environments: [ + { name: "local" }, + { name: "other" }, + ], + providers: [ + { name: "test-plugin", environments: ["local"] }, + { name: "test-plugin-b" }, + ], }, - taskConfigs: [], - testConfigs: [], - }]) + ]) }) it("should throw an error when parsing a flat-style config using an unknown/invalid kind", async () => { @@ -303,9 +307,9 @@ describe("loadConfig", () => { }) }) - it("should return undefined if config file is not found", async () => { + it("should return [] if config file is not found", async () => { const parsed = await loadConfig("/thisdoesnotexist", "/thisdoesnotexist") - expect(parsed).to.be.undefined + expect(parsed).to.eql([]) }) }) diff --git a/garden-service/test/unit/src/config/config-context.ts b/garden-service/test/unit/src/config/config-context.ts index fcd1570c5c..36bb0f3a87 100644 --- a/garden-service/test/unit/src/config/config-context.ts +++ b/garden-service/test/unit/src/config/config-context.ts @@ -233,7 +233,9 @@ describe("ModuleConfigContext", () => { await garden.scanModules() c = new ModuleConfigContext( garden, - garden.environment, + garden.environmentName, + await garden.resolveProviders(), + garden.variables, Object.values((garden).moduleConfigs), ) }) @@ -249,7 +251,7 @@ describe("ModuleConfigContext", () => { }) it("should should resolve the environment config", async () => { - expect(await c.resolve({ key: ["environment", "name"], nodePath: [], opts: {} })).to.equal(garden.environment.name) + expect(await c.resolve({ key: ["environment", "name"], nodePath: [], opts: {} })).to.equal(garden.environmentName) }) it("should should resolve the path of a module", async () => { diff --git a/garden-service/test/unit/src/garden.ts b/garden-service/test/unit/src/garden.ts index 754819a65a..04b85b5d02 100644 --- a/garden-service/test/unit/src/garden.ts +++ b/garden-service/test/unit/src/garden.ts @@ -13,6 +13,7 @@ import { cleanProject, stubGitCli, testModuleVersion, + TestGarden, } from "../../helpers" import { getNames } from "../../../src/util/util" import { MOCK_CONFIG } from "../../../src/cli/cli" @@ -20,6 +21,15 @@ import { LinkedSource } from "../../../src/config-store" import { ModuleVersion } from "../../../src/vcs/vcs" import { hashRepoUrl } from "../../../src/util/ext-source-util" import { getModuleCacheContext } from "../../../src/types/module" +import { Plugins, GardenPlugin, PluginFactory } from "../../../src/types/plugin/plugin" +import { ConfigureProviderParams } from "../../../src/types/plugin/provider/configureProvider" +import { ProjectConfig } from "../../../src/config/project" +import { ModuleConfig } from "../../../src/config/module" +import { DEFAULT_API_VERSION } from "../../../src/constants" +import * as Joi from "joi" +import { providerConfigBaseSchema } from "../../../src/config/provider" +import { keyBy } from "lodash" +import stripAnsi from "strip-ansi" describe("Garden", () => { beforeEach(async () => { @@ -27,15 +37,12 @@ describe("Garden", () => { }) describe("factory", () => { - it("should throw when initializing with missing plugins", async () => { - await expectError(async () => await Garden.factory(projectRootA), "configuration") - }) - it("should initialize and add the action handlers for a plugin", async () => { const garden = await makeTestGardenA() + const actions = await garden.getActionHelper() - expect((garden).actions.actionHandlers.prepareEnvironment["test-plugin"]).to.be.ok - expect((garden).actions.actionHandlers.prepareEnvironment["test-plugin-b"]).to.be.ok + expect((actions).actionHandlers.prepareEnvironment["test-plugin"]).to.be.ok + expect((actions).actionHandlers.prepareEnvironment["test-plugin-b"]).to.be.ok }) it("should initialize with MOCK_CONFIG", async () => { @@ -45,19 +52,53 @@ describe("Garden", () => { it("should parse and resolve the config from the project root", async () => { const garden = await makeTestGardenA() + const projectRoot = garden.projectRoot expect(garden.projectName).to.equal("test-project-a") - expect(garden.environment).to.eql({ - name: "local", - providers: [ - { name: "exec", config: { name: "exec" } }, - { name: "container", config: { name: "container" } }, - { name: "test-plugin", config: { name: "test-plugin" } }, - { name: "test-plugin-b", config: { name: "test-plugin-b" } }, - ], - variables: { - some: "variable", + + expect(await garden.resolveProviders()).to.eql([ + { + name: "exec", + config: { + name: "exec", + path: projectRoot, + }, + dependencies: [], + moduleConfigs: [], + }, + { + name: "container", + config: { + name: "container", + path: projectRoot, + }, + dependencies: [], + moduleConfigs: [], }, + { + name: "test-plugin", + config: { + name: "test-plugin", + environments: ["local"], + path: projectRoot, + }, + dependencies: [], + moduleConfigs: [], + }, + { + name: "test-plugin-b", + config: { + name: "test-plugin-b", + environments: ["local"], + path: projectRoot, + }, + dependencies: [], + moduleConfigs: [], + }, + ]) + + expect(garden.variables).to.eql({ + some: "variable", }) }) @@ -72,17 +113,40 @@ describe("Garden", () => { delete process.env.TEST_PROVIDER_TYPE delete process.env.TEST_VARIABLE - expect(garden.environment).to.eql({ - name: "local", - providers: [ - { name: "exec", config: { name: "exec" } }, - { name: "container", config: { name: "container" } }, - { name: "test-plugin", config: { name: "test-plugin" } }, - ], - variables: { - "some": "banana", - "service-a-build-command": "OK", + expect(await garden.resolveProviders()).to.eql([ + { + name: "exec", + config: { + name: "exec", + path: projectRoot, + }, + dependencies: [], + moduleConfigs: [], }, + { + name: "container", + config: { + name: "container", + path: projectRoot, + }, + dependencies: [], + moduleConfigs: [], + }, + { + name: "test-plugin", + config: { + name: "test-plugin", + path: projectRoot, + environments: ["local"], + }, + dependencies: [], + moduleConfigs: [], + }, + ]) + + expect(garden.variables).to.eql({ + "some": "banana", + "service-a-build-command": "OK", }) }) @@ -90,14 +154,10 @@ describe("Garden", () => { await expectError(async () => Garden.factory(projectRootA, { environmentName: "bla" }), "parameter") }) - it("should throw if namespace starts with 'garden-'", async () => { + it("should throw if environment starts with 'garden-'", async () => { await expectError(async () => Garden.factory(projectRootA, { environmentName: "garden-bla" }), "parameter") }) - it("should throw if no provider is configured for the environment", async () => { - await expectError(async () => Garden.factory(projectRootA, { environmentName: "other" }), "configuration") - }) - it("should throw if plugin module exports invalid name", async () => { const pluginPath = join(__dirname, "plugins", "invalid-exported-name.js") const plugins = { foo: pluginPath } @@ -120,6 +180,433 @@ describe("Garden", () => { }) }) + describe("resolveProviders", () => { + it("should throw when when plugins are missing", async () => { + const garden = await Garden.factory(projectRootA) + await expectError(() => garden.resolveProviders(), "configuration") + }) + + it("should pass through a basic provider config", async () => { + const garden = await makeTestGardenA() + const projectRoot = garden.projectRoot + + expect(await garden.resolveProviders()).to.eql([ + { + name: "exec", + config: { + name: "exec", + path: projectRoot, + }, + dependencies: [], + moduleConfigs: [], + }, + { + name: "container", + config: { + name: "container", + path: projectRoot, + }, + dependencies: [], + moduleConfigs: [], + }, + { + name: "test-plugin", + config: { + name: "test-plugin", + environments: ["local"], + path: projectRoot, + }, + dependencies: [], + moduleConfigs: [], + }, + { + name: "test-plugin-b", + config: { + name: "test-plugin-b", + environments: ["local"], + path: projectRoot, + }, + dependencies: [], + moduleConfigs: [], + }, + ]) + }) + + it("should call a configureProvider handler if applicable", async () => { + const test: PluginFactory = (): GardenPlugin => { + return { + actions: { + async configureProvider({ config }: ConfigureProviderParams) { + expect(config).to.eql({ + name: "test", + path: projectRootA, + foo: "bar", + }) + return { config } + }, + }, + } + } + + const projectConfig: ProjectConfig = { + apiVersion: "garden.io/v0", + kind: "Project", + name: "test", + path: projectRootA, + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "test", foo: "bar" }, + ], + variables: {}, + } + + const plugins: Plugins = { test } + const garden = await TestGarden.factory(projectRootA, { config: projectConfig, plugins }) + await garden.resolveProviders() + }) + + it("should give a readable error if provider configs have invalid template strings", async () => { + const test: PluginFactory = (): GardenPlugin => { + return {} + } + + const projectConfig: ProjectConfig = { + apiVersion: "garden.io/v0", + kind: "Project", + name: "test", + path: projectRootA, + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "test", foo: "\${bla.ble}" }, + ], + variables: {}, + } + + const plugins: Plugins = { test } + const garden = await TestGarden.factory(projectRootA, { config: projectConfig, plugins }) + await expectError( + () => garden.resolveProviders(), + err => expect(err.message).to.equal( + "Failed resolving one or more provider configurations:\n- test: Could not find key: bla.ble", + ), + ) + }) + + it("should give a readable error if providers reference non-existent providers", async () => { + const test: PluginFactory = (): GardenPlugin => { + return { + dependencies: ["foo"], + } + } + + const projectConfig: ProjectConfig = { + apiVersion: "garden.io/v0", + kind: "Project", + name: "test", + path: projectRootA, + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "test" }, + ], + variables: {}, + } + + const plugins: Plugins = { test } + const garden = await TestGarden.factory(projectRootA, { config: projectConfig, plugins }) + await expectError( + () => garden.resolveProviders(), + err => expect(err.message).to.equal( + "Missing provider dependency 'foo' in configuration for provider 'test'. " + + "Are you missing a provider configuration?", + ), + ) + }) + + it("should add plugin modules if returned by the provider", async () => { + const pluginModule: ModuleConfig = { + apiVersion: DEFAULT_API_VERSION, + allowPublish: false, + build: { dependencies: [] }, + name: "foo", + outputs: {}, + path: "/tmp", + serviceConfigs: [], + taskConfigs: [], + spec: {}, + testConfigs: [], + type: "exec", + } + + const test: PluginFactory = (): GardenPlugin => { + return { + actions: { + async configureProvider({ config }: ConfigureProviderParams) { + return { config, moduleConfigs: [pluginModule] } + }, + }, + moduleActions: { + test: { + configure: async ({ moduleConfig }) => { + return moduleConfig + }, + }, + }, + } + } + + const projectConfig: ProjectConfig = { + apiVersion: "garden.io/v0", + kind: "Project", + name: "test", + path: projectRootA, + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "test", foo: "bar" }, + ], + variables: {}, + } + + const plugins: Plugins = { test } + const garden = await TestGarden.factory(projectRootA, { config: projectConfig, plugins }) + + const graph = await garden.getConfigGraph() + expect(await graph.getModule("test--foo")).to.exist + }) + + it("should throw if plugins have declared circular dependencies", async () => { + const testA: PluginFactory = (): GardenPlugin => { + return { + dependencies: ["test-b"], + } + } + + const testB: PluginFactory = (): GardenPlugin => { + return { + dependencies: ["test-a"], + } + } + + const projectConfig: ProjectConfig = { + apiVersion: "garden.io/v0", + kind: "Project", + name: "test", + path: projectRootA, + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "test-a" }, + { name: "test-b" }, + ], + variables: {}, + } + + const plugins: Plugins = { "test-a": testA, "test-b": testB } + const garden = await TestGarden.factory(projectRootA, { config: projectConfig, plugins }) + + await expectError( + () => garden.resolveProviders(), + err => expect(err.message).to.equal( + "One or more circular dependencies found between providers " + + "or their configurations: test-a <- test-b <- test-a", + ), + ) + }) + + it("should throw if plugins reference themselves as dependencies", async () => { + const testA: PluginFactory = (): GardenPlugin => { + return { + dependencies: ["test-a"], + } + } + + const projectConfig: ProjectConfig = { + apiVersion: "garden.io/v0", + kind: "Project", + name: "test", + path: projectRootA, + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "test-a" }, + ], + variables: {}, + } + + const plugins: Plugins = { "test-a": testA } + const garden = await TestGarden.factory(projectRootA, { config: projectConfig, plugins }) + + await expectError( + () => garden.resolveProviders(), + err => expect(err.message).to.equal( + "One or more circular dependencies found between providers " + + "or their configurations: test-a <- test-a", + ), + ) + }) + + it("should throw if provider configs have implicit circular dependencies", async () => { + const testA: PluginFactory = (): GardenPlugin => { + return {} + } + + const testB: PluginFactory = (): GardenPlugin => { + return {} + } + + const projectConfig: ProjectConfig = { + apiVersion: "garden.io/v0", + kind: "Project", + name: "test", + path: projectRootA, + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "test-a", foo: "\${provider.test-b.outputs.foo}" }, + { name: "test-b", foo: "\${provider.test-a.outputs.foo}" }, + ], + variables: {}, + } + + const plugins: Plugins = { "test-a": testA, "test-b": testB } + const garden = await TestGarden.factory(projectRootA, { config: projectConfig, plugins }) + + await expectError( + () => garden.resolveProviders(), + err => expect(err.message).to.equal( + "One or more circular dependencies found between providers " + + "or their configurations: test-a <- test-b <- test-a", + ), + ) + }) + + it("should throw if provider configs have combined implicit and declared circular dependencies", async () => { + const testA: PluginFactory = (): GardenPlugin => { + return {} + } + + const testB: PluginFactory = (): GardenPlugin => { + return { + dependencies: ["test-a"], + } + } + + const projectConfig: ProjectConfig = { + apiVersion: "garden.io/v0", + kind: "Project", + name: "test", + path: projectRootA, + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "test-a", foo: "\${provider.test-b.outputs.foo}" }, + { name: "test-b" }, + ], + variables: {}, + } + + const plugins: Plugins = { "test-a": testA, "test-b": testB } + const garden = await TestGarden.factory(projectRootA, { config: projectConfig, plugins }) + + await expectError( + () => garden.resolveProviders(), + err => expect(err.message).to.equal( + "One or more circular dependencies found between providers " + + "or their configurations: test-b <- test-a <- test-b", + ), + ) + }) + + it("should apply default values from a plugin's configuration schema if specified", async () => { + const test: PluginFactory = (): GardenPlugin => { + return { + configSchema: providerConfigBaseSchema + .keys({ + foo: Joi.string().default("bar"), + }), + } + } + + const projectConfig: ProjectConfig = { + apiVersion: "garden.io/v0", + kind: "Project", + name: "test", + path: projectRootA, + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "test" }, + ], + variables: {}, + } + + const plugins: Plugins = { test } + const garden = await TestGarden.factory(projectRootA, { config: projectConfig, plugins }) + const providers = keyBy(await garden.resolveProviders(), "name") + + expect(providers.test).to.exist + expect(providers.test.config.foo).to.equal("bar") + }) + + it("should throw if a config doesn't match a plugin's configuration schema", async () => { + const test: PluginFactory = (): GardenPlugin => { + return { + configSchema: providerConfigBaseSchema + .keys({ + foo: Joi.string(), + }), + } + } + + const projectConfig: ProjectConfig = { + apiVersion: "garden.io/v0", + kind: "Project", + name: "test", + path: projectRootA, + defaultEnvironment: "default", + environments: [ + { name: "default", variables: {} }, + ], + providers: [ + { name: "test", foo: 123 }, + ], + variables: {}, + } + + const plugins: Plugins = { test } + const garden = await TestGarden.factory(projectRootA, { config: projectConfig, plugins }) + + await expectError( + () => garden.resolveProviders(), + err => expect(stripAnsi(err.message)).to.equal( + "Failed resolving one or more provider configurations:\n- " + + "test: Error validating provider (/garden.yml): key .foo must be a string", + ), + ) + }) + }) + describe("scanModules", () => { // TODO: assert that gitignore in project root is respected it("should scan the project root for modules and add to the context", async () => { @@ -190,17 +677,6 @@ describe("Garden", () => { const module = (await (garden).loadModuleConfigs("./module-a"))[0] expect(module!.name).to.equal("module-a") }) - - it("should resolve module path to external sources dir if module has a remote source", async () => { - const projectRoot = resolve(dataDir, "test-project-ext-module-sources") - const garden = await makeTestGarden(projectRoot) - stubGitCli(garden) - - const module = (await (garden).loadModuleConfigs("./module-a"))[0] - const repoUrlHash = hashRepoUrl(module!.repositoryUrl!) - - expect(module!.path).to.equal(join(projectRoot, ".garden", "sources", "module", `module-a--${repoUrlHash}`)) - }) }) describe("resolveModuleConfigs", () => { @@ -214,6 +690,21 @@ describe("Garden", () => { ), ) }) + + it("should resolve module path to external sources dir if module has a remote source", async () => { + const projectRoot = resolve(dataDir, "test-project-ext-module-sources") + const garden = await makeTestGarden(projectRoot) + stubGitCli(garden) + + const module = await garden.resolveModuleConfig("module-a") + const repoUrlHash = hashRepoUrl(module!.repositoryUrl!) + + expect(module!.path).to.equal(join(projectRoot, ".garden", "sources", "module", `module-a--${repoUrlHash}`)) + }) + + it.skip("should set default values properly", async () => { + throw new Error("TODO") + }) }) describe("resolveVersion", () => { @@ -312,7 +803,7 @@ describe("Garden", () => { name: "source-a", path: localPath, }] - await garden.localConfigStore.set(["linkedProjectSources"], linked) + await garden.configStore.set(["linkedProjectSources"], linked) const path = await garden.loadExtSourcePath({ name: "source-a", @@ -332,7 +823,7 @@ describe("Garden", () => { name: "module-a", path: localPath, }] - await garden.localConfigStore.set(["linkedModuleSources"], linked) + await garden.configStore.set(["linkedModuleSources"], linked) const path = await garden.loadExtSourcePath({ name: "module-a", diff --git a/garden-service/test/unit/src/plugins/container.ts b/garden-service/test/unit/src/plugins/container.ts index 37d8a7e9ba..8a0cac5fb4 100644 --- a/garden-service/test/unit/src/plugins/container.ts +++ b/garden-service/test/unit/src/plugins/container.ts @@ -63,7 +63,7 @@ describe("plugins.container", () => { beforeEach(async () => { garden = await makeTestGarden(projectRoot, { container: gardenPlugin }) log = garden.log - ctx = garden.getPluginContext("container") + ctx = await garden.getPluginContext("container") td.replace(garden.buildDir, "syncDependencyProducts", () => null) diff --git a/garden-service/test/unit/src/plugins/exec.ts b/garden-service/test/unit/src/plugins/exec.ts index 68a82bd49d..daea885b7a 100644 --- a/garden-service/test/unit/src/plugins/exec.ts +++ b/garden-service/test/unit/src/plugins/exec.ts @@ -145,7 +145,8 @@ describe("exec plugin", () => { await writeModuleVersionFile(versionFilePath, version) - const result = await garden.actions.getBuildStatus({ log, module }) + const actions = await garden.getActionHelper() + const result = await actions.getBuildStatus({ log, module }) expect(result.ready).to.be.true }) @@ -159,7 +160,8 @@ describe("exec plugin", () => { const versionFilePath = join(buildMetadataPath, GARDEN_BUILD_VERSION_FILENAME) await garden.buildDir.syncFromSrc(module, log) - await garden.actions.build({ log, module }) + const actions = await garden.getActionHelper() + await actions.build({ log, module }) const versionFileContents = await readModuleVersionFile(versionFilePath) diff --git a/garden-service/test/unit/src/plugins/kubernetes/container/ingress.ts b/garden-service/test/unit/src/plugins/kubernetes/container/ingress.ts index 8934625788..ea741f0e78 100644 --- a/garden-service/test/unit/src/plugins/kubernetes/container/ingress.ts +++ b/garden-service/test/unit/src/plugins/kubernetes/container/ingress.ts @@ -55,6 +55,8 @@ const basicConfig: KubernetesConfig = { const basicProvider: KubernetesProvider = { name: "kubernetes", config: basicConfig, + dependencies: [], + moduleConfigs: [], } const singleTlsConfig: KubernetesConfig = { @@ -72,6 +74,8 @@ const singleTlsConfig: KubernetesConfig = { const singleTlsProvider: KubernetesProvider = { name: "kubernetes", config: singleTlsConfig, + dependencies: [], + moduleConfigs: [], } const multiTlsConfig: KubernetesConfig = { @@ -105,6 +109,8 @@ const multiTlsConfig: KubernetesConfig = { const multiTlsProvider: KubernetesProvider = { name: "kubernetes", config: multiTlsConfig, + dependencies: [], + moduleConfigs: [], } // generated with `openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 365 -out certificate.pem` @@ -636,6 +642,8 @@ describe("createIngressResources", () => { secretRef: { name: "foo", namespace: "default" }, }], }, + dependencies: [], + moduleConfigs: [], } const err: any = new Error("nope") @@ -663,6 +671,8 @@ describe("createIngressResources", () => { secretRef: { name: "foo", namespace: "default" }, }], }, + dependencies: [], + moduleConfigs: [], } const api = await getKubeApi(basicConfig.context) @@ -696,6 +706,8 @@ describe("createIngressResources", () => { secretRef: { name: "foo", namespace: "default" }, }], }, + dependencies: [], + moduleConfigs: [], } const api = await getKubeApi(basicConfig.context) @@ -785,6 +797,8 @@ describe("createIngressResources", () => { secretRef: { name: "somesecret", namespace: "somenamespace" }, }], }, + dependencies: [], + moduleConfigs: [], } td.when(api.core.readNamespacedSecret("foo", "default")).thenResolve({ diff --git a/garden-service/test/unit/src/plugins/kubernetes/helm/common.ts b/garden-service/test/unit/src/plugins/kubernetes/helm/common.ts index ee2ceb75e4..3e9adba2c6 100644 --- a/garden-service/test/unit/src/plugins/kubernetes/helm/common.ts +++ b/garden-service/test/unit/src/plugins/kubernetes/helm/common.ts @@ -32,7 +32,7 @@ describe("Helm common functions", () => { const projectRoot = resolve(dataDir, "test-projects", "helm") garden = await makeTestGarden(projectRoot) graph = await garden.getConfigGraph() - ctx = garden.getPluginContext("local-kubernetes") + ctx = await garden.getPluginContext("local-kubernetes") log = garden.log await buildModules() }) diff --git a/garden-service/test/unit/src/plugins/kubernetes/helm/config.ts b/garden-service/test/unit/src/plugins/kubernetes/helm/config.ts index f49162da49..bd2140f78d 100644 --- a/garden-service/test/unit/src/plugins/kubernetes/helm/config.ts +++ b/garden-service/test/unit/src/plugins/kubernetes/helm/config.ts @@ -4,35 +4,33 @@ import { cloneDeep } from "lodash" import { TestGarden, dataDir, makeTestGarden, expectError } from "../../../../../helpers" import { PluginContext } from "../../../../../../src/plugin-context" -import { validateHelmModule, helmModuleSpecSchema } from "../../../../../../src/plugins/kubernetes/helm/config" import { deline } from "../../../../../../src/util/string" -import { LogEntry } from "../../../../../../src/logger/log-entry" -import { validate } from "../../../../../../src/config/common" +import { ModuleConfig } from "../../../../../../src/config/module" +import { apply } from "json-merge-patch" describe("validateHelmModule", () => { let garden: TestGarden let ctx: PluginContext - let log: LogEntry + let moduleConfigs: { [key: string]: ModuleConfig } before(async () => { const projectRoot = resolve(dataDir, "test-projects", "helm") garden = await makeTestGarden(projectRoot) - log = garden.log - ctx = garden.getPluginContext("local-kubernetes") + ctx = await garden.getPluginContext("local-kubernetes") await garden.resolveModuleConfigs() + moduleConfigs = cloneDeep((garden).moduleConfigs) + }) + + beforeEach(() => { + (garden).moduleConfigs = cloneDeep(moduleConfigs) }) after(async () => { await garden.close() }) - function getModuleConfig(name: string) { - const config = cloneDeep((garden).moduleConfigs[name]) - config.spec = validate(config.spec, helmModuleSpecSchema) - config.serviceConfigs = [] - config.taskConfigs = [] - config.testConfigs = [] - return config + function patchModuleConfig(name: string, patch: any) { + apply((garden).moduleConfigs[name], patch) } it("should validate a Helm module", async () => { @@ -129,17 +127,15 @@ describe("validateHelmModule", () => { }) it("should not return a serviceConfig if skipDeploy=true", async () => { - const moduleConfig = getModuleConfig("api") - moduleConfig.spec.skipDeploy = true - const config = await validateHelmModule({ ctx, moduleConfig, log }) + patchModuleConfig("api", { spec: { skipDeploy: true } }) + const config = await garden.resolveModuleConfig("api") expect(config.serviceConfigs).to.eql([]) }) it("should add the module specified under 'base' as a build dependency", async () => { - const moduleConfig = getModuleConfig("postgres") - moduleConfig.spec.base = "foo" - const config = await validateHelmModule({ ctx, moduleConfig, log }) + patchModuleConfig("postgres", { spec: { base: "foo" } }) + const config = await garden.resolveModuleConfig("postgres") expect(config.build.dependencies).to.eql([ { name: "foo", copy: [{ source: "*", target: "." }] }, @@ -147,10 +143,11 @@ describe("validateHelmModule", () => { }) it("should add copy spec to build dependency if it's already a dependency", async () => { - const moduleConfig = getModuleConfig("postgres") - moduleConfig.build.dependencies = [{ name: "foo", copy: [] }] - moduleConfig.spec.base = "foo" - const config = await validateHelmModule({ ctx, moduleConfig, log }) + patchModuleConfig("postgres", { + build: { dependencies: [{ name: "foo", copy: [] }] }, + spec: { base: "foo" }, + }) + const config = await garden.resolveModuleConfig("postgres") expect(config.build.dependencies).to.eql([ { name: "foo", copy: [{ source: "*", target: "." }] }, @@ -158,11 +155,14 @@ describe("validateHelmModule", () => { }) it("should add module specified under tasks[].resource.containerModule as a build dependency", async () => { - const moduleConfig = getModuleConfig("api") - moduleConfig.spec.tasks = [ - { name: "my-task", resource: { kind: "Deployment", containerModule: "foo" } }, - ] - const config = await validateHelmModule({ ctx, moduleConfig, log }) + patchModuleConfig("api", { + spec: { + tasks: [ + { name: "my-task", resource: { kind: "Deployment", containerModule: "foo" } }, + ], + }, + }) + const config = await garden.resolveModuleConfig("api") expect(config.build.dependencies).to.eql([ { name: "foo", copy: [] }, @@ -170,11 +170,14 @@ describe("validateHelmModule", () => { }) it("should add module specified under tests[].resource.containerModule as a build dependency", async () => { - const moduleConfig = getModuleConfig("api") - moduleConfig.spec.tests = [ - { name: "my-task", resource: { kind: "Deployment", containerModule: "foo" } }, - ] - const config = await validateHelmModule({ ctx, moduleConfig, log }) + patchModuleConfig("api", { + spec: { + tests: [ + { name: "my-task", resource: { kind: "Deployment", containerModule: "foo" } }, + ], + }, + }) + const config = await garden.resolveModuleConfig("api") expect(config.build.dependencies).to.eql([ { name: "foo", copy: [] }, @@ -182,10 +185,10 @@ describe("validateHelmModule", () => { }) it("should throw if chart both contains sources and specifies base", async () => { - const moduleConfig = getModuleConfig("api") - moduleConfig.spec.base = "foo" + patchModuleConfig("api", { spec: { base: "foo" } }) + await expectError( - () => validateHelmModule({ ctx, moduleConfig, log }), + () => garden.resolveModuleConfig("api"), err => expect(err.message).to.equal(deline` Helm module 'api' both contains sources and specifies a base module. Since Helm charts cannot currently be merged, please either remove the sources or @@ -195,10 +198,10 @@ describe("validateHelmModule", () => { }) it("should throw if chart contains no sources and doesn't specify chart name nor base", async () => { - const moduleConfig = getModuleConfig("postgres") - delete moduleConfig.spec.chart + patchModuleConfig("postgres", { spec: { chart: null } }) + await expectError( - () => validateHelmModule({ ctx, moduleConfig, log }), + () => garden.resolveModuleConfig("postgres"), err => expect(err.message).to.equal(deline` Chart neither specifies a chart name, base module, nor contains chart sources at \`chartPath\`. `), diff --git a/garden-service/test/unit/src/task-graph.ts b/garden-service/test/unit/src/task-graph.ts index fb9831c010..91d56d2f83 100644 --- a/garden-service/test/unit/src/task-graph.ts +++ b/garden-service/test/unit/src/task-graph.ts @@ -1,14 +1,9 @@ import { join } from "path" import { expect } from "chai" import { BaseTask, TaskType } from "../../../src/tasks/base" -import { - TaskGraph, - TaskResult, - TaskResults, -} from "../../../src/task-graph" +import { TaskGraph, TaskResult, TaskResults } from "../../../src/task-graph" import { makeTestGarden, freezeTime, dataDir } from "../../helpers" import { Garden } from "../../../src/garden" -import { DependencyGraphNodeType } from "../../../src/config-graph" const projectRoot = join(dataDir, "test-project-empty") @@ -23,7 +18,6 @@ interface TestTaskOptions { class TestTask extends BaseTask { type: TaskType = "test" - depType: DependencyGraphNodeType = "test" name: string callback: TestTaskCallback | null uid: string @@ -32,7 +26,7 @@ class TestTask extends BaseTask { constructor( garden: Garden, name: string, - force, + force: boolean, options?: TestTaskOptions, ) { super({ @@ -107,6 +101,7 @@ describe("task-graph", () => { type: "test", description: "a", key: "a", + name: "a", output: { result: "result-a", dependencyResults: {}, @@ -118,7 +113,7 @@ describe("task-graph", () => { expect(results).to.eql(expected) }) - it("should emit events when processing and completing a task", async () => { + it("should emit a taskPending event when adding a task", async () => { const now = freezeTime() const garden = await getGarden() @@ -136,12 +131,15 @@ describe("task-graph", () => { ]) }) - it("should not emit a taskPending event when adding a task with a cached result", async () => { + it.skip("should throw if tasks have circular dependencies", async () => { + throw new Error("TODO") + }) + + it("should emit events when processing and completing a task", async () => { const now = freezeTime() const garden = await getGarden() const graph = new TaskGraph(garden, garden.log) - const task = new TestTask(garden, "a", false) await graph.process([task]) @@ -238,6 +236,7 @@ describe("task-graph", () => { type: "test", description: "a.a1", key: "a", + name: "a", output: { result: "result-a.a1", dependencyResults: {}, @@ -247,6 +246,7 @@ describe("task-graph", () => { const resultB: TaskResult = { type: "test", key: "b", + name: "b", description: "b.b1", output: { result: "result-b.b1", @@ -258,6 +258,7 @@ describe("task-graph", () => { type: "test", description: "c.c1", key: "c", + name: "c", output: { result: "result-c.c1", dependencyResults: { b: resultB }, @@ -273,6 +274,7 @@ describe("task-graph", () => { type: "test", description: "d.d1", key: "d", + name: "d", output: { result: "result-d.d1", dependencyResults: { @@ -335,6 +337,7 @@ describe("task-graph", () => { type: "test", description: "a", key: "a", + name: "a", output: { result: "result-a", dependencyResults: {}, diff --git a/garden-service/test/unit/src/template-string.ts b/garden-service/test/unit/src/template-string.ts index 4fcb22d941..2245897e32 100644 --- a/garden-service/test/unit/src/template-string.ts +++ b/garden-service/test/unit/src/template-string.ts @@ -1,5 +1,5 @@ import { expect } from "chai" -import { resolveTemplateString, resolveTemplateStrings } from "../../../src/template-string" +import { resolveTemplateString, resolveTemplateStrings, collectTemplateReferences } from "../../../src/template-string" import { ConfigContext } from "../../../src/config/config-context" import { expectError } from "../../helpers" @@ -279,3 +279,22 @@ describe("resolveTemplateStrings", () => { }) }) }) + +describe("collectTemplateReferences", () => { + it("should return and sort all template string references in an object", async () => { + const obj = { + foo: "\${my.reference}", + nested: { + boo: "\${moo}", + foo: "lalalla\${moo}\${moo}", + banana: "\${banana.rama.llama}", + }, + } + + expect(await collectTemplateReferences(obj)).to.eql([ + ["banana", "rama", "llama"], + ["moo"], + ["my", "reference"], + ]) + }) +}) diff --git a/garden-service/test/unit/src/util/ext-source-util.ts b/garden-service/test/unit/src/util/ext-source-util.ts index f67401ff71..19e157c0d6 100644 --- a/garden-service/test/unit/src/util/ext-source-util.ts +++ b/garden-service/test/unit/src/util/ext-source-util.ts @@ -55,12 +55,12 @@ describe("ext-source-util", () => { describe("getLinkedSources", () => { it("should get linked project sources", async () => { - await garden.localConfigStore.set(["linkedProjectSources"], sources) + await garden.configStore.set(["linkedProjectSources"], sources) expect(await getLinkedSources(garden, "project")).to.eql(sources) }) it("should get linked module sources", async () => { - await garden.localConfigStore.set(["linkedModuleSources"], sources) + await garden.configStore.set(["linkedModuleSources"], sources) expect(await getLinkedSources(garden, "module")).to.eql(sources) }) @@ -69,22 +69,22 @@ describe("ext-source-util", () => { describe("addLinkedSources", () => { it("should add linked project sources to local config", async () => { await addLinkedSources({ garden, sourceType: "project", sources }) - expect(await garden.localConfigStore.get(["linkedProjectSources"])).to.eql(sources) + expect(await garden.configStore.get(["linkedProjectSources"])).to.eql(sources) }) it("should add linked module sources to local config", async () => { await addLinkedSources({ garden, sourceType: "module", sources }) - expect(await garden.localConfigStore.get(["linkedModuleSources"])).to.eql(sources) + expect(await garden.configStore.get(["linkedModuleSources"])).to.eql(sources) }) it("should append sources to local config if key already has value", async () => { - const { localConfigStore } = garden + const { configStore: localConfigStore } = garden await localConfigStore.set(["linkedModuleSources"], sources) const newSources = [{ name: "name-c", path: "path-c" }] await addLinkedSources({ garden, sourceType: "module", sources: newSources }) - expect(await garden.localConfigStore.get(["linkedModuleSources"])).to.eql(sources.concat(newSources)) + expect(await garden.configStore.get(["linkedModuleSources"])).to.eql(sources.concat(newSources)) }) @@ -92,34 +92,34 @@ describe("ext-source-util", () => { describe("removeLinkedSources", () => { it("should remove linked project sources from local config", async () => { - await garden.localConfigStore.set(["linkedModuleSources"], sources) + await garden.configStore.set(["linkedModuleSources"], sources) const names = ["name-a"] await removeLinkedSources({ garden, sourceType: "module", names }) - expect(await garden.localConfigStore.get(["linkedModuleSources"])).to.eql([{ + expect(await garden.configStore.get(["linkedModuleSources"])).to.eql([{ name: "name-b", path: "path-b", }]) }) it("should remove linked module sources from local config", async () => { - await garden.localConfigStore.set(["linkedProjectSources"], sources) + await garden.configStore.set(["linkedProjectSources"], sources) const names = ["name-a"] await removeLinkedSources({ garden, sourceType: "project", names }) - expect(await garden.localConfigStore.get(["linkedProjectSources"])).to.eql([{ + expect(await garden.configStore.get(["linkedProjectSources"])).to.eql([{ name: "name-b", path: "path-b", }]) }) it("should remove multiple sources from local config", async () => { - await garden.localConfigStore.set(["linkedModuleSources"], sources.concat({ name: "name-c", path: "path-c" })) + await garden.configStore.set(["linkedModuleSources"], sources.concat({ name: "name-c", path: "path-c" })) const names = ["name-a", "name-b"] await removeLinkedSources({ garden, sourceType: "module", names }) - expect(await garden.localConfigStore.get(["linkedModuleSources"])).to.eql([{ + expect(await garden.configStore.get(["linkedModuleSources"])).to.eql([{ name: "name-c", path: "path-c", }]) }) diff --git a/garden-service/test/unit/src/util/validate-dependencies.ts b/garden-service/test/unit/src/util/validate-dependencies.ts index 8384de6681..8d4521f8bc 100644 --- a/garden-service/test/unit/src/util/validate-dependencies.ts +++ b/garden-service/test/unit/src/util/validate-dependencies.ts @@ -3,7 +3,7 @@ import { join } from "path" import { detectCycles, detectMissingDependencies, - detectCircularDependencies, + detectCircularModuleDependencies, } from "../../../../src/util/validate-dependencies" import { makeTestGarden, dataDir } from "../../../helpers" import { ModuleConfig } from "../../../../src/config/module" @@ -62,7 +62,7 @@ describe("validate-dependencies", () => { const circularProjectRoot = join(dataDir, "test-project-circular-deps") const garden = await makeTestGarden(circularProjectRoot) const { moduleConfigs } = await scanAndGetConfigs(garden) - const err = detectCircularDependencies(moduleConfigs) + const err = detectCircularModuleDependencies(moduleConfigs) expect(err).to.be.an.instanceOf(ConfigurationError) }) @@ -70,7 +70,7 @@ describe("validate-dependencies", () => { const nonCircularProjectRoot = join(dataDir, "test-project-b") const garden = await makeTestGarden(nonCircularProjectRoot) const { moduleConfigs } = await scanAndGetConfigs(garden) - const err = detectCircularDependencies(moduleConfigs) + const err = detectCircularModuleDependencies(moduleConfigs) expect(err).to.eql(null) }) }) @@ -79,7 +79,7 @@ describe("validate-dependencies", () => { it("should detect self-to-self cycles", () => { const cycles = detectCycles({ a: { a: { distance: 1, next: "a" } }, - }, ["a"]) + }) expect(cycles).to.deep.eq([["a"]]) }) @@ -89,7 +89,7 @@ describe("validate-dependencies", () => { foo: { bar: { distance: 1, next: "bar" } }, bar: { baz: { distance: 1, next: "baz" } }, baz: { foo: { distance: 1, next: "foo" } }, - }, ["foo", "bar", "baz"]) + }) expect(cycles).to.deep.eq([["foo", "bar", "baz"]]) }) diff --git a/garden-service/test/unit/src/vcs/vcs.ts b/garden-service/test/unit/src/vcs/vcs.ts index c242189903..ef4ce70734 100644 --- a/garden-service/test/unit/src/vcs/vcs.ts +++ b/garden-service/test/unit/src/vcs/vcs.ts @@ -135,10 +135,13 @@ describe("VcsHandler", () => { moduleABefore = await templateGarden.resolveModuleConfig("module-a") // uses the echo-string variable moduleBBefore = await templateGarden.resolveModuleConfig("module-b") // does not use the echo-string variable - const moduleAAfterEnv = cloneDeep(templateGarden.environment) - moduleAAfterEnv.variables["echo-string"] = "something else" const configContext = new ModuleConfigContext( - templateGarden, moduleAAfterEnv, await templateGarden.getRawModuleConfigs()) + templateGarden, + templateGarden.environmentName, + await templateGarden.resolveProviders(), + { ...templateGarden.variables, "echo-string": "something else" }, + await templateGarden.getRawModuleConfigs(), + ) moduleAAfter = await templateGarden.resolveModuleConfig("module-a", { configContext }) moduleBAfter = await templateGarden.resolveModuleConfig("module-b", { configContext })