feat(config): allow multiple config files in same directory

We now allow Garden configuration files to have custom names, in the
form of `*.garden.yml` or `*.garden.yaml`. This means that a single
directory can contain any number of configuration files.

For example, in your project root you might have separate
`project.garden.yml` and `workflows.garden.yml` files. If you only have
a few modules, you could even lump those together in a
`modules.garden.yml` file.

You can also use this to make module configs easier to find, by
naming them individually, e.g. `my-module.garden.yml`.

docs/reference/commands.md

@@ -317,6 +317,7 @@ Examples:
 | Argument | Alias | Type | Description |
 | -------- | ----- | ---- | ----------- |
   | `--dir` |  | path | Directory to place the project in (defaults to current directory).
+  | `--filename` |  | string | Filename to place the project config in (defaults to project.garden.yml).
   | `--interactive` | `-i` | boolean | Set to false to disable interactive prompts.
   | `--name` |  | string | Name of the project (defaults to current directory name).
 
@@ -348,6 +349,7 @@ Examples:
 | Argument | Alias | Type | Description |
 | -------- | ----- | ---- | ----------- |
   | `--dir` |  | path | Directory to place the module in (defaults to current directory).
+  | `--filename` |  | string | Filename to place the module config in (defaults to garden.yml).
   | `--interactive` | `-i` | boolean | Set to false to disable interactive prompts.
   | `--name` |  | string | Name of the module (defaults to current directory name).
   | `--type` |  | string | The module type to create. Required if --interactive=false.

docs/using-garden/configuration-overview.md

@@ -5,16 +5,14 @@ title: Configuration Overview
 
 # Configuration Overview
 
-Garden is configured via `garden.yml` configuration files, which Garden collects and compiles into a
+Garden is configured via `garden.yml` (or `*.garden.yml`) configuration files, which Garden collects and compiles into a
 [Stack Graph](../basics/stack-graph.md) of your project.
 
-The [project configuration](./projects.md) `garden.yml` file should be located in the top-level directory of the
-project's Git repository.
+The [project configuration](./projects.md) file should be located in the top-level directory of the project's Git repository. We suggest naming it `project.garden.yml` for clarity, but you can also use `garden.yml` or any filename ending with `.garden.yml`.
 
-In addition, each of the project's [modules](../reference/glossary.md#module)' `garden.yml` should be located in that
-module's top-level directory. Modules define all the individual components of your project, including [services](./services.md), [tasks](./tasks.md) and [tests](./tests.md).
+In addition, each of the project's [modules](../reference/glossary.md#module)' should be located in that module's top-level directory. Modules define all the individual components of your project, including [services](./services.md), [tasks](./tasks.md) and [tests](./tests.md).
 
-Lastly, you can define [workflows](./workflows.md), to codify sequences of Garden commands and custom scripts.
+Lastly, you can define [workflows](./workflows.md), to codify sequences of Garden commands and custom scripts. We suggest placing those in a `workflows.garden.yml` file in your project root.
 
 The other docs under the _Using Garden_ go into more details, and we highly recommend reading through all of them.
 

docs/using-garden/modules.md

@@ -26,11 +26,11 @@ tests:
 
 ## How it Works
 
-A Garden project is usually split up into the project-level `garden.yml` file and several module-level configuration files:
+A Garden project is usually split up into the project-level configuration file, and several module-level configuration files, each in the root directory of the respective module:
 
 ```console
 .
-├── garden.yml
+├── project.garden.yml
 ├── module-a
 │   ├── garden.yml
 │   └── ...
@@ -42,8 +42,10 @@ A Garden project is usually split up into the project-level `garden.yml` file an
      └── ...
 ```
 
+You can also choose any `*.garden.yml` filename for each module configuration file. For example, you might prefer to set the module name in the filename, e.g. `my-module.garden.yml` to make it easier to find in a large project.
+
 {% hint style="info" %}
-It's also possible to [define several modules in the same `garden.yml` file](#multiple-modules-in-the-same-directory) and/or in the same file as the the project-level configuration.
+It's also possible to [define several modules in the same `garden.yml` file](#multiple-modules-in-the-same-directory) and/or in the same file as the the project-level configuration. If you only have a couple of modules, you might for example define them together in a single `modules.garden.yml` file. See [below](#multiple-modules-in-the-same-directory) for more details.
 {% endhint %}
 
 Modules must have a type. Different [module _types_](#module-types) behave in different ways. For example, the `container` module type corresponds to a Docker image, either built from a local Dockerfile or pulled from a remote repository.
@@ -94,14 +96,11 @@ You can also use [.gardenignore files](./configuration-overview.md#ignore-files)
 
 ### Multiple modules in the same directory
 
-Sometimes, it's useful to define several modules in the same `garden.yml` file. One common situation is where more than
-one Dockerfile is in use (e.g. one for a development build and one for a production build).
+Sometimes, it's useful to define several modules in the same `garden.yml` file. One common situation is where more than one Dockerfile is in use (e.g. one for a development build and one for a production build). You may only have a handful of modules, and it may be the cleanest approach to define all of them in a `modules.garden.yml` in your project root.
 
-Another is when the dev configuration and the production configuration have different integration testing suites,
-which may depend on different external services being available.
+Another example is when the dev configuration and the production configuration have different integration testing suites, which may depend on different external services being available.
 
-To do this, add a document separator (`---`) between the module definitions. Here's a simple (if a bit contrived)
-example:
+To do this, add a document separator (`---`) between the module definitions. Here's a simple (if a bit contrived) example:
 
 ```yaml
 kind: Module
@@ -135,7 +134,9 @@ tests:
       - b-integration-testing-backend
 ```
 
-Note that you must use the `include` and/or `exclude` directives (described above) when module paths overlap. This is to help users steer away from subtle bugs that can occur when modules unintentionally consume source files from other modules. See the next section for details on including and excluding files.
+{% hint style="warning" %}
+Note that you **must** use the `include` and/or `exclude` directives (described above) when module paths overlap. This is to help users steer away from subtle bugs that can occur when modules unintentionally consume source files from other modules. See the next section for details on including and excluding files.
+{% endhint %}
 
 ## Modules in the Stack Graph
 

docs/using-garden/projects.md

@@ -5,10 +5,10 @@ title: Projects
 
 # Projects
 
-The first step to using Garden is to create a project. You can use the `garden create project` helper command, or manually create a `garden.yml` file in the root directory of your project:
+The first step to using Garden is to create a project. You can use the `garden create project` helper command, or manually create a `project.garden.yml` file in the root directory of your project:
 
 ```yaml
-# garden.yml - located in the top-level directory of your project
+# project.garden.yml - located in the top-level directory of your project
 kind: Project
 name: my-project
 environments:
@@ -18,17 +18,19 @@ providers:
     environments: ["local"]
 ```
 
+We suggest naming it `project.garden.yml` for clarity, but you can also use `garden.yml` or any filename ending with `.garden.yml`.
+
 The helper command has the benefit of including all the possible fields you can configure, and their documentation, so you can quickly scan through the available options and uncomment as needed.
 
 ## How it Works
 
-The top-level `garden.yml` file is where project-wide configuration takes place. This includes environment configurations and project variables. Most importantly, it's where you declare and configure the *providers* you want to use for your project ([see below](#providers)).
+The top-level `project.garden.yml` file is where project-wide configuration takes place. This includes environment configurations and project variables. Most importantly, it's where you declare and configure the *providers* you want to use for your project ([see below](#providers)).
 
 Garden treats the directory containing the project configuration as the project's top-level directory. Garden commands that run in subdirectories of the project root are assumed to apply to that project, and commands above/outside a project root will fail—similarly to how Git uses the location of the repo's `.git` directory as the repo's root directory.
 
 ### Environments and namespaces
 
-Every Garden command is run against one of the environments defined in the project-level `garden.yml` file. You can specify the environment with the `--env` flag or by setting a `defaultEnvironment`. Alternatively, Garden defaults to the first environment in the `garden.yml` file.
+Every Garden command is run against one of the environments defined in the project-level configuration file. You can specify the environment with the `--env` flag or by setting a `defaultEnvironment`. Alternatively, Garden defaults to the first environment defined in your configuration.
 
 An environment can be partitioned using _namespaces_. A common use-case is to split a shared development or testing environment by namespace, between e.g. users or different branches of code.
 

docs/using-garden/workflows.md

@@ -16,13 +16,14 @@ A sequence of commands executed in a workflow is also generally more efficent th
 Workflows are defined with a separate _kind_ of configuration file, with a list of _steps_:
 
 ```yaml
+# workflows.garden.yml
 kind: Workflow
 name: my-workflow
 steps:
   - ...
 ```
 
-You can place your workflow definitions in your project root `garden.yml` file (with a `---` separator after the project configuration), or in a separate `garden.yml` in a different directory, e.g. in `workflows/garden.yml`.
+We suggest making a `workflows.garden.yml` next to your project configuration in your project root. You can also place your workflow definitions in your project root `project.garden.yml`/`garden.yml` file (with a `---` separator after the project configuration).
 
 Each step in your workflow can either trigger Garden commands, or run custom scripts. The steps are executed in succession. If a step fails, the remainder of the workflow is aborted.
 

garden-service/src/commands/base.ts

@@ -141,7 +141,7 @@ export class StringsParameter extends Parameter<string[] | undefined> {
 
 export class PathParameter extends Parameter<string> {
   type = "path"
-  schema = joi.posixPath()
+  schema = joi.string()
 }
 
 export class PathsParameter extends Parameter<string[]> {

garden-service/src/commands/create/create-module.ts

@@ -18,11 +18,12 @@ import {
   PathParameter,
   BooleanParameter,
   StringOption,
+  StringParameter,
 } from "../base"
 import { printHeader } from "../../logger/util"
-import { getConfigFilePath, isDirectory } from "../../util/fs"
-import { loadConfig, findProjectConfig } from "../../config/base"
-import { resolve, basename, relative } from "path"
+import { isDirectory, defaultConfigFilename } from "../../util/fs"
+import { loadConfigResources, findProjectConfig } from "../../config/base"
+import { resolve, basename, relative, join } from "path"
 import { GardenBaseError, ParameterError } from "../../exceptions"
 import { getModuleTypes, getPluginBaseNames } from "../../plugins"
 import { addConfig } from "./helpers"
@@ -46,6 +47,10 @@ const createModuleOpts = {
     help: "Directory to place the module in (defaults to current directory).",
     defaultValue: ".",
   }),
+  filename: new StringParameter({
+    help: "Filename to place the module config in (defaults to garden.yml).",
+    defaultValue: defaultConfigFilename,
+  }),
   interactive: new BooleanParameter({
     alias: "i",
     help: "Set to false to disable interactive prompts.",
@@ -113,7 +118,7 @@ export class CreateModuleCommand extends Command<CreateModuleArgs, CreateModuleO
       throw new ParameterError(`${configDir} is not a directory`, { configDir })
     }
 
-    const configPath = await getConfigFilePath(configDir)
+    const configPath = join(configDir, opts.filename)
 
     let name = opts.name || basename(configDir)
     let type = opts.type
@@ -164,7 +169,7 @@ export class CreateModuleCommand extends Command<CreateModuleArgs, CreateModuleO
 
     // Throw if module with same name already exists
     if (await pathExists(configPath)) {
-      const configs = await loadConfig(configDir, configDir)
+      const configs = await loadConfigResources(configDir, configPath)
 
       if (configs.filter((c) => c.kind === "Module" && c.name === name).length > 0) {
         throw new CreateError(

garden-service/src/commands/create/create-project.ts

@@ -18,11 +18,12 @@ import {
   PathParameter,
   BooleanParameter,
   StringOption,
+  StringParameter,
 } from "../base"
 import { printHeader } from "../../logger/util"
-import { getConfigFilePath, isDirectory } from "../../util/fs"
-import { loadConfig } from "../../config/base"
-import { resolve, basename, relative } from "path"
+import { isDirectory } from "../../util/fs"
+import { loadConfigResources } from "../../config/base"
+import { resolve, basename, relative, join } from "path"
 import { GardenBaseError, ParameterError } from "../../exceptions"
 import { renderProjectConfigReference } from "../../docs/config"
 import { addConfig } from "./helpers"
@@ -36,12 +37,18 @@ const defaultIgnorefile = dedent`
 # For more info, see https://docs.garden.io/using-garden/configuration-overview#including-excluding-files-and-directories
 `
 
+export const defaultProjectConfigFilename = "project.garden.yml"
+
 const createProjectArgs = {}
 const createProjectOpts = {
   dir: new PathParameter({
     help: "Directory to place the project in (defaults to current directory).",
     defaultValue: ".",
   }),
+  filename: new StringParameter({
+    help: "Filename to place the project config in (defaults to project.garden.yml).",
+    defaultValue: defaultProjectConfigFilename,
+  }),
   interactive: new BooleanParameter({
     alias: "i",
     help: "Set to false to disable interactive prompts.",
@@ -107,11 +114,11 @@ export class CreateProjectCommand extends Command<CreateProjectArgs, CreateProje
       throw new ParameterError(`${configDir} is not a directory`, { configDir })
     }
 
-    const configPath = await getConfigFilePath(configDir)
+    const configPath = join(configDir, opts.filename)
 
     // Throw if a project config already exists in the config path
     if (await pathExists(configPath)) {
-      const configs = await loadConfig(configDir, configDir)
+      const configs = await loadConfigResources(configDir, configPath)
 
       if (configs.filter((c) => c.kind === "Project").length > 0) {
         throw new CreateError(`A Garden project already exists in ${configPath}`, { configDir, configPath })

garden-service/src/commands/get/get-debug-info.ts

@@ -13,8 +13,7 @@ import { getPackageVersion, exec, safeDumpYaml } from "../../util/util"
 import { platform, release } from "os"
 import { join, relative, basename, dirname } from "path"
 import { LogEntry } from "../../logger/log-entry"
-import { deline } from "../../util/string"
-import { findConfigPathsInPath, getConfigFilePath, defaultDotIgnoreFiles } from "../../util/fs"
+import { findConfigPathsInPath, defaultDotIgnoreFiles } from "../../util/fs"
 import { ERROR_LOG_FILENAME } from "../../constants"
 import dedent = require("dedent")
 import { Garden } from "../../garden"
@@ -40,12 +39,10 @@ export const PROVIDER_INFO_FILENAME_NO_EXT = "info"
  */
 export async function collectBasicDebugInfo(root: string, gardenDirPath: string, log: LogEntry) {
   // Find project definition
-  const config = await findProjectConfig(root, true)
-  if (!config) {
+  const projectConfig = await findProjectConfig(root, true)
+  if (!projectConfig) {
     throw new ValidationError(
-      deline`
-      Couldn't find a garden.yml with a project definition.
-      Please run this command from the root of your Garden project.`,
+      "Couldn't find a Project definition. Please run this command from the root of your Garden project.",
       {}
     )
   }
@@ -56,18 +53,19 @@ export async function collectBasicDebugInfo(root: string, gardenDirPath: string,
   await ensureDir(tempPath)
 
   // Copy project definition in tmp folder
-  const projectConfigFilePath = await getConfigFilePath(root)
+  const projectConfigFilePath = projectConfig.configPath!
   const projectConfigFilename = basename(projectConfigFilePath)
   await copy(projectConfigFilePath, join(tempPath, projectConfigFilename))
+
   // Check if error logs exist and copy it over if it does
   if (await pathExists(join(root, ERROR_LOG_FILENAME))) {
     await copy(join(root, ERROR_LOG_FILENAME), join(tempPath, ERROR_LOG_FILENAME))
   }
 
   // Find all services paths
-  const vcs = new GitHandler(root, config.dotIgnoreFiles || defaultDotIgnoreFiles)
-  const include = config.modules && config.modules.include
-  const exclude = config.modules && config.modules.exclude
+  const vcs = new GitHandler(root, projectConfig.dotIgnoreFiles || defaultDotIgnoreFiles)
+  const include = projectConfig.modules && projectConfig.modules.include
+  const exclude = projectConfig.modules && projectConfig.modules.exclude
   const paths = await findConfigPathsInPath({ vcs, dir: root, include, exclude, log })
 
   // Copy all the service configuration files
@@ -80,14 +78,13 @@ export async function collectBasicDebugInfo(root: string, gardenDirPath: string,
     })
     const tempServicePath = join(tempPath, relative(root, servicePath))
     await ensureDir(tempServicePath)
-    const moduleConfigFilePath = await getConfigFilePath(servicePath)
-    const moduleConfigFilename = basename(moduleConfigFilePath)
+    const moduleConfigFilename = basename(configPath)
     const gardenLog = gardenPathLog.info({
       section: moduleConfigFilename,
       msg: "collecting garden.yml",
       status: "active",
     })
-    await copy(moduleConfigFilePath, join(tempServicePath, moduleConfigFilename))
+    await copy(configPath, join(tempServicePath, moduleConfigFilename))
     gardenLog.setSuccess({ msg: chalk.green(`Done (took ${log.getDuration(1)} sec)`), append: true })
     // Check if error logs exist and copy them over if they do
     if (await pathExists(join(servicePath, ERROR_LOG_FILENAME))) {