Add support for a configuration file & environment variable settings (#…

…75)

README.md

@@ -7,20 +7,21 @@
     - [Using `krew`](#using-krew)
     - [Download and install manually](#download-and-install-manually)
   - [Usage](#usage)
-    - [Flags](#flags)
   - [Why `kubectl-slice`?](#why-kubectl-slice)
+  - [Passing configuration options to `kubectl-slice`](#passing-configuration-options-to-kubectl-slice)
+  - [Including and excluding manifests from the output](#including-and-excluding-manifests-from-the-output)
   - [Examples](#examples)
-  - [Contributing & Roadmap](#contributing--roadmap)
+  - [Contributing \& Roadmap](#contributing--roadmap)
 
-`kubectl-slice` is a neat tool that allows you to split a single multi-YAML Kubernetes manifest into multiple subfiles using a naming convention you choose. This is done by parsing the YAML code and giving you the option to access any key from the YAML object [using Go Templates](https://pkg.go.dev/text/template).
+`kubectl-slice` is a neat tool that allows you to split a single multi-YAML Kubernetes manifest into multiple subfiles using a naming convention you choose. This is done by parsing the YAML code and allowing you to access any key from the YAML object [using Go Templates](https://pkg.go.dev/text/template).
 
-By default, `kubectl-slice` will split your files into multiple subfiles following this naming convention:
+By default, `kubectl-slice` will split your files into multiple subfiles following this naming convention that you can configure to your liking:
 
 ```handlebars
 {{.kind | lower}}-{{.metadata.name}}.yaml
 ```
 
-That is, the Kubernets kind -- say, `Namespace` -- lowercased, followed by a dash, followed by the resource name -- say, `production`:
+That is, the Kubernetes kind -- in this case, the value `Namespace` -- lowercased, followed by a dash, followed by the resource name -- in this case, the value `production`:
 
 ```text
 namespace-production.yaml
@@ -83,66 +84,42 @@ Examples:
   kubectl-slice -f foo.yaml -o ./ --exclude-kind Pod
   kubectl-slice -f foo.yaml -o ./ --exclude-name *-svc
   kubectl-slice -f foo.yaml --exclude-name *-svc --stdout
+  kubectl-slice -f foo.yaml --include Pod/* --stdout
+  kubectl-slice -f foo.yaml --exclude deployment/kube* --stdout
+  kubectl-slice --config config.yaml
 
 Flags:
+  -c, --config string          path to the config file
       --dry-run                if true, no files are created, but the potentially generated files will be printed as the command output
+      --exclude strings        resource name to exclude in the output (format <kind>/<name>, case insensitive, glob supported)
       --exclude-kind strings   resource kind to exclude in the output (singular, case insensitive, glob supported)
       --exclude-name strings   resource name to exclude in the output (singular, case insensitive, glob supported)
   -h, --help                   help for kubectl-slice
+      --include strings        resource name to include in the output (format <kind>/<name>, case insensitive, glob supported)
       --include-kind strings   resource kind to include in the output (singular, case insensitive, glob supported)
       --include-name strings   resource name to include in the output (singular, case insensitive, glob supported)
   -f, --input-file string      the input file used to read the initial macro YAML file; if empty or "-", stdin is used
   -o, --output-dir string      the output directory used to output the splitted files
-  -q, --quiet                  silences all output to stdout/err when writing to files
+  -q, --quiet                  if true, no output is written to stdout/err
   -s, --skip-non-k8s           if enabled, any YAMLs that don't contain at least an "apiVersion", "kind" and "metadata.name" will be excluded from the split
       --sort-by-kind           if enabled, resources are sorted by Kind, a la Helm, before saving them to disk
       --stdout                 if enabled, no resource is written to disk and all resources are printed to stdout instead
   -t, --template string        go template used to generate the file name when creating the resource files in the output directory (default "{{.kind | lower}}-{{.metadata.name}}.yaml")
   -v, --version                version for kubectl-slice
 ```
 
-### Flags
-
-* `--dry-run`:
-  * Allows the program to execute but not save anything to files. The output will show what potential files would be created.
-* `--input-file`:
-  * The input file to read as YAML multi-file. If this value is empty or set to `-`, `stdin` is used instead. Even after processing, the original file is preserved as much as possible, and that includes comments, YAML arrays, and formatting.
-* `--output-dir`:
-  * The output directory where the files must be saved. By default is set to the current directory. You can use this in combination with `--template` to control where your files will land once split. If the folder does not exist, it will be created.
-* `--template`:
-  * A Go Text Template used to generate the splitted file names. You can access any field from your YAML files -- even fields that don't exist, although they will render as `""` -- and use this to your advantage. Consider the following:
-    * There's a check to validate that, after rendering the file name, there's at least a file name.
-    * Unix linebreaks (`\n`) are removed from the generated file name, thus allowing you to use multiline Go Templates if needed.
-    * You can use any of the built-in [Template Functions](docs/template_functions.md#template-functions) to your advantage.
-    * If multiple files from your YAML generate the same file name, all YAMLs that match this file name will be appended.
-    * If the rendered file name includes a path separator, subfolders under `--output-dir` will be created.
-    * If a file already exists in `--output-directory` under this generated file name, their contents will be replaced.
-* `--exclude-kind`:
-  * A case-insensitive, comma-separated list of Kubernetes object kinds to exclude from the output. Globs are supported.
-  * You can also repeat the parameter multiple times to achieve the same effect (`--exclude-kind pod --exclude-kind deployment`)
-* `--include-kind`:
-  * A case-insensitive, comma-separated list of Kubernetes object kinds to include in the output. Globs are supported. Any other Kubernetes object kinds will be excluded.
-  * You can also repeat the parameter multiple times to achieve the same effect (`--include-kind pod --include-kind deployment`)
-* `--skip-non-k8s`:
-  * If enabled, any YAMLs that don't contain at least an `apiVersion`, `kind` and `metadata.name` will be excluded from the split
-  * There are no attempts to validate how correct these fields are. For example, there's no check to validate that `apiVersion` exists in a Kubernetes cluster, or whether this `apiVersion` is valid: `"example\foo"`.
-    * It's useful, however, if alongside the original YAML you suspect there might be some non Kubernetes YAMLs being generated.
-* `--sort-by-kind`:
-  * If enabled, resources are sorted by Kind, like Helm does, before saving them to disk or printing them to `stdout`.
-  * If this flag is not present, resources are outputted following the order in which they were found in the YAML file.
-* `--stdout`:
-  * If enabled, no resource is written to disk and all resources are printed to `stdout` instead, useful if you want to pipe the output of `kubectl-slice` to another command or to itself. File names are still generated, but used as reference and prepended at the top of each file in the multi-YAML output. Other than that, the file name template has no effect -- it won't create any subfolders, for example.
-* `--include-name`:
-  * A case-insensitive, comma-separated list of Kubernetes object names to include in the output. Globs are supported. Any other Kubernetes object names will be excluded.
-  * You can also repeat the parameter multiple times to achieve the same effect (`--include-name foo --include-name bar`)
-* `--exclude-name`:
-  * A case-insensitive, comma-separated list of Kubernetes object names to exclude from the output. Globs are supported.
-  * You can also repeat the parameter multiple times to achieve the same effect (`--exclude-name foo --exclude-name bar`)
-
 ## Why `kubectl-slice`?
 
 See [why `kubectl-slice`?](docs/why.md) for more information.
 
+## Passing configuration options to `kubectl-slice`
+
+Besides command-line flags, you can also use environment variables and a YAML configuration file to pass options to `kubectl-slice`. See [the documentation for configuration options](docs/configuring-cli.md) for details about both, including precedence.
+
+## Including and excluding manifests from the output
+
+Including or excluding manifests from the output via `metadata.name` or `kind` is possible. Globs are supported in both cases. See [the documentation for including and excluding items](docs/including-excluding-items.md) for more information.
+
 ## Examples
 
 See [examples](docs/examples.md) for more information.

app.go

@@ -4,9 +4,12 @@ import (
 	"bytes"
 	"fmt"
 	"os"
+	"strings"
 
 	"github.com/patrickdappollonio/kubectl-slice/slice"
 	"github.com/spf13/cobra"
+	"github.com/spf13/pflag"
+	"github.com/spf13/viper"
 )
 
 var version = "development"
@@ -25,6 +28,7 @@ var examples = []string{
 	"kubectl-slice -f foo.yaml --exclude-name *-svc --stdout",
 	"kubectl-slice -f foo.yaml --include Pod/* --stdout",
 	"kubectl-slice -f foo.yaml --exclude deployment/kube* --stdout",
+	"kubectl-slice --config config.yaml",
 }
 
 func generateExamples([]string) string {
@@ -42,6 +46,7 @@ func generateExamples([]string) string {
 
 func root() *cobra.Command {
 	opts := slice.Options{}
+	var configFile string
 
 	rootCommand := &cobra.Command{
 		Use:           "kubectl-slice",
@@ -51,6 +56,11 @@ func root() *cobra.Command {
 		SilenceUsage:  true,
 		SilenceErrors: true,
 		Example:       generateExamples(examples),
+
+		PreRunE: func(cmd *cobra.Command, args []string) error {
+			return bindCobraAndViper(cmd, configFile)
+		},
+
 		RunE: func(cmd *cobra.Command, args []string) error {
 			// Bind to the appropriate stdout/stderr
 			opts.Stdout = cmd.OutOrStdout()
@@ -60,6 +70,18 @@ func root() *cobra.Command {
 			// point the app to stdin
 			if opts.InputFile == "" || opts.InputFile == "-" {
 				opts.InputFile = os.Stdin.Name()
+
+				// Check if we're receiving data from the terminal
+				// or from piped content. Users from piped content
+				// won't see this message. Users that might have forgotten
+				// setting the flags correctly will see this message.
+				if !opts.Quiet {
+					if fi, err := os.Stdin.Stat(); err == nil && fi.Mode()&os.ModeNamedPipe == 0 {
+						fmt.Fprintln(opts.Stderr, "Receiving data from the terminal. Press CTRL+D when you're done typing or CTRL+C")
+						fmt.Fprintln(opts.Stderr, "to exit without processing the content. If you're seeing this by mistake, make")
+						fmt.Fprintln(opts.Stderr, "sure the command line flags, environment variables or config file are correct.")
+					}
+				}
 			}
 
 			// Create a new instance. This will also perform a basic validation.
@@ -87,7 +109,93 @@ func root() *cobra.Command {
 	rootCommand.Flags().BoolVarP(&opts.StrictKubernetes, "skip-non-k8s", "s", false, "if enabled, any YAMLs that don't contain at least an \"apiVersion\", \"kind\" and \"metadata.name\" will be excluded from the split")
 	rootCommand.Flags().BoolVar(&opts.SortByKind, "sort-by-kind", false, "if enabled, resources are sorted by Kind, a la Helm, before saving them to disk")
 	rootCommand.Flags().BoolVar(&opts.OutputToStdout, "stdout", false, "if enabled, no resource is written to disk and all resources are printed to stdout instead")
+	rootCommand.Flags().StringVarP(&configFile, "config", "c", "", "path to the config file")
 
 	_ = rootCommand.Flags().MarkHidden("debug")
 	return rootCommand
 }
+
+// envVarPrefix is the prefix used for environment variables.
+// Using underscores to ensure compatibility with the shell.
+const envVarPrefix = "KUBECTL_SLICE"
+
+// skippedFlags is a list of flags that are not bound through
+// Viper. These include things like "help", "version", and of
+// course, "config", since it doesn't make sense to say where
+// the config file is located in the config file itself.
+var skippedFlags = [...]string{
+	"help",
+	"version",
+	"config",
+}
+
+// bindCobraAndViper binds the settings loaded by Viper
+// to the flags defined in Cobra.
+func bindCobraAndViper(cmd *cobra.Command, configFileLocation string) error {
+	v := viper.New()
+
+	// If a configuration file has been passed...
+	if cmd.Flags().Lookup("config").Changed {
+		// ... then set it as the configuration file
+		v.SetConfigFile(configFileLocation)
+
+		// then read the configuration file
+		if err := v.ReadInConfig(); err != nil {
+			return fmt.Errorf("failed to read configuration file: %w", err)
+		}
+	}
+
+	// Handler for potential error
+	var err error
+
+	// Recurse through all the variables
+	cmd.Flags().VisitAll(func(flag *pflag.Flag) {
+		// Skip the flags that are not bound through Viper
+		for _, v := range skippedFlags {
+			if v == flag.Name {
+				return
+			}
+		}
+
+		// Normalize key names with underscores instead of dashes
+		nameUnderscored := strings.ReplaceAll(flag.Name, "-", "_")
+		envVarName := strings.ToUpper(fmt.Sprintf("%s_%s", envVarPrefix, nameUnderscored))
+
+		// Bind the flag to the environment variable
+		if val, found := os.LookupEnv(envVarName); found {
+			v.Set(nameUnderscored, val)
+		}
+
+		// If the CLI flag hasn't been changed, but the value is set in
+		// the configuration file, then set the CLI flag to the value
+		// from the configuration file
+		if !flag.Changed && v.IsSet(nameUnderscored) {
+			// Type check for all the supported types
+			switch val := v.Get(nameUnderscored).(type) {
+
+			case string:
+				_ = cmd.Flags().Set(flag.Name, val)
+
+			case []interface{}:
+				var stringified []string
+				for _, v := range val {
+					stringified = append(stringified, fmt.Sprintf("%v", v))
+				}
+				_ = cmd.Flags().Set(flag.Name, strings.Join(stringified, ","))
+
+			case bool:
+				_ = cmd.Flags().Set(flag.Name, fmt.Sprintf("%t", val))
+
+			case int:
+				_ = cmd.Flags().Set(flag.Name, fmt.Sprintf("%d", val))
+
+			default:
+				err = fmt.Errorf("unsupported type %T for flag %q", val, nameUnderscored)
+				return
+			}
+		}
+	})
+
+	// If an error occurred, return it
+	return err
+}

docs/configuring-cli.md

@@ -0,0 +1,79 @@
+# Configuring the CLI
+
+- [Configuring the CLI](#configuring-the-cli)
+  - [Using a configuration file](#using-a-configuration-file)
+  - [Using environment variables](#using-environment-variables)
+  - [Using command-line flags](#using-command-line-flags)
+
+There are three ways to configure the CLI. Providing configuration to it has the following processing order:
+
+1. Configuration file
+2. Environment variables
+3. Command-line flags
+
+The order of precedence dictates how `kubectl-slice` will handle being provided with configuration from multiple sources.
+
+The following example can illustrate this by providing `kubectl-slice` with an input file to process via three different ways:
+
+```bash
+KUBECTL_SLICE_INPUT_FILE=2.yaml kubectl-slice -f 3.yaml --config $(echo "input_file: 1.yaml">>config.yaml && echo "config.yaml")
+```
+
+You'll notice the error message you get is that the file `3.yaml` doesn't exist. From the configuration file (first precedence), to the environment variable (second precedence), to the command-line flag (third precedence), `kubectl-slice` used `3.yaml`.
+
+Removing now the `-f 3.yaml` you'll see that `kubectl-slice` will use `2.yaml` as the input file, coming from the environment variable. Deleting the environment variable will load the setting from the configuration file.
+
+The order of precedence is useful if you want to provide a default configuration file and then override some of the settings using environment variables or command-line flags.
+
+## Using a configuration file
+
+The configuration file is a YAML file that contains the settings for `kubectl-slice`. The configuration file uses the same format expected by the CLI flags, with the names of the flags being the keys of the YAML file and dashes replaced with underscores.
+
+For example, the `--input-file` flag becomes `input_file:` in the configuration file.
+
+The following is an example of a configuration file with the types defined:
+
+```yaml
+input-file: string
+output-dir: string
+template: string
+dry-run: boolean
+debug: boolean
+quiet: boolean
+include-kind: [string]
+exclude-kind: [string]
+include-name: [string]
+exclude-name: [string]
+include: [string]
+exclude: [string]
+skip-non-k8s: bool
+sort-by-kind: bool
+stdout: bool
+```
+
+You can use this file to provide more complex templates by using multiline strings without having to escape special characters, for example:
+
+```yaml
+template: >
+  {{ .kind | lower }}/{{ .metadata.name | dottodash | replace ":" "-" }}.yaml
+```
+
+## Using environment variables
+
+Similarly to what happens with YAML configuration files, we use the same format for environment variables, with the names of the flags being the keys of the environment variable and dashes replaced with underscores. The environment variable's name is also prefixed with `KUBECTL_SLICE`, and the entire key is uppercased.
+
+Here are a few examples of environment variables and their corresponding flags:
+
+| Environment variable       | Flag           |
+| -------------------------- | -------------- |
+| `KUBECTL_SLICE_INPUT_FILE` | `--input-file` |
+| `KUBECTL_SLICE_OUTPUT_DIR` | `--output-dir` |
+| `KUBECTL_SLICE_TEMPLATE`   | `--template`   |
+| `KUBECTL_SLICE_DRY_RUN`    | `--dry-run`    |
+| `KUBECTL_SLICE_DEBUG`      | `--debug`      |
+
+The same values as the YAML counterpart apply. In the case of booleans, `true` or `false` are valid values. In the case of arrays, the values are comma-separated.
+
+## Using command-line flags
+
+The command-line flags are the most straightforward way to configure `kubectl-slice`. You can get an up-to-date list of the available flags by running `kubectl-slice --help`.