Separate Config and secrets aligning to 12 factor apps

[Bengreen]

In a 12 factor app it is important to separate out the config from secrets.
Typically when using Kubernetes or docker the secrets are provided as files within a mount.

it would be useful if Hoplite provided support for Kubernetes style secrets to be used to configure credentials within an application.

Suggestion

Create a new Preprocessor that takes a Path as part of its constructor. This path is the mount point of the hierarchy of secrets (credentials) required for the application. This should not have a rigid or implied structure but instead provide flexibility to allow the devops patterns to be easily configured.

When executed the Preprocessor should identify the secret filename relative to the secret mount base directory and read in that file as content of the configuration node.

Comment

the base directory for the secret mount needs to be flexible and runtime so that the deployment can change the target dir as necessary. This will also make testing simpler as it does not need any hardcoding or predefined directories.
I would expect the directory of the secret dir to be provided via a CLI arg or ENV var AND is provided by the devops or packaging team who are responsible for the k8s pod definitions.

Example directory structure

/secretmount0/database/password

contents of password is 'MyPassword'

val preproc=SecretFilesPreprocessor("/secretmount0")

config file could then be

abc:
  db:
    url: jdbc:postgresql://localhost:5432/mydatabase
    username: user0
    password: ${database/password}

An initial PR is provided for discussion: #470

[AlexCue987]

I think this problem is usually solved via an already existing addResourceSource method.

val config = ConfigLoaderBuilder.default()
 .addResourceSource("/path/to/config/application.yml")
.addResourceSource("/path/to/secrets/secret.yml")
.build()

Am I missing something?

[Bengreen]

I looked at that pattern and my interpretation was that it provided 'an overlay' of config.

eg "/path/to/secrets/secret.yml" is going to be of the form (same structure) as "/path/to/config/application.yml" albeit with less fields (ie just the secret fields).

That would be the secret file needs quite a complex structure and is not in keeping with typical use in k8s where a secret file normally has JUST the secret in question.
Typically you then have multiple files within your secret mount to provide the various credentials.
eg db username, db password, access_key, access_id, etc.

But I may also be wrong in my understanding and usage of .addResourceSource