Improve secret management workflow

[ilyasotkov]

No description provided.

[ilyasotkov]

https://www.terraform.io/docs/providers/vault/index.html

[ilyasotkov]

https://github.com/kubernetes/charts/tree/master/incubator/vault

[ilyasotkov]

We have a working deployment for Vault in core/vault which we have tested manually. We are now up for a real-world example with Concouse, which supports pulling secrets from a Vault server natively.

[ilyasotkov]

Vault should probably be integrated with GCP KMS one way or another since owner-key.json is something we already use anyway.

hashicorp/terraform-provider-google#495

[ilyasotkov]

https://cloud.google.com/solutions/using-vault-for-secret-management

[ilyasotkov]

I renamed the issue since we might now necessarily want to use Vault. There's no need to write other solutions off.

[ilyasotkov]

✅ We now support a simple solution using Google Cloud KMS and Cloud Storage buckets via the gcp-kms-secret-mgmt module

This is how it works:

• every subdirectory in $TF_VAR_secrets_dir (for every environment of a project) corresponds to a crypto key and also a single cloud storage bucket for storing encrypted secrets
• granular permissions using permissions.tf file (https://github.com/exekube/internal-ops-project/blob/master/live/dev/secrets/terraform.tfvars)
• xk secrets-fetch and xk secrets-push scripts for syncing local (decrypted) copies and remote (encrypted) for distributing (similar to using git)