Add GCP Secret Manager support

.labrador.example.yaml

@@ -56,3 +56,19 @@ aws:
   ssm_param:
   - /path/to/single/param
   - /path/to/wildcard/params/*
+
+
+###########################################################
+# GCP options
+###########################################################
+
+gcp:
+
+  # List of GCP Secret Manager secret names to fetch.
+  # Secrets can specify an explicit version, or default to latest.
+  # Format must be one of:
+  #     projects/<project name>/secrets/<secret name>
+  #     projects/<project name>/secrets/<secret name>/versions/<version>
+  sm_secret:
+    - projects/my-first-project/secrets/name-of-secret1
+    - projects/my-other-project/secrets/name-of-secret2/versions/1

README.md

@@ -34,6 +34,8 @@ Example use cases:
   - [Fetch All AWS SSM Parameter Store Values at Given Base Path (Wildcard)](#fetch-all-aws-ssm-parameter-store-values-at-given-base-path-wildcard)
   - [Fetch Two Sets of AWS SSM Parameter Store Values](#fetch-two-sets-of-aws-ssm-parameter-store-values)
   - [Fetch an AWS Secrets Manager Value with multiple Key/Value Pairs](#fetch-an-aws-secrets-manager-value-with-multiple-keyvalue-pairs)
+
+  - [Fetch GCP Secret Manager Values With or Without Explicit Versions](#fetch-gcp-secret-manager-values-with-or-without-explicit-versions)
   - [Fetch from Multiple Services At Once](#fetch-from-multiple-services-at-once)
   - [Save Fetched Values to an `.env` File](#save-fetched-values-to-an-env-file)
   - [Set Fetched Values as Environment Variables in the Current Shell](#set-fetched-values-as-environment-variables-in-the-current-shell)
@@ -42,33 +44,62 @@ Example use cases:
 - [Reference](#reference)
   - [Labrador Environment Variables](#labrador-environment-variables)
   - [AWS Environment Variables](#aws-environment-variables)
+  - [GCP Environment Variables](#gcp-environment-variables)
 - [Why Go (Golang)?](#why-go-golang)
 - [Similar Projects](#similar-projects)
 
 ## Quickstart
 
 Before running, set environment variables for accessing the services where the
-values are stored.
-For example, if [authenticating to AWS](#aws-environment-variables) using a
-local profile:
+values are stored. Some quick examples:
 
 ```sh
+# AWS with a local profile.
 export AWS_PROFILE="myprofile"
 export AWS_REGION="us-east-1"
+
+# GCP with a file containing service account key.
+export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account-key.json"
 ```
 
-Continuing the example, fetch all key/value pairs from AWS SSM Parameter Store
-at base path `/path/to/params/*`, and saving to the local file `.env`.
+Download and extract the CLI.
 
 ```sh
 curl -sL https://github.com/DivergentCodes/labrador/releases/latest/download/labrador_Linux_x86_64.tar.gz  | tar -zx
+```
+
+Fetch key/value pairs from AWS SSM Parameter Store, AWS Secret Manager,
+and GCP Secret Manager, then save all of them in an `.env` file.
 
-./labrador fetch --aws-param "/path/to/params/*" --outfile ".env"
+```sh
+./labrador fetch \
+  --aws-param "/path/to/test/params/*" \
+  --aws-secret "my/test/aws-secret1" \
+  --gcp-secret "projects/my-test-project/secrets/gcp-secret1" \
+  --outfile ".env"
 ```
 
 You can also copy the `.labrador.example.yaml` example configuration file
-over to `.labrador.yaml`, customize it, and run `labrador fetch` without
-any other arguments needed.
+over to `.labrador.yaml`, customize it, and run `labrador fetch` or
+`labrador export` without any arguments needed.
+
+```yaml
+aws:
+  ssm_param:
+  - /path/to/test/params/*
+  sm_secret:
+  - my/test/aws-secret1
+gcp:
+  sm_secret:
+  - projects/my-test-project/secrets/gcp-secret1
+```
+
+Example of exporting into the current shell with the above configuation.
+
+```sh
+source <(./labrador export)
+```
+
 
 ## Features
 
@@ -91,8 +122,9 @@ any other arguments needed.
 
 ### Supported Value Stores
 
-- **AWS SSM Parameter Store**: this action can pull individual parameters, or recursively pull a wildcard path with all child variables, as individual environment variables.
-- **AWS Secrets Manager**: this action can pull all key/value pairs in a single secret are loaded as individual environment variables.
+- **AWS SSM Parameter Store**: pull individual parameters, or recursively pull a wildcard path with all child variables, as individual environment variables.
+- **AWS Secrets Manager**: pull all key/value pairs in a single secret are loaded as individual environment variables.
+- **GCP Secret Manager**: pull each secret as a key/value pair, with or without explicit versions, and load as individual environment variables.
 
 ### CI/CD pipeline Packages
 
@@ -149,7 +181,7 @@ specific instance.
 labrador fetch --aws-param "/global/shared/params/*" --aws-param "/instance/params/*"
 ```
 
-### Fetch an AWS Secrets Manager Value with multiple Key/Value Pairs
+### Fetch an AWS Secrets Manager Value With multiple Key/Value Pairs
 
 A single secret in AWS Secrets Manager can store multiple key/value pairs.
 Labrador will pull the secret, extract each key/value, and return them as
@@ -159,13 +191,25 @@ individual variables.
 labrador fetch --aws-secret "path/to/secret"
 ```
 
+### Fetch GCP Secret Manager Values With or Without Explicit Versions
+
+Labrador will default to a GCP secret's `versions/latest` version if
+one isn't explicitly specified. Each secret can hold a single key/value
+pair.
+
+```sh
+labrador fetch \
+  --gcp-secret "projects/myproject/secrets/foo" \
+  --gcp-secret "projects/myproject/secrets/bar/versions/1"
+```
+
 ### Fetch from Multiple Services At Once
 
 If your configuration is spread across multiple services (e.g. undergoing
 a migration), you can fetch values from all of them at the same time.
 
 ```sh
-labrador fetch --aws-param "/path/to/params/*" --aws-secret "path/to/secret"
+labrador fetch --aws-param "/path/to/params/*" --gcp-secret "projects/myproject/secrets/mysecret"
 ```
 
 ### Save Fetched Values to an `.env` File
@@ -289,6 +333,15 @@ session with AWS, like Github Actions' `aws-actions/configure-aws-credentials`.
 - `AWS_SECRET_ACCESS_KEY` and `AWS_ACCESS_KEY_ID`
 - `AWS_ROLE_ARN` and `AWS_WEB_IDENTITY_TOKEN_FILE` and `AWS_ROLE_SESSION_NAME`.
 
+### GCP Environment Variables
+
+For Labrador to access secrets stored in GCP, configure the
+[Application Default
+Credentials (ADC)](https://cloud.google.com/docs/authentication/provide-credentials-adc). If the credentials file is not stored in the default
+location, an environment variable can point to the file.
+
+- `GOOGLE_APPLICATION_CREDENTIALS`
+
 
 ## Why Go (Golang)?
 

cmd/labrador/export.go

@@ -25,13 +25,7 @@ func export(cmd *cobra.Command, args []string) {
 	// export implies --quiet
 	viper.Set(core.OptStr_Quiet, true)
 
-	if countRemoteTargets() == 0 {
-		core.PrintFatal("no remote values to fetch were specified", 1)
-	}
-
-	variables := make(map[string]*variable.Variable, 0)
-	variables = fetchAwsSsmParameters(variables)
-	variables = fetchAwsSmSecrets(variables)
+	variables := fetchVariables()
 
 	toLower := viper.GetBool(core.OptStr_ToLower)
 	toUpper := viper.GetBool(core.OptStr_ToUpper)

cmd/labrador/fetch.go

@@ -57,13 +57,17 @@ func init() {
 func fetch(cmd *cobra.Command, args []string) {
 	ShowBanner()
 
-	if countRemoteTargets() == 0 {
-		core.PrintFatal("no remote values to fetch were specified", 1)
-	}
+	/*
+		if countRemoteTargets() == 0 {
+			core.PrintFatal("no remote values to fetch were specified", 1)
+		}
+
+		variables := make(map[string]*variable.Variable, 0)
+		variables = fetchAwsSsmParameters(variables)
+		variables = fetchAwsSmSecrets(variables)
+	*/
 
-	variables := make(map[string]*variable.Variable, 0)
-	variables = fetchAwsSsmParameters(variables)
-	variables = fetchAwsSmSecrets(variables)
+	variables := fetchVariables()
 
 	core.PrintDebug("\n")
 	core.PrintNormal(fmt.Sprintf("\nFetched %d values\n", len(variables)))

cmd/labrador/root.go

@@ -47,6 +47,7 @@ import (
 
 	"github.com/divergentcodes/labrador/internal/aws"
 	"github.com/divergentcodes/labrador/internal/core"
+	"github.com/divergentcodes/labrador/internal/gcp"
 	"github.com/divergentcodes/labrador/internal/variable"
 )
 
@@ -152,9 +153,17 @@ func initRootFlags() {
 	}
 
 	// aws-secret
-	defaultAwsSmSecrets := viper.GetViper().GetStringSlice(core.OptStr_AWS_SecretManager)
+	defaultAwsSmSecrets := viper.GetViper().GetStringSlice(core.OptStr_AWS_SecretsManager)
 	rootCmd.PersistentFlags().StringSlice("aws-secret", defaultAwsSmSecrets, "AWS Secrets Manager secret name")
-	err = viper.BindPFlag(core.OptStr_AWS_SecretManager, rootCmd.PersistentFlags().Lookup("aws-secret"))
+	err = viper.BindPFlag(core.OptStr_AWS_SecretsManager, rootCmd.PersistentFlags().Lookup("aws-secret"))
+	if err != nil {
+		panic(err)
+	}
+
+	// gcp-secret
+	defaultGcpSecret := viper.GetViper().GetStringSlice(core.OptStr_GCP_SecretManager)
+	rootCmd.PersistentFlags().StringSlice("gcp-secret", defaultGcpSecret, "GCP Secret Manager secret name")
+	err = viper.BindPFlag(core.OptStr_GCP_SecretManager, rootCmd.PersistentFlags().Lookup("gcp-secret"))
 	if err != nil {
 		panic(err)
 	}
@@ -187,14 +196,30 @@ func bannerText() string {
 	return fmt.Sprintf("Labrador %s created by %s <%s>\n", core.Version, core.AuthorName, core.AuthorEmail)
 }
 
+// Wrapper to bundle fetching from each service.
+func fetchVariables() map[string]*variable.Variable {
+	if countRemoteTargets() == 0 {
+		core.PrintFatal("no remote values to fetch were specified", 1)
+	}
+
+	variables := make(map[string]*variable.Variable, 0)
+
+	variables = fetchAwsSsmParameters(variables)
+	variables = fetchAwsSmSecrets(variables)
+
+	variables = fetchGcpSmSecrets(variables)
+
+	return variables
+}
+
 // Count the number of user-defined resources to pull values from.
 func countRemoteTargets() int {
 	remoteTargetCount := 0
 
 	awsSsmParameters := viper.GetStringSlice(core.OptStr_AWS_SsmParameterStore)
 	remoteTargetCount += len(awsSsmParameters)
 
-	awsSmSecrets := viper.GetStringSlice(core.OptStr_AWS_SecretManager)
+	awsSmSecrets := viper.GetStringSlice(core.OptStr_AWS_SecretsManager)
 	remoteTargetCount += len(awsSmSecrets)
 
 	return remoteTargetCount
@@ -207,13 +232,14 @@ func fetchAwsSsmParameters(variables map[string]*variable.Variable) map[string]*
 	if len(awsSsmParameters) != 0 {
 		ssmVariables, err := aws.FetchParameterStore()
 		if err != nil {
-			core.PrintFatal("failed to get SSM parameters", 1)
+			core.PrintFatal("failed to get AWS SSM parameters", 1)
 		}
 
 		core.PrintVerbose(fmt.Sprintf("\nFetched %d values from AWS SSM Parameter Store", len(ssmVariables)))
 		for name, variable := range ssmVariables {
 			variables[name] = variable
-			core.PrintVerbose(fmt.Sprintf("\n\t%s", variable.Metadata["arn"]))
+			core.PrintVerbose(fmt.Sprintf("\n\t%s (%s)", variable.Metadata["arn"], variable.Key))
+			core.PrintDebug(fmt.Sprintf("\n\t\tarn: \t\t%s", variable.Metadata["arn"]))
 			core.PrintDebug(fmt.Sprintf("\n\t\ttype: \t\t%s", variable.Metadata["type"]))
 			core.PrintDebug(fmt.Sprintf("\n\t\tversion: \t%s", variable.Metadata["version"]))
 			core.PrintDebug(fmt.Sprintf("\n\t\tmodified: \t%s", variable.Metadata["last-modified"]))
@@ -226,17 +252,18 @@ func fetchAwsSsmParameters(variables map[string]*variable.Variable) map[string]*
 // Fetch AWS Secrets Manager values, convert to variables, add to list, and return the list.
 func fetchAwsSmSecrets(variables map[string]*variable.Variable) map[string]*variable.Variable {
 
-	awsSmSecrets := viper.GetStringSlice(core.OptStr_AWS_SecretManager)
+	awsSmSecrets := viper.GetStringSlice(core.OptStr_AWS_SecretsManager)
 	if len(awsSmSecrets) != 0 {
 		smVariables, err := aws.FetchSecretsManager()
 		if err != nil {
-			core.PrintFatal("failed to get Secrets Manager values", 1)
+			core.PrintFatal("failed to get AWS Secrets Manager values", 1)
 		}
 
 		core.PrintVerbose(fmt.Sprintf("\nFetched %d values from AWS Secrets Manager", len(smVariables)))
 		for name, variable := range smVariables {
 			variables[name] = variable
 			core.PrintVerbose(fmt.Sprintf("\n\t%s (%s)", variable.Metadata["arn"], variable.Key))
+			core.PrintDebug(fmt.Sprintf("\n\t\tarn: \t\t%s", variable.Metadata["arn"]))
 			core.PrintDebug(fmt.Sprintf("\n\t\tsecret-name: \t%s", variable.Metadata["secret-name"]))
 			core.PrintDebug(fmt.Sprintf("\n\t\ttype: \t\t%s", variable.Metadata["type"]))
 			core.PrintDebug(fmt.Sprintf("\n\t\tversion-id: \t%s", variable.Metadata["version-id"]))
@@ -246,3 +273,33 @@ func fetchAwsSmSecrets(variables map[string]*variable.Variable) map[string]*vari
 
 	return variables
 }
+
+// Fetch GCP Secret Manager values, convert to variables, add to list, and return the list.
+func fetchGcpSmSecrets(variables map[string]*variable.Variable) map[string]*variable.Variable {
+
+	gcpSmSecrets := viper.GetStringSlice(core.OptStr_GCP_SecretManager)
+	if len(gcpSmSecrets) != 0 {
+		smVariables, err := gcp.FetchSecretManager()
+		if err != nil {
+			core.PrintFatal("failed to get GCP Secret Manager values", 1)
+		}
+
+		core.PrintVerbose(fmt.Sprintf("\nFetched %d values from GCP Secret Manager", len(smVariables)))
+		for name, variable := range smVariables {
+			variables[name] = variable
+			core.PrintVerbose(fmt.Sprintf("\n\t%s (%s)", variable.Metadata["secret-name"], variable.Key))
+			core.PrintDebug(fmt.Sprintf("\n\t\tsecret-name: \t%s", variable.Metadata["secret-name"]))
+			core.PrintDebug(fmt.Sprintf("\n\t\tcreate-time: \t%s", variable.Metadata["create-time"]))
+			core.PrintDebug(fmt.Sprintf("\n\t\texpire-time: \t%s", variable.Metadata["expire-time"]))
+			core.PrintDebug(fmt.Sprintf("\n\t\tversion: \t%s", variable.Metadata["version"]))
+			core.PrintDebug(fmt.Sprintf("\n\t\tproject: \t%s", variable.Metadata["project"]))
+			core.PrintDebug(fmt.Sprintf("\n\t\tetag: \t\t%s", variable.Metadata["etag"]))
+			core.PrintDebug(fmt.Sprintf("\n\t\trotation: \t%s", variable.Metadata["rotation"]))
+			core.PrintDebug(fmt.Sprintf("\n\t\ttopics: \t%s", variable.Metadata["topics"]))
+			core.PrintDebug(fmt.Sprintf("\n\t\tannotations: \t%s", variable.Metadata["annotations"]))
+			core.PrintDebug(fmt.Sprintf("\n\t\tlabels: \t%s", variable.Metadata["labels"]))
+		}
+	}
+
+	return variables
+}

go.mod

@@ -3,15 +3,20 @@ module github.com/divergentcodes/labrador
 go 1.20
 
 require (
+	cloud.google.com/go/secretmanager v1.11.1
 	github.com/aws/aws-sdk-go-v2 v1.18.1
 	github.com/aws/aws-sdk-go-v2/config v1.18.27
 	github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.19.10
 	github.com/aws/aws-sdk-go-v2/service/ssm v1.36.6
 	github.com/spf13/cobra v1.7.0
 	github.com/spf13/viper v1.15.0
+	google.golang.org/grpc v1.55.0
 )
 
 require (
+	cloud.google.com/go/compute v1.19.3 // indirect
+	cloud.google.com/go/compute/metadata v0.2.3 // indirect
+	cloud.google.com/go/iam v1.1.0 // indirect
 	github.com/aws/aws-sdk-go-v2/credentials v1.13.26 // indirect
 	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.4 // indirect
 	github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.34 // indirect
@@ -23,6 +28,12 @@ require (
 	github.com/aws/aws-sdk-go-v2/service/sts v1.19.2 // indirect
 	github.com/aws/smithy-go v1.13.5 // indirect
 	github.com/fsnotify/fsnotify v1.6.0 // indirect
+	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
+	github.com/golang/protobuf v1.5.3 // indirect
+	github.com/google/go-cmp v0.5.9 // indirect
+	github.com/google/s2a-go v0.1.4 // indirect
+	github.com/googleapis/enterprise-certificate-proxy v0.2.3 // indirect
+	github.com/googleapis/gax-go/v2 v2.11.0 // indirect
 	github.com/hashicorp/hcl v1.0.0 // indirect
 	github.com/inconshreveable/mousetrap v1.1.0 // indirect
 	github.com/jmespath/go-jmespath v0.4.0 // indirect
@@ -34,8 +45,18 @@ require (
 	github.com/spf13/jwalterweatherman v1.1.0 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	github.com/subosito/gotenv v1.4.2 // indirect
-	golang.org/x/sys v0.3.0 // indirect
-	golang.org/x/text v0.5.0 // indirect
+	go.opencensus.io v0.24.0 // indirect
+	golang.org/x/crypto v0.9.0 // indirect
+	golang.org/x/net v0.10.0 // indirect
+	golang.org/x/oauth2 v0.8.0 // indirect
+	golang.org/x/sys v0.8.0 // indirect
+	golang.org/x/text v0.9.0 // indirect
+	google.golang.org/api v0.126.0 // indirect
+	google.golang.org/appengine v1.6.7 // indirect
+	google.golang.org/genproto v0.0.0-20230530153820-e85fd2cbaebc // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20230530153820-e85fd2cbaebc // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20230530153820-e85fd2cbaebc // indirect
+	google.golang.org/protobuf v1.30.0 // indirect
 	gopkg.in/ini.v1 v1.67.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )