plugin tls init: error="error during token unwrap request: secret is nil

[bernielomax]

I am experiencing an issue whilst trying to get a custom plugin to work. My plugin so far is just a copy of the mock plugin defined in the source.

I have been able to successfully build and add the plugin to Vault using:

vault write sys/plugins/catalog/my-plugin sha_256=1164fbec... \
     command="my-plugin -ca-cert=/certs/ca/ca.crt -client-cert=/certs/client/plugin.crt -client-key=/certs/client/plugin.key"

I can also mount the plugin:

vault mount -path=foo -plugin-name=my-plugin plugin

But when I try and read the mount point I get the following:

$ vault read foo/
Error reading foo/: Error making API request.

URL: GET https://vault.local:8200/v1/foo
Code: 500. Errors:

* 1 error occurred:

* plugin exited before we could connect

The log line in Vault shows:

2017/10/23 00:49:45.175905 [ERROR] plugin.my-plugin: plugin tls init: error="error during token unwrap request: secret is nil

I can confirm that my client certificates work when setting them as environment variables and by calling vault API commands via the vault client. I.e. by using:

• VAULT_CLIENT_CERT=/certs/client/plugin.crt
• VAULT_CLIENT_KEY=/certs/client/plugin.key
• VAULT_CACERT=/certs/ca/ca.crt
• VAULT_ADDR=https://vault.local:8200

$ vault read secret
No value found at secret

This appears to be similar to hashicorp/vault-plugin-auth-kubernetes#5 but the solution does not appear to have worked in my case.

I can also confirm that using -tls-skip-verify makes no difference.

I am also using the apiClientMeta.GetTLSConfig()

func (f *APIClientMeta) FlagSet() *flag.FlagSet {
	fs := flag.NewFlagSet("vault plugin settings", flag.ContinueOnError)

	fs.StringVar(&f.flagCACert, "ca-cert", "", "")
	fs.StringVar(&f.flagCAPath, "ca-path", "", "")
	fs.StringVar(&f.flagClientCert, "client-cert", "", "")
	fs.StringVar(&f.flagClientKey, "client-key", "", "")
	fs.BoolVar(&f.flagInsecure, "tls-skip-verify", false, "")

	return fs
}

func (f *APIClientMeta) GetTLSConfig() *api.TLSConfig {
	// If we need custom TLS configuration, then set it
	if f.flagCACert != "" || f.flagCAPath != "" || f.flagClientCert != "" || f.flagClientKey != "" || f.flagInsecure {
		t := &api.TLSConfig{
			CACert:        f.flagCACert,
			CAPath:        f.flagCAPath,
			ClientCert:    f.flagClientCert,
			ClientKey:     f.flagClientKey,
			TLSServerName: "",
			Insecure:      f.flagInsecure,
		}

		return t
	}

	return nil
}

[briankassouf]

Hi @bernielomax

Could you also share the vault configuration file you are using? As well as any vault environment variables that may be set?

[bernielomax]

Hi @briankassouf ,

Sorry for the late reply. My config looks like:

storage "consul" {
  address = "consul:8500"
  path    = "vault"
  scheme  = "http"
}

listener "tcp" {
  address     = "0.0.0.0:8200"
  tls_cert_file = "/certs/server/vault.local.bundle.crt"
  tls_key_file = "/certs/server/vault.local.key"
  tls_client_ca_file = "/certs/ca/ca.crt"
}

plugin_directory = "/plugins"

Environment variables:

• VAULT_ADDR=https://vault.local:8200

I have vault.local as my certs common name and my /etc/hosts has vault.local pointing to 127.0.0.1 as I am just running it locally.

[briankassouf]

Could you try setting the redirect_addr setting in the consul stanza? Like so:

storage "consul" {
  address = "consul:8500"
  path    = "vault"
  scheme  = "http"
  redirect_addr = "https://<replace with ip>:8200"
}

[bernielomax]

Hey @briankassouf,

Thanks for your last post. I finally got a chance to test it out and seems to have fixed the issue. I now get:

vault read foo
No value found at foo

Which should be cool because my plugin doesnt actually do anything yet. Im a little confused about what adding the redirect_addr configuration actually did though?

[briankassouf]

Glad that fixed it! Since plugins use the vault API to unwrap a response wrapping token, they need to know the address of the api. #3499 is filed to make this a little less confusing in the configuration file.

[calvn]

Closing this since the config now (as of 0.9.0) supports the top-level api_addr parameter. #3620 further clarifies the use of this value in the context of plugins.