Panic on plugin.(*GRPCProvider).ConfigureProvider call

[AndreasSko]

When using the provider (v1.16.1) for the first time with only one resource, I encountered the following error:

│ Error: Plugin did not respond
│ 
│   with module.<module-name>.provider["registry.terraform.io/mastercard/restapi"],
│   on modules/<module-name>/providers.tf line 22, in provider "restapi":
│   22: provider "restapi" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ConfigureProvider call. The plugin logs may contain
│ more details.
╵

Stack trace from the terraform-provider-restapi_v1.16.1 plugin:

panic: interface conversion: interface {} is map[string]interface {}, not map[string][]string

goroutine 36 [running]:
github.com/Mastercard/terraform-provider-restapi/restapi.configureProvider(0xc000116300)
	github.com/Mastercard/terraform-provider-restapi/restapi/provider.go:254 +0xc65
github.com/hashicorp/terraform/helper/schema.(*Provider).Configure(0xc000142180, 0xc0004d0070)
	github.com/hashicorp/[email protected]/helper/schema/provider.go:266 +0x78
github.com/hashicorp/terraform/helper/plugin.(*GRPCProviderServer).Configure(0xc00011c018, {0xc000482780, 0x49ad46}, 0xc000482780)
	github.com/hashicorp/[email protected]/helper/plugin/grpc_provider.go:439 +0x1fb
github.com/hashicorp/terraform/internal/tfplugin5._Provider_Configure_Handler({0xc397c0, 0xc00011c018}, {0xda3a70, 0xc000481710}, 0xc00048cc30, 0x0)
	github.com/hashicorp/[email protected]/internal/tfplugin5/tfplugin5.pb.go:2965 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000414300, {0xdafd28, 0xc00018a300}, 0xc0004b4600, 0xc000418360, 0x127d158, 0x0)
	google.golang.org/[email protected]/server.go:966 +0xe47
google.golang.org/grpc.(*Server).handleStream(0xc000414300, {0xdafd28, 0xc00018a300}, 0xc0004b4600, 0x0)
	google.golang.org/[email protected]/server.go:1245 +0x9e5
google.golang.org/grpc.(*Server).serveStreams.func1.1()
	google.golang.org/[email protected]/server.go:685 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/[email protected]/server.go:683 +0xef

Error: The terraform-provider-restapi_v1.16.1 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

The provider block looks like this:

provider "restapi" {
  uri                  = "...."
  write_returns_object = true
  id_attribute         = "id"

  oauth_client_credentials {
    oauth_client_id      = "...."
    oauth_client_secret  = "...."
    oauth_token_endpoint = "...."
    oauth_scopes         = ["...."]
  }
}

and the resource like this:

resource "restapi_object" "app_certificate" {
  path = "PATH"
  data = jsonencode({
    some = "data"
  })
  update_method  = "POST"
}

So nothing really special here, I think 🤔

I already had a quick look and seems to panic here:

terraform-provider-restapi/restapi/provider.go

Lines 253 to 254 in 0918001

	if tmp, ok := oauthConfig["endpoint_params"]; ok {
	m := tmp.(map[string][]string)

However, I'm wondering how this is even possible, as I'm not setting endpoint_params in the oauth_client_credentials block.

I really appreciate any help 🙂

[billgrant]

Hi, I was also was also able to reproduce this. It happens whenever you configure 'oauth_client_credentials'. I was able to get a plan to run by building the provider from this older commit. https://github.com/Mastercard/terraform-provider-restapi/tree/ef7f07a2334a87ff50c42625bc63934affbf6646

[laszlomiklosik]

Hello, I am also reproducing this problem with the latest version of the provider (1.16.1)

[dfry]

Hi, I was also was also able to reproduce this. It happens whenever you configure 'oauth_client_credentials'. I was able to get a plan to run by building the provider from this older commit. https://github.com/Mastercard/terraform-provider-restapi/tree/ef7f07a2334a87ff50c42625bc63934affbf6646

@billgrant how are you making use of your custom build of the provider? The only version that appears to be available on Hashicorp's site is the latest.

[runecalico]

@AndreasSko - My guess (I'm no go dev) is that while the schema has the endpoint_parms as optional, the way the code is written doesn't actually make them optional if you have a oauth_client_credentials block. So the code barfs on trying to do something with the empty interface{}? (line 254), Can you configure endpoint_parms and see if it works? I'm not sure what valid values are for that block (I didn't see anything in the docs or examples to indicate what they would be), but I bet it will at least give a different error :)

Changing that bit of code to the following at least makes endpoint_parms optional, but I don't know what the correct config is to determine if the endpoint_parms code is truely broken, or if my hack broke it.
if tmp, ok := oauthConfig["endpoint_params"].(map[string][]string); ok { m := tmp
yea, I'm sure you don't need to do m := tmp, but I didn't want to change much for my local hack as .. I'm no go developer (aka I don't know what I'm doing)

[runecalico]

if you don't need the endpoint_parms, and don't mind the above hack to make it usable without them in your provider config, I quickly setup a fork of the restapi provider with this modification (and two other PR's in waiting) here: https://registry.terraform.io/providers/runecalico/restapi/latest/docs/resources/object
With the Release notes here: https://github.com/runecalico/terraform-provider-restapi/releases/tag/v1.16.2-1

I am hoping this is just a stop-gap measure and @DRuggeri will be able to fix this error as well as merge in some of the waiting Pull Requests ..

[dfry]

thanks @runecalico, I had already gone done this route with a local developer override of the provider. forking and having a registry version to use was the next step, thanks for taking care of that.

[runecalico]

You are welcome :)

[DRuggeri]

Hey there, @runecalico - appreciate the callout. I've been watching this issue, but was a bit unclear on what the root of the problem is just yet. Your proposed patch does get past the issue when there are no endpoint_params set at all, but I need to dig into what happens when they ARE set (because, in my testing, that part breaks next).

[DRuggeri]

Well... I'm convinced that triaging this has uncovered a bug in terraform. The provider schema allows us to declare this:

...
                                                "endpoint_params": {
                                                        Type:        schema.TypeMap,
                                                        Optional:    true,
                                                        Description: "Additional key/values to pass to the underlying Oauth client library (as EndpointParams)",
                                                        Elem: &schema.Schema{
                                                                Type: schema.TypeList,
                                                                Elem: &schema.Schema{Type: schema.TypeString},
                                                        },
                                                },
...

(which basically says, I have a map with each key pointing to a list of strings)

... but when terraform attempts to validate the values of that list, it doesn't know how to handle a list:
https://github.com/hashicorp/terraform-plugin-sdk/blob/main/helper/schema/schema.go#L2056-L2113

So... either we shouldn't be allowed to set a map of string lists in our schema, or the validator must support that case. Simply fixing the types in this provider is enough to avoid the bug reported, but unfortunately uncovers another bug that isn't ours :-(

The stuff we can control will be fixed in 1.16.2 (which is building now...)

[matpen]

I am just testing this on 1.17.0 by using oauth_client_credentials with endpoint_params and I get the same crash described in the OP: is it possible to reopen this issue?

but when terraform attempts to validate the values of that list, it doesn't know how to handle a list:

Is there an issue open on terraform upstream? If not, the problem will never be worked on by the core team...

either we shouldn't be allowed to set a map of string lists in our schema, or the validator must support that case

Why are we using a map of list of strings in the first place? It seems like a map of strings would be sufficient, and maybe that can (temporarily) workaround the terraform bug?