Provide missing key names in error messages

[ToddG]

What happened?

WHAT
pulumi up fails with missing keys, but does not report what the missing keys actually are.

RELATED

• Including the Mobile Analytics endpoint causes Pulumi to fail to validate the provider configuration localstack/pulumi-local#5
• Provide missing key names in error messages localstack/pulumi-local#7
• https://pulumi-community.slack.com/archives/C026EE6N2VB/p1668542984295379

Steps to reproduce

Following Instructions:

• https://docs.localstack.cloud/integrations/pulumi/

Installed

• https://github.com/localstack/pulumi-local
• https://github.com/localstack/localstack

REPRO

1. Install deps

pip install localstack
pip install pulumi-local

2. Check versions

aws --version
aws-cli/2.8.12 Python/3.9.11 Linux/5.15.0-52-generic exe/x86_64.ubuntu.22 prompt/off
python version
/home/todd/.pyenv/versions/test-3.10.6/bin/python
pulumi version
v3.46.1
pulumilocal version
v3.46.1

3. Start localstack

localstack start -d
docker ps

CONTAINER ID   IMAGE                   COMMAND                  CREATED             STATUS                       PORTS                                                                                                                          NAMES
db15aefc28b3   localstack/localstack   "docker-entrypoint.sh"   About an hour ago   Up About an hour (healthy)   127.0.0.1:4510-4559->4510-4559/tcp, 127.0.0.1:4566->4566/tcp, 127.0.0.1:4571->4571/tcp, 127.0.0.1:12121->12121/tcp, 5678/tcp   localstack_main

4. Check localstack services

localstack status services
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Service                  ┃ Status      ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ acm                      │ ✔ available │
│ apigateway               │ ✔ available │
│ cloudformation           │ ✔ available │
│ cloudwatch               │ ✔ available │
│ config                   │ ✔ available │
│ dynamodb                 │ ✔ available │
│ dynamodbstreams          │ ✔ available │
│ ec2                      │ ✔ available │
│ es                       │ ✔ available │
│ events                   │ ✔ available │
│ firehose                 │ ✔ available │
│ iam                      │ ✔ available │
│ kinesis                  │ ✔ available │
│ kms                      │ ✔ available │
│ lambda                   │ ✔ available │
│ logs                     │ ✔ available │
│ opensearch               │ ✔ available │
│ redshift                 │ ✔ available │
│ resource-groups          │ ✔ available │
│ resourcegroupstaggingapi │ ✔ available │
│ route53                  │ ✔ available │
│ route53resolver          │ ✔ available │
│ s3                       │ ✔ available │
│ s3control                │ ✔ available │
│ secretsmanager           │ ✔ available │
│ ses                      │ ✔ available │
│ sns                      │ ✔ available │
│ sqs                      │ ✔ running   │
│ ssm                      │ ✔ available │
│ stepfunctions            │ ✔ available │
│ sts                      │ ✔ available │
│ support                  │ ✔ available │
│ swf                      │ ✔ available │
│ transcribe               │ ✔ available │
└──────────────────────────┴─────────────┘

5. Create project

mkdir quickstart && cd quickstart
pulumi new aws-python
pulumilocal init
pulumilocal up

ERROR

pulumilocal up
Previewing update (localstack)

View Live: https://app.pulumi.com/ToddG/blargblarg/localstack/previews/45637377-b694-4e2b-a242-27d614bd04df

     Type                 Name                   Plan       Info
 +   pulumi:pulumi:Stack  blargblarg-localstack  create     1 warning
     └─ aws:s3:Bucket     my-bucket                         1 error


Diagnostics:
  aws:s3:Bucket (my-bucket):
    error: could not validate provider configuration: 1 error occurred:
    	* Invalid or unknown key

  pulumi:pulumi:Stack (blargblarg-localstack):
    warning: provider config warning: Use s3_use_path_style instead.

Further Links:
pulumi/examples#185 (comment)
pulumi/pulumi#3546
pulumi/pulumi#4718
pulumi/pulumi#4720

Expected Behavior

At a high level, I'd expect the pulumi up command to succeed. If it doesn't I expect an error message that tells me what the problem is. Here's what we get:

  aws:s3:Bucket (my-bucket):
    error: could not validate provider configuration: 1 error occurred:
    	* Invalid or unknown key

Hello? Bueller? What's the Invalid or unknown key?

Here's what would be helpful

  aws:s3:Bucket (my-bucket):
    error: could not validate provider configuration: 1 error occurred:
    	* Invalid or unknown key : NAME_OF_INVALID_OR_UNKNOWN_KEY

Actual Behavior

See above.

Output of pulumi about

$ pulumi about
CLI
Version 3.46.1
Go Version go1.19.2
Go Compiler gc

Plugins
NAME VERSION
aws 5.20.0
python unknown

Host
OS ubuntu
Version 22.04
Arch x86_64

This project is written in python: executable='/home/todd/.pyenv/shims/python3' version='3.10.6
'

Current Stack: localstack

TYPE URN
pulumi:pulumi:Stack urn:pulumi:localstack::blargblarg::pulumi:pulumi:Stack::blargblarg-localstack
pulumi:providers:aws urn:pulumi:localstack::blargblarg::pulumi:providers:aws::default_5_20_0
aws:s3/bucket:Bucket urn:pulumi:localstack::blargblarg::aws:s3/bucket:Bucket::my-bucket

Found no pending operations associated with localstack

Backend
Name pulumi.com
URL https://app.pulumi.com/ToddG
User ToddG
Organizations ToddG, redzone-co

Dependencies:
NAME VERSION
pip 22.3.1
pulumi-aws 5.20.0
setuptools 65.5.1
wheel 0.38.4

Pulumi locates its logs in /tmp by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[Frassle]

I think this is probably more an issue in the aws provider than the engine? cc @pulumi/platform-providers

Also referencing back to the confusion of pulumi/pulumi#10446, this:

  aws:s3:Bucket (my-bucket):
    error: could not validate provider configuration: 1 error occurred:
    	* Invalid or unknown key

Is a bit odd that it looks like a bucket error but it's actually a provider error. I'm very tempted to remove the async configuration of providers to get better looking diagnostics here.

[thomas11]

The error message originates from terraform-plugin-sdk so I don't think it's provider specific. The key name is added in AttributePath but we seem to lose it somewhere.

[t0yv0]

I have some good news that might be relevant here. This looks like it could potentially be a duplicate #906 fixed by #1153 released in https://github.com/pulumi/pulumi-terraform-bridge/releases/tag/v3.47.4 release of the bridge, but the fix being applied to the bridge it would need pulumi-aws to upgrade to 3.47.4 before we can attempt to repro.

[iwahbe]

I checked, and this problem still presents on pulumi-aws 5.41.0 (on bridge version v3.49.1).

[t0yv0]

I actually cannot reproduce right now.

#!/usr/bin/env bash

set -euo pipefail

mkdir quick
cd quick

pulumi new aws-python
source venv/bin/activate
python -m pip install localstack
python -m pip install pulumi-local
localstack start -d
pulumilocal init
pulumilocal up
localstack stop

$ aws --version 
aws-cli/2.8.12 Python/3.10.8 Darwin/22.4.0 source/x86_64 prompt/off
$ python --version     
Python 3.9.6
$ pulumilocal version               
v3.72.2

Updating (localstack)

View in Browser (Ctrl+O): https://app.pulumi.com/t0yv0/localstack-repro/localstack/updates/1

     Type                 Name                         Status              
 +   pulumi:pulumi:Stack  localstack-repro-localstack  created (1s)        
 +   └─ aws:s3:Bucket     my-bucket                    created (0.53s)     


Outputs:
    bucket_name: "my-bucket-0483902"

Resources:
    + 2 created

Duration: 3s

pulumi about

CLI          
Version      3.72.2
Go Version   go1.20.5
Go Compiler  gc

Plugins
NAME    VERSION
aws     5.41.0
local   0.6.0
python  unknown

Host     
OS       darwin
Version  13.3.1
Arch     x86_64

This project is written in python: executable='/Users/t0yv0/tmp/2023/06/aws-561/quickstart/venv/bin/python3' version='3.9.6
'

Current Stack: t0yv0/localstack-repro/localstack

TYPE                  URN
pulumi:pulumi:Stack   urn:pulumi:localstack::localstack-repro::pulumi:pulumi:Stack::localstack-repro-localstack
pulumi:providers:aws  urn:pulumi:localstack::localstack-repro::pulumi:providers:aws::default_5_41_0
aws:s3/bucket:Bucket  urn:pulumi:localstack::localstack-repro::aws:s3/bucket:Bucket::my-bucket


Found no pending operations associated with localstack

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/t0yv0
User           t0yv0
Organizations  t0yv0, pulumi

Dependencies:
NAME          VERSION
localstack    2.1.0
pip           23.1.2
pulumi-aws    5.41.0
pulumi-local  0.6.0
setuptools    68.0.0
wheel         0.40.0

Pulumi locates its logs in /var/folders/gk/cchgxh512m72f_dmkcc3d09h0000gp/T/ by default

[t0yv0]

All right, looks like localstack/pulumi-local#6 fix has landed and removed the problematic key.

I can still reproduce the issue if I build against

python -m pip install localstack==0.4

However, if I try to rebuild pulumi-aws against the latest bridge version I get this new behavior:

  pulumi:providers:aws (default_5_41_0):
    error: pulumi:providers:aws resource 'default_5_41_0' has a problem: could not validate provider configur
ation: Invalid or unknown key. Check `pulumi config get aws:endpoints`. Did you mean `aws:endpoints`?

The "Did you mean aws:endpoints" bit is an issue with typo detection code that we need to fix, but the original issue appears solved. I've checked the diagnostic we are receiving from the upstream provider and it does not appear that the bridge is dropping any information - unfortunately the diagnostic just says "Invalid or unknown key" and does not state what the problem is.

[t0yv0]

Confirmed fixed as of v3.50.1 version of the bridge. This fix should propagate to pulumi-aws through routine dependency updates.