Connection error while upgrading from 2.9.0 to 3.1.0

[perevernihata]

What happened?

We are upgrading auth0 pulumi package from 2.9.0 to 3.1.0, and encountered this error:
auth0:index:Connection (amip-Username-Password-Authentication-staging):
error: unmarshaling urn:pulumi:staging::ami-pulumi::auth0:index/connection:Connection::amip-Username-Password-Authentication-staging's instance state: could not read field options: '' expected type 'string', got unconvertible type 'map[string]interface {}', value: 'map[]'

As far as I can tell, there is nothing that has changed while between these version for the Connection resource, so I am unsure what is happening here:

Example

The repo is not public, I can not provide the link. I did not try reproducing this issue on the blank repo yet.

Output of pulumi about

CLI
Version 3.99.0
Go Version go1.21.5
Go Compiler gc

Host
OS darwin
Version 14.1.1
Arch x86_64

Pulumi locates its logs in /var/folders/d9/qvsrk_hj2fv1d8cp_21t4kbm0000gn/T/ by default
warning: Failed to read project: no Pulumi.yaml project file found (searching upwards from /Users/ivanperevernykhata/temp/ami-platform-monorepo). If you have not created a project yet, use pulumi new to do so: no project file found
warning: Could not access the backend: read ".pulumi/meta.yaml": blob (key ".pulumi/meta.yaml") (code=Unknown): AccessDenied: Access Denied
status code: 403, request id: KNMXT38BZ29XR3KK, host id: usCOMs1THwJZfrXTWPM7sCZIIRuJOFHEF38EH/s1h//G5D+wfk2df3t6Q1GGJtRLR8MV0h45gc0=
(base) ➜ ami-platform-monorepo git:(AMIP-1439-upgrade-pulumi-auth0-package) ✗ ./scripts/10-provision-pulumi.sh staging

Bucket owned and existsLogging in into Pulumi backend: ami-pulumi-backend-staging

up to date, audited 215 packages in 3s

49 packages are looking for funding
run npm fund for details

2 vulnerabilities (1 moderate, 1 high)

To address all issues, run:
npm audit fix

Run npm audit for details.
stack selected: staging
Updating Pulumi Stack
CLI
Version 3.99.0
Go Version go1.21.5
Go Compiler gc

Plugins
NAME VERSION
auth0 3.1.0
aws 6.17.0
aws-native 0.92.0
ec 0.7.0
mongodbatlas 3.13.0
newrelic 5.16.0
nodejs unknown

Host
OS darwin
Version 14.1.1
Arch x86_64

This project is written in nodejs: executable='/Users/ivanperevernykhata/.nvm/versions/node/v18.13.0/bin/node' version='v18.13.0'

Current Stack: staging

TYPE URN
pulumi:pulumi:Stack urn:pulumi:staging::ami-pulumi::pulumi:pulumi:Stack::ami-pulumi-staging
pulumi:providers:aws urn:pulumi:staging::ami-pulumi::pulumi:providers:aws::default_6_17_0
pulumi:providers:mongodbatlas urn:pulumi:staging::ami-pulumi::pulumi:providers:mongodbatlas::default_3_13_0
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::/copilot/ami/staging/secrets/MONGODB_PASS
mongodbatlas:index/project:Project urn:pulumi:staging::ami-pulumi::mongodbatlas:index/project:Project::AMI Platform
mongodbatlas:index/databaseUser:DatabaseUser urn:pulumi:staging::ami-pulumi::mongodbatlas:index/databaseUser:DatabaseUser::amip-staging-deployment
mongodbatlas:index/cluster:Cluster urn:pulumi:staging::ami-pulumi::mongodbatlas:index/cluster:Cluster::amip-staging
mongodbatlas:index/projectIpAccessList:ProjectIpAccessList urn:pulumi:staging::ami-pulumi::mongodbatlas:index/projectIpAccessList:ProjectIpAccessList::acl
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::/copilot/ami/staging/secrets/MONGODB_USER
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::/copilot/ami/staging/secrets/MONGODB_CONNECTION
aws:ses/domainIdentity:DomainIdentity urn:pulumi:staging::ami-pulumi::aws:ses/domainIdentity:DomainIdentity::sesDomainIdentity
aws:iam/user:User urn:pulumi:staging::ami-pulumi::aws:iam/user:User::sesSmtpUser
aws:ses/emailIdentity:EmailIdentity urn:pulumi:staging::ami-pulumi::aws:ses/emailIdentity:EmailIdentity::sesEmailIdentity
aws:route53/record:Record urn:pulumi:staging::ami-pulumi::aws:route53/record:Record::sesDomainRoute53VerificationRecord
aws:ses/mailFrom:MailFrom urn:pulumi:staging::ami-pulumi::aws:ses/mailFrom:MailFrom::domainMailFrom
aws:ses/domainDkim:DomainDkim urn:pulumi:staging::ami-pulumi::aws:ses/domainDkim:DomainDkim::sesDomainDkim
aws:iam/userPolicy:UserPolicy urn:pulumi:staging::ami-pulumi::aws:iam/userPolicy:UserPolicy::sesSmtpUserPolicy
aws:iam/accessKey:AccessKey urn:pulumi:staging::ami-pulumi::aws:iam/accessKey:AccessKey::sesSmtpUserAccessKey
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::/copilot/ami/staging/public/AWS_SES_EMAIL
aws:route53/record:Record urn:pulumi:staging::ami-pulumi::aws:route53/record:Record::sesDomainMailFromMx
aws:route53/record:Record urn:pulumi:staging::ami-pulumi::aws:route53/record:Record::sesDomainMailFromTxt
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::/copilot/ami/staging/secrets/AWS_SES_SMTP_USER_ID
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::/copilot/ami/staging/secrets/AWS_SES_SMTP_USER_SECRET
aws:route53/record:Record urn:pulumi:staging::ami-pulumi::aws:route53/record:Record::sesDomainDkimRecord-0
aws:route53/record:Record urn:pulumi:staging::ami-pulumi::aws:route53/record:Record::sesDomainDkimRecord-2
aws:route53/record:Record urn:pulumi:staging::ami-pulumi::aws:route53/record:Record::sesDomainDkimRecord-1
pulumi:providers:auth0 urn:pulumi:staging::ami-pulumi::pulumi:providers:auth0::default_2_9_0
pulumi:providers:newrelic urn:pulumi:staging::ami-pulumi::pulumi:providers:newrelic::default_5_16_0
auth0:index/customDomain:CustomDomain urn:pulumi:staging::ami-pulumi::auth0:index/customDomain:CustomDomain::ami_custom_domain
newrelic:index/alertPolicy:AlertPolicy urn:pulumi:staging::ami-pulumi::newrelic:index/alertPolicy:AlertPolicy::frontend-policy
auth0:index/client:Client urn:pulumi:staging::ami-pulumi::auth0:index/client:Client::amip-client-staging
auth0:index/action:Action urn:pulumi:staging::ami-pulumi::auth0:index/action:Action::Add tenantId to user_metadata
auth0:index/resourceServer:ResourceServer urn:pulumi:staging::ami-pulumi::auth0:index/resourceServer:ResourceServer::amip-backend-staging
newrelic:index/alertPolicy:AlertPolicy urn:pulumi:staging::ami-pulumi::newrelic:index/alertPolicy:AlertPolicy::backend-policy
auth0:index/email:Email urn:pulumi:staging::ami-pulumi::auth0:index/email:Email::sesEmailProvider
newrelic:index/alertPolicy:AlertPolicy urn:pulumi:staging::ami-pulumi::newrelic:index/alertPolicy:AlertPolicy::worker-policy
auth0:index/client:Client urn:pulumi:staging::ami-pulumi::auth0:index/client:Client::amip-machine-to-machine-staging
newrelic:index/alertPolicy:AlertPolicy urn:pulumi:staging::ami-pulumi::newrelic:index/alertPolicy:AlertPolicy::admin-worker-policy
auth0:index/client:Client urn:pulumi:staging::ami-pulumi::auth0:index/client:Client::amip-native-client-staging
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::/copilot/ami/staging/secrets/AUTH0_CLIENT_ID
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::/copilot/ami/staging/secrets/AUTH0_CLIENT_SECRET
auth0:index/connection:Connection urn:pulumi:staging::ami-pulumi::auth0:index/connection:Connection::amip-Username-Password-Authentication-staging
auth0:index/triggerBinding:TriggerBinding urn:pulumi:staging::ami-pulumi::auth0:index/triggerBinding:TriggerBinding::postLoginTriggerBinding
auth0:index/emailTemplate:EmailTemplate urn:pulumi:staging::ami-pulumi::auth0:index/emailTemplate:EmailTemplate::welcomeEmailTemplate
auth0:index/emailTemplate:EmailTemplate urn:pulumi:staging::ami-pulumi::auth0:index/emailTemplate:EmailTemplate::resetPasswordTemplate
auth0:index/emailTemplate:EmailTemplate urn:pulumi:staging::ami-pulumi::auth0:index/emailTemplate:EmailTemplate::verifyEmailTemplate
auth0:index/tenant:Tenant urn:pulumi:staging::ami-pulumi::auth0:index/tenant:Tenant::amip-tenant
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::/copilot/ami/staging/secrets/AUTH0_CONNECTION_NAME
newrelic:index/notificationDestination:NotificationDestination urn:pulumi:staging::ami-pulumi::newrelic:index/notificationDestination:NotificationDestination::slack-webhook
newrelic:index/notificationChannel:NotificationChannel urn:pulumi:staging::ami-pulumi::newrelic:index/notificationChannel:NotificationChannel::slack-webhook-channel
newrelic:index/nrqlAlertCondition:NrqlAlertCondition urn:pulumi:staging::ami-pulumi::newrelic:index/nrqlAlertCondition:NrqlAlertCondition::backend-apm-error-rate
newrelic:index/nrqlAlertCondition:NrqlAlertCondition urn:pulumi:staging::ami-pulumi::newrelic:index/nrqlAlertCondition:NrqlAlertCondition::worker-apm-error-rate
newrelic:index/nrqlAlertCondition:NrqlAlertCondition urn:pulumi:staging::ami-pulumi::newrelic:index/nrqlAlertCondition:NrqlAlertCondition::workers-admin-apm-error-rate
pulumi:providers:aws urn:pulumi:staging::ami-pulumi::pulumi:providers:aws::healthcheckAlarmRegion
aws:s3/bucket:Bucket urn:pulumi:staging::ami-pulumi::aws:s3/bucket:Bucket::cloudtrailBucket
aws:sns/topic:Topic urn:pulumi:staging::ami-pulumi::aws:sns/topic:Topic::eu-west2-alarms-topic
aws:route53/healthCheck:HealthCheck urn:pulumi:staging::ami-pulumi::aws:route53/healthCheck:HealthCheck::appHealthcheck
aws:route53/healthCheck:HealthCheck urn:pulumi:staging::ami-pulumi::aws:route53/healthCheck:HealthCheck::apiHealthcheck
newrelic:index/nrqlAlertCondition:NrqlAlertCondition urn:pulumi:staging::ami-pulumi::newrelic:index/nrqlAlertCondition:NrqlAlertCondition::browser-error-rate
newrelic:index/nrqlAlertCondition:NrqlAlertCondition urn:pulumi:staging::ami-pulumi::newrelic:index/nrqlAlertCondition:NrqlAlertCondition::browser-apdex-rate
aws:route53/healthCheck:HealthCheck urn:pulumi:staging::ami-pulumi::aws:route53/healthCheck:HealthCheck::apiWorkerHealthcheck
newrelic:index/workflow:Workflow urn:pulumi:staging::ami-pulumi::newrelic:index/workflow:Workflow::policy-workflow
pulumi:providers:aws urn:pulumi:staging::ami-pulumi::pulumi:providers:aws::snsRegion
aws:cloudwatch/eventRule:EventRule urn:pulumi:staging::ami-pulumi::aws:cloudwatch/eventRule:EventRule::guardduty-findings-rule
aws:guardduty/detector:Detector urn:pulumi:staging::ami-pulumi::aws:guardduty/detector:Detector::guardduty
pulumi:providers:ec urn:pulumi:staging::ami-pulumi::pulumi:providers:ec::default_0_7_0
aws:iam/role:Role urn:pulumi:staging::ami-pulumi::aws:iam/role:Role::chatbotRole
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::redis-alarm-CPUUtilization
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::jobs-alarm-preprocess-data-queue
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::ecs-worker-task-alarm
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::es-backup-alarm-errors
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::redis-alarm-DatabaseMemoryUsagePercentage
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::jobs-alarm-ms-index-api-file-queue
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::jobs-alarm-ms-index-api-file-since-inception-queue
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::redis-alarm-Evictions
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::jobs-alarm-ms-map-category-performance-xml-queue
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::jobs-alarm-detect-significant-changes-queue
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::jobs-alarm-ms-map-fund-metadata-xml-queue
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::jobs-alarm-gs-padi-queue
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::jobs-alarm-ms-queue
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::jobs-alarm-ms-benchmark-pricing-queue
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::jobs-alarm-ms-store-api-file-queue
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::jobs-alarm-notifications-queue
aws:cloudwatch/eventTarget:EventTarget urn:pulumi:staging::ami-pulumi::aws:cloudwatch/eventTarget:EventTarget::guardduty-findings-rule-target
aws:sns/topic:Topic urn:pulumi:staging::ami-pulumi::aws:sns/topic:Topic::us-east1-alarms-topic
aws:s3/bucketPolicy:BucketPolicy urn:pulumi:staging::ami-pulumi::aws:s3/bucketPolicy:BucketPolicy::s3BucketPolicy
aws:cloudtrail/trail:Trail urn:pulumi:staging::ami-pulumi::aws:cloudtrail/trail:Trail::cloudtrailEvents
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::ami-admin-worker-staging-alarm
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::ami-backend-staging-alarm
pulumi:providers:aws-native urn:pulumi:staging::ami-pulumi::pulumi:providers:aws-native::default_0_92_0
aws:cloudwatch/metricAlarm:MetricAlarm urn:pulumi:staging::ami-pulumi::aws:cloudwatch/metricAlarm:MetricAlarm::ami-app-staging-alarm
aws-native:chatbot:SlackChannelConfiguration urn:pulumi:staging::ami-pulumi::aws-native:chatbot:SlackChannelConfiguration::monitoringChatbot
ec:index/deployment:Deployment urn:pulumi:staging::ami-pulumi::ec:index/deployment:Deployment::ami-search-staging
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::es_username_staging
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::es_password_staging
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::kibana_host_staging
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::es_host_staging
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::es_cloud_id_staging
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::es_api_key_staging
aws:ssm/parameter:Parameter urn:pulumi:staging::ami-pulumi::aws:ssm/parameter:Parameter::es_api_key_id_staging

Found no pending operations associated with staging

Backend
Name Ivans-MacBook-Pro.local
URL s3://ami-pulumi-backend-staging?disableSSL=true
User ivanperevernykhata
Organizations
Token type personal

Dependencies:
NAME VERSION
@pulumi/mongodbatlas 3.13.0
@pulumi/newrelic 5.16.0
@pulumi/pulumi 3.100.0
@pulumi/auth0 3.1.0
@pulumi/aws-native 0.92.0
@pulumi/aws 6.17.0
axios 1.6.5
handlebars 4.7.8
ramda 0.29.1
@pulumi/ec 0.7.0
@types/node 18.19.5
@types/ramda 0.29.9
generate-password 1.7.1

Pulumi locates its logs in /var/folders/d9/qvsrk_hj2fv1d8cp_21t4kbm0000gn/T/ by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[iwahbe]

Hi @perevernihata. I was able to repro your issue with this program:

name: dev-yaml
runtime: yaml
resources:
  googleOauth2:
    type: auth0:Connection
    properties:
      options:
        allowedAudiences:
          - example.com
          - api.example.com
        nonPersistentAttrs:
          - ethnicity
          - gender
        scopes:
          - email
          - profile
          - gmail
          - youtube
        setUserRootAttributes: on_each_login
      strategy: google-oauth2
    options:
      version: v2.9.0 # Change to v3.1.0 after up

[iwahbe]

Digging further, this happens whenever upgrading from v2.9.0 to a higher version. This is called out by the release notes on v2.10.0:

Breaking Changes:

Type "auth0:index/ConnectionOptions:ConnectionOptions" input "fieldsMap" type changed from "object" to "string"

Please make sure to do the following when upgrading:

pulumi state delete $urn
pulumi import auth0:index/connection:Connection

[perevernihata]

I read about this import command, and it kinda spooks me a bit, I do not fully understand how it will detect the resource.
So if the existing arn is:

urn:pulumi:staging::ami-pulumi::auth0:index/connection:Connection::amip-Username-Password-Authentication-staging

The commands will look like:

pulumi state delete urn:pulumi:staging::ami-pulumi::auth0:index/connection:Connection::amip-Username-Password-Authentication-staging
pulumi import auth0:index/connection:Connection amip-Username-Password-Authentication-staging con_rf5ixUn2w0xbmTW9

Where con_rf5ixUn2w0xbmTW9 is an connection identifier I found in auth0, and name is the one I specify in the code while provisioning resource. Is that correct?

[perevernihata]

There is a number of resources that depend on the connection, including tenant (which I am very afraid of deleting using --target-dependents

[iwahbe]

I'll see what I can do to make this easier.

[iwahbe]

The next release (v3.1.2) should have have this fixed. The release process is kicking off now.