Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to terraform provider v5.46.0 #1271

Closed

Conversation

mbbush
Copy link
Collaborator

@mbbush mbbush commented Apr 20, 2024

Description of your changes

Upgrade to the latest terraform provider version. There have been several complications, discussed below.

I have:

  • Read and followed Crossplane's [contribution process].
  • Run make reviewable to ensure this PR is ready for review.

Complications

This is integrating several separate pieces, which conflict with each other.

  • AWSClient.Session is now private
    • Since Count external API calls #1241 we're invoking the .Session property on conns.AWSClient
    • In hashicorp/terraform-provider-aws@b87c156 the terraform AWS provider made the .Session property private (which isn't a breaking change to their public API since it's in internal, but we use a forked version of the TF provider that exposes some internal objects to make our no-fork architecture possible).
    • Thanks to a suggestion from @mergenci I added a getter in my fork of the terraform provider. This appears to be working but I would like help verifying that
    • metrics are still working properly for both go sdk v1 and v2
  • EKS ClusterAuth
  • EMR Serverless Application type change

Potentially incompatible schema changes:

From https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8792472872/job/24128687844

Report breaking CRD OpenAPI v3 schema changes:

  • package/crds/cloud9.aws.upbound.io_environmentec2s.yaml
    • Uptest successful
  • package/crds/dms.aws.upbound.io_eventsubscriptions.yaml
    • This is not currently uptestable, and looks not trivial to make uptestable. The change is marking a field which was erroneously made required in TF provider 5.31 back to optional. Based on looking at the change, I don't think this will be a problem.
  • package/crds/fsx.aws.upbound.io_ontapfilesystems.yaml
    • Uptest successful

Report native schema version changes

  • aws_secretsmanager_secret_rotation
    • IUptest successful

Uptest runs

✔️ EKS ClusterAuth: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8794111828
✔️ FSX OnTapFileSystem: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8794383020
✔️ EMR Serverless Application: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8794565770
✔️ Cloud9 EnvironmentEc2: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8795118229
✔️ Secretsmanager SecretRotation: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/8795106214

Terraform provider changelog lines to look into:

This is a very light filter through the tf provider changelog, with things I though might be slightly notable or worth checking. I'm sure there are a bunch of things on here that just won't matter at all.

  • 5.42.0 BUG FIXES * resource/aws_appautoscaling_scheduled_action: Read correct resource by using scalable_dimension as an additional filter
  • 5.37.0 BUG FIXES * resource/aws_autoscaling_group: Fix version to computed for inconsistent final plan issue
  • 5.44.0 ENHANCEMENTS * resource/aws_batch_job_definition: Add update functions instead of ForceNew. Add deregister_on_new_revision to allow keeping prior versions ACTIVE when a new revision is published.
  • 5.46.0 BUG FIXES * resource/aws_ce_anomaly_monitor: Change monitor_dimension to [ForceNew]
  • 5.32.0 BUG FIXES * resource/aws_cloud9_environment_ec2: image_id is Required
  • 5.46.0 ENHANCEMENTS * resource/aws_cloudformation_stack_set: Add retry when creating to potentially help with eventual consistency problems
  • 5.36.0 ENHANCEMENTS * resource/aws_cloudwatch_metric_stream: Add plan-time validation of output_format
  • 5.34.0 ENHANCEMENTS * resource/aws_codecommit_repository: Add kms_key_id argument
    • (This one is causing issues with the codegen pipeline)
  • 5.34.0 ENHANCEMENTS * resource/aws_codecommit_trigger: Add plan-time validation of trigger.destination_arn and trigger.events
  • 5.39.0 ENHANCEMENTS * resource/aws_config_configuration_recorder: Add plan-time validation of aws_config_organization_custom_rule.lambda_function_arn
  • 5.42.0 BUG FIXES * resource/aws_datasync_location_s3: Fix missing s3_bucket_arn attribute value from state read/refresh
  • 5.38.0 ENHANCEMENTS * resource/aws_db_instance: Add plan-time validation of performance_insights_retention_period
  • 5.40.0 BUG FIXES * resource/aws_db_instance: Correctly sets parameter_group_name when replicate_source_db is in different region.
  • 5.44.0 ENHANCEMENTS * resource/aws_db_instance: Adds warning when setting character_set_name when replicate_source_db, restore_to_point_in_time, or snapshot_identifier is set
  • 5.37.0 BUG FIXES * resource/aws_db_proxy: Change auth from TypeList to TypeSet as order is not significant
  • 5.35.0 BUG FIXES * resource/aws_dms_event_subscription: Mark source_ids as Optional. This fixes a regression introduced in [v5.31.0]
  • 5.46.0 BUG FIXES * resource/aws_dms_replication_task: Allows leaving replication_task_settings unset to use default settings.
  • 5.41.0 ENHANCEMENTS * resource/aws_ecs_cluster: Add default value (DEFAULT) for configuration.execute_command_configuration.logging
  • 5.41.0 BUG FIXES * resource/aws_ecs_cluster: Fix panic: interface conversion: interface {} is nil, not map[string]interface {} when configuration, configuration.execute_command_configuration, or configuration.execute_command_configuration.log_configuration are empty
  • 5.41.0 BUG FIXES * resource/aws_ecs_service: service_connect_configuration.service.tls.issuer_cert_authority.aws_pca_authority_arn is Required
  • 5.35.0 BUG FIXES * resource/aws_efs_file_system: Increase lifecycle_policy maximum item limit to 3
  • 5.32.0 ENHANCEMENTS * resource/aws_fsx_ontap_file_system: throughput_capacity is Optional
  • 5.46.0 BUG FIXES * resource/aws_glue_job: Mark number_of_workers and worker_type as optional/computed, preventing persistent differences when max_capacity is set.
  • 5.39.0 ENHANCEMENTS * resource/aws_instance: Apply default tags to volumes/block devices managed through an aws_instance, add ebs_block_device.*.tags_all and root_block_device.*.tags_all attributes which include default tags
  • 5.36.0 BUG FIXES * resource/aws_kinesis_firehose_delivery_stream: Change extended_s3_configuration.processing_configuration.processors.parameters from TypeList to TypeSet as order is not significant
  • 5.32.0 BUG FIXES * resource/aws_lambda_function: Ensure lambda does not get deployed if source_code_hash does not change.
  • 5.38.0 ENHANCEMENTS * resource/aws_lb_target_group: Add load_balancer_arns attribute
  • 5.36.0 NOTES * resource/aws_media_convert_queue: The AWS Elemental MediaConvert service has been converted to use standard Regional endpoints instead of deprecated per-account endpoints
  • 5.41.0 BUG FIXES * resource/aws_network_acl_rule: Fix InvalidNetworkAclID.NotFound errors on resource Delete
  • 5.41.0 BUG FIXES * resource/aws_network_acl_rule: Prevent creation of duplicate Terraform resources
  • 5.41.0 BUG FIXES * resource/aws_ram_principal_association: Prevent creation of duplicate Terraform resources
  • 5.41.0 BUG FIXES * resource/aws_ram_resource_association: Prevent creation of duplicate Terraform resources
  • 5.45.0 NOTES * resource/aws_redshift_cluster: The logging argument is now deprecated. Use the aws_redshift_logging resource instead.
  • 5.45.0 NOTES * resource/aws_redshift_cluster: The snapshot_copy argument is now deprecated. Use the aws_redshift_snapshot_copy resource instead.
  • 5.41.0 BUG FIXES * resource/aws_route: Prevent creation of duplicate Terraform resources
  • 5.43.0 BUG FIXES * resource/aws_route53_zone: Prevent re-creation when name casing changes
  • 5.32.0 ENHANCEMENTS * resource/aws_s3_bucket: Modify resource Read to support third-party S3 API implementations. Because we cannot easily test this functionality, it is best effort and we ask for community help in testing
  • 5.32.0 BUG FIXES * resource/aws_s3_bucket_object: Remove any leading ./ from key to maintain AWS SDK for Go v1 (pre-v5.17.0) compatibility
  • 5.32.0 BUG FIXES * resource/aws_s3_object: Remove any leading ./ from key to maintain AWS SDK for Go v1 (pre-v5.17.0) compatibility
  • 5.32.0 BUG FIXES * resource/aws_s3_object_copy: Remove any leading ./ from key to maintain AWS SDK for Go v1 (pre-v5.17.0) compatibility
  • 5.33.0 BUG FIXES * resource/aws_secretsmanager_secret_version: Fix InvalidParameterException: The parameter RemoveFromVersionId can't be empty. Staging label AWSCURRENT is currently attached to version ..., so you must explicitly reference that version in RemoveFromVersionId errors when a secret is updated outside Terraform
  • 5.43.0 BUG FIXES * resource/aws_secretsmanager_secret_version: Fix to handle versions deleted out-of-band without raising an InvalidRequestException
  • 5.43.0 BUG FIXES * resource/aws_ssm_parameter: force create a new SSM parameter when data_type is updated.

@mbbush mbbush force-pushed the terraform-upgrade/v5.46.0 branch from 588a1d8 to ccb0fe8 Compare April 20, 2024 07:12
@mbbush mbbush force-pushed the terraform-upgrade/v5.46.0 branch 5 times, most recently from 7ca93f7 to 724c288 Compare April 22, 2024 14:11
haarchri and others added 6 commits April 22, 2024 15:49
Signed-off-by: Christopher Haar <[email protected]>
Signed-off-by: Christopher Haar <[email protected]>
Signed-off-by: Matt Bush <[email protected]>
Signed-off-by: Matt Bush <[email protected]>
@mbbush mbbush force-pushed the terraform-upgrade/v5.46.0 branch from 724c288 to b296213 Compare April 22, 2024 23:50
@mbbush
Copy link
Collaborator Author

mbbush commented Apr 23, 2024

/test-examples="examples/eks/v1beta1/clusterauth.yaml"

@mbbush
Copy link
Collaborator Author

mbbush commented Apr 23, 2024

/test-examples="examples/fsx/v1beta1/ontapfilesystem.yaml"

@mbbush
Copy link
Collaborator Author

mbbush commented Apr 23, 2024

/test-examples="examples/emrserverless/v1beta1/application.yaml"

@mbbush
Copy link
Collaborator Author

mbbush commented Apr 23, 2024

/test-examples="examples/secretsmanager/v1beta1/secretrotation.yaml"

@mbbush
Copy link
Collaborator Author

mbbush commented Apr 23, 2024

/test-examples="examples/cloud9/v1beta1/environmentec2.yaml"

@mbbush
Copy link
Collaborator Author

mbbush commented Apr 23, 2024

/test-examples="examples/ecs/v1beta1/cluster.yaml"

@mbbush
Copy link
Collaborator Author

mbbush commented Apr 23, 2024

/test-examples="examples/appautoscaling/v1beta1/scheduledaction.yaml"

@johnathan-sq
Copy link

Any updates on this PR? Happy to help out where I can.

@jeanduplessis
Copy link
Collaborator

@johnathan-sq we will be focusing on finalizing the TF provider upgrade next week.

@turkenf
Copy link
Collaborator

turkenf commented May 24, 2024

Superseded by #1315

@turkenf turkenf closed this May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants