Add support for AlloyDB Cluster Major Version Upgrade #11639

GauravJain21 · 2024-09-05T06:56:10Z

Add support for AlloyDB Cluster Major Version Upgrade

This PR adds support for allowing users to upgrade their AlloyDB cluster's PG major version by just updating the database_version field inside the cluster.

Release Note Template for Downstream PRs (will be copied)

alloydb: added support for upgrading `database_version` in `google_alloydb_cluster` resource

github-actions · 2024-09-05T06:56:51Z

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@zli82016, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

modular-magician · 2024-09-05T07:08:01Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 126 insertions(+), 6 deletions(-))
google-beta provider: Diff ( 2 files changed, 126 insertions(+), 6 deletions(-))

modular-magician · 2024-09-05T07:11:36Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 133 insertions(+), 11 deletions(-))
google-beta provider: Diff ( 3 files changed, 133 insertions(+), 11 deletions(-))

modular-magician · 2024-09-05T07:12:14Z

Tests analytics

Total tests: 74
Passed tests: 68
Skipped tests: 4
Affected tests: 2

Click here to see the affected service packages

alloydb

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAlloydbCluster_update

TestAccAlloydbCluster_upgrade

Get to know how VCR tests work

modular-magician · 2024-09-05T07:14:54Z

Tests analytics

Total tests: 74
Passed tests: 68
Skipped tests: 4
Affected tests: 2

Click here to see the affected service packages

alloydb

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAlloydbCluster_update

TestAccAlloydbCluster_upgrade

Get to know how VCR tests work

modular-magician · 2024-09-05T07:25:46Z

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccAlloydbCluster_update[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccAlloydbCluster_upgrade[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

modular-magician · 2024-09-05T07:27:53Z

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccAlloydbCluster_update[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccAlloydbCluster_upgrade[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

GauravJain21 · 2024-09-05T07:45:17Z

This is failing because the beta provider is trying to hit the v1beta endpoint which isn't rolled out yet. The API rollout is in progress and v1beta is supposed to be visible this week.

@zli82016 Can you please review this CL ignoring the VCR test?

mmv1/templates/terraform/pre_update/alloydb_cluster.go.erb

zli82016 · 2024-09-06T15:49:25Z

mmv1/templates/terraform/pre_update/alloydb_cluster.go.erb

+    }
+    upgradeUrl = strings.Replace(upgradeUrl, "/v1/", "/v1alpha/", 1)
+
+    upgradeClusterTimeout := 72 * time.Hour


Is the timeout 72 hour? It is too long for an operation's wait time.

Yes. Actually, AlloyDB cluster upgrade is an LRO which can take up to 3 days in worst case.

Are there any repercussions of keeping this long a wait time? Will the TF code break or will there be any issues that can come due to this?

Confirmed with the team, we have 58 hours of wait time out on this LRO.

Thanks for the information. Do you know how long will it cost on average?

There are no average numbers as we haven't done the profiling on the AlloyDB fleet.

The upgrades in our shadow testing takes around 1 to 3 hours for initial numbers.

The total time depends on the size of the database and the number of database objects.

A client side field skip_await_version_upgrade defaulted to true can be used to indicated waiting for the LRO or not.
Doc: https://googlecloudplatform.github.io/magic-modules/develop/client-side-fields/

Example: skip_await_rollout

Thanks, Zhenhua for helping me with this flag! Can you help me understand the there a problem with a long wait time?

The purpose of this flag is to allow the users to do other tasks with Terraform instead of having to wait for long time. Without the flag, the users have to wait until the update is complete and cannot do anything else.

@GauravJain21 , do you have any questions about this flag? Thanks.

zli82016 · 2024-09-06T15:53:37Z

There is a yaml lint error:

repo/mmv1/products/alloydb/Cluster.yaml
  Error: 269:116 [trailing-spaces] trailing spaces

modular-magician · 2024-09-09T06:01:52Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))
google-beta provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))

modular-magician · 2024-09-09T06:05:24Z

Tests analytics

Total tests: 74
Passed tests: 69
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

alloydb

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAlloydbCluster_upgrade

Get to know how VCR tests work

modular-magician · 2024-09-09T06:07:07Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))
google-beta provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))

modular-magician · 2024-09-09T06:10:22Z

Tests analytics

Total tests: 74
Passed tests: 69
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

alloydb

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAlloydbCluster_upgrade

Get to know how VCR tests work

modular-magician · 2024-09-09T06:19:55Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))
google-beta provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))

modular-magician · 2024-09-09T06:22:30Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccAlloydbCluster_upgrade[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

modular-magician · 2024-09-09T06:23:15Z

Tests analytics

Total tests: 74
Passed tests: 69
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

alloydb

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAlloydbCluster_upgrade

Get to know how VCR tests work

GauravJain21 · 2024-09-09T06:29:52Z

This PR will likely keep failing till the API rollout is not complete. I will update here when the API rollout is complete and we have allow listed the TF project used to run the VCR tests.

modular-magician · 2024-09-09T06:35:13Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccAlloydbCluster_upgrade[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

zli82016 · 2024-09-09T19:18:02Z

/gcbrun

zli82016 · 2024-09-09T19:25:25Z

/gcbrun

zli82016 · 2024-09-09T22:11:21Z

Do you mind removing output_upgrade.log, which caused the presubmit checks being cancelled? Thanks.

GauravJain21 · 2024-09-10T06:28:36Z

@zli82016 ohh, my bad! Removed the file from the PR.

Thanks!

modular-magician · 2024-09-10T06:39:39Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))
google-beta provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))

modular-magician · 2024-09-10T06:43:09Z

Tests analytics

Total tests: 74
Passed tests: 69
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

alloydb

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAlloydbCluster_upgrade

Get to know how VCR tests work

modular-magician · 2024-09-10T06:55:15Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccAlloydbCluster_upgrade[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

zli82016

Add a flag to skip the long LRO updating.

GauravJain21 · 2024-09-23T05:05:01Z

Add a flag to skip the long LRO updating.

Can you please help me understand what this means and why to do it?

zli82016 · 2024-09-23T19:42:48Z

Add a flag to skip the long LRO updating.

Can you please help me understand what this means and why to do it?

Please see the comment .

skip_await_version_upgrade is defaulted to true. The users can set skip_await_version_upgrade to false in the Terraform configuration. When skip_await_version_upgrade is set to true, the users don't need to wait for the long LRO to complete. When it is set to false, the users will wait for the long LRO to complete. Because of the version upgrading may take too long, the flag provides the users an option to wait or not.

c2thorn · 2024-09-24T20:34:03Z

Hi @GauravJain21

Tomorrow, the Magic Modules repository is scheduled to undergo a language migration from Ruby to Go. You can view more details about this in our announcement here: hashicorp/terraform-provider-google#19583 (or go/mm-migration-announcement if you are a Googler)

This open pull request may become incompatible due to most YAML and .erb files converting to Go-compatible files.

Our team (Magic Modules repository maintainers) has tooling to automatically convert changes to the new language, and we can prepare a new commit for this pull request that is compatible with the migration.

In order to push the new changes to your pull request, we would need to force push the commit to your fork's branch. Our tooling saves a backup branch before converting, so we could rollback or open a new pull request if needed. We would also work with you and the PR reviewer in the event additional changes are needed.

You also have the option to update the pull request yourself after the migration. You can view a preview branch and updated documentation related to the migration changes.

We will take no action until we have your explicit permission to push changes to your fork's branch used for this pull request. Let me or your reviewer know if you have any further questions!

modular-magician · 2024-09-25T21:38:58Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))
google-beta provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))

modular-magician · 2024-09-25T21:42:18Z

Tests analytics

Total tests: 74
Passed tests: 69
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

alloydb

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAlloydbCluster_upgrade

Get to know how VCR tests work

modular-magician · 2024-09-25T22:30:21Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccAlloydbCluster_upgrade[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

github-actions · 2024-09-26T09:42:54Z

@zli82016 This PR has been waiting for review for 3 weekdays. Please take a look! Use the label disable-review-reminders to disable these notifications.

zli82016

Add the field skip_await_version_upgrade

github-actions · 2024-10-10T09:42:24Z

@GauravJain21, this PR is waiting for action from you. If no action is taken, this PR will be closed in 28 days.

Please address any comments or change requests, or re-request review from a core reviewer if no action is required.

This notification can be disabled with the disable-automatic-closure label.

modular-magician · 2024-10-14T07:03:49Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))
google-beta provider: Diff ( 3 files changed, 132 insertions(+), 11 deletions(-))

modular-magician · 2024-10-14T07:08:08Z

Tests analytics

Total tests: 74
Passed tests: 69
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

alloydb

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAlloydbCluster_upgrade

Get to know how VCR tests work

modular-magician · 2024-10-14T08:12:54Z

🟢 Tests passed during RECORDING mode:
TestAccAlloydbCluster_upgrade[Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.

🟢 All tests passed!

View the build log or the debug log for each test

zli82016 · 2024-10-15T17:22:28Z

This is failing because the beta provider is trying to hit the v1beta endpoint which isn't rolled out yet. The API rollout is in progress and v1beta is supposed to be visible this week.

@zli82016 Can you please review this CL ignoring the VCR test?

@GauravJain21 , what is the status of the API rollout? Is this PR ready for review? Thanks.

github-actions · 2024-10-29T09:42:25Z

@GauravJain21, this PR is waiting for action from you. If no action is taken, this PR will be closed in 28 days.

Please address any comments or change requests, or re-request review from a core reviewer if no action is required.

This notification can be disabled with the disable-automatic-closure label.

github-actions · 2024-11-12T09:42:23Z

@GauravJain21, this PR is waiting for action from you. If no action is taken, this PR will be closed in 14 days.

Please address any comments or change requests, or re-request review from a core reviewer if no action is required.

This notification can be disabled with the disable-automatic-closure label.

github-actions · 2024-11-22T09:42:28Z

@GauravJain21, this PR is waiting for action from you. If no action is taken, this PR will be closed in 2 weekdays.

Please address any comments or change requests, or re-request review from a core reviewer if no action is required.

This notification can be disabled with the disable-automatic-closure label.

github-actions · 2024-11-26T09:43:04Z

@GauravJain21, this PR is being closed due to inactivity.

github-actions bot requested a review from zli82016 September 5, 2024 06:56

modular-magician added the service/alloydb label Sep 5, 2024

zli82016 reviewed Sep 6, 2024

View reviewed changes

github-actions bot requested a review from zli82016 September 9, 2024 05:50

zli82016 requested changes Sep 16, 2024

View reviewed changes

github-actions bot requested a review from zli82016 September 23, 2024 05:05

c2thorn added the not-go-migrated label Sep 24, 2024

c2thorn assigned zli82016 Sep 25, 2024

Add support for AlloyDB Cluster Major Version Upgrade

36c4829

zli82016 force-pushed the main branch from c4a7844 to 36c4829 Compare September 25, 2024 21:28

c2thorn removed the not-go-migrated label Sep 25, 2024

zli82016 requested changes Sep 26, 2024

View reviewed changes

Merge branch 'GoogleCloudPlatform:main' into main

fb1d255

github-actions bot requested a review from zli82016 October 14, 2024 06:51

github-actions bot closed this Nov 26, 2024

Add support for AlloyDB Cluster Major Version Upgrade #11639

Add support for AlloyDB Cluster Major Version Upgrade #11639

Conversation

GauravJain21 commented Sep 5, 2024 • edited by zli82016 Loading

github-actions bot commented Sep 5, 2024

modular-magician commented Sep 5, 2024

Diff report

modular-magician commented Sep 5, 2024

Diff report

modular-magician commented Sep 5, 2024

Tests analytics

Action taken

modular-magician commented Sep 5, 2024

Tests analytics

Action taken

modular-magician commented Sep 5, 2024

modular-magician commented Sep 5, 2024

GauravJain21 commented Sep 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zli82016 Sep 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zli82016 commented Sep 6, 2024

modular-magician commented Sep 9, 2024

Diff report

modular-magician commented Sep 9, 2024

Tests analytics

Action taken

modular-magician commented Sep 9, 2024

Diff report

modular-magician commented Sep 9, 2024

Tests analytics

Action taken

modular-magician commented Sep 9, 2024

Diff report

modular-magician commented Sep 9, 2024

modular-magician commented Sep 9, 2024

Tests analytics

Action taken

GauravJain21 commented Sep 9, 2024

modular-magician commented Sep 9, 2024

zli82016 commented Sep 9, 2024

zli82016 commented Sep 9, 2024

zli82016 commented Sep 9, 2024

GauravJain21 commented Sep 10, 2024

modular-magician commented Sep 10, 2024

Diff report

modular-magician commented Sep 10, 2024

Tests analytics

Action taken

modular-magician commented Sep 10, 2024

zli82016 left a comment

Choose a reason for hiding this comment

GauravJain21 commented Sep 23, 2024

zli82016 commented Sep 23, 2024

c2thorn commented Sep 24, 2024

modular-magician commented Sep 25, 2024

Diff report

modular-magician commented Sep 25, 2024

Tests analytics

Action taken

modular-magician commented Sep 25, 2024

github-actions bot commented Sep 26, 2024

zli82016 left a comment

Choose a reason for hiding this comment

github-actions bot commented Oct 10, 2024

modular-magician commented Oct 14, 2024

Diff report

modular-magician commented Oct 14, 2024

Tests analytics

Action taken

modular-magician commented Oct 14, 2024

zli82016 commented Oct 15, 2024

github-actions bot commented Oct 29, 2024

GauravJain21 commented Sep 5, 2024 •

edited by zli82016

Loading

zli82016 Sep 16, 2024 •

edited

Loading