Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add resource Dataproc batch #11750

Merged
merged 4 commits into from
Sep 30, 2024
Merged

Conversation

zli82016
Copy link
Member

@zli82016 zli82016 commented Sep 18, 2024

fixes hashicorp/terraform-provider-google#11775

Release Note Template for Downstream PRs (will be copied)

`google_dataproc_batch`
dataproc: switched to the v1 API for `google_dataproc_autoscaling_policy` resource (beta)

@modular-magician

This comment was marked as outdated.

1 similar comment
@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

1 similar comment
@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@zli82016 zli82016 force-pushed the dataproc-batch branch 5 times, most recently from 6672fee to a7c65dc Compare September 25, 2024 23:33
@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

@modular-magician

This comment was marked as outdated.

Fix tests
Address comments
Rename resource_dataproc_batch_test.go to resource_dataproc_batch_test.go.erb
Fixed package name & switched to func
Fixed imports & test file name
Fixed function names
Fix PR review comments
Fixed yaml lint errors
Add a test for SparkR batch
Resolve version and properties permadiff for batch resource
Add more fields to test examples
Resolve peripheral_configs permadiff.
Making fields exatly_one_off/
Bootstrap new subnet with firewall for Dataproc Batches
Added utility method to bootstrap subnet for Dataproc batches
Added utility method to bootstrap subnet for Dataproc batches
Updated async block for batch resource.
Terraform Support for Dataproc Batches.
@modular-magician

This comment was marked as outdated.

@melinath
Copy link
Member

If they can get it set up before merge that would definitely be preferable, but I think we can go ahead otherwise if the overall resource works. There's just a higher risk of bugs.

@modular-magician

This comment was marked as outdated.

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 8 files changed, 3463 insertions(+), 4 deletions(-))
google-beta provider: Diff ( 10 files changed, 3466 insertions(+), 7 deletions(-))
terraform-google-conversion: Diff ( 2 files changed, 640 insertions(+), 2 deletions(-))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_dataproc_batch (5 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_dataproc_batch" "primary" {
  runtime_config {
    container_image = # value needed
  }
  spark_batch {
    archive_uris      = # value needed
    file_uris         = # value needed
    main_jar_file_uri = # value needed
  }
  spark_r_batch {
    archive_uris = # value needed
    file_uris    = # value needed
  }
}

@modular-magician

This comment was marked as outdated.

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 4105
Passed tests: 3691
Skipped tests: 413
Affected tests: 1

Click here to see the affected service packages

All service packages are affected

#### Non-exercised tests

Tests were added that are skipped in VCR:

  • TestCloudDataprocBatchRuntimeConfigVersionDiffSuppress

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataprocCluster_withAutoscalingPolicy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccDataprocCluster_withAutoscalingPolicy[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

Copy link
Member

@melinath melinath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - Adding the tests would be great if it works out before the next cut, otherwise it's probably fine to go ahead with this.

@melinath
Copy link
Member

melinath commented Sep 27, 2024

Actually it would be great to merge this Monday if possible (at the latest)

@zli82016
Copy link
Member Author

I am going to merge this PR now and add the missing test later.

@zli82016 zli82016 merged commit a0b7099 into GoogleCloudPlatform:main Sep 30, 2024
13 of 14 checks passed
@zli82016 zli82016 deleted the dataproc-batch branch September 30, 2024 17:11
anavada pushed a commit to anavada/magic-modules that referenced this pull request Sep 30, 2024
karolgorc pushed a commit to karolgorc/magic-modules that referenced this pull request Oct 4, 2024
niharika-98 pushed a commit to niharika-98/magic-modules that referenced this pull request Oct 7, 2024
trodge pushed a commit to trodge/magic-modules that referenced this pull request Oct 10, 2024
karolgorc pushed a commit to karolgorc/magic-modules that referenced this pull request Oct 11, 2024
BBBmau pushed a commit to BBBmau/magic-modules that referenced this pull request Oct 23, 2024
BBBmau pushed a commit to BBBmau/magic-modules that referenced this pull request Oct 24, 2024
Philip-Jonany pushed a commit to Philip-Jonany/magic-modules that referenced this pull request Nov 4, 2024
BBBmau pushed a commit to BBBmau/magic-modules that referenced this pull request Nov 5, 2024
akshat-jindal-nit pushed a commit to akshat-jindal-nit/magic-modules that referenced this pull request Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dataproc Serverless: google_dataproc_batch
3 participants