Updategenproviders (GoogleCloudPlatform#12350)

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

docs/content/_index.md

@@ -1,6 +1,6 @@
 ---
 title: "Overview"
-weight: 20
+weight: 10
 aliases:
   - /docs/how-to/types-of-resources
   - /how-to/types-of-resources

docs/content/contribution-process.md

@@ -1,6 +1,6 @@
 ---
 title: "Contribution process"
-weight: 9
+weight: 11
 aliases:
   - /docs/getting-started/contributing
   - /getting-started/contributing
@@ -22,14 +22,15 @@ This page explains how you can contribute code and documentation to the	`magic-m
 
 ## Contribute code
 
+1. [Set up your development environment]({{< ref "/develop/set-up-dev-environment" >}})
 1. [Create a new branch for your change](https://docs.github.com/en/get-started/quickstart/github-flow#create-a-branch)
 1. Make the code change. For example:
    - [Add or modify a resource]({{< ref "/develop/resource" >}})
    - [Add resource tests]({{< ref "/develop/test/test" >}})
    - [Add a datasource]({{< ref "/develop/add-handwritten-datasource" >}})
    - [Promote to GA]({{< ref "/develop/promote-to-ga" >}})
    - [Make a breaking change]({{< ref "/develop/breaking-changes/make-a-breaking-change" >}})
-1. [Generate the providers]({{< ref "/get-started/generate-providers" >}}) that include your change.
+1. [Generate the providers]({{< ref "/develop/generate-providers" >}}) that include your change.
 1. [Run provider tests locally]({{< ref "/develop/test/run-tests" >}}) that are relevant to the change you made
 1. [Create a pull request (PR)]({{< ref "/contribute/create-pr" >}})
 1. Make changes in response to [code review]({{< ref "/contribute/create-pr#code-review" >}})

docs/content/develop/add-handwritten-datasource.md

@@ -1,7 +1,7 @@
 ---
 title: "Add a datasource"
 summary: "Datasources are like terraform resources except they don't *create* anything."
-weight: 40
+weight: 14
 aliases:
   - /docs/how-to/add-handwritten-datasource
   - /how-to/add-handwritten-datasource
@@ -52,5 +52,5 @@ library, or the raw HTTP client used in MMV1 through `SendRequest`.
 1. Open the data source documentation in [`magic-modules/third_party/terraform/website/docs/d/`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/third_party/terraform/website/docs/d) using an editor of your choice.
    - The name of the file is the name of the data source without a `google_` prefix. For example, for `google_compute_instance`, the file is called `compute_instance.html.markdown`
 2. Modify the documentation as needed according to [Handwritten documentation style guide]({{< ref "/develop/handwritten-docs-style-guide" >}}).
-4. [Generate the providers]({{< ref "/get-started/generate-providers.md" >}})
+4. [Generate the providers]({{< ref "/develop/generate-providers" >}})
 5. Copy and paste the generated documentation into the Hashicorp Registry's [Doc Preview Tool](https://registry.terraform.io/tools/doc-preview) to see how it is rendered.

docs/content/develop/custom-code.md

@@ -1,6 +1,6 @@
 ---
 title: "Add custom resource code"
-weight: 39
+weight: 15
 ---
 
 # Add custom resource code
@@ -17,7 +17,7 @@ custom_code:
 
 By convention, the template files are stored in a directory matching the type of custom code, and the name of the file includes the resource (and, if relevant, field) impacted by the custom code. Like handwritten resource and test code, custom code is written as go templates which render go code.
 
-When in doubt about the behavior of custom code, write the custom code, [generate the providers]({{< ref "/get-started/generate-providers" >}}), and inspect what changed in the providers using `git diff`.
+When in doubt about the behavior of custom code, write the custom code, [generate the providers]({{< ref "/develop/generate-providers" >}}), and inspect what changed in the providers using `git diff`.
 
 The following sections describe types of custom code in more detail.
 

docs/content/develop/diffs.md

@@ -1,6 +1,6 @@
 ---
 title: "Fix diffs"
-weight: 60
+weight: 18
 aliases:
   - /develop/permadiff
 ---

docs/content/develop/generate-providers.md

@@ -0,0 +1,96 @@
+---
+title: "Generate the providers"
+weight: 19
+aliases:
+  - /docs/getting-started/setup
+  - /getting-started/setup
+  - /docs/getting-started/generate-providers
+  - /getting-started/generate-providers
+  - /get-started/generate-providers
+---
+
+# Generate `google` and `google-beta` providers
+
+After making a change to the Terraform providers for Google Cloud, you must
+integrate your changes with the providers. This page explains how to generate
+provider changes to the `google` and `google-beta` Terraform providers.
+
+## Before you begin
+
+1. [Set up your development environment]({{< ref "/develop/set-up-dev-environment" >}}).
+1. Update `magic-modules` as needed. These updates could be any of the following changes:
+  + [Adding or modifying a resource]({{< ref "/develop/resource" >}}).
+  + [Adding a datasource]({{< ref "/develop/add-handwritten-datasource" >}}).
+  + [Adding custom resource code]({{< ref "/develop/custom-code" >}}).
+  + [Promoting a resource to GA]({{< ref "/develop/promote-to-ga" >}}).
+
+## Generate a provider change
+
+1. Clone the `google` and `google-beta` provider repositories with the following commands:
+
+   ```bash
+   git clone https://github.com/hashicorp/terraform-provider-google.git $GOPATH/src/github.com/hashicorp/terraform-provider-google
+   git clone https://github.com/hashicorp/terraform-provider-google-beta.git $GOPATH/src/github.com/hashicorp/terraform-provider-google-beta
+   ```
+1. Generate changes for the `google` provider:
+   ```bash
+   make provider VERSION=ga OUTPUT_PATH="$GOPATH/src/github.com/hashicorp/terraform-provider-google" PRODUCT=[PRODUCT_NAME]
+   ```
+    Where `[PRODUCT_NAME]` is one of the folder names in
+    https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/products.
+
+    For example, if your product is `bigqueryanalyticshub`, the command would be
+    the following:
+
+     ```bash
+     make provider VERSION=ga OUTPUT_PATH="$GOPATH/src/github.com/hashicorp/terraform-provider-google" PRODUCT=bigqueryanalyticshub
+     ```
+
+1. Generate changes for the `google-beta` provider:
+   ```bash
+   make provider VERSION=beta OUTPUT_PATH="$GOPATH/src/github.com/hashicorp/terraform-provider-google-beta" PRODUCT=[PRODUCT_NAME]
+   ```
+
+    Where `[PRODUCT_NAME]` is one of the folder names in https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/products.
+
+    For example, if your product name is `bigqueryanalyticshub`, the command would be the following:
+
+     ```bash
+     make provider VERSION=beta OUTPUT_PATH="$GOPATH/src/github.com/hashicorp/terraform-provider-google-beta" PRODUCT=bigqueryanalyticshub
+     ```
+
+1. Confirm that the expected changes were generated:
+   ```bash
+   cd $GOPATH/src/github.com/hashicorp/terraform-provider-google
+   git diff -U0
+   cd $GOPATH/src/github.com/hashicorp/terraform-provider-google-beta
+   git diff -U0
+   ```
+
+
+   {{< hint info >}}
+   **Note**: There may be additional changes present due to specifying a
+   `PRODUCT=` value or due to the `magic-modules` repository being out of sync
+   with the provider repositories.
+   {{< /hint >}}
+
+
+## Troubleshoot
+
+### Too many open files {#too-many-open-files}
+
+If you are getting “Too many open files” ulimit needs to be raised.
+
+{{< tabs "ulimit" >}}
+{{< tab "Mac OS" >}}
+```bash
+ulimit -n 8192
+```
+{{< /tab >}}
+{{< /tabs >}}
+
+## What's next
+
++ [Learn how to add resource tests]({{< ref "/develop/test/test" >}})
++ [Learn how to run tests]({{< ref "/develop/test/run-tests" >}})
++ [Learn about `make` commands]({{< ref "/reference/make-commands" >}})

docs/content/develop/promote-to-ga.md

@@ -1,6 +1,6 @@
 ---
 title: "Promote to GA"
-weight: 50
+weight: 16
 ---
 
 # Promote from beta to GA
@@ -13,7 +13,7 @@ For more information about types of resources and the generation process overall
 
 ## Before you begin
 
-1. Complete the [Generate the providers]({{< ref "/get-started/generate-providers" >}}) quickstart to set up your environment and your Google Cloud project.
+1. Complete the steps in [Set up your development environment]({{< ref "/develop/set-up-dev-environment" >}}) to set up your environment and your Google Cloud project.
 2. Ensure that your `magic-modules`, `terraform-provider-google`, and `terraform-provider-google-beta` repositories are up to date.
    ```
    cd ~/magic-modules

docs/content/develop/resource.md

@@ -1,6 +1,6 @@
 ---
 title: "Add or modify a resource"
-weight: 30
+weight: 13
 aliases:
   - /docs/how-to/add-mmv1-resource
   - /how-to/add-mmv1-resource
@@ -31,7 +31,7 @@ For more information about types of resources and the generation process overall
 
 ## Before you begin
 
-1. Complete the steps in [Generate the providers]({{< ref "/get-started/generate-providers" >}}) to set up your environment and your Google Cloud project.
+1. Complete the steps in [Set up your development environment]({{< ref "/develop/set-up-dev-environment" >}}) to set up your environment and your Google Cloud project.
 2. Ensure that your `magic-modules`, `terraform-provider-google`, and `terraform-provider-google-beta` repositories are up to date.
    ```
    cd ~/magic-modules
@@ -151,7 +151,7 @@ For more information about types of resources and the generation process overall
 > **Warning:** Handwritten resources are more difficult to develop and maintain. New handwritten resources will only be accepted if implementing the resource in MMv1 would require entirely overriding two or more CRUD methods.
 
 1. Add the resource in MMv1.
-2. [Generate the beta provider]({{< ref "/get-started/generate-providers.md" >}})
+2. [Generate the beta provider]({{< ref "/develop/generate-providers.md" >}})
 3. From the beta provider, copy the files generated for the resource to the following locations:
    - Resource: Copy to the appropriate service folder inside [`magic-modules/mmv1/third_party/terraform/services`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/third_party/terraform/services)
    - Documentation: [`magic-modules/mmv1/third_party/terraform/website/docs/r`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/third_party/terraform/website/docs/r)
@@ -420,7 +420,7 @@ iam_policy:
 
 ### Convert to handwritten (not usually necessary)
 
-1. [Generate the beta provider]({{< ref "/get-started/generate-providers.md" >}})
+1. [Generate the beta provider]({{< ref "/develop/generate-providers.md" >}})
 2. From the beta provider, copy the files generated for the IAM resources to the following locations:
    - Resource: Copy to the appropriate service folder inside [`magic-modules/mmv1/third_party/terraform/services`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/third_party/terraform/services)
    - Documentation: [`magic-modules/mmv1/third_party/terraform/website/docs/r`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/third_party/terraform/website/docs/r)
@@ -442,7 +442,7 @@ iam_policy:
 {{< tab "MMv1" >}}
 Documentation is autogenerated based on the resource and field configurations. To preview the documentation:
 
-1. [Generate the providers]({{< ref "/get-started/generate-providers.md" >}})
+1. [Generate the providers]({{< ref "/develop/generate-providers.md" >}})
 2. Copy and paste the generated documentation into the Hashicorp Registry's [Doc Preview Tool](https://registry.terraform.io/tools/doc-preview) to see how it is rendered.
 {{< /tab >}}
 {{< tab "Handwritten" >}}
@@ -451,7 +451,7 @@ Documentation is autogenerated based on the resource and field configurations. T
 1. Open the resource documentation in [`magic-modules/third_party/terraform/website/docs/r/`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/third_party/terraform/website/docs/r) using an editor of your choice.
    - The name of the file is the name of the resource without a `google_` prefix. For example, for `google_compute_instance`, the file is called `compute_instance.html.markdown`
 2. Modify the documentation as needed according to [Handwritten documentation style guide]({{< ref "/develop/handwritten-docs-style-guide" >}}).
-3. [Generate the providers]({{< ref "/get-started/generate-providers.md" >}})
+3. [Generate the providers]({{< ref "/develop/generate-providers.md" >}})
 4. Copy and paste the generated documentation into the Hashicorp Registry's [Doc Preview Tool](https://registry.terraform.io/tools/doc-preview) to see how it is rendered.
 {{< /tab >}}
 {{< /tabs >}}

docs/content/develop/set-up-dev-environment.md

@@ -0,0 +1,77 @@
+---
+title: "Set up your development environment"
+weight: 12
+---
+
+# Set up your development environment
+
+Before you start adding or updating a Terraform resource using
+[magic-modules]({{< ref "/" >}}), you must first set up your environment by
+installing the necessary tools. This page explains the steps for setting up your
+development environment.
+
+1. [Install the gcloud CLI.](https://cloud.google.com/sdk/docs/install)
+1. In the Google Cloud console, on the project selector page, select or
+   [create a Google Cloud project](https://cloud.google.com/resource-manager/docs/creating-managing-projects).
+   {{< hint info >}}
+
+   **Note:** If you don't already have a project to use for testing changes to
+   the Terraform providers, create a project instead of selecting an existing
+   poject. After you finish these steps, you can delete the project, removing
+   all resources associated with the project.
+
+   {{< /hint >}}
+   {{< button href="https://console.cloud.google.com/projectselector2/home/dashboard" >}}Go to project selector{{< /button >}}
+1. Make sure that billing is enabled for your Google Cloud project. Learn how to
+   [check if billing is enabled on a project](https://cloud.google.com/billing/docs/how-to/verify-billing-enabled).
+
+
+1. [Install git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+1. [Install go](https://go.dev/doc/install)
+1. Add the following values to your environment settings such as `.bashrc`:
+   ```bash
+   # Add GOPATH variable for convenience
+   export GOPATH=$(go env GOPATH)
+   # Add Go binaries to PATH
+   export PATH=$PATH:$(go env GOPATH)/bin
+   ```
+1. Install goimports
+   ```bash
+   go install golang.org/x/tools/cmd/goimports@latest
+   ```
+1. [Install terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli)
+1. Clone the `magic-modules` repository
+   ```bash
+   cd ~
+   git clone https://github.com/GoogleCloudPlatform/magic-modules.git
+   ```
+
+1. Run the following command from the root of your cloned `magic-modules` repository.
+
+   ```bash
+   cd magic-modules
+   ./scripts/doctor
+   ```
+
+   Expected output if everything is installed properly:
+
+   ```
+   Check for go in path...
+      found!
+   Check for goimports in path...
+      found!
+   Check for git in path...
+      found!
+   Check for terraform in path...
+      found!
+   Check for make in path...
+      found!
+   ```
+
+## What's next
+
++ [Learn how to add or modify a resource]({{< ref "/develop/resource" >}})
++ [Learn how to add custom resource code]({{< ref "/develop/custom-code" >}})
++ [Learn how to add a datasource]({{< ref "/develop/add-handwritten-datasource" >}})
++ [Learn how to promote a resource to GA]({{< ref "/develop/promote-to-ga" >}})
+

docs/content/develop/test/run-tests.md

@@ -15,7 +15,7 @@ aliases:
 
 ## Before you begin
 
-[Generate the modified provider(s)]({{< ref "/get-started/generate-providers" >}})
+[Generate the modified provider(s)]({{< ref "/develop/generate-providers" >}})
 
 
 1. Set up application default credentials for Terraform
@@ -243,7 +243,7 @@ Configure Terraform to use locally-built binaries for `google` and `google-beta`
 
 ### Run manual tests
 
-1. [Generate the provider(s) you want to test]({{< ref "/get-started/generate-providers" >}})
+1. [Generate the provider(s) you want to test]({{< ref "/develop/generate-providers" >}})
 2. Build the provider(s) you want to test
 
     ```bash

docs/content/develop/test/test.md

@@ -100,7 +100,7 @@ This section assumes you've used the [Add a resource]({{< ref "/develop/resource
 > **Note:** If not, you can create one now, or skip this guide and construct the test by hand. Writing tests by hand can sometimes be a better option if there is a similar test you can copy from.
 
 1. Add the test in MMv1. Repeat for all the create tests you will need.
-2. [Generate the beta provider]({{< ref "/get-started/generate-providers.md" >}}).
+2. [Generate the beta provider]({{< ref "/develop/generate-providers.md" >}}).
 3. From the beta provider, copy and paste the generated `*_generated_test.go` file into the appropriate service folder inside [`magic-modules/mmv1/third_party/terraform/services`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/third_party/terraform/services/) as a new file call `*_test.go`.
 4. Modify the tests as needed.
    - Replace all occurrences of `github.com/hashicorp/terraform-provider-google-beta/google-beta` with `github.com/hashicorp/terraform-provider-google/google`
@@ -120,7 +120,7 @@ An update test is a test that creates the target resource and then makes updates
 
 {{< tabs "update" >}}
 {{< tab "MMv1" >}}
-1. [Generate the beta provider]({{< ref "/get-started/generate-providers.md" >}}).
+1. [Generate the beta provider]({{< ref "/develop/generate-providers" >}}).
 2. From the beta provider, copy and paste the generated `*_generated_test.go` file into the appropriate service folder inside [`magic-modules/mmv1/third_party/terraform/services`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/third_party/terraform/services) as a new file call `*_test.go`.
 3. Using an editor of your choice, delete the `*DestroyProducer` function, and all but one test. The remaining test should be the "full" test, or if there is no "full" test, the "basic" test. This will be the starting point for your new update test.
 4. Modify the `TestAcc*` *test function* to support updates.
@@ -274,4 +274,4 @@ func TestSignatureAlgorithmDiffSuppress(t *testing.T) {
 
 ## What's next?
 
-- [Run your tests]({{< ref "/develop/test/run-tests.md" >}})
+- [Run your tests]({{< ref "/develop/test/run-tests" >}})

docs/content/develop/update-dependencies.md

@@ -1,6 +1,6 @@
 ---
 title: "Update dependencies"
-weight: 300
+weight: 17
 aliases:
   - /docs/update-dependencies
 ---