CDPCP-9951 Clone swagger files locally (#29)

I added a clone-swaggers Makefile target to copy the swagger files from the upstream repo and added
them to this git repo. We can now reflect upstream changes to the local copies of these files (by
sending a PR against the swaggeers) before waiting on the public API release which can take 2-3
weeks. When the API changes are released upstream, changing the target API release version in
Makefile and running:
```
  make clone-swaggers
```
will just override the existing local copies with the upstream files so that we can keep the
swaggers in sync and will not diverge much.

Added more details in the README.

Co-authored-by: Enis Soztutar <[email protected]>

cdp-sdk-go/Makefile

@@ -14,13 +14,6 @@ SWAGGER_REPO_RAW=https://raw.githubusercontent.com/cloudera/cdp-dev-docs/
 API_DEFINITION_TAG ?= cdp-api-0.9.88
 SWAGGER=$(SWAGGER_REPO_RAW)/$(API_DEFINITION_TAG)/api-docs-beta/swagger
 
-IAM_SWAGGER_YAML=$(SWAGGER)/iam.yaml
-ENVIRONMENTS_SWAGGER_YAML=$(SWAGGER)/environments.yaml
-DATALAKE_SWAGGER_YAML=$(SWAGGER)/datalake.yaml
-DATAHUB_SWAGGER_YAML=$(SWAGGER)/datahub.yaml
-ML_SWAGGER_YAML=$(SWAGGER)/ml.yaml
-DW_SWAGGER_YAML=$(SWAGGER)/dw.yaml
-
 all: check-go test build
 
 check-go:
@@ -53,30 +46,30 @@ vet:
 generate:
 	go generate ./common/... ./cdp/...
 
-mkdirs:
-	mkdir -p gen/iam gen/environments gen/datalake gen/datahub gen/ml gen/dw
+mkdir-%:
+	mkdir -p gen/$*
 
 # TODO: Manage this via go mod
 install-go-swagger:
 	go get github.com/go-swagger/go-swagger/cmd/swagger
 
-# TODO: should run this as a part of main build
-swagger-gen: mkdirs install-go-swagger swagger-gen-iam swagger-gen-environments swagger-gen-datalake swagger-gen-datahub swagger-gen-ml swagger-gen-dw
-
-swagger-gen-iam: mkdirs
-	go run github.com/go-swagger/go-swagger/cmd/swagger generate client -f $(IAM_SWAGGER_YAML) -A iam -t gen/iam/
-
-swagger-gen-environments: mkdirs
-	go run github.com/go-swagger/go-swagger/cmd/swagger generate client -f $(ENVIRONMENTS_SWAGGER_YAML) -A environments -t gen/environments/
-
-swagger-gen-datalake: mkdirs
-	go run github.com/go-swagger/go-swagger/cmd/swagger generate client -f $(DATALAKE_SWAGGER_YAML) -A datalake -t gen/datalake/
-
-swagger-gen-datahub: mkdirs
-	go run github.com/go-swagger/go-swagger/cmd/swagger generate client -f $(DATAHUB_SWAGGER_YAML) -A datahub -t  gen/datahub/
-
-swagger-gen-ml: mkdirs
-	go run github.com/go-swagger/go-swagger/cmd/swagger generate client -f $(ML_SWAGGER_YAML) -A ml -t  gen/ml/
-
-swagger-gen-dw: mkdirs
-	go run github.com/go-swagger/go-swagger/cmd/swagger generate client -f $(DW_SWAGGER_YAML) -A dw -t  gen/dw/
+swagger-gen: install-go-swagger
+swagger-gen: swagger-gen-iam
+swagger-gen: swagger-gen-environments
+swagger-gen: swagger-gen-datalake
+swagger-gen: swagger-gen-datahub
+swagger-gen: swagger-gen-ml
+swagger-gen: swagger-gen-dw
+
+clone-swaggers: clone-swagger-iam
+clone-swaggers: clone-swagger-environments
+clone-swaggers: clone-swagger-datalake
+clone-swaggers: clone-swagger-datahub
+clone-swaggers: clone-swagger-ml
+clone-swaggers: clone-swagger-dw
+
+clone-swagger-%:
+	wget $(SWAGGER)/$*.yaml -O resources/swagger/$*.yaml
+
+swagger-gen-%: mkdir-%
+	go run github.com/go-swagger/go-swagger/cmd/swagger generate client -f resources/swagger/$*.yaml -A $* -t gen/$*/

cdp-sdk-go/README.md

@@ -19,19 +19,38 @@ API release. These tags correspond to CDP API release versions. The tag can be f
 ```
 API_DEFINITION_TAG ?= cdp-api-0.9.88
 ```
+Then run:
+```
+    make clone-swaggers
+```
+which will copy the swagger files from the upstream repo to this repo.
 
-**Note:** Only publicly-released APIs can be consumed from the Terraform provider. We use the Beta API swaggers.
+We have a local copy of the swagger files from the upstream repos since we sometimes need the changes to the API that
+are yet to be released publicly. The local copies of the swagger files can be found at
+[./resources/swagger](./resources/swagger). A PR can be sent to reflect the API changes to these swagger files for
+upcoming API changes that will be released soon, but since we have a local copy, we can generate the swagger code before
+waiting for the API release. Once the API changes are released publicly, simply update the local copy of the swagger files
+by running:
+```
+    make clone-swaggers
+```
+which will override any local changes from the upstream repo.
 
 To upgrade the version, simply get a new git tag from https://github.com/cloudera/cdp-dev-docs/, and update it in the
 Makefile. Afterwards, run:
 ```
-  make swagger-gen
+  make clone-swaggers swagger-gen
 ```
 to generate for the new code. Then send a PR for the new changes to check in the code.
 
+**Note:** Only publicly-released (or soon-to-be-released) APIs can be consumed from the Terraform provider. We use the
+Beta API swaggers.
+
 # Supporting a new Service
-Adding a new CDP service (CML, CDW, CDE, etc) should be trivial and should follow the existing model from `Makefile`:
+Adding a new CDP service (CML, CDW, CDE, etc) should be trivial and should follow the existing model from `Makefile`.
+Add two new directives like this:
 ```
-swagger-gen-<newservice>: mkdirs
-	go run github.com/go-swagger/go-swagger/cmd/swagger generate client -f $(NEWSERVICE_SWAGGER_YAML) -A dw -t  gen/<newservice>/
+swagger-gen: swagger-gen-<service-name>
+
+clone-swaggers: clone-swagger-<service-name>
 ```