Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Codegen and support for transform endpoints #783

Merged
merged 14 commits into from
Jul 6, 2020

Conversation

tyrannosaurus-becks
Copy link
Contributor

@tyrannosaurus-becks tyrannosaurus-becks commented May 28, 2020

This PR adds support for a nascent code generation functionality. The intent of code generation is to speed up the development process, and to provide more standardized (and hopefully less buggy) code. As it's used for more resources and data sources, templates and logic will likely need to be tweaked to accommodate them. The codegen package includes a README.md describing how to use it properly.

This PR also adds support for the following generated endpoints:

  • "/auth/jwt/config" as a resource Update from @catsby 2020-07-01: This duplicated the existing resource_jwt_auth and was removed in remove the jwt/config resource from the code generated files #807
  • "/transform/alphabet/{name}" as a resource
  • "/transform/decode/{role_name}" as a data source
  • "/transform/encode/{role_name}" as a data source
  • "/transform/role/{name}" as a resource
  • "/transform/template/{name}" as a resource
  • "/transform/transformation/{name}" as a resource

Since this is a large PR, it's likely there will be a large number of comments/changes on the code. For those, please either 1) PR or commit them directly into this feature branch to add them to the present PR, or 2) close this PR, add more to the branch, and re-PR.

Thank you!

Related test output
=== RUN   TestCodeFilePath
--- PASS: TestCodeFilePath (0.00s)
=== RUN   TestDocFilePath
--- PASS: TestDocFilePath (0.00s)
=== RUN   TestStripCurlyBraces
=== RUN   TestStripCurlyBraces/{test}
=== RUN   TestStripCurlyBraces/{{name}}
=== RUN   TestStripCurlyBraces/name
=== RUN   TestStripCurlyBraces/{name
--- PASS: TestStripCurlyBraces (0.00s)
    --- PASS: TestStripCurlyBraces/{test} (0.00s)
    --- PASS: TestStripCurlyBraces/{{name}} (0.00s)
    --- PASS: TestStripCurlyBraces/name (0.00s)
    --- PASS: TestStripCurlyBraces/{name (0.00s)
=== RUN   TestFormat
=== RUN   TestFormat/alphabet
=== RUN   TestFormat/{name}
=== RUN   TestFormat/{role_name}
=== RUN   TestFormat/{name}#01
=== RUN   TestFormat/{version}
=== RUN   TestFormat/unlikely
=== RUN   TestFormat/{role_name_}
=== RUN   TestFormat/{role__name}
=== RUN   TestFormat/{role_name_here}
=== RUN   TestFormat/{rOlE_nAmE}
=== RUN   TestFormat/{ROLE_NAME}
--- PASS: TestFormat (0.00s)
    --- PASS: TestFormat/alphabet (0.00s)
    --- PASS: TestFormat/{name} (0.00s)
    --- PASS: TestFormat/{role_name} (0.00s)
    --- PASS: TestFormat/{name}#01 (0.00s)
    --- PASS: TestFormat/{version} (0.00s)
    --- PASS: TestFormat/unlikely (0.00s)
    --- PASS: TestFormat/{role_name_} (0.00s)
    --- PASS: TestFormat/{role__name} (0.00s)
    --- PASS: TestFormat/{role_name_here} (0.00s)
    --- PASS: TestFormat/{rOlE_nAmE} (0.00s)
    --- PASS: TestFormat/{ROLE_NAME} (0.00s)
=== RUN   TestValidate
=== RUN   TestValidate/nil_inputs_error
=== RUN   TestValidate/blank_endpoints_error
=== RUN   TestValidate/blank_dirnames_error
=== RUN   TestValidate/blank_upper_case_differentiators_error
=== RUN   TestValidate/blank_lower_case_differentiators_error
=== RUN   TestValidate/valid_endpoint
=== RUN   TestValidate/bad_parameter_type
=== RUN   TestValidate/good_parameter_type
=== RUN   TestValidate/array_of_strings_param
=== RUN   TestValidate/array_of_objects_param
--- PASS: TestValidate (0.00s)
    --- PASS: TestValidate/nil_inputs_error (0.00s)
    --- PASS: TestValidate/blank_endpoints_error (0.00s)
    --- PASS: TestValidate/blank_dirnames_error (0.00s)
    --- PASS: TestValidate/blank_upper_case_differentiators_error (0.00s)
    --- PASS: TestValidate/blank_lower_case_differentiators_error (0.00s)
    --- PASS: TestValidate/valid_endpoint (0.00s)
    --- PASS: TestValidate/bad_parameter_type (0.00s)
    --- PASS: TestValidate/good_parameter_type (0.00s)
    --- PASS: TestValidate/array_of_strings_param (0.00s)
    --- PASS: TestValidate/array_of_objects_param (0.00s)
=== RUN   TestToTemplatableParam
--- PASS: TestToTemplatableParam (0.00s)
=== RUN   TestParseParameters
=== RUN   TestParseParameters//transform/role/{name}
=== RUN   TestParseParameters//transform/alphabet/{name}
--- PASS: TestParseParameters (0.00s)
    --- PASS: TestParseParameters//transform/role/{name} (0.00s)
    --- PASS: TestParseParameters//transform/alphabet/{name} (0.00s)
=== RUN   TestTemplateHandler
--- PASS: TestTemplateHandler (0.00s)
PASS
ok  	github.com/terraform-providers/terraform-provider-vault/codegen	(cached)
?   	github.com/terraform-providers/terraform-provider-vault/generated	[no test files]
=== RUN   TestDecodeBasic
--- PASS: TestDecodeBasic (0.26s)
=== RUN   TestDecodeBatch
--- PASS: TestDecodeBatch (0.30s)
PASS
ok  	github.com/terraform-providers/terraform-provider-vault/generated/datasources/transform/decode	0.580s
=== RUN   TestEncodeBasic
--- PASS: TestEncodeBasic (0.27s)
=== RUN   TestEncodeBatch
--- PASS: TestEncodeBatch (0.30s)
PASS
ok  	github.com/terraform-providers/terraform-provider-vault/generated/datasources/transform/encode	0.594s
=== RUN   TestConfig
--- PASS: TestConfig (0.79s)
PASS
ok  	github.com/terraform-providers/terraform-provider-vault/generated/resources/auth/jwt	0.813s
=== RUN   TestAlphabetName
--- PASS: TestAlphabetName (0.59s)
PASS
ok  	github.com/terraform-providers/terraform-provider-vault/generated/resources/transform/alphabet	0.608s
=== RUN   TestRoleName
--- PASS: TestRoleName (0.59s)
PASS
ok  	github.com/terraform-providers/terraform-provider-vault/generated/resources/transform/role	0.614s
=== RUN   TestTemplateName
--- PASS: TestTemplateName (0.48s)
PASS
ok  	github.com/terraform-providers/terraform-provider-vault/generated/resources/transform/template	0.690s
=== RUN   TestTransformationName
--- PASS: TestTransformationName (0.38s)
PASS
ok  	github.com/terraform-providers/terraform-provider-vault/generated/resources/transform/transformation	0.396s
=== RUN   TestParsePath
=== RUN   TestParsePath/my/transform/hello
=== RUN   TestParsePath/jwt-1914071788362821795
=== RUN   TestParsePath/accounting-transit
--- PASS: TestParsePath (0.00s)
    --- PASS: TestParsePath/my/transform/hello (0.00s)
    --- PASS: TestParsePath/jwt-1914071788362821795 (0.00s)
    --- PASS: TestParsePath/accounting-transit (0.00s)
=== RUN   TestPathParameters
=== RUN   TestPathParameters//transform/role/{name}
=== RUN   TestPathParameters//transit/sign/{name}/{urlalgorithm}
=== RUN   TestPathParameters//transit/sign/{name}/{urlalgorithm}#01
=== RUN   TestPathParameters//auth/approle/tidy/secret-id
=== RUN   TestPathParameters//sys/mfa/method/totp/{name}/admin-generate
--- PASS: TestPathParameters (0.00s)
    --- PASS: TestPathParameters//transform/role/{name} (0.00s)
    --- PASS: TestPathParameters//transit/sign/{name}/{urlalgorithm} (0.00s)
    --- PASS: TestPathParameters//transit/sign/{name}/{urlalgorithm}#01 (0.00s)
    --- PASS: TestPathParameters//auth/approle/tidy/secret-id (0.00s)
    --- PASS: TestPathParameters//sys/mfa/method/totp/{name}/admin-generate (0.00s)
PASS
ok  	github.com/terraform-providers/terraform-provider-vault/util	0.008s

tyrannosaurus-becks and others added 5 commits April 23, 2020 13:36
* move provider to separate package

* rename provider package to schema
* add make dev command

* add code for path structure

* add trailing slash to mv

* Update codegen/generate.go

Co-Authored-By: Michael Golowka <[email protected]>

* Update codegen/generate.go

Co-Authored-By: Michael Golowka <[email protected]>

* Update codegen/generate.go

Co-Authored-By: Michael Golowka <[email protected]>

* add unset template type

* resolve naming clash to fix tests

* refactor test to use testcases rather than indexes

* strip dev command because its redundant

Co-authored-by: Michael Golowka <[email protected]>
* add code generation for a resource

* Update cmd/generate/main.go

Co-authored-by: Michael Golowka <[email protected]>

* Update cmd/generate/main.go

Co-authored-by: Michael Golowka <[email protected]>

* Update cmd/generate/main.go

Co-authored-by: Michael Golowka <[email protected]>

* Update codegen/generate.go

Co-authored-by: Michael Golowka <[email protected]>

* incorporate feedback

* Update codegen/templates_test.go

Co-authored-by: Michael Golowka <[email protected]>

* Update codegen/templates_test.go

Co-authored-by: Michael Golowka <[email protected]>

* Update codegen/templates.go

Co-authored-by: Michael Golowka <[email protected]>

* Update codegen/generate_test.go

Co-authored-by: Michael Golowka <[email protected]>

* changes from feedback

* update tests

* update var name

* return err from parsing home dir

* put test in t.Run

* use errwrap

* use filepath.Dir

* more changes from feedback

* more feedback

* break parameter validation into its on method

* more feedback

* test table for Validate

* use stringbuilder

* rename file

* expand upon path description

* return err

* expand upon err

* add tests for path param func

* strip unused test

Co-authored-by: Michael Golowka <[email protected]>
* add code generation for a resource

* generate /transform/transformation/{name}

* 3. Add a generated data source (#750)

* add a generated data source

* add data source to endpoint registry

* 4. Generate docs (#754)

* generate docs

* improve example hcl

* 5. Add a /transform/alphabet/{name} resource (#755)

* add a /transform/alphabet/{name} resource

* 6. Add a /transform/encode/{role_name} data source (#756)

* add a /transform/encode/{role_name} data source

* 7. Add a resource for /transform/template/{name} (#757)

* add a resource for /transform/template/{name}

* strip TODO

* improve README

* run gen and fmt at once

* improve script output

* add tfType

* encapsulate parameter grooming

* increase test for format

* add TestEntPreCheck

* avoid pointers to templatable params

* fix os.Exits from moving code
@kalafut kalafut changed the title Feature: Codegen and support for multiple endpoints Feature: Codegen and support for transform endpoints Jun 29, 2020
@catsby catsby merged commit c4d5ee7 into master Jul 6, 2020
@catsby
Copy link
Contributor

catsby commented Jul 6, 2020

Merging this in, thanks @tyrannosaurus-becks !

catsby added a commit that referenced this pull request Jul 20, 2020
Add new resources and data sources from #783 to CHANGELOG
@jasonodonnell jasonodonnell deleted the feature-generated-transform-engine branch October 21, 2020 13:46
dandandy pushed a commit to dandandy/terraform-provider-vault that referenced this pull request Jun 17, 2021
* Move provider to separate package (hashicorp#737)

* move provider to separate package

* rename provider package to schema

* Add directory structure (hashicorp#738)

* add make dev command

* add code for path structure

* add trailing slash to mv

* Update codegen/generate.go

Co-Authored-By: Michael Golowka <[email protected]>

* Update codegen/generate.go

Co-Authored-By: Michael Golowka <[email protected]>

* Update codegen/generate.go

Co-Authored-By: Michael Golowka <[email protected]>

* add unset template type

* resolve naming clash to fix tests

* refactor test to use testcases rather than indexes

* strip dev command because its redundant

Co-authored-by: Michael Golowka <[email protected]>

* 1. Add code generation for a resource (hashicorp#746)

* add code generation for a resource

* Update cmd/generate/main.go

Co-authored-by: Michael Golowka <[email protected]>

* Update cmd/generate/main.go

Co-authored-by: Michael Golowka <[email protected]>

* Update cmd/generate/main.go

Co-authored-by: Michael Golowka <[email protected]>

* Update codegen/generate.go

Co-authored-by: Michael Golowka <[email protected]>

* incorporate feedback

* Update codegen/templates_test.go

Co-authored-by: Michael Golowka <[email protected]>

* Update codegen/templates_test.go

Co-authored-by: Michael Golowka <[email protected]>

* Update codegen/templates.go

Co-authored-by: Michael Golowka <[email protected]>

* Update codegen/generate_test.go

Co-authored-by: Michael Golowka <[email protected]>

* changes from feedback

* update tests

* update var name

* return err from parsing home dir

* put test in t.Run

* use errwrap

* use filepath.Dir

* more changes from feedback

* more feedback

* break parameter validation into its on method

* more feedback

* test table for Validate

* use stringbuilder

* rename file

* expand upon path description

* return err

* expand upon err

* add tests for path param func

* strip unused test

Co-authored-by: Michael Golowka <[email protected]>

* Remaining code generation and endpoints (hashicorp#749)

* add code generation for a resource

* generate /transform/transformation/{name}

* 3. Add a generated data source (hashicorp#750)

* add a generated data source

* add data source to endpoint registry

* 4. Generate docs (hashicorp#754)

* generate docs

* improve example hcl

* 5. Add a /transform/alphabet/{name} resource (hashicorp#755)

* add a /transform/alphabet/{name} resource

* 6. Add a /transform/encode/{role_name} data source (hashicorp#756)

* add a /transform/encode/{role_name} data source

* 7. Add a resource for /transform/template/{name} (hashicorp#757)

* add a resource for /transform/template/{name}

* strip TODO

* improve README

* run gen and fmt at once

* improve script output

* add tfType

* encapsulate parameter grooming

* increase test for format

* add TestEntPreCheck

* avoid pointers to templatable params

* fix os.Exits from moving code

* add resource that supports importing jwt config (hashicorp#780)

* Remove _name suffix

* remove the jwt/config resource from the code generated files (hashicorp#807)

* remove the jwt/config resource from the code generated files

* remove jwt/config from the registry

* update documentation to remove _name, jwt config generated docs

* update docs to remove _name and -name suffix

* update links

* update link src

* doc format update

* update docs to fix formatting

* update template docs and add template to sidebar

Co-authored-by: Michael Golowka <[email protected]>
Co-authored-by: Clint <[email protected]>
Co-authored-by: catsby <[email protected]>
dandandy pushed a commit to dandandy/terraform-provider-vault that referenced this pull request Jun 17, 2021
Add new resources and data sources from hashicorp#783 to CHANGELOG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants