Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Introduce assertions generators part2 #2956

Merged
merged 97 commits into from
Jul 25, 2024
Merged
Changes from 1 commit
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
3e158bc
Add PoC of snowflake parameters assertions
sfc-gh-asawicki Jul 19, 2024
751d730
Add all user parameters assertions
sfc-gh-asawicki Jul 19, 2024
79cf4a0
Use assertions in user integration tests
sfc-gh-asawicki Jul 19, 2024
86b6890
Fix the tests
sfc-gh-asawicki Jul 19, 2024
262f776
Add generic checks
sfc-gh-asawicki Jul 19, 2024
077a861
Add generic default checks
sfc-gh-asawicki Jul 19, 2024
47bb554
Add TODOs
sfc-gh-asawicki Jul 19, 2024
74412d2
Add TODOs
sfc-gh-asawicki Jul 19, 2024
ed54823
Add default checks for user parameters
sfc-gh-asawicki Jul 19, 2024
52f99e9
Add level assertions
sfc-gh-asawicki Jul 19, 2024
ff89925
Move gen utils to separate gencommons
sfc-gh-asawicki Jul 19, 2024
b4a8c71
Move struct details extractor to separate gencommons
sfc-gh-asawicki Jul 19, 2024
cbb2026
Extract write to file helper func
sfc-gh-asawicki Jul 19, 2024
50fbd3a
Rename to struct details
sfc-gh-asawicki Jul 19, 2024
b4cf5ef
Extract a common func for template execution for all objects
sfc-gh-asawicki Jul 19, 2024
2b8f725
Move common func to generate and print
sfc-gh-asawicki Jul 19, 2024
93fe57c
Prepare a generator skeleton (WIP)
sfc-gh-asawicki Jul 19, 2024
e2143e3
Rewrite show output generator using the generator skeleton
sfc-gh-asawicki Jul 19, 2024
0d3f120
Fix the gen
sfc-gh-asawicki Jul 19, 2024
f5ae021
Move helper funcs to the generator
sfc-gh-asawicki Jul 19, 2024
97e861d
Add flags to the generator skeleton
sfc-gh-asawicki Jul 19, 2024
f9fd204
Allow passing the arguments to the generator
sfc-gh-asawicki Jul 19, 2024
088b5c4
Extract common template functions
sfc-gh-asawicki Jul 19, 2024
da74f1e
Extract mapper commons
sfc-gh-asawicki Jul 19, 2024
423fd33
Take template helper func by name
sfc-gh-asawicki Jul 19, 2024
a3702a6
Remove already done TODOs
sfc-gh-asawicki Jul 19, 2024
a3e1004
Allow using multiple flags (hacky)
sfc-gh-asawicki Jul 19, 2024
6c10a18
Add possibility to filter objects
sfc-gh-asawicki Jul 19, 2024
43fd094
Add example filter
sfc-gh-asawicki Jul 19, 2024
0bae099
Prepare snowflake object assertions generator (WIP)
sfc-gh-asawicki Jul 21, 2024
f8cb936
Add add assertion func
sfc-gh-asawicki Jul 21, 2024
4e50d80
Add assertion through method
sfc-gh-asawicki Jul 21, 2024
a3a0525
Add additional standard imports (quick and dirty)
sfc-gh-asawicki Jul 21, 2024
67b13cc
Extract list of objects
sfc-gh-asawicki Jul 21, 2024
fc9d1ef
Move generate snowflake object to the destination package
sfc-gh-asawicki Jul 21, 2024
0c7ce27
Reorganize
sfc-gh-asawicki Jul 21, 2024
0968d59
Prepare snowflake object parameters generation
sfc-gh-asawicki Jul 21, 2024
c01ec37
Rename templates
sfc-gh-asawicki Jul 21, 2024
94e58f4
Continue objects parameters assertions generation
sfc-gh-asawicki Jul 21, 2024
1ab5384
Add generic checks generation
sfc-gh-asawicki Jul 21, 2024
c0ad76c
Expose snowflake parameters assertions and helpers
sfc-gh-asawicki Jul 21, 2024
22af4ff
Add aggregated generic checks
sfc-gh-asawicki Jul 21, 2024
66b1831
Add value checks
sfc-gh-asawicki Jul 21, 2024
043e9bf
Add all other specific object parameter checks
sfc-gh-asawicki Jul 21, 2024
d450052
Clean up templates a bit
sfc-gh-asawicki Jul 21, 2024
8e2d238
Add all user parameters
sfc-gh-asawicki Jul 21, 2024
839b2db
Change names
sfc-gh-asawicki Jul 21, 2024
0d768e9
Update comment
sfc-gh-asawicki Jul 22, 2024
351797f
Generate snowflake object assertion for warehouse and user
sfc-gh-asawicki Jul 22, 2024
202d92a
Fix the package and use generated user assertions
sfc-gh-asawicki Jul 22, 2024
341001d
Add a quick workaround for user param
sfc-gh-asawicki Jul 22, 2024
38b77ed
Remove manually created user parameters
sfc-gh-asawicki Jul 22, 2024
f3c55f2
Remove manually created warehouse object assertions
sfc-gh-asawicki Jul 22, 2024
9646487
Update readme
sfc-gh-asawicki Jul 22, 2024
8c81ebb
Add README to the generator commons
sfc-gh-asawicki Jul 22, 2024
7f7d4fa
Run make pre-push
sfc-gh-asawicki Jul 22, 2024
bd561e9
Add explicit field names
sfc-gh-asawicki Jul 22, 2024
edfddf2
Use the generated assertions in warehouse acceptance tests
sfc-gh-asawicki Jul 22, 2024
3015e4b
Update readme and TODOs
sfc-gh-asawicki Jul 22, 2024
b358270
Fix the tests
sfc-gh-asawicki Jul 23, 2024
cef6917
Run acceptance tests regardless of the integration tests result
sfc-gh-asawicki Jul 23, 2024
849c199
assertions generators part 2 starts here
sfc-gh-asawicki Jul 22, 2024
fe4e849
Add skeleton for resource assertions generation
sfc-gh-asawicki Jul 22, 2024
0005b6e
Add definitions generation (WIP)
sfc-gh-asawicki Jul 22, 2024
438d197
Generate definitions
sfc-gh-asawicki Jul 22, 2024
0ff1ab1
Prepare assertions generation
sfc-gh-asawicki Jul 22, 2024
57846fb
Add AddAssertion helper method
sfc-gh-asawicki Jul 22, 2024
1f3df44
Generate resource assertions
sfc-gh-asawicki Jul 22, 2024
09b851a
Use generated warehouse resource assertions in tests
sfc-gh-asawicki Jul 22, 2024
e05025e
Set up resource parameters assertions generation
sfc-gh-asawicki Jul 23, 2024
54aa31d
Generate the checks for resource parameters assertions
sfc-gh-asawicki Jul 23, 2024
129ceb8
Replace warehouse resource parameters assertions with the generated ones
sfc-gh-asawicki Jul 23, 2024
7f0bd31
Prepare resource show output assertions
sfc-gh-asawicki Jul 23, 2024
ff74df7
Aggregate all cleanups and generators
sfc-gh-asawicki Jul 23, 2024
e20d3ed
Generate definition for show output assertions
sfc-gh-asawicki Jul 23, 2024
aedc7b8
Add assertions generation for resource show output
sfc-gh-asawicki Jul 23, 2024
3bc8f1e
Generate assertions for resource show output
sfc-gh-asawicki Jul 23, 2024
1e5bcc7
Use generated assertions for resource show output
sfc-gh-asawicki Jul 23, 2024
2b2b552
Fix the generated content message
sfc-gh-asawicki Jul 23, 2024
c6eb309
Prepare config model builder generator
sfc-gh-asawicki Jul 23, 2024
58a10aa
Prepare config model builder constructors (WIP)
sfc-gh-asawicki Jul 23, 2024
2c3879d
Finish config model builder constructors
sfc-gh-asawicki Jul 23, 2024
bb86356
Add builder generation
sfc-gh-asawicki Jul 23, 2024
e6187c0
Handle basic types
sfc-gh-asawicki Jul 23, 2024
a2cea72
Use generated warehouse model
sfc-gh-asawicki Jul 23, 2024
8f2b5d9
Remove unnecessary sdk import
sfc-gh-asawicki Jul 23, 2024
6e2b034
Fix after review
sfc-gh-asawicki Jul 24, 2024
962dbdd
Merge branch 'main' into assertions-generators-part1
sfc-gh-asawicki Jul 24, 2024
e7bdf8f
Merge branch 'assertions-generators-part1' into assertions-generators…
sfc-gh-asawicki Jul 24, 2024
8ddd8fc
Fix after merge
sfc-gh-asawicki Jul 24, 2024
7b3163a
Add tabulators to template files
sfc-gh-asawicki Jul 24, 2024
2fd1cd5
Add example extension with reasoning
sfc-gh-asawicki Jul 24, 2024
34e4872
Update readme and regenerate
sfc-gh-asawicki Jul 24, 2024
8845282
Improve the makefile and the documentation
sfc-gh-asawicki Jul 24, 2024
cbbfc82
Generate user to improve handling of unsupported attribute type
sfc-gh-asawicki Jul 24, 2024
2866e09
Update TODOs and README
sfc-gh-asawicki Jul 24, 2024
680555c
Run pre-push
sfc-gh-asawicki Jul 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Improve the makefile and the documentation
sfc-gh-asawicki committed Jul 24, 2024
commit 8845282a409aac247bcea49ef239ffcfccf937a9
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -159,8 +159,8 @@ generate-resource-model-builders: ## Generate resource model builders
clean-resource-model-builder: ## Clean resource model builders
rm -f ./pkg/acceptance/bettertestspoc/config/model/*_gen.go

clean-all-assertions: clean-snowflake-object-assertions clean-snowflake-object-parameters-assertions clean-resource-assertions clean-resource-parameters-assertions clean-resource-show-output-assertions ## clean all generated assertions
clean-all-assertions-and-config-models: clean-snowflake-object-assertions clean-snowflake-object-parameters-assertions clean-resource-assertions clean-resource-parameters-assertions clean-resource-show-output-assertions clean-resource-model-builder ## clean all generated assertions and config models

generate-all-assertions: generate-snowflake-object-assertions generate-snowflake-object-parameters-assertions generate-resource-assertions generate-resource-parameters-assertions generate-resource-show-output-assertions ## generate all assertions
generate-all-assertions-and-config-models: generate-snowflake-object-assertions generate-snowflake-object-parameters-assertions generate-resource-assertions generate-resource-parameters-assertions generate-resource-show-output-assertions generate-resource-model-builders ## generate all assertions and config models

.PHONY: build-local clean-generator-poc dev-setup dev-cleanup docs docs-check fmt fmt-check fumpt help install lint lint-fix mod mod-check pre-push pre-push-check sweep test test-acceptance uninstall-tf
184 changes: 101 additions & 83 deletions pkg/acceptance/bettertestspoc/README.md
Original file line number Diff line number Diff line change
@@ -11,7 +11,107 @@ It contains the following packages:
- `config` - the new `ResourceModel` abstraction resides here. It provides models for objects and the builder methods allowing better config preparation in the acceptance tests.
It aims to be more readable than using `Config:` with hardcoded string or `ConfigFile:` for file that is not directly reachable from the test body. Also, it should be easier to reuse the models and prepare convenience extension methods. The models are already generated.

## Usage
## How it works
### Adding new resource assertions
Resource assertions can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allStructs` slice in the `assert/resourceassert/gen/resource_schema_def.go`
- to add custom (not generated assertions) create file `abc_resource_ext.go` in the `assert/resourceassert` package. Example would be:
```go
func (w *WarehouseResourceAssert) HasDefaultMaxConcurrencyLevel() *WarehouseResourceAssert {
w.AddAssertion(assert.ValueSet("max_concurrency_level", "8"))
return w
}
```

### Adding new resource show output assertions
Resource show output assertions can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allResourceSchemaDefs` slice in the `assert/objectassert/gen/sdk_object_def.go`
- to add custom (not generated assertions) create file `abc_show_output_ext.go` in the `assert/resourceshowoutputassert` package. Example would be:
```go
func (u *UserShowOutputAssert) HasNameAndLoginName(expected string) *UserShowOutputAssert {
return u.
HasName(expected).
HasLoginName(expected)
}
```

### Adding new resource parameters assertions
Resource parameters assertions can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allObjectsParameters` slice in the `assert/objectparametersassert/gen/object_parameters_def.go`
- to add custom (not generated assertions) create file `warehouse_resource_parameters_ext.go` in the `assert/resourceparametersassert` package. Example would be:
```go
func (w *WarehouseResourceParametersAssert) HasDefaultMaxConcurrencyLevel() *WarehouseResourceParametersAssert {
return w.
HasMaxConcurrencyLevel(8).
HasMaxConcurrencyLevelLevel("")
}
```

### Adding new Snowflake object assertions
Snowflake object assertions can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allStructs` slice in the `assert/objectassert/gen/main/main.go`
- to add custom (not generated assertions) create file `abc_snowflake_ext.go` in the `objectassert` package. Example would be:
```go
func (w *WarehouseAssert) HasStateOneOf(expected ...sdk.WarehouseState) *WarehouseAssert {
w.assertions = append(w.assertions, func(t *testing.T, o *sdk.Warehouse) error {
t.Helper()
if !slices.Contains(expected, o.State) {
return fmt.Errorf("expected state one of: %v; got: %v", expected, string(o.State))
}
return nil
})
return w
}
```

### Adding new Snowflake object parameters assertions
Snowflake object parameters assertions can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allObjectsParameters` slice in the `assert/objectparametersassert/gen/main/main.go`
- make sure that test helper method `acc.TestClient().Parameter.ShowAbcParameters` exists in `/pkg/acceptance/helpers/parameter_client.go`
- to add custom (not generated) assertions create file `abc_parameters_snowflake_ext.go` in the `objectparametersassert` package. Example would be:
```go
func (w *WarehouseParametersAssert) HasDefaultMaxConcurrencyLevel() *WarehouseParametersAssert {
return w.
HasMaxConcurrencyLevel(8).
HasMaxConcurrencyLevelLevel("")
}
```

### Adding new models
Resource config model builders can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allResourceSchemaDefs` slice in the `assert/resourceassert/gen/resource_schema_def.go`
- to add custom (not generated) config builder methods create file `warehouse_model_ext` in the `config/model` package. Example would be:
```go
func BasicWarehouseModel(
name string,
comment string,
) *WarehouseModel {
return WarehouseWithDefaultMeta(name).WithComment(comment)
}

func (w *WarehouseModel) WithWarehouseSizeEnum(warehouseSize sdk.WarehouseSize) *WarehouseModel {
return w.WithWarehouseSize(string(warehouseSize))
}
```

### Running the generators
Each of the above assertion types/config models has its own generator and cleanup entry in our Makefile.
You can generate config models with:
```shell
make clean-resource-model-builder generate-resource-model-builder
```

You can use cli flags:
```shell
make clean-resource-model-builder generate-resource-model-builder SF_TF_GENERATOR_ARGS='--dry-run --verbose'
```

To clean/generate all from this package run
```shell
make clean-all-assertions-and-config-models generate-all-assertions-and-config-models
```

### Example usage in practice
You can check the current example usage in `TestAcc_Warehouse_BasicFlows` and the `create: complete` inside `TestInt_Warehouses`. To see the output after invalid assertions:
- add the following to the first step of `TestAcc_Warehouse_BasicFlows`
```go
@@ -195,88 +295,6 @@ it will result in:
Test: TestInt_Warehouses/create:_complete
```

## Adding new resource assertions
Resource assertions can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allStructs` slice in the `assert/resourceassert/gen/resource_schema_def.go`
- to add custom (not generated assertions) create file `abc_resource_ext.go` in the `assert/resourceassert` package. Example would be:
```go
func (w *WarehouseResourceAssert) HasDefaultMaxConcurrencyLevel() *WarehouseResourceAssert {
w.AddAssertion(assert.ValueSet("max_concurrency_level", "8"))
return w
}
```

## Adding new resource show output assertions
Resource show output assertions can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allResourceSchemaDefs` slice in the `assert/objectassert/gen/sdk_object_def.go`
- to add custom (not generated assertions) create file `abc_show_output_ext.go` in the `assert/resourceshowoutputassert` package. Example would be:
```go
func (u *UserShowOutputAssert) HasNameAndLoginName(expected string) *UserShowOutputAssert {
return u.
HasName(expected).
HasLoginName(expected)
}
```

## Adding new resource parameters assertions
Resource parameters assertions can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allObjectsParameters` slice in the `assert/objectparametersassert/gen/object_parameters_def.go`
- to add custom (not generated assertions) create file `warehouse_resource_parameters_ext.go` in the `assert/resourceparametersassert` package. Example would be:
```go
func (w *WarehouseResourceParametersAssert) HasDefaultMaxConcurrencyLevel() *WarehouseResourceParametersAssert {
return w.
HasMaxConcurrencyLevel(8).
HasMaxConcurrencyLevelLevel("")
}
```

## Adding new Snowflake object assertions
Snowflake object assertions can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allStructs` slice in the `assert/objectassert/gen/main/main.go`
- to add custom (not generated assertions) create file `abc_snowflake_ext.go` in the `objectassert` package. Example would be:
```go
func (w *WarehouseAssert) HasStateOneOf(expected ...sdk.WarehouseState) *WarehouseAssert {
w.assertions = append(w.assertions, func(t *testing.T, o *sdk.Warehouse) error {
t.Helper()
if !slices.Contains(expected, o.State) {
return fmt.Errorf("expected state one of: %v; got: %v", expected, string(o.State))
}
return nil
})
return w
}
```

## Adding new Snowflake object parameters assertions
Snowflake object parameters assertions can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allObjectsParameters` slice in the `assert/objectparametersassert/gen/main/main.go`
- make sure that test helper method `acc.TestClient().Parameter.ShowAbcParameters` exists in `/pkg/acceptance/helpers/parameter_client.go`
- to add custom (not generated) assertions create file `abc_parameters_snowflake_ext.go` in the `objectparametersassert` package. Example would be:
```go
func (w *WarehouseParametersAssert) HasDefaultMaxConcurrencyLevel() *WarehouseParametersAssert {
return w.
HasMaxConcurrencyLevel(8).
HasMaxConcurrencyLevelLevel("")
}
```

## Adding new models
Resource config model builders can be generated automatically. For object `abc` do the following:
- add object you want to generate to `allResourceSchemaDefs` slice in the `assert/resourceassert/gen/resource_schema_def.go`
- to add custom (not generated) config builder methods create file `warehouse_model_ext` in the `config/model` package. Example would be:
```go
func BasicWarehouseModel(
name string,
comment string,
) *WarehouseModel {
return WarehouseWithDefaultMeta(name).WithComment(comment)
}

func (w *WarehouseModel) WithWarehouseSizeEnum(warehouseSize sdk.WarehouseSize) *WarehouseModel {
return w.WithWarehouseSize(string(warehouseSize))
}
```

## Known limitations/planned improvements
- Test all the utilities for assertion/model construction (public interfaces, methods, functions).
- Verify if all the config types are supported.