[Elastic Agent] Golden files for program tests #27862
Conversation
Currently it seems the expected config files for each program were manually generated. This makes adding a new test cumbersome and in case of small changes it means adjusting many files. This changes the test that it is possible to run with `go test . -generate` and new golden files are generated. Ideally it also means most of the additional tests become obsolete like how many program files should be generated. To make sure no left over files exist, on -generate all the generated files are first cleaned up.
botelastic
bot
added
the
needs_team
|
@blakerouse @michalpristas Would be great to get your feedback on this idea. |
botelastic
bot
removed
the
needs_team
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪💚 Flaky test reportTests succeeded. Expand to view the summary
Test stats 🧪
|
| @@ -436,6 +442,16 @@ func TestConfiguration(t *testing.T) { | |||
| }, | |||
| } | |||
|
|
|||
| // Cleanup all generated files to make sure not having any left overs | |||
| if *generateFlag { | |||
| generatedFiles, err := filepath.Glob(filepath.Join("testdata", "*.generated.yml")) | |||
There was a problem hiding this comment.
Better name? Maybe .golden.yml?
There was a problem hiding this comment.
Some what confused on what you are changing here? It seems like you are changing the code to generated the expected results of the tests. To me this will negate the point of the tests. The tests are ensuring that the inputs, match the outputs. The outputs are hand written to ensure that the output is what is expected with the given input.
|
The test files are only generated if the The problem with the current approach is that it is really cumbersome to add new examples. With the new approach, the first time the generate script can be run and the output can be reviewed. If in the future there is a "mass change" needed, this also make it easier. The important part is that any time the |
|
To make this PR easier to review I did not change the old tests and kept them as is. I think we can remove some of these as the new ones replace part of the old ones that have at least one program but we can do this in a follow up PR. Marking this as ready for review. |
|
Pinging @elastic/agent (Team:Agent) |
Currently it seems the expected config files for each program were manually generated. This makes adding a new test cumbersome and in case of small changes it means adjusting many files. This changes the test that it is possible to run with `go test . -generate` and new golden files are generated. Ideally it also means most of the additional tests become obsolete like how many program files should be generated. To make sure no left over files exist, on -generate all the generated files are first cleaned up. (cherry picked from commit d898e3d)
Currently it seems the expected config files for each program were manually generated. This makes adding a new test cumbersome and in case of small changes it means adjusting many files. This changes the test that it is possible to run with `go test . -generate` and new golden files are generated. Ideally it also means most of the additional tests become obsolete like how many program files should be generated. To make sure no left over files exist, on -generate all the generated files are first cleaned up. (cherry picked from commit d898e3d) Co-authored-by: Nicolas Ruflin <[email protected]>
* upstream/master: (658 commits) Add complete k8s metadata through composable provider (elastic#27691) Revert "Fix issue where --insecure didn't propogate to Fleet Server ES connection (elastic#27969)" (elastic#27997) Remove deprecated kafka fields (elastic#27938) [Filebeat] Add Base64 encoded HMAC & UUID template functions to httpjson input (elastic#27873) Improve httpjson template function join (elastic#27996) Remove kubernetes.container.image alias (elastic#27898) [Elastic Agent] Golden files for program tests (elastic#27862) [Elastic Agent] Disable modules.d in metricbeat (elastic#27860) libbeat/common/seccomp: provide default policy for linux arm64 (elastic#27955) Fix logger statement in aws-s3 input (elastic#27982) Fix wrong merge (elastic#27976) Fix issue where --insecure didn't propogate to Fleet Server ES connection (elastic#27969) Forward-port 7.14.2 changelog to master (elastic#27975) [Filebeat] Removing duplicate modules (aliases) Observability (elastic#27919) Fix path in vagrant windows script (elastic#27966) [Filebeat] Removing duplicate modules (aliases) and Cyberark (elastic#27915) No changelog for 8.0.0-alpha2 (elastic#27961) Add write access to 'url.value' from 'request.transforms'. (elastic#27937) Docker: remove deprecated fields (elastic#27933) Filebeat: Make all filesets disabled in default configuration (elastic#27762) ...
Currently it seems the expected config files for each program were manually generated. This makes adding a new test cumbersome and in case of small changes it means adjusting many files. This changes the test that it is possible to run with `go test . -generate` and new golden files are generated. Ideally it also means most of the additional tests become obsolete like how many program files should be generated. To make sure no left over files exist, on -generate all the generated files are first cleaned up.
Currently it seems the expected config files for each program were manually generated. This makes adding a new test cumbersome and in case of small changes it means adjusting many files.
This changes the test that it is possible to run with
go test . -generateand new golden files are generated. Ideally it also means most of the additional tests become obsolete like how many program files should be generated. To make sure no left over files exist, on -generate all the generated files are first cleaned up.NOTE: This PR is not complete yet, for example removal of old files is missing and cleanup of generated files. I wanted to discuss the approach first before going further.