Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: use script to run tests and sync results to cloud #1871

Merged
merged 1 commit into from
Sep 16, 2024

Conversation

i4ki
Copy link
Contributor

@i4ki i4ki commented Sep 13, 2024

What this PR does / why we need it:

Experiment: Uses Terramate to run its own tests and sync the results to Terramate Cloud.

Which issue(s) this PR fixes:

none

Special notes for your reviewer:

Does this PR introduce a user-facing change?

no

@i4ki i4ki force-pushed the i4k-parallel-terramate-tests branch 2 times, most recently from 1ec738b to cd06f35 Compare September 14, 2024 00:01
Copy link

github-actions bot commented Sep 14, 2024

metric: time/op
CloudReadLines-4: old 1.03ms ± 6%: new 1.02ms ± 6%: delta: 0.00%
CloudReadLine-4: old 7.29ms ± 1%: new 7.29ms ± 1%: delta: 0.00%
ListFiles-4: old 80.1µs ± 2%: new 80.6µs ± 1%: delta: 0.58%
Generate-4: old 4.93s ± 2%: new 4.94s ± 5%: delta: 0.00%
GenerateRegex-4: old 3.29s ± 2%: new 3.41s ± 1%: delta: 3.69%
TokensForExpressionComplex-4: old 1.26ms ± 0%: new 1.26ms ± 1%: delta: 0.33%
TokensForExpressionPlainStringNoNewline-4: old 787ns ± 1%: new 805ns ± 1%: delta: 2.18%
TokensForExpressionStringWith100Newlines-4: old 23.7µs ± 1%: new 24.6µs ± 2%: delta: 3.93%
TokensForExpressionObjectWith100KeysWithNumberValues-4: old 1.44ms ± 1%: new 1.47ms ± 0%: delta: 1.59%
TokensForExpression-4: old 1.26ms ± 2%: new 1.28ms ± 1%: delta: 1.20%
PartialEvalComplex-4: old 527µs ± 2%: new 545µs ± 1%: delta: 3.48%
PartialEvalSmallString-4: old 3.83µs ± 3%: new 3.86µs ± 1%: delta: 0.81%
PartialEvalHugeString-4: old 1.92ms ± 1%: new 1.93ms ± 1%: delta: 0.81%
PartialEvalHugeInterpolatedString-4: old 4.86ms ± 3%: new 4.89ms ± 2%: delta: 0.00%
PartialEvalObject-4: old 25.7µs ± 1%: new 26.2µs ± 2%: delta: 1.91%
TmAllTrueLiteralList-4: old 630µs ± 2%: new 636µs ± 2%: delta: 0.92%
TmAllTrueFuncall-4: old 20.9µs ± 3%: new 21.2µs ± 1%: delta: 1.24%
TmAnyTrueLiteralList-4: old 4.99ms ± 4%: new 4.98ms ± 1%: delta: 0.00%
TmAnyTrueFuncall-4: old 21.3µs ± 2%: new 21.5µs ± 3%: delta: 0.64%
TmTernary-4: old 2.63µs ± 2%: new 2.67µs ± 1%: delta: 1.37%
TmTryUnknownFunc-4: old 2.27µs ± 1%: new 2.33µs ± 2%: delta: 2.41%
TmTryUnknownVariable-4: old 2.19µs ± 1%: new 2.21µs ± 1%: delta: 0.87%
TmTryUnknownObjectKey-4: old 2.49µs ± 1%: new 2.52µs ± 1%: delta: 1.33%
metric: alloc/op
CloudReadLines-4: old 3.12MB ± 0%: new 3.12MB ± 0%: delta: 0.00%
CloudReadLine-4: old 3.37MB ± 0%: new 3.37MB ± 0%: delta: 0.00%
ListFiles-4: old 23.3kB ± 0%: new 23.3kB ± 0%: delta: 0.00%
Generate-4: old 4.47GB ± 0%: new 4.47GB ± 0%: delta: 0.00%
GenerateRegex-4: old 1.84GB ± 0%: new 1.84GB ± 0%: delta: 0.00%
TokensForExpressionComplex-4: old 394kB ± 0%: new 394kB ± 0%: delta: 0.00%
TokensForExpressionPlainStringNoNewline-4: old 512B ± 0%: new 512B ± 0%: delta: 0.00%
TokensForExpressionStringWith100Newlines-4: old 12.5kB ± 0%: new 12.5kB ± 0%: delta: 0.00%
TokensForExpressionObjectWith100KeysWithNumberValues-4: old 395kB ± 0%: new 395kB ± 0%: delta: 0.00%
TokensForExpression-4: old 394kB ± 0%: new 395kB ± 0%: delta: 0.00%
PartialEvalComplex-4: old 338kB ± 0%: new 338kB ± 0%: delta: 0.00%
PartialEvalSmallString-4: old 1.74kB ± 0%: new 1.74kB ± 0%: delta: 0.00%
PartialEvalHugeString-4: old 196kB ± 0%: new 196kB ± 0%: delta: 0.00%
PartialEvalHugeInterpolatedString-4: old 4.05MB ± 0%: new 4.05MB ± 0%: delta: 0.00%
PartialEvalObject-4: old 17.2kB ± 0%: new 17.2kB ± 0%: delta: 0.00%
TmAllTrueLiteralList-4: old 321kB ± 0%: new 321kB ± 0%: delta: 0.00%
TmAllTrueFuncall-4: old 10.6kB ± 0%: new 10.6kB ± 0%: delta: 0.00%
TmAnyTrueLiteralList-4: old 2.09MB ± 0%: new 2.09MB ± 0%: delta: 0.00%
TmAnyTrueFuncall-4: old 10.7kB ± 0%: new 10.7kB ± 0%: delta: 0.00%
TmTernary-4: old 1.20kB ± 0%: new 1.20kB ± 0%: delta: 0.00%
TmTryUnknownFunc-4: old 784B ± 0%: new 784B ± 0%: delta: 0.00%
TmTryUnknownVariable-4: old 768B ± 0%: new 768B ± 0%: delta: 0.00%
TmTryUnknownObjectKey-4: old 952B ± 0%: new 952B ± 0%: delta: 0.00%
metric: allocs/op
CloudReadLines-4: old 5.54k ± 0%: new 5.54k ± 0%: delta: 0.00%
CloudReadLine-4: old 60.0k ± 0%: new 60.0k ± 0%: delta: 0.00%
ListFiles-4: old 321 ± 0%: new 321 ± 0%: delta: 0.00%
Generate-4: old 51.4M ± 0%: new 51.4M ± 0%: delta: 0.00%
GenerateRegex-4: old 36.8M ± 0%: new 36.8M ± 0%: delta: 0.00%
TokensForExpressionComplex-4: old 4.83k ± 0%: new 4.83k ± 0%: delta: 0.00%
TokensForExpressionPlainStringNoNewline-4: old 20.0 ± 0%: new 20.0 ± 0%: delta: 0.00%
TokensForExpressionStringWith100Newlines-4: old 227 ± 0%: new 227 ± 0%: delta: 0.00%
TokensForExpressionObjectWith100KeysWithNumberValues-4: old 3.19k ± 0%: new 3.19k ± 0%: delta: 0.01%
TokensForExpression-4: old 4.83k ± 0%: new 4.83k ± 0%: delta: 0.00%
PartialEvalComplex-4: old 3.48k ± 0%: new 3.48k ± 0%: delta: 0.00%
PartialEvalSmallString-4: old 23.0 ± 0%: new 23.0 ± 0%: delta: 0.00%
PartialEvalHugeString-4: old 35.0 ± 0%: new 35.0 ± 0%: delta: 0.00%
PartialEvalHugeInterpolatedString-4: old 23.1k ± 0%: new 23.1k ± 0%: delta: 0.00%
PartialEvalObject-4: old 152 ± 0%: new 152 ± 0%: delta: 0.00%
TmAllTrueLiteralList-4: old 5.94k ± 0%: new 5.94k ± 0%: delta: 0.00%
TmAllTrueFuncall-4: old 275 ± 0%: new 275 ± 0%: delta: 0.00%
TmAnyTrueLiteralList-4: old 59.6k ± 0%: new 59.6k ± 0%: delta: 0.00%
TmAnyTrueFuncall-4: old 277 ± 0%: new 277 ± 0%: delta: 0.00%
TmTernary-4: old 28.0 ± 0%: new 28.0 ± 0%: delta: 0.00%
TmTryUnknownFunc-4: old 21.0 ± 0%: new 21.0 ± 0%: delta: 0.00%
TmTryUnknownVariable-4: old 20.0 ± 0%: new 20.0 ± 0%: delta: 0.00%
TmTryUnknownObjectKey-4: old 23.0 ± 0%: new 23.0 ± 0%: delta: 0.00%

@i4ki i4ki force-pushed the i4k-parallel-terramate-tests branch 21 times, most recently from 1d4f2f8 to a1a09c3 Compare September 15, 2024 23:07
@i4ki i4ki marked this pull request as ready for review September 16, 2024 12:03
@i4ki i4ki requested review from a team as code owners September 16, 2024 12:03
@i4ki i4ki force-pushed the i4k-parallel-terramate-tests branch 3 times, most recently from db4e6d7 to 48f7148 Compare September 16, 2024 13:47
@terramate-io terramate-io deleted a comment from github-actions bot Sep 16, 2024
@terramate-io terramate-io deleted a comment from github-actions bot Sep 16, 2024
@terramate-io terramate-io deleted a comment from github-actions bot Sep 16, 2024
@terramate-io terramate-io deleted a comment from github-actions bot Sep 16, 2024
@i4ki i4ki force-pushed the i4k-parallel-terramate-tests branch 5 times, most recently from a79ec0f to 0d9055d Compare September 16, 2024 14:02
Copy link

github-actions bot commented Sep 16, 2024

Preview of macos-ventura/go1.21 tests in bc6bc16

🔍 View Details on Terramate Cloud

.
cloud
cloud/deployment
cloud/drift
cloud/preview
cloud/stack
cloud/testserver
cloud/testserver/cloudstore
cloud/testserver/cmd/testserver
cmd/terramate
cmd/terramate-ls
cmd/terramate/cli
cmd/terramate/cli/cliconfig
cmd/terramate/cli/clitest
cmd/terramate/cli/github
cmd/terramate/cli/gitlab
cmd/terramate/cli/out
cmd/tgdeps
config
config/filter
config/tag
e2etests/cloud
e2etests/cmd/helper
e2etests/core
e2etests/internal/runner
errors
errors/errlog
errors/verbosity
event
fs
generate
generate/genfile
generate/genhcl
generate/sharing
git
globals
hcl
hcl/ast
hcl/eval
hcl/fmt
hcl/info
lets
ls
mapexpr
mapexpr/test
modvendor
modvendor/download
modvendor/manifest
printer
project
run
run/dag
safeguard
scheduler
scheduler/resource
stack
stack/trigger
stdlib
strconv
test
test/cloud
test/errors
test/hclutils
test/hclutils/info
test/hclwrite
test/hclwrite/hclutils
test/ls
test/sandbox
tf
tg
versions

Copy link

github-actions bot commented Sep 16, 2024

Preview of ubuntu-focal/go1.21 tests in bc6bc16

🔍 View Details on Terramate Cloud

.
cloud
cloud/deployment
cloud/drift
cloud/preview
cloud/stack
cloud/testserver
cloud/testserver/cloudstore
cloud/testserver/cmd/testserver
cmd/terramate
cmd/terramate-ls
cmd/terramate/cli
cmd/terramate/cli/cliconfig
cmd/terramate/cli/clitest
cmd/terramate/cli/github
cmd/terramate/cli/gitlab
cmd/terramate/cli/out
cmd/tgdeps
config
config/filter
config/tag
e2etests/cloud
e2etests/cmd/helper
e2etests/core
e2etests/internal/runner
errors
errors/errlog
errors/verbosity
event
fs
generate
generate/genfile
generate/genhcl
generate/sharing
git
globals
hcl
hcl/ast
hcl/eval
hcl/fmt
hcl/info
lets
ls
mapexpr
mapexpr/test
modvendor
modvendor/download
modvendor/manifest
printer
project
run
run/dag
safeguard
scheduler
scheduler/resource
stack
stack/trigger
stdlib
strconv
test
test/cloud
test/errors
test/hclutils
test/hclutils/info
test/hclwrite
test/hclwrite/hclutils
test/ls
test/sandbox
tf
tg
versions

@i4ki i4ki force-pushed the i4k-parallel-terramate-tests branch 3 times, most recently from 8a24db8 to 2e324aa Compare September 16, 2024 14:29
@i4ki i4ki force-pushed the i4k-parallel-terramate-tests branch from 2e324aa to bc6bc16 Compare September 16, 2024 15:07
@i4ki i4ki merged commit 0877f8c into main Sep 16, 2024
6 checks passed
@i4ki i4ki deleted the i4k-parallel-terramate-tests branch September 16, 2024 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants