Skip to content

Commit

Permalink
Add tc39/test262 tests in k6 (#1747)
Browse files Browse the repository at this point in the history
  • Loading branch information
mstoykov authored Dec 14, 2020
1 parent 1443e5e commit 15e15e5
Show file tree
Hide file tree
Showing 8 changed files with 1,691 additions and 1 deletion.
32 changes: 32 additions & 0 deletions .github/workflows/tc39.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: TC39
on:
workflow_dispatch:
push:
branches:
- master
tags:
- v*
pull_request:
paths:
- 'js/**'

defaults:
run:
shell: bash

jobs:
tc39:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Install Go
uses: actions/setup-go@v2
with:
go-version: 1.15.x
- name: Run tests
run: |
set -x
cd js/tc39
sh checkout.sh
go test -timeout 1h
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
/k6.exe
/dist
/pkg-build
/js/tc39/TestTC39

.vscode
*.sublime-workspace
*.log
*.syso
.idea

*.DS_Store

/versioninfo.json
Expand Down
34 changes: 34 additions & 0 deletions js/tc39/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# This a WIP

The point of this module is to test k6 goja+babel+core.js combo against the tc39 test suite.

Ways to use it:
1. run ./checkout.sh to checkout the last commit sha of [test262](https://github.com/tc39/test262)
that was tested with
2. Run `go test &> out.log`

If there are failures there will be a JSON with what failed.
The full list of failing tests is in `breaking_test_errors.json` in order to regenerate it (in case
of changes) it needs to become an empty JSON object `{}` and then the test should be rerun and the
new json should be put there.

TODO:
1. ~Enable more test currently only es5 and es6 tests are enabled but babel supports some ES2016 and
ES2017~
2. disable tests that we know won't work and .. don't care
3. Make this faster and better
4. ~Move it to inside k6~


This is obviously a modified version of [the code in the goja
repo](https://github.com/dop251/goja/blob/master/tc39_test.go)


# Reasons for recording breaking_test_errors.json

Unfortunately k6 doesn't pass all the test that are currently defined as "interesting" and probably
won't even more. Goja decided to just not run the ones that it knows it fails currently, but this
means that if they stop failing, someone needs to go re-enable these tests. This also means that, if the
previous breakage was something a user can work around in a certain way, it now might be something
else that the user can't workaround or have another problem. For this reasons I decided that
actually recording what breaks and checking that it doesn't change is a better idea.
1,005 changes: 1,005 additions & 0 deletions js/tc39/breaking_test_errors.json

Large diffs are not rendered by default.

9 changes: 9 additions & 0 deletions js/tc39/checkout.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/sh
sha=72154b17fc99a26e79b2586960f059360d4ce43d # this is just the commit it was last tested with
mkdir -p ./TestTC39/test262
cd ./TestTC39/test262
git init
git remote add origin https://github.com/tc39/test262.git
git fetch origin --depth=1 "${sha}"
git reset --hard "${sha}"
cd -
16 changes: 16 additions & 0 deletions js/tc39/tc39_norace_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
// +build !race
// Heavily influenced by the fantastic work by @dop251 for https://github.com/dop251/goja

package tc39

import "testing"

func (ctx *tc39TestCtx) runTest(name string, f func(t *testing.T)) {
ctx.t.Run(name, func(t *testing.T) {
// t.Parallel()
f(t)
})
}

func (ctx *tc39TestCtx) flush() {
}
32 changes: 32 additions & 0 deletions js/tc39/tc39_race_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// +build race
// Heavily influenced by the fantastic work by @dop251 for https://github.com/dop251/goja

package tc39

import (
"testing"
)

const (
tc39MaxTestGroupSize = 1000 // to prevent race detector complaining about too many goroutines
)

func (ctx *tc39TestCtx) runTest(name string, f func(t *testing.T)) {
ctx.testQueue = append(ctx.testQueue, tc39Test{name: name, f: f})
if len(ctx.testQueue) >= tc39MaxTestGroupSize {
ctx.flush()
}
}

func (ctx *tc39TestCtx) flush() {
ctx.t.Run("tc39", func(t *testing.T) {
for _, tc := range ctx.testQueue {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
tc.f(t)
})
}
})
ctx.testQueue = ctx.testQueue[:0]
}
Loading

0 comments on commit 15e15e5

Please sign in to comment.