Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: scaledata/filesystem_simulator/nodes=3 failed #46291

Closed
cockroach-teamcity opened this issue Mar 19, 2020 · 4 comments
Closed

roachtest: scaledata/filesystem_simulator/nodes=3 failed #46291

cockroach-teamcity opened this issue Mar 19, 2020 · 4 comments
Assignees
@andreimatei
Labels
C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Milestone
20.1

Comments

@cockroach-teamcity
Copy link
Member

(roachtest).scaledata/filesystem_simulator/nodes=3 failed on release-19.1@9d9b9a1a3417ae3460b8bdc7c77bf8c9df54d899:

		    main.(*cluster).RunE
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2010
		    main.runSqlapp.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/scaledata.go:108
		    main.(*monitor).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2344
		    github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
		    runtime.goexit
		    	/usr/local/go/src/runtime/asm_amd64.s:1357
		  - error with embedded safe details: output in %s
		    -- arg 1: <string>
		  - output in run_062335.330_n4_filesystemsimulator_:
		  - error with attached stack trace:
		    main.execCmd
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:406
		    main.(*cluster).RunL
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2019
		    main.(*cluster).RunE
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2000
		    main.runSqlapp.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/scaledata.go:108
		    main.(*monitor).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2344
		    github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
		    runtime.goexit
		    	/usr/local/go/src/runtime/asm_amd64.s:1357
		  - error with embedded safe details: %s returned:
		    stderr:
		    %s
		    stdout:
		    %s
		    -- arg 1: <string>
		    -- arg 2: <string>
		    -- arg 3: <string>
		  - /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1816983-1584598768-09-n4cpu4:4 -- ./filesystem_simulator  --duration_secs=600 --num_workers=16 --cockroach_ip_addresses_csv='10.128.15.244:26257,10.128.15.239:26257,10.128.15.248:26257'  returned:
		    stderr:
		    B.RandomDB chose DB at index 1
		    2020/03/19 06:33:12 ExecuteTx retry attempt 1 failed, started at 2020-03-19 06:33:12.628380669 +0000 UTC m=+576.562469098, now = 2020-03-19 06:33:12.847515376 +0000 UTC m=+576.781603818, took 219.13472ms
		    2020/03/19 06:33:12 Attempt failed with error driver: bad connection: ... Retrying after sleeping 5ns
		    2020/03/19 06:33:12 ExecuteTx retry attempt 1 failed, started at 2020-03-19 06:33:12.801005451 +0000 UTC m=+576.735093878, now = 2020-03-19 06:33:12.84778251 +0000 UTC m=+576.781870955, took 46.777077ms
		    2020/03/19 06:33:12 Attempt failed with error driver: bad connection: ... Retrying after sleeping 5ns
		    2020/03/19 06:33:12 ExecuteTx retry attempt 1 failed, started at 2020-03-19 06:33:10.828762181 +0000 UTC m=+574.762850609, now = 2020-03-19 06:33:12.848008983 +0000 UTC m=+576.782097425, took 2.019246816s
		    2020/03/19 06:33:12 Aborting Retries because this error of type *errors.errorString is not retryable : unexpected EOF
		    2020/03/19 06:33:12 unexpected EOF
		    Error:  exit status 255
		    
		    stdout::
		  - exit status 1
More

Artifacts: /scaledata/filesystem_simulator/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues
@cockroach-teamcity cockroach-teamcity added branch-release-19.1 C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Mar 19, 2020
@cockroach-teamcity cockroach-teamcity added this to the 20.1 milestone Mar 19, 2020
This was referenced Mar 19, 2020
Closed
Closed
@cockroach-teamcity
Copy link
Member Author

(roachtest).scaledata/filesystem_simulator/nodes=3 failed on release-19.1@cfdc5f37cad427dcc5e36aa630f9d49e588f5cf3:

		    main.(*cluster).RunE
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2010
		    main.runSqlapp.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/scaledata.go:108
		    main.(*monitor).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2344
		    github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
		    runtime.goexit
		    	/usr/local/go/src/runtime/asm_amd64.s:1357
		  - error with embedded safe details: output in %s
		    -- arg 1: <string>
		  - output in run_061842.223_n4_filesystemsimulator_:
		  - error with attached stack trace:
		    main.execCmd
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:406
		    main.(*cluster).RunL
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2019
		    main.(*cluster).RunE
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2000
		    main.runSqlapp.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/scaledata.go:108
		    main.(*monitor).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2344
		    github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
		    runtime.goexit
		    	/usr/local/go/src/runtime/asm_amd64.s:1357
		  - error with embedded safe details: %s returned:
		    stderr:
		    %s
		    stdout:
		    %s
		    -- arg 1: <string>
		    -- arg 2: <string>
		    -- arg 3: <string>
		  - /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1819536-1584684878-15-n4cpu4:4 -- ./filesystem_simulator  --duration_secs=600 --num_workers=16 --cockroach_ip_addresses_csv='10.128.0.147:26257,10.128.0.101:26257,10.128.0.135:26257'  returned:
		    stderr:
		    andomDB chose DB at index 0
		    2020/03/20 06:26:17 ExecuteTx retry attempt 1 failed, started at 2020-03-20 06:26:17.073795048 +0000 UTC m=+454.135973835, now = 2020-03-20 06:26:17.795431371 +0000 UTC m=+454.857610238, took 721.636403ms
		    2020/03/20 06:26:17 Attempt failed with error driver: bad connection: ... Retrying after sleeping 5ns
		    2020/03/20 06:26:17 ExecuteTx retry attempt 1 failed, started at 2020-03-20 06:26:16.976434875 +0000 UTC m=+454.038613649, now = 2020-03-20 06:26:17.795452344 +0000 UTC m=+454.857631200, took 819.017551ms
		    2020/03/20 06:26:17 Attempt failed with error driver: bad connection: ... Retrying after sleeping 5ns
		    2020/03/20 06:26:17 ExecuteTx retry attempt 1 failed, started at 2020-03-20 06:26:16.695879714 +0000 UTC m=+453.758058490, now = 2020-03-20 06:26:17.795482191 +0000 UTC m=+454.857661021, took 1.099602531s
		    2020/03/20 06:26:17 Aborting Retries because this error of type *errors.errorString is not retryable : unexpected EOF
		    2020/03/20 06:26:17 unexpected EOF
		    Error:  exit status 255
		    
		    stdout::
		  - exit status 1
More

Artifacts: /scaledata/filesystem_simulator/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

This was referenced Mar 20, 2020
Closed
Closed
@cockroach-teamcity
Copy link
Member Author

(roachtest).scaledata/filesystem_simulator/nodes=3 failed on release-19.1@d2ca82a9f898bd8ef7bd2e9660e24605f2d0b25d:

		  - error with attached stack trace:
		    main.(*cluster).RunE
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2010
		    main.runSqlapp.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/scaledata.go:100
		    main.(*monitor).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2344
		    github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
		    runtime.goexit
		    	/usr/local/go/src/runtime/asm_amd64.s:1357
		  - error with embedded safe details: output in %s
		    -- arg 1: <string>
		  - output in run_062145.813_n4_filesystemsimulator:
		  - error with attached stack trace:
		    main.execCmd
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:406
		    main.(*cluster).RunL
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2019
		    main.(*cluster).RunE
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2000
		    main.runSqlapp.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/scaledata.go:100
		    main.(*monitor).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2344
		    github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
		    runtime.goexit
		    	/usr/local/go/src/runtime/asm_amd64.s:1357
		  - error with embedded safe details: %s returned:
		    stderr:
		    %s
		    stdout:
		    %s
		    -- arg 1: <string>
		    -- arg 2: <string>
		    -- arg 3: <string>
		  - /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1829279-1585203477-09-n4cpu4:4 -- ./filesystem_simulator --install_schema --cockroach_ip_addresses_csv='10.128.0.34:26257,10.128.0.30:26257,10.128.0.46:26257'  returned:
		    stderr:
		    2020/03/26 06:21:46 numWorkers: 16, numCheckers: 16
		    2020/03/26 06:21:46 pq: column "username" does not exist
		    Error: SSH_PROBLEM:
		      - error with user detail: Node 4. Command with error:
		        ```
		        ./filesystem_simulator --install_schema --cockroach_ip_addresses_csv='10.128.0.34:26257,10.128.0.30:26257,10.128.0.46:26257'
		        ```
		      - exit status 255
		    
		    stdout::
		  - exit status 10
More

Artifacts: /scaledata/filesystem_simulator/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).scaledata/filesystem_simulator/nodes=3 failed on release-19.1@8b5a5d6333a3d7ac7bdf96b7f90d6272247551cc:

		  - error with attached stack trace:
		    main.(*cluster).RunE
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2010
		    main.runSqlapp.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/scaledata.go:100
		    main.(*monitor).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2344
		    github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
		    runtime.goexit
		    	/usr/local/go/src/runtime/asm_amd64.s:1357
		  - error with embedded safe details: output in %s
		    -- arg 1: <string>
		  - output in run_062200.647_n4_filesystemsimulator:
		  - error with attached stack trace:
		    main.execCmd
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:406
		    main.(*cluster).RunL
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2019
		    main.(*cluster).RunE
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2000
		    main.runSqlapp.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/scaledata.go:100
		    main.(*monitor).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2344
		    github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
		    runtime.goexit
		    	/usr/local/go/src/runtime/asm_amd64.s:1357
		  - error with embedded safe details: %s returned:
		    stderr:
		    %s
		    stdout:
		    %s
		    -- arg 1: <string>
		    -- arg 2: <string>
		    -- arg 3: <string>
		  - /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1834629-1585376297-04-n4cpu4:4 -- ./filesystem_simulator --install_schema --cockroach_ip_addresses_csv='10.128.0.2:26257,10.128.0.70:26257,10.128.0.68:26257'  returned:
		    stderr:
		    2020/03/28 06:22:01 numWorkers: 16, numCheckers: 16
		    2020/03/28 06:22:01 pq: column "username" does not exist
		    Error: SSH_PROBLEM:
		      - error with user detail: Node 4. Command with error:
		        ```
		        ./filesystem_simulator --install_schema --cockroach_ip_addresses_csv='10.128.0.2:26257,10.128.0.70:26257,10.128.0.68:26257'
		        ```
		      - exit status 255
		    
		    stdout::
		  - exit status 10
More

Artifacts: /scaledata/filesystem_simulator/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@irfansharif
Copy link
Contributor

cockroachdb/rksql#1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andreimatei andreimatei

Labels
C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Projects
None yet
Milestone
20.1
Development

No branches or pull requests
3 participants
@andreimatei @irfansharif @cockroach-teamcity