Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: acceptance/rapid-restart failed #52060

Closed
cockroach-teamcity opened this issue Jul 29, 2020 · 1 comment · Fixed by #52269
Closed

roachtest: acceptance/rapid-restart failed #52060

cockroach-teamcity opened this issue Jul 29, 2020 · 1 comment · Fixed by #52269
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Milestone
20.2

Comments

@cockroach-teamcity
Copy link
Member

(roachtest).acceptance/rapid-restart failed on master@c9c2aa454d5428d5327993a696464072d8224507:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: artifacts/acceptance/rapid-restart/run_1
	cluster.go:2092,rapid_restart.go:67,acceptance.go:96,test_runner.go:754: /go/src/github.com/cockroachdb/cockroach/bin/roachprod stop --sig=2 local:1 returned: exit status 1
		(1) /go/src/github.com/cockroachdb/cockroach/bin/roachprod stop --sig=2 local:1 returned
		  | stderr:
		  | 0: exit status 1: /bin/bash: line 8: kill: (16016) - No such process
		  |
		  | I200729 08:46:31.024834 1 cluster_synced.go:1732  command failed
		  |
		  | stdout:
		  | local: stopping
		Wraps: (2) exit status 1
		Error types: (1) *main.withCommandDetails (2) *exec.ExitError
More

Artifacts: /acceptance/rapid-restart

See this test on roachdash
powered by pkg/cmd/internal/issues
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Jul 29, 2020
@cockroach-teamcity cockroach-teamcity added this to the 20.2 milestone Jul 29, 2020
@andreimatei andreimatei mentioned this issue Aug 3, 2020
Merged
craig bot pushed a commit that referenced this issue Aug 4, 2020
@pbardea @andreimatei @irfansharif
Merge #51959 #52269 #52273 #52343
986e308 
51959: backupccl: distribute RESTORE work using DistSQL r=dt,yuzefovich a=pbardea

This commit replaces restore's old method of coordinating work on a
single node to instead use DistSQL. It creates a 2 stage DistSQL flow.
The first stage assigns chunks to SplitAndScatter processors which
forward spans that they scattered to the second stage made up of the
RestoreData processors which ingest the data. The SplitAndScatter
processors attempt to send the work to the RestoreData colocated with
the leaseholder of the span after the scattering.

Release note: None.

52269: roachtest: correctly start crdb in acceptance/rapid_restart r=andreimatei a=andreimatei

--start-single-node was needed. Without it, I think the test's killing
of a node raced with that node dieing by itself, and sometimes the race
resulted in `cockroach stop` first seeing the process but then
`/bin/bash: line 8: kill: (16016) - No such proces`

Fixes #52060

Release note: None

52273: roachtest: remove confusing "--- PASS" log lines r=andreimatei a=andreimatei

Two workloads were printing PASS/FAIL lines from inside the predicates
passed to Searcher.Search. This is really confusing when reading the
test's output, because they don't correspond to the test's disposition.

Release note: None

52343: roachtest: reflake (skip) disk-stalled test for release-19.1 r=irfansharif a=irfansharif

Fixes #52181. This test doesn't make use of all the `roachtest start`
smartness to figure out what sub-command to use and instead constructs
the raw start command by hand.

Given our support policy, let's just bump the minimum version this test
expects to run. This feels like another instance where #51897 would be 
nice to have.

Release note: None

Co-authored-by: Paul Bardea <[email protected]>
Co-authored-by: Andrei Matei <[email protected]>
Co-authored-by: irfan sharif <[email protected]>
@cockroach-teamcity
Copy link
Member Author

(roachtest).acceptance/rapid-restart failed on master@1d12a54a77971781775aa11697159d2914b05cd9:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: artifacts/acceptance/rapid-restart/run_1
	cluster.go:2092,rapid_restart.go:67,acceptance.go:96,test_runner.go:754: /go/src/github.com/cockroachdb/cockroach/bin/roachprod stop --sig=2 local:1 returned: exit status 1
		(1) /go/src/github.com/cockroachdb/cockroach/bin/roachprod stop --sig=2 local:1 returned
		  | stderr:
		  | 0: exit status 1: /bin/bash: line 8: kill: (16396) - No such process
		  |
		  | I200804 17:18:55.430741 1 cluster_synced.go:1732  command failed
		  |
		  | stdout:
		  | local: stopping
		Wraps: (2) exit status 1
		Error types: (1) *main.withCommandDetails (2) *exec.ExitError
More

Artifacts: /acceptance/rapid-restart

See this test on roachdash
powered by pkg/cmd/internal/issues

@craig craig bot closed this as completed in dd0cdfe Aug 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Projects
None yet
Milestone
20.2
Development

Successfully merging a pull request may close this issue.

roachtest: correctly start crdb in acceptance/rapid_restart andreimatei/cockroach
1 participant
@cockroach-teamcity