Skip to content

Commit

Permalink
test: Show cluster logs on failure for 21million systest. (#5507)
Browse files Browse the repository at this point in the history
test-21million.sh can show failures like the following in CI builds:

```
------- Stdout: -------
=== RUN   TestQueries/query-044
    --- FAIL: TestQueries/query-044 (4.74s)
        run_test.go:86: 
            	Error Trace:	run_test.go:86
            	Error:      	Received unexpected error:
            	            	rpc error: code = Unknown desc = : rpc error: code = Unavailable desc = transport is closing
            	Test:       	TestQueries/query-044
```
```
------- Stdout: -------
=== RUN   TestQueries/query-045
    --- FAIL: TestQueries/query-045 (0.01s)
        run_test.go:86: 
            	Error Trace:	run_test.go:86
            	Error:      	Received unexpected error:
            	            	rpc error: code = Unknown desc = : rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial tcp 172.24.0.4:7182: connect: connection refused"
            	Test:       	TestQueries/query-045
```

It looks like something happened to the Dgraph cluster, but we're
lost without any cluster logs.

This change prints out the cluster logs for alpha1, alpha2,
alpha3, and zero1 if there are test failures in
test-21million.sh.

Changes
* Print out cluster logs on test failure
* Move cluster tear down to the very end of the test so logs can
  be captured beforehand.
  • Loading branch information
danielmai authored May 26, 2020
1 parent 0ce7ff9 commit 36be40c
Showing 1 changed file with 14 additions and 6 deletions.
20 changes: 14 additions & 6 deletions systest/21million/test-21million.sh
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,20 @@ if [[ ! -z "$TEAMCITY_VERSION" ]]; then
fi
go test -v -tags standalone $SAVEDIR $QUIET || FOUND_DIFFS=1

if [[ $FOUND_DIFFS -eq 0 ]]; then
Info "no diffs found in query results"
else
Info "Cluster logs for alpha1"
docker logs alpha1
Info "Cluster logs for alpha2"
docker logs alpha2
Info "Cluster logs for alpha3"
docker logs alpha3
Info "Cluster logs for zero1"
docker logs zero1
Info "found some diffs in query results"
fi

if [[ $CLEANUP == all ]]; then
Info "bringing down zero and alpha and data volumes"
DockerCompose down -v
Expand All @@ -154,10 +168,4 @@ else
DockerCompose down
fi

if [[ $FOUND_DIFFS -eq 0 ]]; then
Info "no diffs found in query results"
else
Info "found some diffs in query results"
fi

exit $FOUND_DIFFS

0 comments on commit 36be40c

Please sign in to comment.