roachtest: tpchbench/tpch/nodes=3/cpu=4/sf=1 failed

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/d01a95b1ee71dfb36eed374619a8ed30de057ed2

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=tpchbench/tpch/nodes=3/cpu=4/sf=1 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1312970&tab=buildLog

The test failed on branch=release-2.1, cloud=gce:
	tpchbench.go:114,cluster.go:1854,errgroup.go:57: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1312970-tpchbench-tpch-nodes-3-cpu-4-sf-1:4 -- ./workload run querybench --db=tpch --concurrency=1 --query-file=tpch --num-runs=3 --max-ops=63 --vectorized=false {pgurl:1-3} --histograms=logs/stats.json returned:
		stderr:
		
		stdout:
		        0.0      0.0      0.0      0.0      0.0  8: SELECT o_year, sum(CASE WHEN nation = 'BRAZIL' THEN volume ELSE 0 END) / sum(volume) AS mkt_share FROM ( SELECT EXTRACT(year FROM o_orderdate) AS o_year, l_extendedprice * (1 - l_discount) AS volume, n2.n_name AS nation FROM part, supplier, lineitem, orders, customer, nation n1, nation n2, region WHERE p_partkey = l_partkey AND s_suppkey = l_suppkey AND l_orderkey = o_orderkey AND o_custkey = c_custkey AND c_nationkey = n1.n_nationkey AND n1.n_regionkey = r_regionkey AND r_name = 'AMERICA' AND s_nationkey = n2.n_nationkey AND o_orderdate BETWEEN DATE '1995-01-01' AND DATE '1996-12-31' AND p_type = 'ECONOMY ANODIZED STEEL') AS all_nations GROUP BY o_year ORDER BY o_year
		Error: pq: this query requires additional disk space: temp disk storage: disk budget exceeded: 1048576 bytes requested, 34359738368 currently allocated, 34359738368 bytes in budget
		Error:  ssh verbose log retained in /root/.roachprod/debug/ssh_35.237.56.114_2019-05-29T14:38:56Z: exit status 1
		: exit status 1
	cluster.go:1875,tpchbench.go:118,tpchbench.go:235,test.go:1251: Goexit() was called

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/24b7b441f56ce8d559380bd6a42bb8104c5cd68a

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=tpchbench/tpch/nodes=3/cpu=4/sf=1 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1318721&tab=buildLog

The test failed on branch=release-2.1, cloud=gce:
	tpchbench.go:114,cluster.go:1854,errgroup.go:57: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1318721-tpchbench-tpch-nodes-3-cpu-4-sf-1:4 -- ./workload run querybench --db=tpch --concurrency=1 --query-file=tpch --num-runs=3 --max-ops=63 --vectorized=false {pgurl:1-3} --histograms=logs/stats.json returned:
		stderr:
		
		stdout:
		ion, cust_nation, l_year
		3h19m50s        0            0.0            0.0      0.0      0.0      0.0      0.0  8: SELECT o_year, sum(CASE WHEN nation = 'BRAZIL' THEN volume ELSE 0 END) / sum(volume) AS mkt_share FROM ( SELECT EXTRACT(year FROM o_orderdate) AS o_year, l_extendedprice * (1 - l_discount) AS volume, n2.n_name AS nation FROM part, supplier, lineitem, orders, customer, nation n1, nation n2, region WHERE p_partkey = l_partkey AND s_suppkey = l_suppkey AND l_orderkey = o_orderkey AND o_custkey = c_custkey AND c_nationkey = n1.n_nationkey AND n1.n_regionkey = r_regionkey AND r_name = 'AMERICA' AND s_nationkey = n2.n_nationkey AND o_orderdate BETWEEN DATE '1995-01-01' AND DATE '1996-12-31' AND p_type = 'ECONOMY ANODIZED STEEL') AS all_nations GROUP BY o_year ORDER BY o_year
		Error: pq: disk budget exceeded: 1048576 bytes requested, 34359738368 currently allocated, 34359738368 bytes in budget
		Error:  ssh verbose log retained in /root/.roachprod/debug/ssh_35.231.190.109_2019-06-01T14:17:29Z: exit status 1
		: exit status 1
	cluster.go:1875,tpchbench.go:118,tpchbench.go:235,test.go:1251: Goexit() was called