roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70011

cockroach-teamcity · 2021-09-10T09:14:37Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ 62c5be51bbb3beeb51638032871365585d69f75d:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:657,follower_reads.go:319,follower_reads.go:57,test_runner.go:777: too many intervals with more than 2 nodes with low follower read ratios: 5 intervals > 4 threshold. Bad intervals:
		interval 09:08:40-09:08:50: n1 ratio: 0.000 n2 ratio: 0.355 n3 ratio: 0.724 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 1.000 
		interval 09:09:30-09:09:40: n1 ratio: 1.000 n2 ratio: 0.694 n3 ratio: 0.120 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.650 
		interval 09:10:00-09:10:10: n1 ratio: 1.000 n2 ratio: 0.468 n3 ratio: 0.000 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.627 
		interval 09:10:20-09:10:30: n1 ratio: 1.000 n2 ratio: 0.853 n3 ratio: 0.161 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.686 
		interval 09:10:50-09:11:00: n1 ratio: 1.000 n2 ratio: 0.508 n3 ratio: 0.124 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.628

Reproduce

See: roachtest README

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!

Epic CRDB-10559}

The text was updated successfully, but these errors were encountered:

cockroach-teamcity · 2021-09-12T10:03:19Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ 6eb9df1f3d460f904b5254c3d32c745eecbedd3f:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:657,follower_reads.go:319,follower_reads.go:57,test_runner.go:777: too many intervals with more than 2 nodes with low follower read ratios: 6 intervals > 4 threshold. Bad intervals:
		interval 09:55:10-09:55:20: n1 ratio: 0.201 n2 ratio: 1.032 n3 ratio: 2.073 n4 ratio: 1.551 n5 ratio: 0.000 n6 ratio: 0.300 
		interval 09:55:20-09:55:30: n1 ratio: 0.000 n2 ratio: 0.412 n3 ratio: 2.475 n4 ratio: 1.000 n5 ratio: 0.895 n6 ratio: 0.449 
		interval 09:56:00-09:56:10: n1 ratio: 0.000 n2 ratio: 0.000 n3 ratio: 1.978 n4 ratio: 1.000 n5 ratio: 0.856 n6 ratio: 1.000 
		interval 09:57:10-09:57:20: n1 ratio: 0.312 n2 ratio: 0.119 n3 ratio: 0.567 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 1.000 
		interval 09:57:30-09:57:40: n1 ratio: 1.000 n2 ratio: 0.000 n3 ratio: 0.000 n4 ratio: 1.000 n5 ratio: 0.875 n6 ratio: 1.000 
		interval 09:58:30-09:58:40: n1 ratio: 1.000 n2 ratio: 0.000 n3 ratio: 0.000 n4 ratio: 1.000 n5 ratio: 0.783 n6 ratio: 1.000

Reproduce

See: roachtest README

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-09-13T09:04:52Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ 394e35cbbb7ae466383c3ee6f2bbcc76857afeed:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:657,follower_reads.go:319,follower_reads.go:57,test_runner.go:777: too many intervals with more than 2 nodes with low follower read ratios: 6 intervals > 4 threshold. Bad intervals:
		interval 08:58:30-08:58:40: n1 ratio: 0.000 n2 ratio: 0.154 n3 ratio: 0.816 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 1.000 
		interval 08:59:10-08:59:20: n1 ratio: 0.000 n2 ratio: 0.141 n3 ratio: 1.000 n4 ratio: 1.000 n5 ratio: 0.883 n6 ratio: 1.000 
		interval 08:59:30-08:59:40: n1 ratio: 0.260 n2 ratio: 0.000 n3 ratio: 1.000 n4 ratio: 1.000 n5 ratio: 0.808 n6 ratio: 1.000 
		interval 08:59:50-09:00:00: n1 ratio: 0.276 n2 ratio: 0.011 n3 ratio: 1.179 n4 ratio: 1.000 n5 ratio: 0.807 n6 ratio: 1.000 
		interval 09:00:40-09:00:50: n1 ratio: 0.000 n2 ratio: 0.152 n3 ratio: 1.000 n4 ratio: 1.000 n5 ratio: 0.818 n6 ratio: 1.000 
		interval 09:01:20-09:01:30: n1 ratio: 0.000 n2 ratio: 0.000 n3 ratio: 1.000 n4 ratio: 1.000 n5 ratio: 0.850 n6 ratio: 1.000

Reproduce

See: roachtest README

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-09-20T10:37:32Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ d9278939d4be4d1109e9ed0a84c6e9cd2a8705bb:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:657,follower_reads.go:319,follower_reads.go:57,test_runner.go:777: too many intervals with more than 2 nodes with low follower read ratios: 6 intervals > 4 threshold. Bad intervals:
		interval 10:30:10-10:30:20: n1 ratio: 0.499 n2 ratio: 0.000 n3 ratio: 1.000 n4 ratio: 1.654 n5 ratio: 0.107 n6 ratio: 1.000 
		interval 10:30:40-10:30:50: n1 ratio: 0.116 n2 ratio: 0.133 n3 ratio: 1.000 n4 ratio: 1.000 n5 ratio: 0.844 n6 ratio: 1.000 
		interval 10:32:00-10:32:10: n1 ratio: 0.000 n2 ratio: 0.107 n3 ratio: 1.217 n4 ratio: 1.000 n5 ratio: 0.777 n6 ratio: 1.000 
		interval 10:32:30-10:32:40: n1 ratio: 0.000 n2 ratio: 0.183 n3 ratio: 1.000 n4 ratio: 1.000 n5 ratio: 0.805 n6 ratio: 1.000 
		interval 10:33:10-10:33:20: n1 ratio: 0.000 n2 ratio: 0.077 n3 ratio: 1.169 n4 ratio: 1.000 n5 ratio: 0.880 n6 ratio: 1.000 
		interval 10:33:40-10:33:50: n1 ratio: 0.000 n2 ratio: 0.089 n3 ratio: 1.000 n4 ratio: 1.000 n5 ratio: 0.892 n6 ratio: 0.996

Reproduce

See: roachtest README

Same failure on other branches

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70110 roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed [C-test-failure O-roachtest O-robot branch-release-21.2 release-blocker]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-09-21T09:51:12Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ 676dc076dc6a137d88eac9217764bbb63beae86f:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:657,follower_reads.go:319,follower_reads.go:57,test_runner.go:777: too many intervals with more than 2 nodes with low follower read ratios: 6 intervals > 4 threshold. Bad intervals:
		interval 09:43:50-09:44:00: n1 ratio: 1.640 n2 ratio: 0.682 n3 ratio: 0.000 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.509 
		interval 09:45:20-09:45:30: n1 ratio: 0.172 n2 ratio: 1.000 n3 ratio: 0.000 n4 ratio: 0.779 n5 ratio: 1.000 n6 ratio: 1.000 
		interval 09:45:50-09:46:00: n1 ratio: 0.149 n2 ratio: 1.000 n3 ratio: 0.473 n4 ratio: 0.727 n5 ratio: 1.000 n6 ratio: 1.000 
		interval 09:46:30-09:46:40: n1 ratio: 0.277 n2 ratio: 1.000 n3 ratio: 0.000 n4 ratio: 0.753 n5 ratio: 1.000 n6 ratio: 1.003 
		interval 09:46:40-09:46:50: n1 ratio: 0.121 n2 ratio: 1.000 n3 ratio: 0.243 n4 ratio: 0.834 n5 ratio: 1.000 n6 ratio: 0.997 
		interval 09:47:30-09:47:40: n1 ratio: 0.109 n2 ratio: 1.000 n3 ratio: 0.050 n4 ratio: 0.886 n5 ratio: 1.000 n6 ratio: 1.000

Reproduce

See: roachtest README

Same failure on other branches

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70110 roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed [C-test-failure O-roachtest O-robot branch-release-21.2 release-blocker]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-09-26T09:54:02Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ ecaa4920192549af1418923648769ede3de1c9c8:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:545,follower_reads.go:327,follower_reads.go:57,test_runner.go:777: 20 latency values ([21.03705ms 20.578298ms 20.05401ms 20.840442ms 20.250618ms 20.316154ms 20.840442ms 23.199738ms 22.806522ms 22.544378ms 21.102586ms 21.233658ms 21.102586ms 21.233658ms 21.233658ms 21.233658ms 22.413306ms 23.396346ms 23.265274ms 20.119546ms]) are above target latency 20ms, 4 permitted

Reproduce

See: roachtest README

Same failure on other branches

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70110 roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed [C-test-failure O-roachtest O-robot branch-release-21.2]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-09-28T09:32:37Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ 80844e44f4f9d0e565c6e810bd4c782be61899bf:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:545,follower_reads.go:327,follower_reads.go:57,test_runner.go:777: 5 latency values ([21.102586ms 20.971514ms 20.512762ms 20.119546ms 20.905978ms]) are above target latency 20ms, 4 permitted

Reproduce

See: roachtest README

Same failure on other branches

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70110 roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed [C-test-failure O-roachtest O-robot branch-release-21.2]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-09-29T09:38:13Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ ab0c462672cff1e6928aba2dfc3a5429466aea39:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:657,follower_reads.go:319,follower_reads.go:57,test_runner.go:777: too many intervals with more than 2 nodes with low follower read ratios: 5 intervals > 4 threshold. Bad intervals:
		interval 09:30:50-09:31:00: n1 ratio: 0.865 n2 ratio: 0.330 n3 ratio: 0.551 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 1.000 
		interval 09:31:50-09:32:00: n1 ratio: 0.149 n2 ratio: 0.495 n3 ratio: 1.186 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.770 
		interval 09:32:00-09:32:10: n1 ratio: 0.141 n2 ratio: 0.097 n3 ratio: 1.000 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.825 
		interval 09:32:40-09:32:50: n1 ratio: 0.186 n2 ratio: 0.322 n3 ratio: 1.343 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.647 
		interval 09:33:30-09:33:40: n1 ratio: 0.226 n2 ratio: 0.396 n3 ratio: 1.000 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.678

Reproduce

See: roachtest README

Same failure on other branches

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70110 roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed [C-test-failure O-roachtest O-robot branch-release-21.2]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-09-30T10:12:54Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ ac885449c1fe74de23ea5d82858b3aa3674496f9:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:545,follower_reads.go:327,follower_reads.go:57,test_runner.go:777: 5 latency values ([20.447226ms 20.70937ms 22.151162ms 20.643834ms 20.643834ms]) are above target latency 20ms, 4 permitted

Reproduce

See: roachtest README

Same failure on other branches

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70110 roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed [C-test-failure O-roachtest O-robot branch-release-21.2]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-10-01T09:38:32Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ 5f53feb1ce7070453e9ce52d99013934718cc9d7:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:545,follower_reads.go:327,follower_reads.go:57,test_runner.go:777: 22 latency values ([21.495802ms 21.561338ms 21.495802ms 22.02009ms 21.954554ms 21.954554ms 21.561338ms 22.872058ms 24.575994ms 24.117242ms 23.65849ms 22.806522ms 21.430266ms 21.36473ms 21.69241ms 24.379386ms 24.182778ms 24.182778ms 24.96921ms 24.379386ms 24.510458ms 23.461882ms]) are above target latency 20ms, 4 permitted

Reproduce

See: roachtest README

Same failure on other branches

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70110 roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed [C-test-failure O-roachtest O-robot branch-release-21.2]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-10-03T10:04:01Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ bc42b6626366da8b2cdd470382dd5fb8202ba0c5:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:657,follower_reads.go:319,follower_reads.go:57,test_runner.go:777: too many intervals with more than 2 nodes with low follower read ratios: 5 intervals > 4 threshold. Bad intervals:
		interval 09:57:30-09:57:40: n1 ratio: 0.414 n2 ratio: 0.302 n3 ratio: 1.000 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.679 
		interval 09:57:50-09:58:00: n1 ratio: 0.116 n2 ratio: 0.184 n3 ratio: 1.637 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.792 
		interval 09:58:30-09:58:40: n1 ratio: 0.209 n2 ratio: 0.127 n3 ratio: 1.218 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 0.772 
		interval 09:59:00-09:59:10: n1 ratio: 0.220 n2 ratio: 0.083 n3 ratio: 1.501 n4 ratio: 1.000 n5 ratio: 1.003 n6 ratio: 0.858 
		interval 09:59:30-09:59:40: n1 ratio: 0.294 n2 ratio: 0.248 n3 ratio: 0.732 n4 ratio: 1.000 n5 ratio: 1.000 n6 ratio: 1.000

Reproduce

See: roachtest README

Same failure on other branches

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70110 roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed [C-test-failure O-roachtest O-robot branch-release-21.2]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-10-05T10:19:30Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ 9013165fa81c4cddbe38b73c8b429faa3de226e3:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:545,follower_reads.go:327,follower_reads.go:57,test_runner.go:777: 17 latency values ([21.823482ms 21.69241ms 22.609914ms 24.248314ms 23.855098ms 21.102586ms 22.282234ms 22.478842ms 22.413306ms 21.823482ms 23.068666ms 23.920634ms 24.182778ms 23.920634ms 24.117242ms 21.823482ms 21.69241ms]) are above target latency 20ms, 4 permitted

Reproduce

See: roachtest README

Same failure on other branches

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70110 roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed [C-test-failure O-roachtest O-robot branch-release-21.2]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-10-06T10:18:39Z

roachtest.follower-reads/survival=region/locality=regional/reads=bounded-staleness failed with artifacts on master @ aef9aa08e12d50ca6401440cac5b57d5a5d6b720:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/survival=region/locality=regional/reads=bounded-staleness/run_1
	follower_reads.go:752,follower_reads.go:364,follower_reads.go:71,test_runner.go:777: too many intervals with more than 2 nodes with low follower read ratios: 5 intervals > 4 threshold. Bad intervals:
		interval 10:11:30-10:11:40: n1 ratio: 1.000 n2 ratio: 0.000 n3 ratio: 1.476 n4 ratio: 0.318 n5 ratio: 0.167 n6 ratio: 1.000 
		interval 10:12:20-10:12:30: n1 ratio: 1.000 n2 ratio: 0.000 n3 ratio: 1.698 n4 ratio: 0.000 n5 ratio: 0.870 n6 ratio: 1.000 
		interval 10:13:10-10:13:20: n1 ratio: 1.000 n2 ratio: 0.000 n3 ratio: 1.095 n4 ratio: 0.000 n5 ratio: 0.766 n6 ratio: 1.000 
		interval 10:13:20-10:13:30: n1 ratio: 1.000 n2 ratio: 0.000 n3 ratio: 1.556 n4 ratio: 0.676 n5 ratio: 0.247 n6 ratio: 1.000 
		interval 10:14:10-10:14:20: n1 ratio: 1.000 n2 ratio: 0.000 n3 ratio: 1.000 n4 ratio: 0.000 n5 ratio: 0.705 n6 ratio: 1.000

Reproduce

See: roachtest README

Same failure on other branches

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70110 roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed [C-test-failure O-roachtest O-robot branch-release-21.2]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

tbg · 2021-10-20T13:18:35Z

The "x latency values [...] are above target latency 20ms" failure mode is described here: #70110 (comment)

Its resolution is tracked in #71694.

tbg · 2021-10-20T13:22:08Z

Note: these tests are currently skipped.

This setting had rotted, leading to a span use-after-Finish. This was caught by the recently-unskipped follower reads roachtests (roachtests panic on use-after-finish), accounting for all the issues below. The last entry in these issues refers to the nodes crashing because of the bug fixed here. Some of these issues have been open for a long time for more relevant failures - those have been fixed in cockroachdb#72296. Fixes cockroachdb#75716 Fixes cockroachdb#75715 Fixes cockroachdb#75714 Fixes cockroachdb#71244 Fixes cockroachdb#71126 Fixes cockroachdb#70818 Fixes cockroachdb#70191 Fixes cockroachdb#70011 Fixes cockroachdb#70010 Fixes cockroachdb#69952 Fixes cockroachdb#69951 Release note: None

75739: sql: fix the sql.trace.stmt.enable_threshold cluster setting r=andreimatei a=andreimatei This setting had rotted, leading to a span use-after-Finish. This was caught by the recently-unskipped follower reads roachtests (roachtests panic on use-after-finish), accounting for all the issues below. The last entry in these issues refers to the nodes crashing because of the bug fixed here. Some of these issues have been open for a long time for more relevant failures - those have been fixed in #72296. Fixes #75716 Fixes #75715 Fixes #75714 Fixes #71244 Fixes #71126 Fixes #70818 Fixes #70191 Fixes #70011 Fixes #70010 Fixes #69952 Fixes #69951 Release note: None 75764: roachtest: add a variant of tpch_concurrency with no admission control r=yuzefovich a=yuzefovich Informs: #74836. Release note: None Co-authored-by: Andrei Matei <[email protected]> Co-authored-by: Yahor Yuzefovich <[email protected]>

cockroach-teamcity mentioned this issue Sep 13, 2021

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70110

Closed

nvanbenschoten mentioned this issue Sep 14, 2021

roachtest: follower-reads/survival=zone/locality=regional/reads=exact-staleness failed #69817

Closed

nvanbenschoten removed the release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. label Sep 21, 2021

exalate-issue-sync bot added the T-kv KV Team label Oct 11, 2021

tbg added the skipped-test label Oct 20, 2021

cockroach-teamcity mentioned this issue Dec 18, 2021

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #74020

Closed

cockroach-teamcity mentioned this issue Jan 13, 2022

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #74793

Closed

tbg assigned andreimatei Jan 31, 2022

andreimatei mentioned this issue Jan 31, 2022

sql: fix the sql.trace.stmt.enable_threshold cluster setting #75739

Merged

craig bot closed this as completed in #75739 Feb 1, 2022

craig bot closed this as completed in f5d78e2 Feb 1, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70011

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70011

cockroach-teamcity commented Sep 10, 2021 •

edited by exalate-issue-sync bot

Loading

cockroach-teamcity commented Sep 12, 2021

cockroach-teamcity commented Sep 13, 2021

cockroach-teamcity commented Sep 20, 2021

cockroach-teamcity commented Sep 21, 2021

cockroach-teamcity commented Sep 26, 2021

cockroach-teamcity commented Sep 28, 2021

cockroach-teamcity commented Sep 29, 2021

cockroach-teamcity commented Sep 30, 2021

cockroach-teamcity commented Oct 1, 2021

cockroach-teamcity commented Oct 3, 2021

cockroach-teamcity commented Oct 5, 2021

cockroach-teamcity commented Oct 6, 2021

tbg commented Oct 20, 2021

tbg commented Oct 20, 2021

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70011

roachtest: follower-reads/survival=region/locality=regional/reads=bounded-staleness failed #70011

Comments

cockroach-teamcity commented Sep 10, 2021 • edited by exalate-issue-sync bot Loading

cockroach-teamcity commented Sep 12, 2021

cockroach-teamcity commented Sep 13, 2021

cockroach-teamcity commented Sep 20, 2021

cockroach-teamcity commented Sep 21, 2021

cockroach-teamcity commented Sep 26, 2021

cockroach-teamcity commented Sep 28, 2021

cockroach-teamcity commented Sep 29, 2021

cockroach-teamcity commented Sep 30, 2021

cockroach-teamcity commented Oct 1, 2021

cockroach-teamcity commented Oct 3, 2021

cockroach-teamcity commented Oct 5, 2021

cockroach-teamcity commented Oct 6, 2021

tbg commented Oct 20, 2021

tbg commented Oct 20, 2021

cockroach-teamcity commented Sep 10, 2021 •

edited by exalate-issue-sync bot

Loading