-
Notifications
You must be signed in to change notification settings - Fork 3.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
25297: build: add roachtests to be run on every PR r=benesch,petermattis a=danhhz
Release note: None
25434: opt: Change how Any operator is represented and flattened r=andy-kimball a=andy-kimball
The Any operator currently takes a single input rowset, and expects
it to return a single boolean column. The hoister flattens the Any
operator by testing and projecting that boolean column. The problem
is that this representation cannot easily be decorrelated. Example:
z = ANY(SELECT x FROM xy)
This is currently represented as:
(Any (Project xy [ z=x ]))
The z=x projection field cannot easily be hoisted over a left join.
This commit uses an alternate representation:
(Any xy z EqOp)
The new representation keeps the input, scalar, and comparison op
components separate, so they can be combined in ways that it easier
to decorrelate.
25456: storage: fix deadlock in consistency queue r=bdarnell,a-robinson a=tschottdorf
When `CheckConsistency` returns an error, the queue checks whether the
store is draining to decide whether the error is worth logging.
Unfortunately this check was incorrect and would block until the store
actually started draining.
A toy example of this problem is below (this will deadlock). The dual
return form of chan receive isn't non-blocking -- the second parameter
indicates whether the received value corresponds to a closing of the
channel.
Switch to a `select` instead.
```go
package main
import (
"fmt"
)
func main() {
ch := make(chan struct{})
_, ok := <-ch
fmt.Println(ok)
}
```
Touches #21824.
Release note (bug fix): Prevent the consistency checker from
deadlocking. This would previously manifest itself as a steady number of
replicas queued for consistency checking on one or more nodes and would
resolve by restarting the affected nodes.
Co-authored-by: Daniel Harrison <[email protected]>
Co-authored-by: Andrew Kimball <[email protected]>
Co-authored-by: Tobias Schottdorf <[email protected]>- Loading branch information
Showing
43 changed files
with
1,935 additions
and
1,100 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| source "$(dirname "${0}")/teamcity-support.sh" | ||
|
|
||
| tc_start_block "Prepare environment" | ||
| run mkdir -p artifacts | ||
| maybe_ccache | ||
| tc_end_block "Prepare environment" | ||
|
|
||
| tc_start_block "Install roachprod" | ||
| run build/builder.sh go get -u -v github.com/cockroachdb/roachprod | ||
| tc_end_block "Install roachprod" | ||
|
|
||
| tc_start_block "Compile CockroachDB" | ||
| run build/builder.sh make build | ||
| tc_end_block "Compile CockroachDB" | ||
|
|
||
| tc_start_block "Compile workload/roachtest" | ||
| run build/builder.sh make bin/workload bin/roachtest | ||
| tc_end_block "Compile workload/roachtest" | ||
|
|
||
| tc_start_block "Run local roachtests" | ||
| # TODO(dan): Run kv/splits as a proof of concept of running roachtest on every | ||
| # PR. After we're sure this is stable, curate a suite of the tests that work | ||
| # locally. | ||
| run build/builder.sh ./bin/roachtest run kv/splits \ | ||
| --local \ | ||
| --cockroach "cockroach" \ | ||
| --workload "bin/workload" \ | ||
| --artifacts artifacts \ | ||
| --teamcity | ||
| tc_end_block "Run local roachtests" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,64 +1,128 @@ | ||
| exec-ddl | ||
| CREATE TABLE a (x INT PRIMARY KEY, y INT) | ||
| CREATE TABLE xy (x INT PRIMARY KEY, y INT) | ||
| ---- | ||
| TABLE a | ||
| TABLE xy | ||
| ├── x int not null | ||
| ├── y int | ||
| └── INDEX primary | ||
| └── x int not null | ||
|
|
||
| exec-ddl | ||
| CREATE TABLE b (x INT, z INT NOT NULL) | ||
| CREATE TABLE uv (u INT, v INT NOT NULL) | ||
| ---- | ||
| TABLE b | ||
| ├── x int | ||
| ├── z int not null | ||
| TABLE uv | ||
| ├── u int | ||
| ├── v int not null | ||
| ├── rowid int not null (hidden) | ||
| └── INDEX primary | ||
| └── rowid int not null (hidden) | ||
|
|
||
| build | ||
| SELECT * FROM a WHERE x < 5 | ||
| SELECT * FROM xy WHERE x < 5 | ||
| ---- | ||
| select | ||
| ├── columns: x:1(int!null) y:2(int) | ||
| ├── stats: [rows=333] | ||
| ├── keys: (1) | ||
| ├── scan a | ||
| │ ├── columns: a.x:1(int!null) a.y:2(int) | ||
| ├── scan xy | ||
| │ ├── columns: xy.x:1(int!null) xy.y:2(int) | ||
| │ ├── stats: [rows=1000] | ||
| │ └── keys: (1) | ||
| └── lt [type=bool, outer=(1), constraints=(/1: (/NULL - /4]; tight)] | ||
| ├── variable: a.x [type=int, outer=(1)] | ||
| ├── variable: xy.x [type=int, outer=(1)] | ||
| └── const: 5 [type=int] | ||
|
|
||
| build | ||
| SELECT a.x + 1 = length('foo') + a.y, b.rowid * a.x FROM a, b | ||
| SELECT xy.x + 1 = length('foo') + xy.y, uv.rowid * xy.x FROM xy, uv | ||
| ---- | ||
| project | ||
| ├── columns: column6:6(bool) column7:7(int) | ||
| ├── stats: [rows=1000000] | ||
| ├── inner-join | ||
| │ ├── columns: a.x:1(int!null) a.y:2(int) b.x:3(int) b.z:4(int!null) b.rowid:5(int!null) | ||
| │ ├── columns: xy.x:1(int!null) xy.y:2(int) uv.u:3(int) uv.v:4(int!null) uv.rowid:5(int!null) | ||
| │ ├── stats: [rows=1000000] | ||
| │ ├── scan a | ||
| │ │ ├── columns: a.x:1(int!null) a.y:2(int) | ||
| │ ├── scan xy | ||
| │ │ ├── columns: xy.x:1(int!null) xy.y:2(int) | ||
| │ │ ├── stats: [rows=1000] | ||
| │ │ └── keys: (1) | ||
| │ ├── scan b | ||
| │ │ ├── columns: b.x:3(int) b.z:4(int!null) b.rowid:5(int!null) | ||
| │ ├── scan uv | ||
| │ │ ├── columns: uv.u:3(int) uv.v:4(int!null) uv.rowid:5(int!null) | ||
| │ │ ├── stats: [rows=1000] | ||
| │ │ └── keys: (5) | ||
| │ └── true [type=bool] | ||
| └── projections [outer=(1,2,5)] | ||
| ├── eq [type=bool, outer=(1,2)] | ||
| │ ├── plus [type=int, outer=(1)] | ||
| │ │ ├── variable: a.x [type=int, outer=(1)] | ||
| │ │ ├── variable: xy.x [type=int, outer=(1)] | ||
| │ │ └── const: 1 [type=int] | ||
| │ └── plus [type=int, outer=(2)] | ||
| │ ├── function: length [type=int] | ||
| │ │ └── const: 'foo' [type=string] | ||
| │ └── variable: a.y [type=int, outer=(2)] | ||
| │ └── variable: xy.y [type=int, outer=(2)] | ||
| └── mult [type=int, outer=(1,5)] | ||
| ├── variable: b.rowid [type=int, outer=(5)] | ||
| └── variable: a.x [type=int, outer=(1)] | ||
| ├── variable: uv.rowid [type=int, outer=(5)] | ||
| └── variable: xy.x [type=int, outer=(1)] | ||
|
|
||
| build | ||
| SELECT * FROM xy WHERE EXISTS(SELECT * FROM uv WHERE u=x) | ||
| ---- | ||
| select | ||
| ├── columns: x:1(int!null) y:2(int) | ||
| ├── stats: [rows=333] | ||
| ├── keys: (1) | ||
| ├── scan xy | ||
| │ ├── columns: xy.x:1(int!null) xy.y:2(int) | ||
| │ ├── stats: [rows=1000] | ||
| │ └── keys: (1) | ||
| └── exists [type=bool, outer=(1)] | ||
| └── project | ||
| ├── columns: uv.u:3(int) uv.v:4(int!null) | ||
| ├── outer: (1) | ||
| ├── stats: [rows=333] | ||
| ├── select | ||
| │ ├── columns: uv.u:3(int) uv.v:4(int!null) uv.rowid:5(int!null) | ||
| │ ├── outer: (1) | ||
| │ ├── stats: [rows=333] | ||
| │ ├── keys: (5) | ||
| │ ├── scan uv | ||
| │ │ ├── columns: uv.u:3(int) uv.v:4(int!null) uv.rowid:5(int!null) | ||
| │ │ ├── stats: [rows=1000] | ||
| │ │ └── keys: (5) | ||
| │ └── eq [type=bool, outer=(1,3)] | ||
| │ ├── variable: uv.u [type=int, outer=(3)] | ||
| │ └── variable: xy.x [type=int, outer=(1)] | ||
| └── projections [outer=(3,4)] | ||
| ├── variable: uv.u [type=int, outer=(3)] | ||
| └── variable: uv.v [type=int, outer=(4)] | ||
|
|
||
| build | ||
| SELECT * FROM xy WHERE y IN (SELECT v FROM uv WHERE u=x) | ||
| ---- | ||
| select | ||
| ├── columns: x:1(int!null) y:2(int) | ||
| ├── stats: [rows=333] | ||
| ├── keys: (1) | ||
| ├── scan xy | ||
| │ ├── columns: xy.x:1(int!null) xy.y:2(int) | ||
| │ ├── stats: [rows=1000] | ||
| │ └── keys: (1) | ||
| └── any: eq [type=bool, outer=(1,2)] | ||
| ├── project | ||
| │ ├── columns: uv.v:4(int!null) | ||
| │ ├── outer: (1) | ||
| │ ├── stats: [rows=333] | ||
| │ ├── select | ||
| │ │ ├── columns: uv.u:3(int) uv.v:4(int!null) uv.rowid:5(int!null) | ||
| │ │ ├── outer: (1) | ||
| │ │ ├── stats: [rows=333] | ||
| │ │ ├── keys: (5) | ||
| │ │ ├── scan uv | ||
| │ │ │ ├── columns: uv.u:3(int) uv.v:4(int!null) uv.rowid:5(int!null) | ||
| │ │ │ ├── stats: [rows=1000] | ||
| │ │ │ └── keys: (5) | ||
| │ │ └── eq [type=bool, outer=(1,3)] | ||
| │ │ ├── variable: uv.u [type=int, outer=(3)] | ||
| │ │ └── variable: xy.x [type=int, outer=(1)] | ||
| │ └── projections [outer=(4)] | ||
| │ └── variable: uv.v [type=int, outer=(4)] | ||
| └── variable: xy.y [type=int, outer=(2)] |
Oops, something went wrong.