Support for composite indexes #376

jhchabran · 2021-03-18T11:53:39Z

This pull-request adds support for composite indexes. It uses document.Array to wrap the multiple values into one for the index stores. The changes are not that large as the line changes count suggests, this is mainly due to having added a lot of table tests on the indexes.

I have also rewritten the tests for the index iteration methods, which are now quite exhaustive.

The Benchmarks are here and are quite good (0% change on simple indexes).

database/catalog.go

database/catalog_test.go

database/config.go

database/index.go

planner/optimizer.go

query/select_test.go

stream/iterator.go

database/index.go

codecov-io · 2021-03-19T13:10:09Z

Codecov Report

Merging #376 (de1363d) into main (e19ca7a) will increase coverage by 0.26%.
The diff coverage is 77.57%.

@@            Coverage Diff             @@
##             main     #376      +/-   ##
==========================================
+ Coverage   68.53%   68.80%   +0.26%     
==========================================
  Files          74       74              
  Lines        7479     7716     +237     
==========================================
+ Hits         5126     5309     +183     
- Misses       1678     1717      +39     
- Partials      675      690      +15

Impacted Files	Coverage Δ
document/document.go	`65.18% <0.00%> (-0.74%)`	⬇️
expr/comparison.go	`70.32% <0.00%> (-3.24%)`	⬇️
database/table.go	`51.72% <61.11%> (+0.46%)`	⬆️
database/index.go	`71.21% <77.00%> (+1.12%)`	⬆️
planner/optimizer.go	`78.36% <79.84%> (+1.39%)`	⬆️
stream/iterator.go	`66.13% <80.26%> (+2.80%)`	⬆️
database/config.go	`66.95% <83.33%> (+1.38%)`	⬆️
database/catalog.go	`81.38% <86.66%> (+0.29%)`	⬆️
query/create.go	`84.61% <100.00%> (ø)`
sql/parser/create.go	`75.88% <100.00%> (-0.29%)`	⬇️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e19ca7a...de1363d. Read the comment docs.

planner/optimizer.go

database/index.go

codecov-commenter · 2021-04-26T17:11:02Z

Codecov Report

Merging #376 (d404b59) into main (e19ca7a) will decrease coverage by 0.58%.
The diff coverage is 58.10%.

@@            Coverage Diff             @@
##             main     #376      +/-   ##
==========================================
- Coverage   68.53%   67.95%   -0.59%     
==========================================
  Files          74       75       +1     
  Lines        7479     7805     +326     
==========================================
+ Hits         5126     5304     +178     
- Misses       1678     1800     +122     
- Partials      675      701      +26

Impacted Files	Coverage Δ
document/array.go	`60.86% <0.00%> (-6.72%)`	⬇️
document/document.go	`65.18% <0.00%> (-0.74%)`	⬇️
document/value.go	`73.23% <0.00%> (ø)`
expr/comparison.go	`70.32% <0.00%> (-3.24%)`	⬇️
stream/range.go	`34.04% <34.04%> (ø)`
database/table.go	`51.42% <48.27%> (+0.17%)`	⬆️
database/index.go	`65.21% <65.11%> (-4.87%)`	⬇️
planner/optimizer.go	`76.94% <80.89%> (-0.04%)`	⬇️
database/config.go	`66.95% <83.33%> (+1.38%)`	⬆️
database/catalog.go	`81.38% <86.66%> (+0.29%)`	⬆️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e19ca7a...d404b59. Read the comment docs.

asdine

I'm really excited about this, thanks @jhchabran !
I'm approving to unblock, feel free to merge once you feel it's ready

database/index.go

planner/optimizer.go

database/catalog.go

cmd/genji/dbutil/dump_test.go

database/config.go

database/table.go

query/reindex_test.go

sql/parser/create.go

asdine · 2021-04-29T08:53:11Z

planner/optimizer.go

+				// what the index says this node type must be
+				typ := idx.Info.Types[i]
+
+				fno.v, ok, err = operandCanUseIndex(typ, fno.path, info.FieldConstraints, fno.v)


This can be removed as it's already done above right?

We cannot, because the only other one is just for the primary key because it builds the candidate on the fly.

Co-authored-by: Asdine El Hrychy <[email protected]>