colexec: messed up batch type schema (probably with CASE operator - the implementation of OR) #48622

kocoten1992 · 2020-05-09T01:39:30Z

Got this error when run a complicated query (query below)

ERROR: internal error: unexpected error from the vectorized engine: trying to add a column of bool type at index 6 but batch has width 2
SQLSTATE: XX000
DETAIL: stack trace:
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecbase/colexecerror/error.go:91: func1()
/usr/local/go/src/runtime/panic.go:679: gopanic()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecbase/colexecerror/error.go:179: InternalError()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:194: MaybeAppendColumn()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/operator.go:303: Next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/const.eg.go:149: Next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/simple_project.go:124: Next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/hashjoiner.go:353: exec()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/hashjoiner.go:241: Next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/disk_spiller.go:218: func1()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecbase/colexecerror/error.go:94: CatchVectorizedRuntimeError()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/disk_spiller.go:216: Next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/simple_project.go:124: Next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/hash_aggregator.go:234: Next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:157: next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:148: nextAdapter()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecbase/colexecerror/error.go:94: CatchVectorizedRuntimeError()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:185: Next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/rowexec/noop.go:69: Next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/columnarizer.go:112: Next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:157: next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:148: nextAdapter()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecbase/colexecerror/error.go:94: CatchVectorizedRuntimeError()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:185: Next()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/execinfra/base.go:170: Run()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/execinfra/processorsbase.go:748: Run()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/flowinfra/flow.go:369: Run()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:407: Run()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1017: PlanAndRun()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:891: execWithDistSQLEngine()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:782: dispatchToExecutionEngine()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:485: execStmtInOpenState()

HINT: You have encountered an unexpected error.

Please check the public issue tracker to check whether this problem is
already tracked. If you cannot find it there, please report the error
with details by creating a new issue.

If you would rather not post publicly, please contact us directly
using the support form.

We appreciate your feedback.

select "name" from "biseqs" where (not exists (select * from "endpoints" inner join "biseq_endpoint" on "endpoints"."id" = "biseq_endpoint"."endpoint_id" where "biseqs"."id" = "biseq_endpoint"."biseq_id" and "endpoints"."deleted_at" is null for share) or ((select count(*) from "biseq_pieces" where "biseqs"."id" = "biseq_pieces"."biseq_id") = "biseqs"."max_split" or exists (select * from "biseq_pieces" where "biseqs"."id" = "biseq_pieces"."biseq_id" and not exists (select * from "endpoints" inner join "biseq_piece_endpoint" on "endpoints"."id" = "biseq_piece_endpoint"."endpoint_id" where "biseq_pieces"."id" = "biseq_piece_endpoint"."biseq_piece_id" and "endpoints"."deleted_at" is null for share) and not exists (select * from "accounts" inner join "account_biseq_piece" on "accounts"."id" = "account_biseq_piece"."account_id" where "biseq_pieces"."id" = "account_biseq_piece"."biseq_piece_id" and "accounts"."deleted_at" is null for share)))) for share;

version

Build Tag:    e35af53
Build Time:   2020/05/08 21:27:47
Distribution: CCL
Platform:     linux amd64 (x86_64-pc-linux-gnu)
Go Version:   go1.13.9
C Compiler:   4.2.1 Compatible Clang 3.8.0 (tags/RELEASE_380/final)
Build SHA-1:  e35af53cffd031392da36040529b42af77d17ad1
Build Type:   development

Let me know if we need database schema.

The text was updated successfully, but these errors were encountered:

blathers-crl · 2020-05-09T01:39:32Z

Hello, I am Blathers. I am here to help you get the issue triaged.

It looks like you have not filled out the issue in the format of any of our templates. To best assist you, we advise you to use one of these templates.

I have CC'd a few people who may be able to assist you:

@asubiotto (found keywords: Vectorize)
@jordanlewis (found keywords: join)

If we have not gotten back to your issue within a few business days, you can try the following:

Join our community slack channel and ask on #cockroachdb.
Try find someone from here if you know they worked closely on the area and CC them.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

yuzefovich · 2020-05-09T02:31:14Z

Thanks for the report! Yes, the schema would be very helpful. Also, if you don't mind, running EXPLAIN (vec) on the query would help as well.

yuzefovich · 2020-05-09T02:44:34Z

EXPLAIN (vec, verbose) of the query would be even better.

kocoten1992 · 2020-05-09T02:46:32Z

EXPLAIN (vec, verbose) of the query would be even better.

here go, schema will post later

root@:26257/samehada> explain (vec, verbose) select "name" from "biseqs" where (not exists (select * from "endpoints" inner join "biseq_endpoint" on "endpoints"."id" = "biseq_endpoint"."endpoint_id" where "biseqs"."id" = "biseq_endpoint"."biseq_id" and "endpoints"."deleted_at" is null for share) or ((select count(*) from "biseq_pieces" where "biseqs"."id" = "biseq_pieces"."biseq_id") = "biseqs"."max_split" or exists (select * from "biseq_pieces" where "biseqs"."id" = "biseq_pieces"."biseq_id" and not exists (select * from "endpoints" inner join "biseq_piece_endpoint" on "endpoints"."id" = "biseq_piece_endpoint"."endpoint_id" where
"biseq_pieces"."id" = "biseq_piece_endpoint"."biseq_piece_id" and "endpoints"."deleted_at" is null for share) and not exists (select * from "accounts" inner join "account_biseq_piece" on "accounts"."id" = "account_biseq_piece"."account_id" where "biseq_pieces"."id" = "account_biseq_piece"."biseq_piece_id" and "accounts"."deleted_at" is null for share)))) for share;

                                              text
------------------------------------------------------------------------------------------------
  │
  └ Node 1
    └ *colexec.Materializer
      └ *colexec.Columnarizer
        └ *rowexec.noopProcessor
          └ *colexec.Materializer
            └ *colexec.hashAggregator
              └ *colexec.simpleProjectOp
                └ *colexec.diskSpillerBase
                  ├ *colexec.hashJoiner
                  │ ├ *colexec.simpleProjectOp
                  │ │ └ *colexec.constBoolOp
                  │ │   └ *colexec.vectorTypeEnforcer
                  │ │     └ *colexec.diskSpillerBase
                  │ │       ├ *colexec.hashJoiner
                  │ │       │ ├ *colexec.simpleProjectOp
                  │ │       │ │ └ *colexec.diskSpillerBase
                  │ │       │ │   ├ *colexec.hashJoiner
                  │ │       │ │   │ ├ *colexec.simpleProjectOp
                  │ │       │ │   │ │ └ *colexec.CancelChecker
                  │ │       │ │   │ │   └ *colexec.colBatchScan
                  │ │       │ │   │ └ *colexec.mergeJoinInnerOp
                  │ │       │ │   │   ├ *colexec.simpleProjectOp
                  │ │       │ │   │   │ └ *colexec.isNullSelOp
                  │ │       │ │   │   │   └ *colexec.CancelChecker
                  │ │       │ │   │   │     └ *colexec.colBatchScan
                  │ │       │ │   │   └ *colexec.simpleProjectOp
                  │ │       │ │   │     └ *colexec.CancelChecker
                  │ │       │ │   │       └ *colexec.colBatchScan
                  │ │       │ │   ├ *colexec.simpleProjectOp
                  │ │       │ │   ├ *colexec.mergeJoinInnerOp
                  │ │       │ │   └ *colexec.externalHashJoiner
                  │ │       │ │     ├ *colexec.bufferExportingOperator
                  │ │       │ │     └ *colexec.bufferExportingOperator
                  │ │       │ └ *colexec.Columnarizer
                  │ │       │   └ *rowexec.joinReader
                  │ │       │     └ *colexec.Materializer
                  │ │       │       └ *colexec.simpleProjectOp
                  │ │       │         └ *colexec.CancelChecker
                  │ │       │           └ *colexec.colBatchScan
                  │ │       ├ *colexec.simpleProjectOp
                  │ │       ├ *colexec.Columnarizer
                  │ │       └ *colexec.externalHashJoiner
                  │ │         ├ *colexec.bufferExportingOperator
                  │ │         └ *colexec.bufferExportingOperator
                  │ └ *colexec.orderedAggregator
                  │   └ *colexec.distinctChainOps
                  │     └ colexec.fnOp
                  │       └ *colexec.mergeJoinLeftOuterOp
                  │         ├ *colexec.simpleProjectOp
                  │         │ └ *colexec.isNullProjOp
                  │         │   └ *colexec.vectorTypeEnforcer
                  │         │     └ *colexec.orderedAggregator
                  │         │       └ *colexec.distinctChainOps
                  │         │         └ colexec.fnOp
                  │         │           └ *colexec.simpleProjectOp
                  │         │             └ *colexec.mergeJoinLeftOuterOp
                  │         │               ├ *colexec.simpleProjectOp
                  │         │               │ └ *colexec.CancelChecker
                  │         │               │   └ *colexec.colBatchScan
                  │         │               └ *colexec.diskSpillerBase
                  │         │                 ├ *colexec.sortOp
                  │         │                 │ └ *colexec.allSpooler
                  │         │                 │   └ *colexec.simpleProjectOp
                  │         │                 │     └ *colexec.constBoolOp
                  │         │                 │       └ *colexec.vectorTypeEnforcer
                  │         │                 │         └ *colexec.diskSpillerBase
                  │         │                 │           ├ *colexec.hashJoiner
                  │         │                 │           │ ├ *colexec.simpleProjectOp
                  │         │                 │           │ │ └ *colexec.isNullSelOp
                  │         │                 │           │ │   └ *colexec.CancelChecker
                  │         │                 │           │ │     └ *colexec.colBatchScan
                  │         │                 │           │ └ *colexec.simpleProjectOp
                  │         │                 │           │   └ *colexec.CancelChecker
                  │         │                 │           │     └ *colexec.colBatchScan
                  │         │                 │           ├ *colexec.simpleProjectOp
                  │         │                 │           ├ *colexec.simpleProjectOp
                  │         │                 │           └ *colexec.externalHashJoiner
                  │         │                 │             ├ *colexec.bufferExportingOperator
                  │         │                 │             └ *colexec.bufferExportingOperator
                  │         │                 ├ *colexec.simpleProjectOp
                  │         │                 └ *colexec.externalSorter
                  │         │                   └ *colexec.sortOp
                  │         │                     └ *colexec.allSpooler
                  │         │                       └ *colexec.inputPartitioningOperator
                  │         │                         └ *colexec.bufferExportingOperator
                  │         └ *colexec.simpleProjectOp
                  │           └ *colexec.CancelChecker
                  │             └ *colexec.colBatchScan
                  ├ *colexec.simpleProjectOp
                  ├ *colexec.orderedAggregator
                  └ *colexec.externalHashJoiner
                    ├ *colexec.bufferExportingOperator
                    └ *colexec.bufferExportingOperator
(94 rows)

Time: 16.572324ms

kocoten1992 · 2020-05-09T05:26:38Z

Have you able to deduce from the above @yuzefovich ?

Turn out it real difficult to have a reduce test case, just the schema alone wont be enough, some random data I put in won't crash cockroachdb either, should I just dump database and give it to you ? (it relative small)

P/s: if yes, then can I have your email ?

yuzefovich · 2020-05-09T05:37:54Z

Eye-balling the plan I don't see what the problem is, so having a reproduction would be immensely helpful if you're ok with sharing the data. My email is [email protected].

In the meantime, if this issue is blocking you, you could disable vectorization with SET vectorize=off.

kocoten1992 · 2020-05-10T01:44:00Z

Sent!

Another issue I've mention in email about cockroach dump command, is this an expected behavior though ?

yuzefovich · 2020-05-10T03:53:29Z

Thanks, I'll take a look!

Regardless the difficulties with cockroach dump on non-official binary, I'm not very familiar with the tool, but this is where the error is coming from https://github.com/cockroachdb/cockroach/blob/master/pkg/cli/sql_util.go#L310 which is called by https://github.com/cockroachdb/cockroach/blob/master/pkg/cli/dump.go#L96, so I'm assuming it is "expected" at the moment. However, I do agree with you that such limitation is unfortunate, so please open up a separate issue about that.

yuzefovich · 2020-05-10T05:35:14Z

Thanks to your reproduction I was able to spot a bug pretty quickly, fix incoming :)

48552: geo/geomfn: implement ST_Relate and ST_ContainsProperly r=sumeerbhola a=otan Commit on top of #48522. Added ST_ContainsProperly to the optimizer as well that calls it to use Covers. Also update the RFC to claim ST_ContainsProperly as indexed backed. Release note (sql change): Implemented the geometry based builtins `ST_Relate` and `ST_ContainsProperly`. 48659: colexec: fix type schema for LEFT SEMI and LEFT ANTI joins r=yuzefovich a=yuzefovich LEFT SEMI and LEFT ANTI joins output only all the columns from the left, however, we mistakenly put the columns from the right into `result.ColumnTypes`. Fixes: #48622. Release note (bug fix): Previously, CockroachDB could encounter an internal error when a query with LEFT SEMI or LEFT ANTI join was performed via the vectorized execution engine in some cases, and now this has been fixed. This is likely to occur only with `vectorize=on` setting. Co-authored-by: Oliver Tan <[email protected]> Co-authored-by: Yahor Yuzefovich <[email protected]>

blathers-crl bot added O-community Originated from the community X-blathers-triaged blathers was able to find an owner labels May 9, 2020

yuzefovich changed the title ~~internal error: unexpected error from the vectorized engine: trying to add a column of bool type at index 6 but batch has width 2~~ colexec: messed up batch type schema (probably with CASE operator - the implementation of OR) May 9, 2020

yuzefovich self-assigned this May 9, 2020

yuzefovich mentioned this issue May 10, 2020

colexec: fix type schema for LEFT SEMI and LEFT ANTI joins #48659

Merged

craig bot closed this as completed in 12696df May 12, 2020

yuzefovich mentioned this issue May 12, 2020

release-20.1: colexec: fix type schema for LEFT SEMI and LEFT ANTI joins #48751

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

colexec: messed up batch type schema (probably with CASE operator - the implementation of OR) #48622

colexec: messed up batch type schema (probably with CASE operator - the implementation of OR) #48622

kocoten1992 commented May 9, 2020 •

edited

Loading

blathers-crl bot commented May 9, 2020

yuzefovich commented May 9, 2020

yuzefovich commented May 9, 2020

kocoten1992 commented May 9, 2020

kocoten1992 commented May 9, 2020 •

edited

Loading

yuzefovich commented May 9, 2020

kocoten1992 commented May 10, 2020

yuzefovich commented May 10, 2020

yuzefovich commented May 10, 2020

colexec: messed up batch type schema (probably with CASE operator - the implementation of OR) #48622

colexec: messed up batch type schema (probably with CASE operator - the implementation of OR) #48622

Comments

kocoten1992 commented May 9, 2020 • edited Loading

blathers-crl bot commented May 9, 2020

yuzefovich commented May 9, 2020

yuzefovich commented May 9, 2020

kocoten1992 commented May 9, 2020

kocoten1992 commented May 9, 2020 • edited Loading

yuzefovich commented May 9, 2020

kocoten1992 commented May 10, 2020

yuzefovich commented May 10, 2020

yuzefovich commented May 10, 2020

kocoten1992 commented May 9, 2020 •

edited

Loading

kocoten1992 commented May 9, 2020 •

edited

Loading