Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql: v21.2.14: interface conversion: tree.Datum is tree.dNull, not *tree.DInt #87371

Closed
cockroach-teamcity opened this issue Sep 5, 2022 · 2 comments · Fixed by #88182
Closed

sql: v21.2.14: interface conversion: tree.Datum is tree.dNull, not *tree.DInt #87371

cockroach-teamcity opened this issue Sep 5, 2022 · 2 comments · Fixed by #88182
Assignees
@yuzefovich
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. E-easy Easy issue to tackle, requires little or no CockroachDB experience E-quick-win Likely to be a quick win for someone experienced. O-sentry Originated from an in-the-wild panic report.

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Sep 5, 2022 •
edited by cockroach-jira-scripts
Loading

This issue was autofiled by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry link: https://sentry.io/organizations/cockroach-labs/issues/3561751054/?referrer=webhooks_plugin

Panic message:

conn_executor.go:668: interface conversion: tree.Datum is tree.dNull, not *tree.DInt
(1) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn.func1
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:668
| [...repeated from below...]
Wraps: (2) while executing: ALTER TABLE _ EXPERIMENTAL_RELOCATE VOTERS SELECT ARRAY[_, , more1], _
Wraps: (3) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn.func1
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:668
| runtime.gopanic
| /usr/local/go/src/runtime/panic.go:965
| github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:58
| runtime.gopanic
| /usr/local/go/src/runtime/panic.go:965
| github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:58
| runtime.gopanic
| /usr/local/go/src/runtime/panic.go:965
| runtime.panicdottypeE
| /usr/local/go/src/runtime/iface.go:261
| runtime.panicdottypeI
| /usr/local/go/src/runtime/iface.go:271
| github.com/cockroachdb/cockroach/pkg/sql.(*relocateNode).Next
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/relocate.go:90
| github.com/cockroachdb/cockroach/pkg/sql.(*planNodeToRowSource).Next
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/plan_node_to_row_source.go:187
| github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Columnarizer).Next
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/columnarizer.go:222
| github.com/cockroachdb/cockroach/pkg/sql/colflow.(*batchInfoCollector).next
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:98
| github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:91
| github.com/cockroachdb/cockroach/pkg/sql/colflow.(*batchInfoCollector).Next
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:106
| github.com/cockroachdb/cockroach/pkg/sql/colflow.(*BatchFlowCoordinator).nextAdapter
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:239
| github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:91
| github.com/cockroachdb/cockroach/pkg/sql/colflow.(*BatchFlowCoordinator).next
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:243
| github.com/cockroachdb/cockroach/pkg/sql/colflow.(*BatchFlowCoordinator).Run
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:282
| github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlow).Run
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:258
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:548
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1289
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1386
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1074
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:709
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:129
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd.func1
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1720
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1722
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1644
| github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:670
| github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync.func1
| /go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:651
| runtime.goexit
| /usr/local/go/src/runtime/asm_amd64.s:1371
Wraps: (4) interface conversion: tree.Datum is tree.dNull, not *tree.DInt
Error types: (1) *withstack.withStack (2) *safedetails.withSafeDetails (3) *withstack.withStack (4) *runtime.TypeAssertionError
-- report composition:
*runtime.TypeAssertionError
conn_executor.go:668: *withstack.withStack (top exception)
*safedetails.withSafeDetails: while executing: ALTER TABLE _ EXPERIMENTAL_RELOCATE VOTERS SELECT ARRAY[, _, more1], _
conn_executor.go:668: *withstack.withStack (1)
(check the extra data payloads)

Stacktrace (expand for inline code snippets):

cockroach/pkg/sql/conn_executor.go

Lines 667 to 669 in 8f4dc94

r := recover()
h.ex.closeWrapper(ctx, r)
}()
in pkg/sql.(*Server).ServeConn.func1
/usr/local/go/src/runtime/panic.go#L964-L966 in runtime.gopanic

cockroach/pkg/sql/colexecerror/error.go

Lines 57 to 59 in 8f4dc94

if !shouldCatchPanic(panicEmittedFrom) {
panic(panicObj)
}
in pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1
/usr/local/go/src/runtime/panic.go#L964-L966 in runtime.gopanic

cockroach/pkg/sql/colexecerror/error.go

Lines 57 to 59 in 8f4dc94

if !shouldCatchPanic(panicEmittedFrom) {
panic(panicObj)
}
in pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1
/usr/local/go/src/runtime/panic.go#L964-L966 in runtime.gopanic
/usr/local/go/src/runtime/iface.go#L260-L262 in runtime.panicdottypeE
/usr/local/go/src/runtime/iface.go#L270-L272 in runtime.panicdottypeI

cockroach/pkg/sql/relocate.go

Lines 89 to 91 in 8f4dc94

for i, d := range relocation.Array {
storeID := roachpb.StoreID(*d.(*tree.DInt))
nodeID, ok := n.run.storeMap[storeID]
in pkg/sql.(*relocateNode).Next

cockroach/pkg/sql/plan_node_to_row_source.go

Lines 186 to 188 in 8f4dc94

for p.State == execinfra.StateRunning {
valid, err := p.node.Next(p.params)
if err != nil || !valid {
in pkg/sql.(*planNodeToRowSource).Next

cockroach/pkg/sql/colexec/columnarizer.go

Lines 221 to 223 in 8f4dc94

for ; nRows < c.batch.Capacity(); nRows++ {
row, meta := c.input.Next()
if meta != nil {
in pkg/sql/colexec.(*Columnarizer).Next

cockroach/pkg/sql/colflow/stats.go

Lines 97 to 99 in 8f4dc94

func (bic *batchInfoCollector) next() {
bic.batch = bic.Operator.Next()
}
in pkg/sql/colflow.(*batchInfoCollector).next

cockroach/pkg/sql/colexecerror/error.go

Lines 90 to 92 in 8f4dc94

}()
operation()
return retErr
in pkg/sql/colexecerror.CatchVectorizedRuntimeError

cockroach/pkg/sql/colflow/stats.go

Lines 105 to 107 in 8f4dc94

// execution time (e.g. in the statement bundle).
err := colexecerror.CatchVectorizedRuntimeError(bic.next)
bic.stopwatch.Stop()
in pkg/sql/colflow.(*batchInfoCollector).Next

cockroach/pkg/sql/colflow/flow_coordinator.go

Lines 238 to 240 in 8f4dc94

func (f *BatchFlowCoordinator) nextAdapter() {
f.batch = f.input.Root.Next()
}
in pkg/sql/colflow.(*BatchFlowCoordinator).nextAdapter

cockroach/pkg/sql/colexecerror/error.go

Lines 90 to 92 in 8f4dc94

}()
operation()
return retErr
in pkg/sql/colexecerror.CatchVectorizedRuntimeError

cockroach/pkg/sql/colflow/flow_coordinator.go

Lines 242 to 244 in 8f4dc94

func (f *BatchFlowCoordinator) next() error {
return colexecerror.CatchVectorizedRuntimeError(f.nextAdapter)
}
in pkg/sql/colflow.(*BatchFlowCoordinator).next

cockroach/pkg/sql/colflow/flow_coordinator.go

Lines 281 to 283 in 8f4dc94

for status == execinfra.NeedMoreRows {
err := f.next()
if err != nil {
in pkg/sql/colflow.(*BatchFlowCoordinator).Run

cockroach/pkg/sql/colflow/vectorized_flow.go

Lines 257 to 259 in 8f4dc94

log.VEvent(ctx, 1, "running the batch flow coordinator in the flow's goroutine")
f.batchFlowCoordinator.Run(ctx)
}
in pkg/sql/colflow.(*vectorizedFlow).Run

cockroach/pkg/sql/distsql_running.go

Lines 547 to 549 in 8f4dc94

// TODO(radu): this should go through the flow scheduler.
flow.Run(ctx, func() {})
in pkg/sql.(*DistSQLPlanner).Run

cockroach/pkg/sql/distsql_running.go

Lines 1288 to 1290 in 8f4dc94

recv.expectedRowsRead = int64(physPlan.TotalEstimatedScannedRows)
runCleanup := dsp.Run(planCtx, txn, physPlan, recv, evalCtx, nil /* finishedSetupFn */)
return func() {
in pkg/sql.(*DistSQLPlanner).PlanAndRun

cockroach/pkg/sql/conn_executor_exec.go

Lines 1385 to 1387 in 8f4dc94

// the planner whether or not to plan remote table readers.
cleanup := ex.server.cfg.DistSQLPlanner.PlanAndRun(
ctx, evalCtx, planCtx, planner.txn, planner.curPlan.main, recv,
in pkg/sql.(*connExecutor).execWithDistSQLEngine

cockroach/pkg/sql/conn_executor_exec.go

Lines 1073 to 1075 in 8f4dc94

ex.sessionTracing.TraceExecStart(ctx, "distributed")
stats, err := ex.execWithDistSQLEngine(
ctx, planner, stmt.AST.StatementReturnType(), res, distributePlan.WillDistribute(), progAtomic,
in pkg/sql.(*connExecutor).dispatchToExecutionEngine

cockroach/pkg/sql/conn_executor_exec.go

Lines 708 to 710 in 8f4dc94

if err := ex.dispatchToExecutionEngine(ctx, p, res); err != nil {
stmtThresholdSpan.Finish()
in pkg/sql.(*connExecutor).execStmtInOpenState

cockroach/pkg/sql/conn_executor_exec.go

Lines 128 to 130 in 8f4dc94

} else {
ev, payload, err = ex.execStmtInOpenState(ctx, parserStmt, prepared, pinfo, res)
}
in pkg/sql.(*connExecutor).execStmt

cockroach/pkg/sql/conn_executor.go

Lines 1719 to 1721 in 8f4dc94

ev, payload, err = ex.execStmt(ctx, tcmd.Statement, nil /* prepared */, nil /* pinfo */, stmtRes)
return err
in pkg/sql.(*connExecutor).execCmd.func1

cockroach/pkg/sql/conn_executor.go

Lines 1721 to 1723 in 8f4dc94

return err
}()
// Note: we write to ex.statsCollector.PhaseTimes, instead of ex.phaseTimes,
in pkg/sql.(*connExecutor).execCmd

cockroach/pkg/sql/conn_executor.go

Lines 1643 to 1645 in 8f4dc94

var err error
if err = ex.execCmd(ex.Ctx()); err != nil {
if errors.IsAny(err, io.EOF, errDrainingComplete) {
in pkg/sql.(*connExecutor).run

cockroach/pkg/sql/conn_executor.go

Lines 669 to 671 in 8f4dc94

}()
return h.ex.run(ctx, s.pool, reserved, cancel)
}
in pkg/sql.(*Server).ServeConn

cockroach/pkg/sql/pgwire/conn.go

Lines 650 to 652 in 8f4dc94

reservedOwned = false // We're about to pass ownership away.
retErr = sqlServer.ServeConn(ctx, connHandler, reserved, cancelConn)
}()
in pkg/sql/pgwire.(*conn).processCommandsAsync.func1
/usr/local/go/src/runtime/asm_amd64.s#L1370-L1372 in runtime.goexit

pkg/sql/conn_executor.go in pkg/sql.(*Server).ServeConn.func1 at line 668
/usr/local/go/src/runtime/panic.go in runtime.gopanic at line 965
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1 at line 58
/usr/local/go/src/runtime/panic.go in runtime.gopanic at line 965
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1 at line 58
/usr/local/go/src/runtime/panic.go in runtime.gopanic at line 965
/usr/local/go/src/runtime/iface.go in runtime.panicdottypeE at line 261
/usr/local/go/src/runtime/iface.go in runtime.panicdottypeI at line 271
pkg/sql/relocate.go in pkg/sql.(*relocateNode).Next at line 90
pkg/sql/plan_node_to_row_source.go in pkg/sql.(*planNodeToRowSource).Next at line 187
pkg/sql/colexec/columnarizer.go in pkg/sql/colexec.(*Columnarizer).Next at line 222
pkg/sql/colflow/stats.go in pkg/sql/colflow.(*batchInfoCollector).next at line 98
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 91
pkg/sql/colflow/stats.go in pkg/sql/colflow.(*batchInfoCollector).Next at line 106
pkg/sql/colflow/flow_coordinator.go in pkg/sql/colflow.(*BatchFlowCoordinator).nextAdapter at line 239
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 91
pkg/sql/colflow/flow_coordinator.go in pkg/sql/colflow.(*BatchFlowCoordinator).next at line 243
pkg/sql/colflow/flow_coordinator.go in pkg/sql/colflow.(*BatchFlowCoordinator).Run at line 282
pkg/sql/colflow/vectorized_flow.go in pkg/sql/colflow.(*vectorizedFlow).Run at line 258
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).Run at line 548
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).PlanAndRun at line 1289
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execWithDistSQLEngine at line 1386
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).dispatchToExecutionEngine at line 1074
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmtInOpenState at line 709
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt at line 129
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd.func1 at line 1720
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd at line 1722
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).run at line 1644
pkg/sql/conn_executor.go in pkg/sql.(*Server).ServeConn at line 670
pkg/sql/pgwire/conn.go in pkg/sql/pgwire.(*conn).processCommandsAsync.func1 at line 651
/usr/local/go/src/runtime/asm_amd64.s in runtime.goexit at line 1371
Tag Value
Cockroach Release v21.2.14
Cockroach SHA: 8f4dc94
Platform linux amd64
Distribution CCL
Environment v21.2.14
Command server
Go Version ``
# of CPUs
# of Goroutines

Jira issue: CRDB-19305
@cockroach-teamcity cockroach-teamcity added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report. labels Sep 5, 2022
@yuzefovich yuzefovich changed the title sentry: conn_executor.go:668: interface conversion: tree.Datum is tree.dNull, not *tree.DInt (1) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn.func1 ... sql: v21.2.14: interface conversion: tree.Datum is tree.dNull, not *tree.DInt Sep 9, 2022
@yuzefovich
Copy link
Member

Easily reproducible:

create table t ();
alter table t experimental_relocate voters select array[null], null;

@yuzefovich yuzefovich added E-easy Easy issue to tackle, requires little or no CockroachDB experience E-quick-win Likely to be a quick win for someone experienced. labels Sep 9, 2022
@blathers-crl blathers-crl bot added the T-kv KV Team label Sep 9, 2022
@yuzefovich yuzefovich added E-starter Might be suitable for a starter project for new employees or team members. and removed E-starter Might be suitable for a starter project for new employees or team members. labels Sep 9, 2022
@exalate-issue-sync exalate-issue-sync bot added T-sql-execution and removed T-kv KV Team labels Sep 19, 2022
@nvanbenschoten
Copy link
Member

@yuzefovich I think it will be faster for someone on SQL to fix this and add testing for it, even if KV owns this specific statement. Do you mind if we move this back to SQL queries?

@blathers-crl blathers-crl bot added the T-sql-queries SQL Queries Team label Sep 19, 2022
@yuzefovich yuzefovich self-assigned this Sep 19, 2022
@yuzefovich yuzefovich mentioned this issue Sep 19, 2022
Merged
@craig craig bot closed this as completed in f363dba Sep 20, 2022
@blathers-crl blathers-crl bot mentioned this issue Sep 20, 2022
Merged
This was referenced Sep 20, 2022
Merged
Merged
@exalate-issue-sync exalate-issue-sync bot removed the T-sql-queries SQL Queries Team label Sep 30, 2022
@mgartner mgartner moved this to Done in SQL Queries Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yuzefovich yuzefovich

Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. E-easy Easy issue to tackle, requires little or no CockroachDB experience E-quick-win Likely to be a quick win for someone experienced. O-sentry Originated from an in-the-wild panic report.
Projects
SQL Queries
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sql: fix relocate commands with NULLs yuzefovich/cockroach
3 participants
@yuzefovich @nvanbenschoten @cockroach-teamcity