Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kvflowcontrol: race with kvflowhandle.(*Handle) #104837

Closed
Tracked by #98703
yuzefovich opened this issue Jun 14, 2023 · 0 comments · Fixed by #104855
Closed
Tracked by #98703

kvflowcontrol: race with kvflowhandle.(*Handle) #104837

yuzefovich opened this issue Jun 14, 2023 · 0 comments · Fixed by #104855
Assignees
@irfansharif
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-kv KV Team

Comments

@yuzefovich
Copy link
Member

yuzefovich commented Jun 14, 2023 •
edited by cockroach-jira-scripts
Loading

Seen here (on #104827 which seems to be unrelated):

=== RUN   TestEnsureSQLStatsAreFlushedDuringDrain
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/5b2c9b3a394428c7572d34050aad8975/logTestEnsureSQLStatsAreFlushedDuringDrain276054599
    test_log_scope.go:79: use -show-logs to present logs inline
==================
WARNING: DATA RACE
Write at 0x00c004c70d08 by goroutine 2619627:
  internal/reflectlite.Swapper.func3()
      GOROOT/src/internal/reflectlite/swapper.go:42 +0xf2
  sort.insertionSort_func()
      GOROOT/src/sort/zsortfunc.go:13 +0x94
  sort.pdqsort_func()
      GOROOT/src/sort/zsortfunc.go:73 +0x3cb
  sort.Slice()
      GOROOT/src/sort/slice.go:23 +0xc4
  github.com/cockroachdb/cockroach/pkg/kv/kvserver/kvflowcontrol/kvflowhandle.(*Handle).connectStreamLocked()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/kvflowcontrol/kvflowhandle/kvflowhandle.go:238 +0x3ca
  github.com/cockroachdb/cockroach/pkg/kv/kvserver/kvflowcontrol/kvflowhandle.(*Handle).ConnectStream()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/kvflowcontrol/kvflowhandle/kvflowhandle.go:228 +0x1a4
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaFlowControlIntegrationImpl).tryReconnect()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/flow_control_replica_integration.go:423 +0x518
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaFlowControlIntegrationImpl).refreshStreams()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/flow_control_replica_integration.go:289 +0x84
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaFlowControlIntegrationImpl).onRaftTicked()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/flow_control_replica_integration.go:257 +0x8b
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).updateProposalQuotaRaftMuLocked()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_proposal_quota.go:268 +0xe7c
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReadyRaftMuLocked()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_raft.go:1203 +0x2344
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReady()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_raft.go:807 +0x20f
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).processReady()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/store_raft.go:646 +0x1cf
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftSchedulerShard).worker()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/scheduler.go:418 +0x2f5
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftSchedulerShard).worker()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/scheduler.go:418 +0x2f5
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftSchedulerShard).worker()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/scheduler.go:418 +0x2f5
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftSchedulerShard).worker()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/scheduler.go:418 +0x2f5
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftSchedulerShard).worker()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/scheduler.go:418 +0x2f5
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftSchedulerShard).worker()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/scheduler.go:418 +0x2f5
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Start.func2()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/scheduler.go:321 +0x9a
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
      github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:484 +0x1f6

Previous read at 0x00c004c70d08 by goroutine 2632354:
  github.com/cockroachdb/cockroach/pkg/kv/kvserver/kvflowcontrol/kvflowhandle.(*Handle).Admit()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/kvflowcontrol/kvflowhandle/kvflowhandle.go:110 +0x2a4
  github.com/cockroachdb/cockroach/pkg/kv/kvserver/kvadmission.(*controllerImpl).AdmitKVWork()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/kvadmission/kvadmission.go:310 +0x5ec
  github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal()
      github.com/cockroachdb/cockroach/pkg/server/node.go:1171 +0x4ac
  github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch()
      github.com/cockroachdb/cockroach/pkg/server/node.go:1278 +0x35d
  github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func1()
      github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:636 +0x7c
  github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor.ServerInterceptor.func1()
      github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:96 +0x1d4
  github.com/cockroachdb/cockroach/pkg/rpc.bindUnaryServerInterceptorToHandler.func1()
      github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:747 +0x8e
  github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3()
      github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:166 +0xf8
  github.com/cockroachdb/cockroach/pkg/rpc.bindUnaryServerInterceptorToHandler.func1()
      github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:747 +0x8e
  github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1()
      github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:133 +0x70
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr()
      github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:336 +0x147
  github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1()
      github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:131 +0x137
  github.com/cockroachdb/cockroach/pkg/rpc.bindUnaryServerInterceptorToHandler.func1()
      github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:747 +0x8e
  github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func2()
      github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:646 +0x81
  github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor.ClientInterceptor.func2()
      github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:227 +0x163
  github.com/cockroachdb/cockroach/pkg/rpc.getChainUnaryInvoker.func1()
      github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:831 +0x19c
  github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func3()
      github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:716 +0x4ed
  github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.Batch()
      github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:839 +0xf9
  github.com/cockroachdb/cockroach/pkg/rpc.(*internalClientAdapter).Batch()
      <autogenerated>:1 +0x33
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*grpcTransport).sendBatch()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/transport.go:210 +0x1e6
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*grpcTransport).SendNext()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/transport.go:189 +0x151
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.raceTransport.SendNext()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/transport_race.go:76 +0x1a8
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*raceTransport).SendNext()
      <autogenerated>:1 +0x75
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).sendToReplicas()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender.go:2181 +0x1af6
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).sendPartialBatch()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender.go:1703 +0xa44
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).divideAndSendBatchToRanges()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender.go:1274 +0x592
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).Send()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender.go:894 +0xa79
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnLockGatekeeper).SendLocked()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_lock_gatekeeper.go:82 +0x229
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnMetricRecorder).SendLocked()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_interceptor_metric_recorder.go:47 +0x1ce
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnSpanRefresher).sendLockedWithRefreshAttempts()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_interceptor_span_refresher.go:220 +0x315
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnSpanRefresher).SendLocked()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_interceptor_span_refresher.go:148 +0x1e6
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnCommitter).SendLocked()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_interceptor_committer.go:145 +0xa45
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnPipeliner).SendLocked()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_interceptor_pipeliner.go:292 +0x245
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnSeqNumAllocator).SendLocked()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go:110 +0xd4
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnHeartbeater).SendLocked()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_interceptor_heartbeater.go:246 +0x7cc
  github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*TxnCoordSender).Send()
      github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_coord_sender.go:529 +0x9b2
  github.com/cockroachdb/cockroach/pkg/kv.(*DB).sendUsingSender()
      github.com/cockroachdb/cockroach/pkg/kv/db.go:1011 +0x190
  github.com/cockroachdb/cockroach/pkg/kv.(*Txn).Send()
      github.com/cockroachdb/cockroach/pkg/kv/txn.go:1087 +0x334
  github.com/cockroachdb/cockroach/pkg/kv.(*Txn).Send-fm()
      <autogenerated>:1 +0x64
  github.com/cockroachdb/cockroach/pkg/kv.sendAndFill()
      github.com/cockroachdb/cockroach/pkg/kv/db.go:841 +0x269
  github.com/cockroachdb/cockroach/pkg/kv.(*Txn).Run()
      github.com/cockroachdb/cockroach/pkg/kv/txn.go:697 +0xa4
  github.com/cockroachdb/cockroach/pkg/sql.(*tableWriterBase).finalize()
      github.com/cockroachdb/cockroach/pkg/sql/tablewriter.go:232 +0x1b7
  github.com/cockroachdb/cockroach/pkg/sql.(*insertFastPathNode).BatchedNext()
      github.com/cockroachdb/cockroach/pkg/sql/insert_fast_path.go:314 +0x285
  github.com/cockroachdb/cockroach/pkg/sql.(*rowCountNode).startExec()
      github.com/cockroachdb/cockroach/pkg/sql/plan_batch.go:173 +0x199
  github.com/cockroachdb/cockroach/pkg/sql.startExec.func2()
      github.com/cockroachdb/cockroach/pkg/sql/plan.go:519 +0x186
  github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visitInternal.func1()
      github.com/cockroachdb/cockroach/pkg/sql/walk.go:112 +0xa9
  runtime.deferreturn()
      GOROOT/src/runtime/panic.go:476 +0x32
  github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visit()
      github.com/cockroachdb/cockroach/pkg/sql/walk.go:79 +0x135
  github.com/cockroachdb/cockroach/pkg/sql.walkPlan()
      github.com/cockroachdb/cockroach/pkg/sql/walk.go:43 +0x165
  github.com/cockroachdb/cockroach/pkg/sql.startExec()
      github.com/cockroachdb/cockroach/pkg/sql/plan.go:522 +0x180
  github.com/cockroachdb/cockroach/pkg/sql.(*planNodeToRowSource).Start()
      github.com/cockroachdb/cockroach/pkg/sql/plan_node_to_row_source.go:174 +0x1e9
  github.com/cockroachdb/cockroach/pkg/sql/colflow.(*FlowCoordinator).Start.func1()
      github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:120 +0x74
  github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError()
      github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92 +0x75
  github.com/cockroachdb/cockroach/pkg/sql/colflow.(*FlowCoordinator).Start()
      github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:119 +0x9a
  github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBaseNoHelper).Run()
      github.com/cockroachdb/cockroach/pkg/sql/execinfra/processorsbase.go:725 +0x7a
  github.com/cockroachdb/cockroach/pkg/sql/colflow.(*FlowCoordinator).Run()
      <autogenerated>:1 +0x6b
  github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowBase).Run()
      github.com/cockroachdb/cockroach/pkg/sql/flowinfra/flow.go:575 +0x727
  github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlow).Run()
      github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:302 +0x38c
  github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run()
      github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:902 +0x1555
  github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun()
      github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1913 +0x2e7
  github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll.func3()
      github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1645 +0x144
  github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll()
      github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1648 +0x2f7
  github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine()
      github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:2098 +0xe14
  github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine()
      github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1656 +0x1fe6
  github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState()
      github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:964 +0x640c
  github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt.func1()
      github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:142 +0x18e
  github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithProfiling()
      github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:2986 +0x50f
  github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt()
      github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:141 +0x724
  github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd.func1()
      github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2165 +0x7e4
  github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd()
      github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2170 +0x1da4
  github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run()
      github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2087 +0x424
  github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn()
      github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:896 +0x1be
  github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync.func1()
      github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:778 +0x607

Goroutine 2619627 (running) created at:
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx()
      github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:475 +0x619
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Start()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/scheduler.go:313 +0x406
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).processRaft()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/store_raft.go:714 +0xc4
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).Start()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/store.go:2063 +0x127a
  github.com/cockroachdb/cockroach/pkg/server.(*Node).start()
      github.com/cockroachdb/cockroach/pkg/server/node.go:508 +0x1539
  github.com/cockroachdb/cockroach/pkg/server.(*Server).PreStart()
      github.com/cockroachdb/cockroach/pkg/server/server.go:1801 +0x347a
  github.com/cockroachdb/cockroach/pkg/server.(*TestServer).Start()
      github.com/cockroachdb/cockroach/pkg/server/testserver.go:622 +0x8f
  github.com/cockroachdb/cockroach/pkg/testutils/testcluster.(*TestCluster).startServer()
      github.com/cockroachdb/cockroach/pkg/testutils/testcluster/testcluster.go:593 +0xd4
  github.com/cockroachdb/cockroach/pkg/testutils/testcluster.(*TestCluster).Start()
      github.com/cockroachdb/cockroach/pkg/testutils/testcluster/testcluster.go:389 +0x890
  github.com/cockroachdb/cockroach/pkg/testutils/testcluster.StartTestCluster()
      github.com/cockroachdb/cockroach/pkg/testutils/testcluster/testcluster.go:233 +0x9a
  github.com/cockroachdb/cockroach/pkg/testutils/testcluster.NewTestCluster()
      github.com/cockroachdb/cockroach/pkg/testutils/testcluster/testcluster.go:329 +0xcec
  github.com/cockroachdb/cockroach/pkg/testutils/testcluster.NewTestCluster()
      github.com/cockroachdb/cockroach/pkg/testutils/testcluster/testcluster.go:329 +0xcec
  github.com/cockroachdb/cockroach/pkg/testutils/testcluster.StartTestCluster()
      github.com/cockroachdb/cockroach/pkg/testutils/testcluster/testcluster.go:232 +0x7d
  github.com/cockroachdb/cockroach/pkg/server_test.newTestDrainContext()
      github.com/cockroachdb/cockroach/pkg/server_test/pkg/server/drain_test.go:178 +0x1db
  github.com/cockroachdb/cockroach/pkg/server_test.TestEnsureSQLStatsAreFlushedDuringDrain()
      github.com/cockroachdb/cockroach/pkg/server_test/pkg/server/drain_test.go:116 +0x144
  testing.tRunner()
      GOROOT/src/testing/testing.go:1446 +0x216
  testing.(*T).Run.func1()
      GOROOT/src/testing/testing.go:1493 +0x47

Goroutine 2632354 (running) created at:
  github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync()
      github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:687 +0x3c4
  github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).serveImpl()
      github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:393 +0xd15
  github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*Server).serveConn()
      github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:178 +0x48c
  github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*Server).ServeConn()
      github.com/cockroachdb/cockroach/pkg/sql/pgwire/server.go:819 +0xbea
  github.com/cockroachdb/cockroach/pkg/server.(*systemServerWrapper).serveConn()
      github.com/cockroachdb/cockroach/pkg/server/server_controller_sql.go:93 +0x1f7
  github.com/cockroachdb/cockroach/pkg/server.(*serverController).sqlMux()
      github.com/cockroachdb/cockroach/pkg/server/server_controller_sql.go:73 +0x4a9
  github.com/cockroachdb/cockroach/pkg/server.(*serverController).sqlMux-fm()
      <autogenerated>:1 +0xc4
  github.com/cockroachdb/cockroach/pkg/server.startServeSQL.func1.1()
      github.com/cockroachdb/cockroach/pkg/server/server_sql.go:1779 +0x28f
  github.com/cockroachdb/cockroach/pkg/util/netutil.(*TCPServer).ServeWith.func1()
      github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:188 +0x111
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
      github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:484 +0x1f6
==================

Jira issue: CRDB-28742
@yuzefovich yuzefovich added the C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. label Jun 14, 2023
@blathers-crl blathers-crl bot added the T-kv KV Team label Jun 14, 2023
This was referenced Jun 14, 2023
Closed
Closed
Merged
irfansharif added a commit to irfansharif/cockroach that referenced this issue Jun 14, 2023
@irfansharif
kvflowcontrol: squash data race
8ac33ee 
Fixes cockroachdb#104837.

Release note: None
@THardy98 THardy98 mentioned this issue Jun 27, 2023
Merged
@irfansharif irfansharif mentioned this issue Jun 29, 2023
Closed
craig bot pushed a commit that referenced this issue Jul 4, 2023
@irfansharif
Merge #104855
c77d127 
104855: kvflowcontrol: squash data race r=irfansharif a=irfansharif

Fixes #104837.
Fixes #105762.

Release note: None

Co-authored-by: irfan sharif <[email protected]>
@craig craig bot closed this as completed in edd1e1b Jul 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@irfansharif irfansharif

Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-kv KV Team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

kvflowcontrol: squash data race irfansharif/cockroach
2 participants
@yuzefovich @irfansharif