Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ticdc: when start a redo changefeed, ticdc panicked #10101

Closed
AkiraXie opened this issue Nov 16, 2023 · 4 comments · Fixed by #10102
Closed

ticdc: when start a redo changefeed, ticdc panicked #10101

AkiraXie opened this issue Nov 16, 2023 · 4 comments · Fixed by #10102
Assignees
Labels
affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. severity/critical type/bug The issue is confirmed as a bug.

Comments

@AkiraXie
Copy link

What did you do?

  1. open a mysql-sink changefeed with redo log

What did you expect to see?

No response

What did you see instead?

[WARN] TiCDC server data-dir is not set. Please use cdc server --data-dir to start the cdc server if possible.
panic: non-positive interval for NewTicker

goroutine 1587 [running]:
time.NewTicker(0xc00025ed60?)
time/tick.go:22 +0xe5
github.com/pingcap/tiflow/cdc/redo.(*logManager).bgUpdateLog(0xc00cc782a0, {0x7fef9294bb10, 0xc00ce2d660}, 0x0)
github.com/pingcap/tiflow/cdc/redo/manager.go:497 +0xb8
github.com/pingcap/tiflow/cdc/redo.(*logManager).Run(0xc00cc782a0, {0x7fef9294bb10, 0xc00ce2d660}, {0x4447f00?, 0x462f9c0?, 0x0?})
github.com/pingcap/tiflow/cdc/redo/manager.go:277 +0x13c
github.com/pingcap/tiflow/cdc/owner.(*changefeed).initialize.func5()
github.com/pingcap/tiflow/cdc/owner/changefeed.go:596 +0xaa
created by github.com/pingcap/tiflow/cdc/owner.(*changefeed).initialize in goroutine 724
github.com/pingcap/tiflow/cdc/owner/changefeed.go:594 +0xe99

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

(paste TiDB cluster version here)

Upstream TiKV version (execute tikv-server --version):

(paste TiKV version here)

TiCDC version (execute cdc version):

release-7.5
sh-5.1# /cdc version
Release Version: v7.5.0
Git Commit Hash: 50b7aa338a042cfd79ea2fd969ff9e0fe4464cb5
Git Branch: heads/refs/tags/v7.5.0
UTC Build Time: 2023-11-14 10:44:31
Go Version: go version go1.21.3 linux/amd64
Failpoint Build: false
@AkiraXie AkiraXie added area/ticdc Issues or PRs related to TiCDC. type/bug The issue is confirmed as a bug. labels Nov 16, 2023
@AkiraXie
Copy link
Author

/severity critical

@AkiraXie
Copy link
Author

AkiraXie commented Nov 16, 2023

/affects 7.5

@nongfushanquan
Copy link
Contributor

/assign @sdojjy

@sdojjy
Copy link
Member

sdojjy commented Nov 16, 2023

some new changefeed redo configuration items are added in new ticdc version, and the changefeed created by old version has no default value.

workaround:

  1. downgrade ticdc and pause changefeed,
  2. upgrade ticdc again,
  3. use ticdc cli to update the changefeed configuration
    consistent.meta-flush-interval
  4. resume changefeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. severity/critical type/bug The issue is confirmed as a bug.
Projects
Development

Successfully merging a pull request may close this issue.

3 participants