Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(pkg/GenDeleteSQL) duplicate calculating columns & values when generate WHERE condition #10311

Closed
zhangjinpeng87 opened this issue Dec 15, 2023 · 0 comments · Fixed by #10312
Closed
Assignees
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@zhangjinpeng87
Copy link
Contributor

https://github.com/pingcap/tiflow/blob/master/pkg/sqlmodel/multirow.go#L86 call whereColumnsAndValues just want to compute the pre-allocate size of a slice. And https://github.com/pingcap/tiflow/blob/master/pkg/sqlmodel/multirow.go#L93 also call whereColumnsAndValues again. Please notice that whereColumnsAndValues is not a trivial function.

When the upstream TiDB is high concurrency OLTP insertion/deletion/update, it means there is not too many queries including multiple rows changes with the same timestamp for each transaction. In that case, TiCDC may call twice whereColumnsAndValues for most row changes.

We can use a fixed pre-allocation size or heuristic number instead of calculating it every row https://github.com/pingcap/tiflow/blob/master/pkg/sqlmodel/multirow.go#L87

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant