integration_tests with kafka_simple_basic arise data inconsistence

[wk989898]

What did you do?

Expand the writing time of upstream and downstream.

	# upstream TiDB disable the cluster index
	run_sql "set global tidb_enable_clustered_index=0;" ${UP_TIDB_HOST} ${UP_TIDB_PORT}

	# upstream tidb cluster enable row level checksum
	run_sql "set global tidb_enable_row_level_checksum=true" ${UP_TIDB_HOST} ${UP_TIDB_PORT}

	TOPIC_NAME="ticdc-simple-basic-$RANDOM"

	run_cdc_server --workdir $WORK_DIR --binary $CDC_BINARY
	start_ts=$(run_cdc_cli_tso_query ${UP_PD_HOST_1} ${UP_PD_PORT_1})

	run_sql_file $CUR/data/pre_ddl.sql ${UP_TIDB_HOST} ${UP_TIDB_PORT}
	run_sql_file $CUR/data/data.sql ${UP_TIDB_HOST} ${UP_TIDB_PORT}

	sleep 10 
	
	SINK_URI="kafka://127.0.0.1:9092/$TOPIC_NAME?protocol=simple"
	run_cdc_cli changefeed create --start-ts $start_ts --sink-uri="$SINK_URI" --config="$CUR/conf/changefeed.toml" -c "simple-basic"

	cdc_kafka_consumer --log-level debug --upstream-uri $SINK_URI --downstream-uri="mysql://[email protected]:3306/?safe-mode=true&batch-dml-enable=false" --upstream-tidb-dsn="root@tcp(${UP_TIDB_HOST}:${UP_TIDB_PORT})/?" --config="$CUR/conf/changefeed.toml" 2>&1 &

	check_table_exists test.finish_mark_for_ddl ${DOWN_TIDB_HOST} ${DOWN_TIDB_PORT} 200

	# pause and resume changefeed makes sure changefeed sending bootstrap events
	# when it is resumed, so the data after pause can be decoded correctly
	run_cdc_cli changefeed pause -c "simple-basic"
	run_cdc_cli changefeed resume -c "simple-basic"


	# sync_diff can't check non-exist table, so we check expected tables are created in downstream first
	check_table_exists test.finish_mark ${DOWN_TIDB_HOST} ${DOWN_TIDB_PORT} 200
	check_sync_diff $WORK_DIR $CUR/conf/diff_config.toml

	run_sql_file $CUR/data/data_gbk.sql ${UP_TIDB_HOST} ${UP_TIDB_PORT}
	check_table_exists test.finish_mark ${DOWN_TIDB_HOST} ${DOWN_TIDB_PORT} 200
	check_sync_diff $WORK_DIR $CUR/conf/diff_config.toml

What did you expect to see?

kafka_simple_basic case success

What did you see instead?

The data of test.tp_time is not equal

-- table: test.tp_time
-- range in sequence: Full
set @@session.time_zone = "+0:00";
/*
  DIFF COLUMNS ╏    `C TIMESTAMP2`      
╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╋╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍
  source data  ╏ '2024-09-10 02:23:24'  
╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╋╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍
  target data  ╏ '2024-09-10 02:24:15'  
╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╋╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍
*/
REPLACE INTO `test`.`tp_time`(`id`,`c_date`,`c_datetime`,`c_timestamp`,`c_time`,`c_year`,`c_timestamp2`) VALUES (1,'2024-07-04','2024-07-04 07:00:00','2024-07-04 04:00:00','15:23:45',2024,'2024-09-10 02:23:24');
/*
  DIFF COLUMNS ╏    `C TIMESTAMP2`      
╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╋╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍
  source data  ╏ '2024-09-10 02:23:24'  
╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╋╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍
  target data  ╏ '2024-09-10 02:24:15'  
╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╋╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍
*/
REPLACE INTO `test`.`tp_time`(`id`,`c_date`,`c_datetime`,`c_timestamp`,`c_time`,`c_year`,`c_timestamp2`) VALUES (5,'2024-03-09','2022-02-22 14:22:22','2020-02-19 18:20:20','02:20:20',2021,'2024-09-10 02:23:24');

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

master

Upstream TiKV version (execute tikv-server --version):

master

TiCDC version (execute cdc version):

master

[wk989898]

The root reason is that TiCDC can't ensure the default value of dynamic generation consistency. some rows before executing alter table tp_time add column c_timestamp2 timestamp default now(); cause this error.