Decode error when "NULL" value in the column with "primary key" flag

[JaySon-Huang]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

TiFlash assumes all values in the column with "primary key" flag are non-null value.

tiflash/dbms/src/Storages/Transaction/RowCodec.cpp

Lines 323 to 330 in 6d0cbc8

	inline bool addDefaultValueToColumnIfPossible(const ColumnInfo & column_info, Block & block, size_t block_column_pos, bool force_decode)
	{
	// We consider a missing column could be safely filled with NULL, unless it has not default value and is NOT NULL.
	// This could saves lots of unnecessary schema syncs for old data with a schema that has newly added columns.
	// for clustered index, if the pk column does not exists, it can still be decoded from the key
	if (column_info.hasPriKeyFlag())
	return true;

However, users could create such "NULL" value in the "primary key" column by following steps

-- create table with `source` is nullable
create table test.t_case (
`case_no` varchar(32) not null,
`source` varchar(20) default null,
`p` varchar(12) DEFAULT NULL,
primary key (`case_no`)
);

-- insert some data and left `source` to be empty
insert into test.t_case(case_no)
values ("1"), ("2"), ("3"), ("4");

-- drop the primary key, fill the `source` to be non-empty
alter table t_case drop primary key;
update test.t_case set `source` = '' where `source` is NULL;
-- add new primary key with case_no and source
alter table t_case add primary key (case_no, source);

-- send the snapshot data to tiflash
alter table t_case set tiflash replica 1;

2. What did you expect to see? (Required)

No exception thrown

3. What did you see instead (Required)

Exception throw with "Parameter out of bound in IColumnString::insertRangeFrom method" in v4.0.x/v5.0.x/v5.1.x/v5.2.x/v5.3.x/v5.4.0~v5.4.1/v6.1.0

[2022/09/13 15:30:02.786 +08:00] [ERROR] [<unknown>] ["bool DB::GCManager::work():Code: 12, e.displayText() = DB::Exception: Parameter out of bound in IColumnString::insertRangeFrom method, start=0, length=8, src_size=4:  while reading DTFile /data2/jaysonhuang/tiup-deploy/data/tiflash-5018/data/t_77/stable/dmf_2: while apply gc Segment [1] [range=[-9223372036854775808,9223372036854775807)] [table=t_77], e.what() = DB::Exception, Stack trace:
0. bin/tiflash/tiflash(StackTrace::StackTrace()+0x16) [0x36d42e6]
1. bin/tiflash/tiflash(DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)+0x26) [0x36c8116]
2. bin/tiflash/tiflash(DB::ColumnString::insertRangeFrom(DB::IColumn const&, unsigned long, unsigned long)+0x2ff) [0x759712f]
3. bin/tiflash/tiflash(DB::DM::copyColumnsData(std::vector<COWPtr<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, COWPtr<DB::IColumn>::immutable_ptr<DB::IColumn> const&, std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long, unsigned long, DB::DM::RowKeyRange const*)+0x7fe) [0x76f690e]
4. bin/tiflash/tiflash(DB::DM::DPFileReader::readRowsOnce(std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long, unsigned long, DB::DM::RowKeyRange const*)+0x135) [0x77059c5]
5. bin/tiflash/tiflash(DB::DM::DPFileReader::readRows(std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long, unsigned long, DB::DM::RowKeyRange const*)+0x55) [0x7706d85]
6. bin/tiflash/tiflash(DB::DM::DeltaValueReader::readRows(std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long, unsigned long, DB::DM::RowKeyRange const*)+0xcf) [0x771c34f]
7. bin/tiflash/tiflash(DB::DM::DeltaMergeBlockInputStream<DB::DM::DeltaValueReader, DB::DM::DTCompactedEntries<55ul, 20ul, 3ul>::Iterator, false>::writeInsertFromDelta(std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long&)+0x94) [0x7698344]
8. bin/tiflash/tiflash(DB::DM::DeltaMergeBlockInputStream<DB::DM::DeltaValueReader, DB::DM::DTCompactedEntries<55ul, 20ul, 3ul>::Iterator, false>::read()+0x863) [0x769b593]
9. bin/tiflash/tiflash(DB::DM::DMRowKeyFilterBlockInputStream<true>::read()+0x5f) [0x7692d2f]
10. bin/tiflash/tiflash(DB::DM::PKSquashingBlockInputStream<false>::read()+0x9f1) [0x7698d61]
11. bin/tiflash/tiflash(DB::DM::DMVersionFilterBlockInputStream<1>::initNextBlock()+0x2f) [0x7a3dcbf]
12. bin/tiflash/tiflash(DB::DM::DMVersionFilterBlockInputStream<1>::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool)+0x1dd0) [0x7a411a0]
13. bin/tiflash/tiflash(DB::DM::DMVersionFilterBlockInputStream<1>::read()+0x18) [0x7681968]
14. bin/tiflash/tiflash(DB::DM::writeIntoNewDMFile(DB::DM::DMContext&, std::shared_ptr<std::vector<DB::DM::ColumnDefine, std::allocator<DB::DM::ColumnDefine> > > const&, std::shared_ptr<DB::IBlockInputStream> const&, unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, DB::DM::DMFileWriter::Flags)+0x20b) [0x7670cdb]
15. bin/tiflash/tiflash(DB::DM::createNewStable(DB::DM::DMContext&, std::shared_ptr<std::vector<DB::DM::ColumnDefine, std::allocator<DB::DM::ColumnDefine> > > const&, std::shared_ptr<DB::IBlockInputStream> const&, unsigned long, DB::DM::WriteBatches&)+0xbd) [0x767143d]
16. bin/tiflash/tiflash(DB::DM::Segment::prepareMergeDelta(DB::DM::DMContext&, std::shared_ptr<std::vector<DB::DM::ColumnDefine, std::allocator<DB::DM::ColumnDefine> > > const&, std::shared_ptr<DB::DM::SegmentSnapshot> const&, DB::DM::WriteBatches&) const+0x17d) [0x767ab7d]
17. bin/tiflash/tiflash(DB::DM::DeltaMergeStore::segmentMergeDelta(DB::DM::DMContext&, std::shared_ptr<DB::DM::Segment> const&, DB::DM::DeltaMergeStore::TaskRunThread, std::shared_ptr<DB::DM::SegmentSnapshot>)+0x3e1) [0x76496b1]
18. bin/tiflash/tiflash(DB::DM::DeltaMergeStore::onSyncGc(long)+0x5c1) [0x7650601]
19. bin/tiflash/tiflash(DB::GCManager::work()+0x7b1) [0x7a09911]
20. bin/tiflash/tiflash(DB::BackgroundProcessingPool::threadFunction()+0x947) [0x75dba17]
21. bin/tiflash/tiflash() [0x8b14f6f]
22. /lib64/libpthread.so.0(+0x6e63) [0x7fe4a07f6e63]
23. /lib64/libc.so.6(clone+0x6d) [0x7fe4a022b55d]
"] [thread_id=10]

Exception message after v5.4.2/v6.1.1/v6.2.0

[2022/09/13 15:59:30.623 +08:00] [ERROR] [Exception.cpp:85] ["DB::RawCppPtr DB::PreHandleSnapshot(DB::EngineStoreServerWrap *, DB::BaseBuffView, uint64_t, DB::SSTViewVec, uint64_t, uint64_t):Code: 9, e.displayText() = DB::Exception: Sizes of columns doesn't match: _INTERNAL_DELMARK(id=-1025): 8, source(id=2): 4, e.what() = DB::Exception, Stack trace:
       0x1cdb0ce    DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) [tiflash+30257358]
                    dbms/src/Common/Exception.h:41
       0x65b93c4    DB::Block::checkNumberOfRows() const [tiflash+106664900]
                    dbms/src/Core/Block.cpp:243
       0x735076e    bool DB::RegionBlockReader::readImpl<(DB::TMTPKType)0>(DB::Block&, std::__1::vector<std::__1::tuple<DB::RawTiDBPK, unsigned char, unsigned long, std::__1::shared_ptr<DB::StringObject<false> const> >, std::__1::allocator<std::__1::tuple<DB::RawTiDBPK, unsigned char, unsigned long, std::__1::shared_ptr<DB::StringObject<false> const> > > > const&, bool) [tiflash+120915822]
                    dbms/src/Storages/Transaction/RegionBlockReader.cpp:211
       0x732cc13    DB::GenRegionBlockDataWithSchema(std::__1::shared_ptr<DB::Region> const&, std::__1::shared_ptr<DB::DecodingStorageSchemaSnapshot const> const&, unsigned long, bool, DB::TMTContext&) [tiflash+120769555]
                    dbms/src/Storages/Transaction/PartitionStreams.cpp:606
       0x72ee5a6    DB::DM::SSTFilesToBlockInputStream::readCommitedBlock() [tiflash+120513958]
                    dbms/src/Storages/DeltaMerge/SSTFilesToBlockInputStream.cpp:238
       0x72edcca    DB::DM::SSTFilesToBlockInputStream::read() [tiflash+120511690]
                    dbms/src/Storages/DeltaMerge/SSTFilesToBlockInputStream.cpp:144
       0x71d25c5    DB::DM::readNextBlock(std::__1::shared_ptr<DB::IBlockInputStream> const&) [tiflash+119350725]
                    dbms/src/Storages/DeltaMerge/DeltaMergeHelpers.h:253
       0x72eeec2    DB::DM::PKSquashingBlockInputStream<true>::read() [tiflash+120516290]
                    dbms/src/Storages/DeltaMerge/PKSquashingBlockInputStream.h:68
       0x71d25c5    DB::DM::readNextBlock(std::__1::shared_ptr<DB::IBlockInputStream> const&) [tiflash+119350725]
                    dbms/src/Storages/DeltaMerge/DeltaMergeHelpers.h:253
       0x1c9f065    DB::DM::DMVersionFilterBlockInputStream<1>::initNextBlock() [tiflash+30011493]
                    dbms/src/Storages/DeltaMerge/DMVersionFilterBlockInputStream.h:134
       0x1c9d16c    DB::DM::DMVersionFilterBlockInputStream<1>::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool) [tiflash+30003564]
                    dbms/src/Storages/DeltaMerge/DMVersionFilterBlockInputStream.cpp:51
       0x72e954c    DB::DM::SSTFilesToDTFilesOutputStream::write() [tiflash+120493388]
                    dbms/src/Storages/DeltaMerge/SSTFilesToDTFilesOutputStream.cpp:162
       0x72e2dc3    DB::KVStore::preHandleSSTsToDTFiles(std::__1::shared_ptr<DB::Region>, DB::SSTViewVec, unsigned long, unsigned long, DB::DM::FileConvertJobType, DB::TMTContext&) [tiflash+120466883]
                    dbms/src/Storages/Transaction/ApplySnapshot.cpp:340
       0x733183a    PreHandleSnapshot [tiflash+120789050]
                    dbms/src/Storages/Transaction/ProxyFFI.cpp:387
  0x7f059e8b3e2a    raftstore::engine_store_ffi::_$LT$impl$u20$raftstore..engine_store_ffi..interfaces..root..DB..EngineStoreServerHelper$GT$::pre_handle_snapshot::h718eff40e0ff22c6 [libtiflash_proxy.so+33197610]
  0x7f059e86be89    raftstore::store::snap::Snapshot::pre_handle_snapshot::h81c04c547dfcb1a8 [libtiflash_proxy.so+32902793]
  0x7f059df01536    yatp::task::future::RawTask$LT$F$GT$::poll::h68cf0c4ec7b4731e [libtiflash_proxy.so+23029046]
  0x7f059f51ec0d    _$LT$yatp..task..future..Runner$u20$as$u20$yatp..pool..runner..Runner$GT$::handle::h6502e9a2b4ccd66e [libtiflash_proxy.so+46218253]
  0x7f059f52143d    std::sys_common::backtrace::__rust_begin_short_backtrace::hc5d6c9c6e9339eea [libtiflash_proxy.so+46228541]
  0x7f059f51413d    core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h8a6505e26346d93a [libtiflash_proxy.so+46174525]
  0x7f059ec5ca0a    std::sys::unix::thread::Thread::new::thread_start::hd39c5f08bdcda277 [libtiflash_proxy.so+37034506]
  0x7f059bfece63    start_thread [libpthread.so.0+28259]
                    /root/yuzhao/glibc-2.17/nptl/pthread_create.c:308
  0x7f059bb0655d    __clone [libc.so.6+968029]
                    /root/yuzhao/glibc-2.17/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113"] [thread_id=48]

4. What is your TiFlash version? (Required)

v5.4.1

[JaySon-Huang]

Fixed in 5.4.3/6.1.2/6.3.0 and later versions