Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix cleaning up broken detached part after timeout #41190

Merged
merged 1 commit into from
Sep 12, 2022

Conversation

kssenii
Copy link
Member

@kssenii kssenii commented Sep 11, 2022

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in official stable or prestable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Fix background clean up of broken detached parts.

@kssenii kssenii force-pushed the fix-clean-up-thread branch from e356734 to 1c38d4f Compare September 11, 2022 13:51
@robot-ch-test-poll robot-ch-test-poll added the pr-bugfix Pull request with bugfix, not backported by default label Sep 11, 2022
@kssenii kssenii force-pushed the fix-clean-up-thread branch from 1c38d4f to 02c3d8b Compare September 11, 2022 14:43
@antonio2368 antonio2368 self-assigned this Sep 12, 2022
@kssenii
Copy link
Member Author

kssenii commented Sep 12, 2022

Stress test (tsan) — Cannot start clickhouse-server

2022.09.11 20:46:49.376505 [ 513684 ] {} <Error> Application: Caught exception while loading metadata: Code: 32. DB::Exception: Attempt to read after eof: Cannot attach table `test_13`.`table` from metadata file /var/lib/clickhouse/metadata/test_13/table.sql from query ATTACH TABLE test_13.table (`x` UInt8) ENGINE = MergeTree ORDER BY x SETTINGS index_granularity = 8192. (ATTEMPT_TO_READ_AFTER_EOF), Stack trace (when copying this message, always include the lines below):

0. ./build_docker/../contrib/libcxx/include/exception:0: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x2010e4db in /usr/bin/clickhouse
1. ./build_docker/../src/Common/Exception.cpp:69: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xb65a21c in /usr/bin/clickhouse
2. ./build_docker/../src/IO/VarInt.h:0: DB::throwReadAfterEOF() @ 0xb63d725 in /usr/bin/clickhouse
3. ./build_docker/../src/IO/ReadHelpers.h:0: void DB::readIntTextImpl<unsigned int, void, (DB::ReadIntTextCheckOverflow)0>(unsigned int&, DB::ReadBuffer&) @ 0xb80c1ab in /usr/bin/clickhouse
4. ./build_docker/../src/IO/ReadHelpers.h:0: DB::DiskObjectStorageMetadata::deserialize(DB::ReadBuffer&) @ 0x1aa66154 in /usr/bin/clickhouse
5. ./build_docker/../src/Disks/ObjectStorages/DiskObjectStorageMetadata.cpp:83: DB::DiskObjectStorageMetadata::deserializeFromString(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x1aa67757 in /usr/bin/clickhouse
6. ./build_docker/../contrib/libcxx/include/string:1445: DB::MetadataStorageFromDisk::readMetadataUnlocked(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_lock<std::__1::shared_mutex>&) const @ 0x1aa60464 in /usr/bin/clickhouse
7. ./build_docker/../src/Disks/ObjectStorages/MetadataStorageFromDisk.cpp:0: DB::MetadataStorageFromDisk::getStorageObjects(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const @ 0x1aa60d78 in /usr/bin/clickhouse
8. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:816: DB::DiskObjectStorage::readFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ReadSettings const&, std::__1::optional<unsigned long>, std::__1::optional<unsigned long>) const @ 0x1aa06867 in /usr/bin/clickhouse
9. ./build_docker/../src/Disks/DiskDecorator.cpp:0: DB::DiskDecorator::readFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ReadSettings const&, std::__1::optional<unsigned long>, std::__1::optional<unsigned long>) const @ 0x1aa51821 in /usr/bin/clickhouse
10. ./build_docker/../src/Disks/DiskRestartProxy.cpp:0: DB::DiskRestartProxy::readFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ReadSettings const&, std::__1::optional<unsigned long>, std::__1::optional<unsigned long>) const @ 0x1aa4d0c8 in /usr/bin/clickhouse

interesting... but not related to changes, will take a look

@kssenii kssenii merged commit df22e8d into ClickHouse:master Sep 12, 2022
@robot-clickhouse robot-clickhouse added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label Sep 12, 2022
kssenii added a commit that referenced this pull request Sep 13, 2022
Backport #41190 to 22.8: Fix cleaning up broken detached part after timeout
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-bugfix Pull request with bugfix, not backported by default
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants