Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[branch-2.1](memory) Pick some memory GC patch #37725

Merged
merged 4 commits into from
Jul 14, 2024
Merged

[branch-2.1](memory) Pick some memory GC patch #37725

merged 4 commits into from
Jul 14, 2024

Conversation

xinyiZzz
Copy link
Contributor

xinyiZzz added 4 commits July 12, 2024 19:24
@xinyiZzz
[fix](memory) Fix free page not tracking correctly (apache#36768)
dcaee24 
## Proposed changes

apache#36235 introduced, index page is not freed in query thread, so saved
tracker ptr when alloc page, so that page only needs to be freed before
query ends.

```
F20240624 18:44:51.880192 162249 mem_tracker_limiter.cpp:125] mem tracker label: Query#Id=f0511107829e465e-a87a985234018b77, consumption: 94, peak consumption: 636540, mem tracker not equal to 0 when mem tracker destruct, this usually means that memory tracking is inaccurate and SCOPED_ATTACH_TASK and SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER are not used correctly. 1. For query and load, memory leaks may have occurred, it is expected that the query mem tracker will be bound to the thread context using SCOPED_ATTACH_TASK and SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER before all memory alloc and free. 2. If a memory alloc is recorded by this tracker, it is expected that be recorded in this tracker when memory is freed. 3. Merge the remaining memory tracking value by this tracker into Orphan, if you observe that Orphan is not equal to 0 in the mem tracker web or log, this indicates that there may be a memory leak. 4. If you need to transfer memory tracking value between two trackers, can use transfer_to..[Address Sanitizer]:
    0x605000160b30, size 46, strack trace:
        0#  Allocator<false, false, false>::alloc_impl(unsigned long, unsigned long)
        1#  doris::PageBase<Allocator<false, false, false> >::PageBase(unsigned long, bool, doris::segment_v2::PageTypePB)
        2#  doris::segment_v2::PageIO::read_and_decompress_page(doris::segment_v2::PageReadOptions const&, doris::segment_v2::PageHandle*, doris::Slice*, doris::segment_v2::PageFooterPB*)
        3#  doris::segment_v2::IndexedColumnReader::read_page(doris::segment_v2::PagePointer const&, doris::segment_v2::PageHandle*, doris::Slice*, doris::segment_v2::PageFooterPB*, doris::segment_v2::PageTypePB, doris::BlockCompressionCodec*, bool) const
        4#  doris::segment_v2::IndexedColumnReader::load_index_page(doris::segment_v2::PagePointerPB const&, doris::segment_v2::PageHandle*, doris::segment_v2::IndexPageReader*)
        5#  doris::segment_v2::IndexedColumnReader::load(bool, bool)
        6#  doris::segment_v2::BitmapIndexReader::_load(bool, bool, std::unique_ptr<doris::segment_v2::BitmapIndexPB, std::default_delete<doris::segment_v2::BitmapIndexPB> >)
        7#  doris::segment_v2::BitmapIndexReader::load(bool, bool)
        8#  doris::segment_v2::ColumnReader::_load_bitmap_index(bool, bool)
        9#  doris::segment_v2::ColumnReader::new_bitmap_index_iterator(doris::segment_v2::BitmapIndexIterator**)
        10# doris::segment_v2::Segment::new_bitmap_index_iterator(doris::TabletColumn const&, std::unique_ptr<doris::segment_v2::BitmapIndexIterator, std::default_delete<doris::segment_v2::BitmapIndexIterator> >*)
        11# doris::segment_v2::SegmentIterator::_init_bitmap_index_iterators()
        12# doris::segment_v2::SegmentIterator::init_iterators()
        13# doris::segment_v2::SegmentIterator::_init_impl(doris::StorageReadOptions const&)
        14# doris::segment_v2::SegmentIterator::init(doris::StorageReadOptions const&)
        15# doris::segment_v2::Segment::new_iterator(std::shared_ptr<doris::Schema const>, doris::StorageReadOptions const&, std::unique_ptr<doris::RowwiseIterator, std::default_delete<doris::RowwiseIterator> >*)
        16# doris::segment_v2::LazyInitSegmentIterator::init(doris::StorageReadOptions const&)
        17# doris::BetaRowsetReader::_init_iterator()
        18# doris::BetaRowsetReader::_init_iterator_once()
        19# doris::BetaRowsetReader::next_block(doris::vectorized::Block*)
        20# doris::vectorized::VCollectIterator::_topn_next(doris::vectorized::Block*)
        21# doris::vectorized::VCollectIterator::next(doris::vectorized::Block*)
        22# doris::vectorized::BlockReader::_direct_next_block(doris::vectorized::Block*, bool*)
        23# doris::vectorized::BlockReader::next_block_with_aggregation(doris::vectorized::Block*, bool*)
        24# doris::vectorized::NewOlapScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*)
        25# doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*)
        26# doris::vectorized::VScanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*)
        27# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)
        28# std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&)
        29# doris::ThreadPool::dispatch_thread()
        30# doris::Thread::supervise_thread(void*)
```
@xinyiZzz
[opt](memory) Purge all unused Jemalloc dirty pages for all arenas wh…
8ae1d9f 
…en exceed limit (apache#37164)

add BE conf je_dirty_pages_mem_limit_percent, default 5% * mem_limit,
purge all unused Jemalloc dirty pages for all arenas when exceed
je_dirty_pages_mem_limit and process exceed soft limit.
@xinyiZzz
[opt](memory) max_sys_mem_available_low_water_mark_bytesdefault val…
7bd8902 
…ue modified to 6.4G (apache#37174)

// The maximum low water mark of the system
`/proc/meminfo/MemAvailable`, Unit byte, default 6.4G,
// actual low water mark=min(6.4G, MemTotal * 10%), avoid wasting too
much memory on machines
// with large memory larger than 64G.
// Turn up max. On machines with more than 64G memory, more memory
buffers will be reserved for Full GC.
// Turn down max. will use as much memory as possible.
@xinyiZzz
[fix](memory) Modified low water mark=min(6.4G, MemTotal * 5%) (apach…
6e2552b 
…e#37525)

On 64G machine, warning water mark is 12.8G. when system available
memory reaches the warning water mark, load memtable will flush, this is
not a good idea.

`mem_limit` default 10%, except for the memory used by the Linux kernel
, buffer may be around 5%, so low water mark=min(6.4G, MemTotal * 5%)
looks better.
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@xinyiZzz
Copy link
Contributor Author

run buildall

HappenLee
HappenLee approved these changes Jul 12, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 12, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.45% (9190/25213)
Line Coverage: 27.98% (75131/268541)
Region Coverage: 26.85% (38748/144339)
Branch Coverage: 23.53% (19637/83464)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6e2552bcc89468fc06d37c27a2759a2b51bb8ce6_6e2552bcc89468fc06d37c27a2759a2b51bb8ce6/report/index.html

@xinyiZzz xinyiZzz merged commit 7471722 into apache:branch-2.1 Jul 14, 2024
20 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HappenLee HappenLee HappenLee approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@xinyiZzz @doris-robot @HappenLee