You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
set set flash.compact_log_min_period to 60 (just and arbitrary value that is not close to 0, default value is 120);
load tpch_10.lineitem to tiflash;
execute the sql "delete from lineitem limit 300000" until all rows is deleted;
wait for some time and restart tiflash;
If we set flash.compact_log_min_period to 0(default value is 120) and redo the above steps 2 - 4, the problem won't occur.
flash.compact_log_min_period controls the min time tiflash can flush region in memory to disk, and we will ignore compact log raft command in the period.
And tikv by default send compact log raft command every 10s for all updated region. So if the config value is too big, there may be too much region data left in memory, and the applied index will left behind a lot.
And after we ignore the compact log raft command, we will never receive new command to flush region data and forward applied index because of no more update. So when we restart tiflash, there will be many raft command to apply, and it will cost a lot of memory.
Reproduce step:
If we set flash.compact_log_min_period to 0(default value is 120) and redo the above steps 2 - 4, the problem won't occur.

flash.compact_log_min_period controls the min time tiflash can flush region in memory to disk, and we will ignore compact log raft command in the period.
And tikv by default send compact log raft command every 10s for all updated region. So if the config value is too big, there may be too much region data left in memory, and the applied index will left behind a lot.
And after we ignore the compact log raft command, we will never receive new command to flush region data and forward applied index because of no more update. So when we restart tiflash, there will be many raft command to apply, and it will cost a lot of memory.
But we cannot simply set flash.compact_log_min_period to 0 because of this issue: https://docs.google.com/document/d/1eg7CDiVkXIgHpN9GdCA-puhi0UBCvs28H44Kj7-QjS0/edit#bookmark=id.avai2iw8w9d3
The text was updated successfully, but these errors were encountered: