Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[config](checksum) Disable consistency checker by default #9699

Merged
merged 1 commit into from
May 22, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions docs/en/admin-manual/config/fe-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -1017,17 +1017,21 @@ IsMutable:true

MasterOnly:true

Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*. Default is from 23:00 to 04:00
Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.

If the two times are the same, no consistency check will be triggered.

### consistency_check_end_time

Default:04
Default:23

IsMutable:true

MasterOnly:true

Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*. Default is from 23:00 to 04:00
Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.

If the two times are the same, no consistency check will be triggered.

### export_tablet_num_per_task

Expand Down
10 changes: 7 additions & 3 deletions docs/zh-CN/admin-manual/config/fe-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -1017,19 +1017,23 @@ colocote join PlanFragment instance 的 memory_limit = exec_mem_limit / min (que

一致性检查开始时间

一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。 默认为 23:00 至 04:00
一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。

如果两个时间相同,则不会触发一致性检查。

### `consistency_check_end_time`

默认值:04
默认值:23

是否可以动态配置:true

是否为 Master FE 节点独有的配置项:true

一致性检查结束时间

一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。 默认为 23:00 至 04:00
一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。

如果两个时间相同,则不会触发一致性检查。

### `export_tablet_num_per_task`

Expand Down
7 changes: 5 additions & 2 deletions fe/fe-core/src/main/java/org/apache/doris/common/Config.java
Original file line number Diff line number Diff line change
Expand Up @@ -856,12 +856,15 @@ public class Config extends ConfigBase {
// Configurations for consistency check
/**
* Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.
* Default is from 23:00 to 04:00
* If start time == end time, the checker will stop scheduling.
* And default is disabled.
* TODO(cmy): Disable by default because current checksum logic has some bugs.
* And it will also bring some overhead.
*/
@ConfField(mutable = true, masterOnly = true)
public static String consistency_check_start_time = "23";
@ConfField(mutable = true, masterOnly = true)
public static String consistency_check_end_time = "4";
public static String consistency_check_end_time = "23";
/**
* Default timeout of a single consistency check task. Set long enough to fit your tablet size.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@
import com.google.common.collect.Maps;
import com.google.common.collect.Multiset;
import com.google.common.collect.Sets;

import org.apache.commons.collections.map.HashedMap;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
Expand Down Expand Up @@ -640,7 +641,8 @@ private void sendFragment() throws TException, RpcException, UserException {
switch (code) {
case TIMEOUT:
throw new RpcException(pair.first.backend.getHost(), "send fragment timeout. backend id: "
+ pair.first.backend.getId());
+ pair.first.backend.getId() + " fragment: " +
DebugUtil.printId(pair.first.rpcParams.params.fragment_instance_id));
case THRIFT_RPC_ERROR:
SimpleScheduler.addToBlacklist(pair.first.backend.getId(), errMsg);
throw new RpcException(pair.first.backend.getHost(), "rpc failed");
Expand Down