Skip to content

Commit

Permalink
[config](checksum) Disable consistency checker by default
Browse files Browse the repository at this point in the history
Disable by default because current checksum logic has some bugs.
And it will also bring some overhead.
  • Loading branch information
morningman committed May 20, 2022
1 parent 61a60d1 commit 2359bdf
Show file tree
Hide file tree
Showing 4 changed files with 22 additions and 9 deletions.
10 changes: 7 additions & 3 deletions docs/en/admin-manual/config/fe-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -1017,17 +1017,21 @@ IsMutable:true

MasterOnly:true

Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*. Default is from 23:00 to 04:00
Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.

If the two times are the same, no consistency check will be triggered.

### consistency_check_end_time

Default:04
Default:23

IsMutable:true

MasterOnly:true

Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*. Default is from 23:00 to 04:00
Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.

If the two times are the same, no consistency check will be triggered.

### export_tablet_num_per_task

Expand Down
10 changes: 7 additions & 3 deletions docs/zh-CN/admin-manual/config/fe-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -1017,19 +1017,23 @@ colocote join PlanFragment instance 的 memory_limit = exec_mem_limit / min (que

一致性检查开始时间

一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。 默认为 23:00 至 04:00
一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`

如果两个时间相同,则不会触发一致性检查。

### `consistency_check_end_time`

默认值:04
默认值:23

是否可以动态配置:true

是否为 Master FE 节点独有的配置项:true

一致性检查结束时间

一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。 默认为 23:00 至 04:00
一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`

如果两个时间相同,则不会触发一致性检查。

### `export_tablet_num_per_task`

Expand Down
7 changes: 5 additions & 2 deletions fe/fe-core/src/main/java/org/apache/doris/common/Config.java
Original file line number Diff line number Diff line change
Expand Up @@ -856,12 +856,15 @@ public class Config extends ConfigBase {
// Configurations for consistency check
/**
* Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.
* Default is from 23:00 to 04:00
* If start time == end time, the checker will stop scheduling.
* And default is disabled.
* TODO(cmy): Disable by default because current checksum logic has some bugs.
* And it will also bring some overhead.
*/
@ConfField(mutable = true, masterOnly = true)
public static String consistency_check_start_time = "23";
@ConfField(mutable = true, masterOnly = true)
public static String consistency_check_end_time = "4";
public static String consistency_check_end_time = "23";
/**
* Default timeout of a single consistency check task. Set long enough to fit your tablet size.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@
import com.google.common.collect.Maps;
import com.google.common.collect.Multiset;
import com.google.common.collect.Sets;

import org.apache.commons.collections.map.HashedMap;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
Expand Down Expand Up @@ -640,7 +641,8 @@ private void sendFragment() throws TException, RpcException, UserException {
switch (code) {
case TIMEOUT:
throw new RpcException(pair.first.backend.getHost(), "send fragment timeout. backend id: "
+ pair.first.backend.getId());
+ pair.first.backend.getId() + " fragment: " +
DebugUtil.printId(pair.first.rpcParams.params.fragment_instance_id));
case THRIFT_RPC_ERROR:
SimpleScheduler.addToBlacklist(pair.first.backend.getId(), errMsg);
throw new RpcException(pair.first.backend.getHost(), "rpc failed");
Expand Down

0 comments on commit 2359bdf

Please sign in to comment.