Skip to content

Commit

Permalink
support victoriametrics as metrics data storage (#1361)
Browse files Browse the repository at this point in the history
Signed-off-by: tomsun28 <[email protected]>
  • Loading branch information
tomsun28 committed Mar 10, 2024
1 parent c79f3bc commit 7312468
Show file tree
Hide file tree
Showing 129 changed files with 5,523 additions and 5,406 deletions.
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,9 @@
### Features

* Combines **monitoring, alarm, and notification** features into one platform, and supports monitoring for web service, program, database, cache, os, webserver, middleware, bigdata, cloud-native, network, custom and more.
* Easy to use and agentless, offering full web-based operations for monitoring and alerting with just a few clicks, all at zero learning cost.
* Easy to use and agentless, offering full gui operations for monitoring and alerting with just a few clicks, all at zero learning cost.
* Makes protocols such as `Http, Jmx, Ssh, Snmp, Jdbc, Prometheus` configurable, allowing you to collect any metrics by simply configuring the template `YML` file online. Imagine being able to quickly adapt to a new monitoring type like K8s or Docker simply by configuring online with HertzBeat.
* Compatible with the `Prometheus` ecosystem and more, can monitoring what `Prometheus` can monitoring with few clicks on gui.
* High performance, supports horizontal expansion of multi-collector clusters, multi-isolated network monitoring and cloud-edge collaboration.
* Provides flexible alarm threshold rules and timely notifications delivered via `Discord` `Slack` `Telegram` `Email` `Dingtalk` `WeChat` `FeiShu` `Webhook` `SMS` `ServerChan`.

Expand Down Expand Up @@ -435,10 +436,10 @@ HertzBeat is a top project under the [Dromara Open Source Community](https://dro

##### Open-Source Project Build From Open-Source

> HertzBeat is built on so many great open source projects, thanks to them!
HertzBeat is built on so many great open source projects, thanks to them!

- Java Spring SpringBoot Jpa Maven Assembly Netty Lombok Sureness Aviator Protobuf HttpClient Guava SnakeYaml JsonPath ...
- TypeScript Angular NG-ZORRO NG-ALAIN NodeJs Npm Html Less Echarts Rxjs ZoneJs MonacoEditor SlickCarousel Docusaurus ...
- `Java Spring SpringBoot Jpa Maven Assembly Netty Lombok Sureness Aviator Protobuf HttpClient Guava SnakeYaml JsonPath ...`
- `TypeScript Angular NG-ZORRO NG-ALAIN NodeJs Npm Html Less Echarts Rxjs ZoneJs MonacoEditor SlickCarousel Docusaurus ...`


## Star History
Expand Down
9 changes: 9 additions & 0 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
-**监控+告警+通知** 为一体,支持对应用服务,应用程序,数据库,缓存,操作系统,大数据,中间件,Web服务器,云原生,网络,自定义等监控阈值告警通知一步到位。
- 易用友好,无需 `Agent`,全 `WEB` 页面操作,鼠标点一点就能监控告警,零上手学习成本。
-`Http, Jmx, Ssh, Snmp, Jdbc, Prometheus` 等协议规范可配置化,只需在浏览器配置监控模版 `YML` 就能使用这些协议去自定义采集想要的指标。您相信只需配置下就能立刻适配一款 `K8s``Docker` 等新的监控类型吗?
- 兼容 `Prometheus` 的系统生态并且更多,只需页面操作就可以监控 `Prometheus` 所能监控的。
- 高性能,支持多采集器集群横向扩展,支持多隔离网络监控,云边协同。
- 自由的告警阈值规则,`邮件` `Discord` `Slack` `Telegram` `钉钉` `微信` `飞书` `短信` `Webhook` `Server酱` 等方式消息及时送达。

Expand Down Expand Up @@ -449,5 +450,13 @@ HertzBeat 赫兹跳动是 [Dromara开源社区](https://dromara.org/) 下顶级
- TypeScript Angular NG-ZORRO NG-ALAIN NodeJs Npm Html Less Echarts Rxjs ZoneJs MonacoEditor SlickCarousel Docusaurus ...


##### Open-Source Project Build From Open-Source

HertzBeat is built on so many great open source projects, thanks to them!

- `Java Spring SpringBoot Jpa Maven Assembly Netty Lombok Sureness Aviator Protobuf HttpClient Guava SnakeYaml JsonPath ...`
- `TypeScript Angular NG-ZORRO NG-ALAIN NodeJs Npm Html Less Echarts Rxjs ZoneJs MonacoEditor SlickCarousel Docusaurus ...`


## 🛡️ License
[`Apache License, Version 2.0`](https://www.apache.org/licenses/LICENSE-2.0.html)
Original file line number Diff line number Diff line change
Expand Up @@ -183,10 +183,7 @@ private void calculate(CollectRep.MetricsData metricsData) {
}
fieldValueMap.clear();
fieldValueMap.put(SYSTEM_VALUE_ROW_COUNT, valueRowCount);
String instance = valueRow.getInstance();
if (!"".equals(instance)) {
fieldValueMap.put("instance", instance);
}
StringBuilder instanceBuilder = new StringBuilder();
for (int index = 0; index < valueRow.getColumnsList().size(); index++) {
String valueStr = valueRow.getColumns(index);
if (CommonConstants.NULL_VALUE.equals(valueStr)) {
Expand All @@ -204,6 +201,9 @@ private void calculate(CollectRep.MetricsData metricsData) {
fieldValueMap.put(field.getName(), valueStr);
}
}
if (field.getLabel()) {
instanceBuilder.append(valueStr).append("-");
}
}
try {
boolean match = execAlertExpression(fieldValueMap, expr);
Expand All @@ -214,7 +214,7 @@ private void calculate(CollectRep.MetricsData metricsData) {
// 若此阈值已被触发,则其它数据行的触发忽略
break;
} else if (define.isRecoverNotice()) {
String notResolvedAlertKey = String.valueOf(monitorId) + define.getId() + (!"".equals(instance) ? instance : null);
String notResolvedAlertKey = String.valueOf(monitorId) + define.getId() + (instanceBuilder.length() == 0 ? null : instanceBuilder.toString());
handleRecoveredAlert(currentTimeMilli, monitorId, app, define, expr, notResolvedAlertKey);
}
} catch (Exception e) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -206,9 +206,11 @@ private List<CollectRep.MetricsData> parseResponseByPrometheusExporter(String re
if (index == 0) {
metric.getLabelPair().forEach(label -> {
metricsFields.add(label.getName());
builder.addFields(CollectRep.Field.newBuilder().setName(label.getName()).setType(CommonConstants.TYPE_STRING).build());
builder.addFields(CollectRep.Field.newBuilder().setName(label.getName())
.setType(CommonConstants.TYPE_STRING).setLabel(true).build());
});
builder.addFields(CollectRep.Field.newBuilder().setName("value").setType(CommonConstants.TYPE_NUMBER).build());
builder.addFields(CollectRep.Field.newBuilder().setName("value")
.setType(CommonConstants.TYPE_NUMBER).setLabel(false).build());
}
Map<String, String> labelMap = metric.getLabelPair()
.stream()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ private void calculateFields(Metrics metrics, CollectRep.MetricsData.Builder col
List<CollectRep.Field> fieldList = new LinkedList<>();
for (Metrics.Field field : metrics.getFields()) {
CollectRep.Field.Builder fieldBuilder = CollectRep.Field.newBuilder();
fieldBuilder.setName(field.getField()).setType(field.getType());
fieldBuilder.setName(field.getField()).setType(field.getType()).setLabel(field.isLabel());
if (field.getUnit() != null) {
fieldBuilder.setUnit(field.getUnit());
}
Expand Down Expand Up @@ -326,13 +326,8 @@ private void calculateFields(Metrics metrics, CollectRep.MetricsData.Builder col
}
realValueRowBuilder.addColumns(value);
fieldValueMap.clear();
if (field.isInstance() && !CommonConstants.NULL_VALUE.equals(value)) {
instanceBuilder.append(value);
}
}
aliasFieldValueMap.clear();
// set instance
realValueRowBuilder.setInstance(instanceBuilder.toString());
collectData.addValues(realValueRowBuilder.build());
realValueRowBuilder.clear();
}
Expand Down Expand Up @@ -377,20 +372,6 @@ private Object[] transformUnit(String unit) {
return new Object[]{field, Pair.of(originUnit, newUnit)};
}

/**
* build CollectRep.Field
* @param field
* @return
*/
private CollectRep.Field doCollectRepField(Metrics.Field field) {
CollectRep.Field.Builder fieldBuilder = CollectRep.Field.newBuilder();
fieldBuilder.setName(field.getField()).setType(field.getType());
if (field.getUnit() != null) {
fieldBuilder.setUnit(field.getUnit());
}
return fieldBuilder.build();
}

private boolean fastFailed() {
return this.timeout == null || this.timeout.isCancelled();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,5 +46,8 @@ public class Field {

@Schema(title = "是否是实例字段")
private Boolean instance;

@Schema(title = "是否是标签字段")
private Boolean label;

}
Original file line number Diff line number Diff line change
Expand Up @@ -24,22 +24,23 @@
import lombok.NoArgsConstructor;

import java.util.List;
import java.util.Map;

/**
* 监控指标组的一行指标数据
* 监控指标集合的一行指标数据
* @author tom
*
*/
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
@Schema(description = "监控指标组的一行指标数据")
@Schema(description = "监控指标集合的一行指标数据")
public class ValueRow {

@Schema(title = "Value Row Labels")
private Map<String, String> labels;

@Schema(title = "此行数据唯一实例")
private String instance;

@Schema(description = "监控指标组指标值")
@Schema(description = "监控指标集合指标值")
private List<Value> values;
}
Original file line number Diff line number Diff line change
Expand Up @@ -273,9 +273,12 @@ public static class Field {
private byte type = 1;
/**
* Whether this field is the instance primary key
* 此字段是否为实例主键
*/
private boolean instance = false;
/**
* Whether this field is the label
*/
private boolean label = false;
/**
* Indicator unit
* 指标单位
Expand Down
Loading

0 comments on commit 7312468

Please sign in to comment.