support victoriametrics as metrics data storage (#1361)

Signed-off-by: tomsun28 <[email protected]>

README.md

@@ -29,8 +29,9 @@
 ### Features
 
 * Combines **monitoring, alarm, and notification** features into one platform, and supports monitoring for web service, program, database, cache, os, webserver, middleware, bigdata, cloud-native, network, custom and more.
-* Easy to use and agentless, offering full web-based operations for monitoring and alerting with just a few clicks, all at zero learning cost.
+* Easy to use and agentless, offering full gui operations for monitoring and alerting with just a few clicks, all at zero learning cost.
 * Makes protocols such as `Http, Jmx, Ssh, Snmp, Jdbc, Prometheus` configurable, allowing you to collect any metrics by simply configuring the template `YML` file online. Imagine being able to quickly adapt to a new monitoring type like K8s or Docker simply by configuring online with HertzBeat.
+* Compatible with the `Prometheus` ecosystem and more, can monitoring what `Prometheus` can monitoring with few clicks on gui.
 * High performance, supports horizontal expansion of multi-collector clusters, multi-isolated network monitoring and cloud-edge collaboration.
 * Provides flexible alarm threshold rules and timely notifications delivered via  `Discord` `Slack` `Telegram` `Email` `Dingtalk` `WeChat` `FeiShu` `Webhook` `SMS` `ServerChan`.
 
@@ -435,10 +436,10 @@ HertzBeat is a top project under the [Dromara Open Source Community](https://dro
 
 ##### Open-Source Project Build From Open-Source
 
-> HertzBeat is built on so many great open source projects, thanks to them!
+HertzBeat is built on so many great open source projects, thanks to them!
 
-- Java Spring SpringBoot Jpa Maven Assembly Netty Lombok Sureness Aviator Protobuf HttpClient Guava SnakeYaml JsonPath ...
-- TypeScript Angular NG-ZORRO NG-ALAIN NodeJs Npm Html Less Echarts Rxjs ZoneJs MonacoEditor SlickCarousel Docusaurus ...
+- `Java Spring SpringBoot Jpa Maven Assembly Netty Lombok Sureness Aviator Protobuf HttpClient Guava SnakeYaml JsonPath ...`
+- `TypeScript Angular NG-ZORRO NG-ALAIN NodeJs Npm Html Less Echarts Rxjs ZoneJs MonacoEditor SlickCarousel Docusaurus ...`
 
 
 ## Star History

README_CN.md

@@ -32,6 +32,7 @@
 - 集 **监控+告警+通知** 为一体，支持对应用服务，应用程序，数据库，缓存，操作系统，大数据，中间件，Web服务器，云原生，网络，自定义等监控阈值告警通知一步到位。
 - 易用友好，无需 `Agent`，全 `WEB` 页面操作，鼠标点一点就能监控告警，零上手学习成本。    
 - 将 `Http, Jmx, Ssh, Snmp, Jdbc, Prometheus` 等协议规范可配置化，只需在浏览器配置监控模版 `YML` 就能使用这些协议去自定义采集想要的指标。您相信只需配置下就能立刻适配一款 `K8s` 或 `Docker` 等新的监控类型吗？  
+- 兼容 `Prometheus` 的系统生态并且更多，只需页面操作就可以监控 `Prometheus` 所能监控的。
 - 高性能，支持多采集器集群横向扩展，支持多隔离网络监控，云边协同。
 - 自由的告警阈值规则，`邮件` `Discord` `Slack` `Telegram` `钉钉` `微信` `飞书` `短信` `Webhook` `Server酱` 等方式消息及时送达。             
 
@@ -449,5 +450,13 @@ HertzBeat 赫兹跳动是 [Dromara开源社区](https://dromara.org/) 下顶级
 - TypeScript Angular NG-ZORRO NG-ALAIN NodeJs Npm Html Less Echarts Rxjs ZoneJs MonacoEditor SlickCarousel Docusaurus ...
 
 
+##### Open-Source Project Build From Open-Source
+
+HertzBeat is built on so many great open source projects, thanks to them!
+
+- `Java Spring SpringBoot Jpa Maven Assembly Netty Lombok Sureness Aviator Protobuf HttpClient Guava SnakeYaml JsonPath ...`
+- `TypeScript Angular NG-ZORRO NG-ALAIN NodeJs Npm Html Less Echarts Rxjs ZoneJs MonacoEditor SlickCarousel Docusaurus ...`
+
+
 ## 🛡️ License
 [`Apache License, Version 2.0`](https://www.apache.org/licenses/LICENSE-2.0.html)

alerter/src/main/java/org/dromara/hertzbeat/alert/calculate/CalculateAlarm.java

@@ -183,10 +183,7 @@ private void calculate(CollectRep.MetricsData metricsData) {
                     }
                     fieldValueMap.clear();
                     fieldValueMap.put(SYSTEM_VALUE_ROW_COUNT, valueRowCount);
-                    String instance = valueRow.getInstance();
-                    if (!"".equals(instance)) {
-                        fieldValueMap.put("instance", instance);
-                    }
+                    StringBuilder instanceBuilder = new StringBuilder();
                     for (int index = 0; index < valueRow.getColumnsList().size(); index++) {
                         String valueStr = valueRow.getColumns(index);
                         if (CommonConstants.NULL_VALUE.equals(valueStr)) {
@@ -204,6 +201,9 @@ private void calculate(CollectRep.MetricsData metricsData) {
                                 fieldValueMap.put(field.getName(), valueStr);
                             }
                         }
+                        if (field.getLabel()) {
+                            instanceBuilder.append(valueStr).append("-");
+                        }
                     }
                     try {
                         boolean match = execAlertExpression(fieldValueMap, expr);
@@ -214,7 +214,7 @@ private void calculate(CollectRep.MetricsData metricsData) {
                             // 若此阈值已被触发，则其它数据行的触发忽略
                             break;
                         } else if (define.isRecoverNotice()) {
-                            String notResolvedAlertKey = String.valueOf(monitorId) + define.getId() + (!"".equals(instance) ? instance : null);
+                            String notResolvedAlertKey = String.valueOf(monitorId) + define.getId() + (instanceBuilder.length() == 0 ? null : instanceBuilder.toString());
                             handleRecoveredAlert(currentTimeMilli, monitorId, app, define, expr, notResolvedAlertKey);
                         }
                     } catch (Exception e) {

collector/src/main/java/org/dromara/hertzbeat/collector/collect/prometheus/PrometheusAutoCollectImpl.java

@@ -206,9 +206,11 @@ private List<CollectRep.MetricsData> parseResponseByPrometheusExporter(String re
                     if (index == 0) {
                         metric.getLabelPair().forEach(label -> {
                             metricsFields.add(label.getName());
-                            builder.addFields(CollectRep.Field.newBuilder().setName(label.getName()).setType(CommonConstants.TYPE_STRING).build());
+                            builder.addFields(CollectRep.Field.newBuilder().setName(label.getName())
+                                    .setType(CommonConstants.TYPE_STRING).setLabel(true).build());
                         });
-                        builder.addFields(CollectRep.Field.newBuilder().setName("value").setType(CommonConstants.TYPE_NUMBER).build());
+                        builder.addFields(CollectRep.Field.newBuilder().setName("value")
+                                .setType(CommonConstants.TYPE_NUMBER).setLabel(false).build());
                     }
                     Map<String, String> labelMap = metric.getLabelPair()
                             .stream()

collector/src/main/java/org/dromara/hertzbeat/collector/dispatch/MetricsCollect.java

@@ -201,7 +201,7 @@ private void calculateFields(Metrics metrics, CollectRep.MetricsData.Builder col
         List<CollectRep.Field> fieldList = new LinkedList<>();
         for (Metrics.Field field : metrics.getFields()) {
             CollectRep.Field.Builder fieldBuilder = CollectRep.Field.newBuilder();
-            fieldBuilder.setName(field.getField()).setType(field.getType());
+            fieldBuilder.setName(field.getField()).setType(field.getType()).setLabel(field.isLabel());
             if (field.getUnit() != null) {
                 fieldBuilder.setUnit(field.getUnit());
             }
@@ -326,13 +326,8 @@ private void calculateFields(Metrics metrics, CollectRep.MetricsData.Builder col
                 }
                 realValueRowBuilder.addColumns(value);
                 fieldValueMap.clear();
-                if (field.isInstance() && !CommonConstants.NULL_VALUE.equals(value)) {
-                    instanceBuilder.append(value);
-                }
             }
             aliasFieldValueMap.clear();
-            // set instance
-            realValueRowBuilder.setInstance(instanceBuilder.toString());
             collectData.addValues(realValueRowBuilder.build());
             realValueRowBuilder.clear();
         }
@@ -377,20 +372,6 @@ private Object[] transformUnit(String unit) {
         return new Object[]{field, Pair.of(originUnit, newUnit)};
     }
 
-    /**
-     * build CollectRep.Field
-     * @param field
-     * @return
-     */
-    private CollectRep.Field doCollectRepField(Metrics.Field field) {
-        CollectRep.Field.Builder fieldBuilder = CollectRep.Field.newBuilder();
-        fieldBuilder.setName(field.getField()).setType(field.getType());
-        if (field.getUnit() != null) {
-            fieldBuilder.setUnit(field.getUnit());
-        }
-        return fieldBuilder.build();
-    }
-
     private boolean fastFailed() {
         return this.timeout == null || this.timeout.isCancelled();
     }

common/src/main/java/org/dromara/hertzbeat/common/entity/dto/Field.java

@@ -46,5 +46,8 @@ public class Field {
 
     @Schema(title = "是否是实例字段")
     private Boolean instance;
+
+    @Schema(title = "是否是标签字段")
+    private Boolean label;
 
 }

common/src/main/java/org/dromara/hertzbeat/common/entity/dto/ValueRow.java

@@ -24,22 +24,23 @@
 import lombok.NoArgsConstructor;
 
 import java.util.List;
+import java.util.Map;
 
 /**
- * 监控指标组的一行指标数据
+ * 监控指标集合的一行指标数据
  * @author tom
  *
  */
 @Data
 @Builder
 @AllArgsConstructor
 @NoArgsConstructor
-@Schema(description = "监控指标组的一行指标数据")
+@Schema(description = "监控指标集合的一行指标数据")
 public class ValueRow {
+
+    @Schema(title = "Value Row Labels")
+    private Map<String, String> labels;
 
-    @Schema(title = "此行数据唯一实例")
-    private String instance;
-
-    @Schema(description = "监控指标组指标值")
+    @Schema(description = "监控指标集合指标值")
     private List<Value> values;
 }

common/src/main/java/org/dromara/hertzbeat/common/entity/job/Metrics.java

@@ -273,9 +273,12 @@ public static class Field {
         private byte type = 1;
         /**
          * Whether this field is the instance primary key
-         * 此字段是否为实例主键
          */
         private boolean instance = false;
+        /**
+         * Whether this field is the label
+         */
+        private boolean label = false;
         /**
          * Indicator unit
          * 指标单位