fix: kafka connector dependency (#3408)

openmldb-jdbc needs commons-io, so pack it
4paradigm · Aug 15, 2023 · e48cac9 · e48cac9
1 parent e2dd0a5
commit e48cac9
Show file tree

Hide file tree

Showing 3 changed files with 24 additions and 1 deletion.
diff --git a/docs/zh/integration/online_datasources/kafka_connector_demo.md b/docs/zh/integration/online_datasources/kafka_connector_demo.md
@@ -24,6 +24,12 @@ OpenMLDB Kafka Connector实现见[extensions/kafka-connect-jdbc](https://github.
 docker run -it -v `pwd`:/work/kafka 4pdosc/openmldb:0.8.2 bash
 ```
 
+### 注意事项
+
+Timestamp列精度为ms，value配置为JsonConvertor，仅支持整型。根据消息的不同，可选配其他Convertor。
+
+Connector可用于较早版本的Kafka Server，例如1.1.1，但注意旧版本的Kafka Broker可能并未默认开启“自动创建topic”，需要开启[此选项](https://kafka.apache.org/documentation/#brokerconfigs_auto.create.topics.enable)。
+
 ### 流程
 
 使用connector的简要流程，如下图所示。我们接下来将详细介绍每一步。

diff --git a/extensions/kafka-connect-jdbc/DEVELOP.md b/extensions/kafka-connect-jdbc/DEVELOP.md
@@ -28,6 +28,23 @@ auto.create=false
 ```
 请确保已经建好了OpenMLDB表。
 
+### Message样式
+
+仅支持Json格式的Message，不用写schema，因此只需要`<col>:<value>`的Map，例如：
+```
+{
+    "c1_int16": 1,
+    "c2_int32": 2,
+    "c3_int64": 3,
+    "c4_float": 4.4,
+    "c5_double": 5.555,
+    "c6_boolean": true,
+    "c7_string": "c77777",
+    "c8_date": 19109,
+    "c9_timestamp": 1651051906000
+}
+```
+
 ### message convert for auto schema
 
 auto schema开启后，主要逻辑在[BufferedRecords](src/main/java/io/confluent/connect/jdbc/sink/BufferedRecords.java)

diff --git a/extensions/kafka-connect-jdbc/pom.xml b/extensions/kafka-connect-jdbc/pom.xml
@@ -172,7 +172,7 @@
             <groupId>commons-io</groupId>
             <artifactId>commons-io</artifactId>
             <version>${commons-io.version}</version>
-            <scope>test</scope>
+            <!-- <scope>test</scope> openmldb-jdbc needs -->
         </dependency>
         <dependency>
             <groupId>org.mockito</groupId>