[dbquery] Initial contribution (openhab#8780)

* Initial commit Intial work history lost due to the repository shrunk done at c53e4ae (intially started from old unshrunked repo) Signed-off-by: Joan Pujol <[email protected]> * Implement reconnect attempts If database can be connected at bridge initialization schedule retry attempts. Prevent query execution scheduling if bridge isn't online Signed-off-by: Joan Pujol <[email protected]> * Minor documentation changes and fix trigger channel name Signed-off-by: Joan Pujol <[email protected]> * Fix NPE bug initializing ThingActions Signed-off-by: Joan Pujol <[email protected]> * Implement query actions and another fixes Implement actions to execute query and get last query result Correctly serialize as JSON non scalar results to result channels Signed-off-by: Joan Pujol <[email protected]> * Update parameters and correct channel Signed-off-by: Joan Pujol <[email protected]> * Fix formatting and forgot part on previous commit Signed-off-by: Joan Pujol <[email protected]> * Improve documentation Signed-off-by: Joan Pujol <[email protected]> * Add javadoc comment and license to all classes Signed-off-by: Joan Pujol <[email protected]> * Code cleanup Signed-off-by: Joan Pujol <[email protected]> * Untrack unused i18n file Signed-off-by: Joan Pujol <[email protected]> * Fix log level for query actions trace information Signed-off-by: Joan Pujol <[email protected]> * Add dbquery addon to bundles pom Signed-off-by: Joan Pujol <[email protected]> * Temporary remove mqtt bindings that make travis build to fail Signed-off-by: Joan Pujol <[email protected]> * Fix formatting issue Signed-off-by: Joan Pujol <[email protected]> * Revert "Temporary remove mqtt bindings that make travis build to fail" This reverts commit 21c0995. Signed-off-by: Joan Pujol <[email protected]> * Code clean up from static analysis Signed-off-by: Joan Pujol <[email protected]> * Update code to be compatible with 3.1.0 Update dependencies version Update Copyright Other minor changes for new static analysis validations. Signed-off-by: Joan Pujol <[email protected]> * Requested PR changes Signed-off-by: Joan Pujol <[email protected]> * Update bundles/org.openhab.binding.dbquery/src/main/java/org/openhab/binding/dbquery/internal/JDBCBridgeHandler.java Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/src/main/java/org/openhab/binding/dbquery/internal/DatabaseBridgeHandler.java Co-authored-by: Matthew Skinner <[email protected]> * Apply suggestions from code review Co-authored-by: Matthew Skinner <[email protected]> * Suggestions from code review Signed-off-by: Joan Pujol <[email protected]> * Update parent version Signed-off-by: Joan Pujol <[email protected]> * Update bundles/org.openhab.binding.dbquery/src/main/resources/OH-INF/thing/thing-types.xml Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/src/main/resources/OH-INF/thing/thing-types.xml Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/src/main/resources/OH-INF/thing/thing-types.xml Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/src/main/resources/OH-INF/thing/jdbc-bridge.xml Co-authored-by: Matthew Skinner <[email protected]> * Update bundles/org.openhab.binding.dbquery/src/main/resources/OH-INF/thing/thing-types.xml Co-authored-by: Matthew Skinner <[email protected]> * Changes asked in PR review Signed-off-by: Joan Pujol <[email protected]> * Update bundles/org.openhab.binding.dbquery/README.md Co-authored-by: Matthew Skinner <[email protected]> * README documentation imporovements Signed-off-by: Joan Pujol <[email protected]> * Fix format issue Signed-off-by: Joan Pujol <[email protected]> Co-authored-by: Matthew Skinner <[email protected]>
bosch-io · Oct 17, 2021 · 8eb1a61 · 8eb1a61
1 parent 4c641ee
commit 8eb1a61
Show file tree

Hide file tree

Showing 49 changed files with 3,701 additions and 0 deletions.
diff --git a/CODEOWNERS b/CODEOWNERS
@@ -60,6 +60,7 @@
 /bundles/org.openhab.binding.dali/ @rs22
 /bundles/org.openhab.binding.danfossairunit/ @pravussum
 /bundles/org.openhab.binding.darksky/ @cweitkamp
+/bundles/org.openhab.binding.dbquery/ @lujop
 /bundles/org.openhab.binding.deconz/ @openhab/add-ons-maintainers
 /bundles/org.openhab.binding.denonmarantz/ @jwveldhuis
 /bundles/org.openhab.binding.digiplex/ @rmichalak

diff --git a/bom/openhab-addons/pom.xml b/bom/openhab-addons/pom.xml
@@ -286,6 +286,11 @@
       <artifactId>org.openhab.binding.darksky</artifactId>
       <version>${project.version}</version>
     </dependency>
+    <dependency>
+      <groupId>org.openhab.addons.bundles</groupId>
+      <artifactId>org.openhab.binding.dbquery</artifactId>
+      <version>${project.version}</version>
+    </dependency>
     <dependency>
       <groupId>org.openhab.addons.bundles</groupId>
       <artifactId>org.openhab.binding.deconz</artifactId>

diff --git a/bundles/org.openhab.binding.dbquery/README.md b/bundles/org.openhab.binding.dbquery/README.md
@@ -0,0 +1,210 @@
+# DBQuery Binding
+
+This binding allows creating items from the result of native database queries.
+It currently only supports InfluxDB 2.X.
+
+You can use the addon in any situation where you want to create an item from a native query.
+The source of the query can be any supported database, and doesn't need to be the one you use as the persistence service in openHAB.
+Some use cases can be:
+
+- Integrate a device that stores its data in a database
+- Query derived data from you openHAB persistence, for example with Influx2 tasks you can process your data to create a new one   
+- Bypass limitations of current openHAB persistence queries
+
+## Supported Things
+
+There are two types of supported things: `influxdb2` and a `query`.
+For each different database you want to connect to, you must define a `Bridge` thing for that database.
+Then each `Bridge` can define as many `Query` things that you want to execute.
+
+## Thing Configuration
+
+### Bridges
+
+#### influxdb2
+
+Defines a connection to an Influx2 database and allows creating queries on it.
+
+| Parameter    | Required | Description                               |
+|--------------|----------|-----------------------------------------  |
+| url          | Yes      | database url                              |
+| user         | Yes      | name of the database user                 |
+| token        | Yes      | token to authenticate to the database  ([Intructions about how to create one](https://v2.docs.influxdata.com/v2.0/security/tokens/create-token/)) |
+| organization | Yes      | database organization name                |
+| bucket       | Yes      | database bucket name                      |
+
+### query
+
+The `Query` thing defines a native query that provides several channels that you can bind to items. 
+
+#### Query parameters
+
+The query items support the following parameters:
+
+| Parameter    | Required | Default  | Description                                                           |
+|--------------|----------|----------|-----------------------------------------------------------------------|
+| query        | true     |          | Query string in native syntax                                         |
+| interval     | false    | 0        | Interval in seconds in which the query is automatically executed      |
+| hasParameters| false    | false    | True if the query has parameters, false otherwise                     | 
+| timeout      | false    | 0        | Query execution timeout in seconds                                    |
+| scalarResult | false    | true     | If query always returns a single value or not                         |
+| scalarColumn | false    |          | In case of multiple columns, it indicates which to use for scalarResult|
+
+These are described further in the following subsections.
+
+##### query  
+
+The query the items represents in the native language of your database:
+
+ - Flux for `influxdb2`
+
+#### hasParameters
+
+If `hasParameters=true` you can use parameters in the query string that can be dynamically set with the `setQueryParameters` action.
+
+ For InfluxDB use the `${paramName}` syntax for each parameter, and keep in mind that the values from that parameters must be from a trusted source as current
+ parameter substitution is subject to query injection attacks.
+
+#### timeout
+
+A time-out in seconds to wait for the query result, if it's exceeded, the result will be discarded and the addon will do its best to cancel the query.
+Currently it's ignored and it will be implemented in a future version.
+
+#### scalarResult 
+
+If `true` the query is expected to return a single scalar value that will be available to `result` channels as string, number, boolean,...
+If the query can return several rows and/or several columns per row then it needs to be set to `false` and the result can be retrieved in `resultString`
+channel as JSON or using the `getLastQueryResult` action.   
+
+#### scalarColumn
+
+In case `scalarResult` is `true` and the select returns multiple columns you can use that parameter to choose which column to use to extract the result.
+
+## Channels
+
+Query items offer the following channels to be able to query / bind them to items:
+
+| Channel Type ID | Item Type | Description                                                                                                                        |
+|-----------------|-----------|------------------------------------------------------------------------------------------------------------------------------------|
+| execute         | Switch    | Send `ON` to execute the query manually. It also indicates if query is currently running (`ON`) or not running (`OFF`)          |
+| resultString    | String    | Result of last executed query as a String |
+| resultNumber    | Number    | Result of last executed query as a Number, query must have `scalarResult=true` |
+| resultDateTime  | DateTime  | Result of last executed query as a DateTime, query must have `scalarResult=true` |
+| resultContact   | Contact   | Result of last executed query as Contact, query must have `scalarResult=true` |
+| resultSwitch    | Switch    | Result of last executed query as Switch, query must have `scalarResult=true` |
+| parameters      | String    | Contains parameters of last executed query as JSON|
+| correct         | Switch    | `ON` if the last executed query completed successfully, `OFF` if the query failed.|
+
+All the channels, except `execute`, are updated when the query execution finishes, and while there is a query in execution they have the values from
+last previous executed query.
+
+The `resultString` channel is the only valid one if `scalarResult=false`, and in that case it contains the query result serialized to JSON in that format:
+
+    {
+        correct : true,
+        data : [
+            { 
+                column1 : value,
+                column2 : value
+            },
+            { ... }, //row2        
+            { ... }  //row3
+        ]
+    }
+
+### Channel Triggers
+
+#### calculateParameters
+
+Triggers when there's a need to calculate parameters before query execution.
+When a query has `hasParameters=true` it fires the `calculateParameters` channel trigger and pauses the execution until `setQueryParameters` action is call in
+ that query.
+
+In the case a query has parameters, it's expected that there is a rule that catches the `calculateParameters` trigger, calculate the parameters with the corresponding logic and then calls the `setQueryParameters` action, after that the query will be executed.
+
+## Actions
+
+### For DatabaseBridge
+
+#### executeQuery 
+
+It allows executing a query synchronously from a script/rule without defining it in a Thing.
+
+To execute the action you need to pass the following parameters:
+
+- String query: The query to execute
+- Map<String,Object>: Query parameters (empty map if not needed)
+- int timeout: Query timeout in seconds
+
+And it returns an `ActionQueryResult` that has the following properties:
+
+- correct (boolean) : True if the query was executed correctly, false otherwise
+- data (List<Map<String,Object>>): A list where each element is a row that is stored in a map with (columnName,value) entries  
+- isScalarResult: It returns if the result is scalar one (only one row with one column)
+- resultAsScalar: It returns the result as a scalar if possible, if not returns null
+
+
+Example (using Jython script):
+
+     from core.log import logging, LOG_PREFIX 
+     log = logging.getLogger("{}.action_example".format(LOG_PREFIX))
+     map = {"time" : "-2h"}
+     influxdb = actions.get("dbquery","dbquery:influxdb2:sampleQuery") //Get bridge thing
+     result = influxdb.executeQuery("from(bucket: \"default\") |> range(start:-2h)  |> filter(fn: (r) => r[\"_measurement\"] == \"go_memstats_frees_total\")  |> filter(fn: (r) => r[\"_field\"] == \"counter\")  |> mean()",{},5)
+     log.info("execute query result is "+str(result.data))
+
+
+Use this action with care, because as the query is executed synchronously, it is not good to execute long-running queries that can block script execution.
+
+### For Queries
+
+#### setQueryParameters
+
+It's used for queries with parameters to set them.
+To execute the action you need to pass the parameters as a Map.
+
+Example (using Jython script):
+
+    params = {"time" : "-2h"}
+    dbquery = actions.get("dbquery","dbquery:query:queryWithParams")  //Get query thing
+    dbquery.setQueryParameters(params)
+
+#### getLastQueryResult
+
+It can be used in scripts to get the last query result.
+It doesn't have any parameters and returns an `ActionQueryResult` as defined in `executeQuery` action.
+
+Example (using Jython script):
+
+    dbquery = actions.get("dbquery","dbquery:query:queryWithParams")  //Get query thing
+    result = dbquery.getLastQueryResult()
+
+
+## Examples
+
+### The Simplest case 
+
+Define a InfluxDB2 database thing and a query with an interval execution.
+That executes the query every 15 seconds and punts the result in `myItem`.
+
+    # Bridge Thing definition
+    Bridge dbquery:influxdb2:mydatabase "InfluxDB2 Bridge" [ bucket="default", user="admin", url="http://localhost:8086", organization="openhab", token="*******" ]
+
+    # Query Thing definition
+    Thing dbquery:query:myquery "My Query" [ interval=15, hasParameters=false, scalarResult=true, timeout=0, query="from(bucket: \"default\") |> range(start:-1h) |> filter(fn: (r) => r[\"_measurement\"] == \"go_memstats_frees_total\")  |> filter(fn: (r) => r[\"_field\"] == \"counter\")  |> mean()", scalarColumn="_value" ]
+
+    # Item definition
+    Number myItem "QueryResult" {channel="dbquery:query:myquery:resultNumber"}
+
+### A query with parameters
+
+Using the previous example you change the `range(start:-1h)` for `range(start:${time})`
+
+Create a rule that is fired 
+
+ - **When** `calculateParameters` is triggered in `myquery`
+ - **Then** executes the following script action (in that example Jython):
+
+            map = {"time" : "-2h"}   
+            dbquery = actions.get("dbquery","dbquery:query:myquery")   
+            dbquery.setQueryParameters(map)
diff --git a/bundles/org.openhab.binding.dbquery/pom.xml b/bundles/org.openhab.binding.dbquery/pom.xml
@@ -0,0 +1,107 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+
+  <modelVersion>4.0.0</modelVersion>
+
+  <parent>
+    <groupId>org.openhab.addons.bundles</groupId>
+    <artifactId>org.openhab.addons.reactor.bundles</artifactId>
+    <version>3.2.0-SNAPSHOT</version>
+  </parent>
+
+  <artifactId>org.openhab.binding.dbquery</artifactId>
+
+  <name>openHAB Add-ons :: Bundles :: DBQuery Binding</name>
+
+  <properties>
+    <bnd.importpackage>
+      !javax.annotation;!android.*,!com.android.*,!com.google.appengine.*,!dalvik.system,!kotlin.*,!kotlinx.*,!org.conscrypt,!sun.security.ssl,!org.apache.harmony.*,!org.apache.http.*,!rx.*,!org.msgpack.*
+    </bnd.importpackage>
+  </properties>
+
+  <dependencies>
+    <!-- influxdb-client-java -->
+    <dependency>
+      <groupId>com.influxdb</groupId>
+      <artifactId>influxdb-client-java</artifactId>
+      <version>1.6.0</version>
+    </dependency>
+    <dependency>
+      <artifactId>influxdb-client-core</artifactId>
+      <groupId>com.influxdb</groupId>
+      <version>1.6.0</version>
+    </dependency>
+    <dependency>
+      <artifactId>converter-gson</artifactId>
+      <groupId>com.squareup.retrofit2</groupId>
+      <version>2.5.0</version>
+    </dependency>
+    <dependency>
+      <artifactId>converter-scalars</artifactId>
+      <groupId>com.squareup.retrofit2</groupId>
+      <version>2.5.0</version>
+    </dependency>
+    <dependency> <!-- also used for querydb library -->
+      <artifactId>gson</artifactId>
+      <groupId>com.google.code.gson</groupId>
+      <version>2.8.5</version>
+    </dependency>
+    <dependency>
+      <artifactId>gson-fire</artifactId>
+      <groupId>io.gsonfire</groupId>
+      <version>1.8.0</version>
+    </dependency>
+    <dependency>
+      <artifactId>okio</artifactId>
+      <groupId>com.squareup.okio</groupId>
+      <version>1.17.3</version>
+    </dependency>
+    <dependency>
+      <artifactId>commons-csv</artifactId>
+      <groupId>org.apache.commons</groupId>
+      <version>1.6</version>
+    </dependency>
+    <dependency>
+      <artifactId>json</artifactId>
+      <groupId>org.json</groupId>
+      <version>20180813</version>
+    </dependency>
+    <dependency>
+      <artifactId>okhttp</artifactId>
+      <groupId>com.squareup.okhttp3</groupId>
+      <version>3.14.4</version>
+    </dependency>
+    <dependency>
+      <artifactId>retrofit</artifactId>
+      <groupId>com.squareup.retrofit2</groupId>
+      <version>2.6.2</version>
+    </dependency>
+    <dependency>
+      <artifactId>jsr305</artifactId>
+      <groupId>com.google.code.findbugs</groupId>
+      <version>3.0.2</version>
+    </dependency>
+    <dependency>
+      <artifactId>logging-interceptor</artifactId>
+      <groupId>com.squareup.okhttp3</groupId>
+      <version>3.14.4</version>
+    </dependency>
+    <dependency>
+      <artifactId>rxjava</artifactId>
+      <groupId>io.reactivex.rxjava2</groupId>
+      <version>2.2.17</version>
+    </dependency>
+    <dependency>
+      <artifactId>reactive-streams</artifactId>
+      <groupId>org.reactivestreams</groupId>
+      <version>1.0.3</version>
+    </dependency>
+    <dependency>
+      <artifactId>swagger-annotations</artifactId>
+      <groupId>io.swagger</groupId>
+      <version>1.5.22</version>
+    </dependency>
+    <!-- end influxdb-client-java -->
+  </dependencies>
+</project>
diff --git a/bundles/org.openhab.binding.dbquery/src/main/feature/feature.xml b/bundles/org.openhab.binding.dbquery/src/main/feature/feature.xml
@@ -0,0 +1,9 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<features name="org.openhab.binding.dbquery-${project.version}" xmlns="http://karaf.apache.org/xmlns/features/v1.4.0">
+	<repository>mvn:org.openhab.core.features.karaf/org.openhab.core.features.karaf.openhab-core/${ohc.version}/xml/features</repository>
+
+	<feature name="openhab-binding-dbquery" description="DBQuery Binding" version="${project.version}">
+		<feature>openhab-runtime-base</feature>
+		<bundle start-level="80">mvn:org.openhab.addons.bundles/org.openhab.binding.dbquery/${project.version}</bundle>
+	</feature>
+</features>