* #402: Updated the documentation about implementing a new dialect. (#…

…406)

* #402: Updated the documentation about implementing a new dialect.
Co-authored-by: chiaradiamarcelo <[email protected]>

doc/changes/changelog.md

@@ -1,6 +1,6 @@
 # Changes
 
-* [4.0.4](changes_4.0.5.md)
+* [4.0.5](changes_4.0.5.md)
 * [4.0.4](changes_4.0.4.md)
 * [4.0.3](changes_4.0.3.md)
 * [4.0.2](changes_4.0.2.md)

doc/changes/changes_4.0.5.md

@@ -1,13 +1,14 @@
 # Exasol Virtual Schemas 4.0.5, released 2020-??-??
 
-##Code name:
+Code name: 
 
 ## Summary
 
 ## Documentation
 
 * #408: Removed PostgreSQL dialect documentation as it has been migrated to https://github.com/exasol/postgresql-virtual-schema.
 * #413: Improved documentation on `ORACLE_CAST_NUMBER_TO_DECIMAL_WITH_PRECISION_AND_SCALE`
+* #402: Updated the documentation about implementing a new dialect.
 
 ## Refactoring
 
@@ -16,4 +17,4 @@
 ## Dependency updates
 
 * Removed org.postgresql:postgresql:42.2.18
-* Removed org.testcontainers:postgresql:1.15.0
+* Removed org.testcontainers:postgresql:1.15.0

doc/development/developing-sql-dialect/developing_a_dialect.md

@@ -23,30 +23,35 @@ The name SQL dialect adapter is derived from the non-standard implementation par
 As an example, PostgreSQL handles some of the data types subtly different from Exasol and the SQL dialect adapter needs to deal with those differences by implementing conversion functions.
 
 Below you can see a layer model of the Virtual Schemas when implemented with the JDBC adapter.
- The layers in the middle — i.e. everything that deals with translating between the source and Exasol — are provided in this repository.
-
-    .-----------------------------------------.
-    |  Exasol    |          Exasol            |
-    |   core     |----------------------------|
-    |            |//// Virtual Schema API ////|
-    |------------|----------------------------|
-    |            |       JDBC  Adapter        |   Common JDBC functions
-    |  In this   |----------------------------|
-    | repository |///// SQL Dialect API //////|
-    |            |----------------------------|
-    |            |    SQL Dialect Adapter     |   Even out specifics of the source database
-    |------------|----------------------------|
-    |            |///////// JDBC API /////////|
-    |            |----------------------------|
-    |            |  PostgresSQL JDBC Driver   |   JDBC compliant access to payload and metadata
-    |  External  |----------------------------|
-    |            |// PostgresSQL Native API //|
-    |            |----------------------------|
-    |            |         PostgreSQL         |   External data source
-    '-----------------------------------------'
+The layers in the middle — i.e. everything that deals with translating between the source and Exasol — are provided in this repository.
+
+    .------------------------------------------------------.
+    |  Exasol                 |          Exasol            |
+    |   core                  |----------------------------|
+    |                         |//// Virtual Schema API ////|
+    |-------------------------|----------------------------|
+    | In Virtual Schema       |    Base for all adapters   |   Foundation of an adapter development
+    | Common Java repository  |----------------------------|
+    | In Virtual Schema       |       JDBC  Adapter        |   Common JDBC functions
+    | Common JDBC repository  |----------------------------|
+    |                         |///// SQL Dialect API //////|
+    |                         |----------------------------|
+    | In dialect repositories |    SQL Dialect Adapter     |   Even out specifics of the source database
+    |-------------------------|----------------------------|
+    |                         |///////// JDBC API /////////|
+    |                         |----------------------------|
+    |                         |  PostgresSQL JDBC Driver   |   JDBC compliant access to payload and metadata
+    |  External               |----------------------------|
+    |                         |// PostgresSQL Native API //|
+    |                         |----------------------------|
+    |                         |         PostgreSQL         |   External data source
+    '------------------------------------------------------'
 
 For more information about the structure of the Virtual Schemas check the UML diagrams provided in the directory [model/diagrams](../../../model/diagrams). You either need [PlantUML](http://plantuml.com/) to render them or an editor that has PlamtUML preview built in.
 
+* [Virtual Schema Common Java Repository](https://github.com/exasol/virtual-schema-common-java)
+* [Virtual Schema Common JDBC Repository](https://github.com/exasol/virtual-schema-common-jdbc)
+
 ## Developing a Dialect
 
 If you want to write an SQL dialect, you need to start by implementing the dialect adapter interfaces.
@@ -89,7 +94,7 @@ The Java package structure of the `virtualschema-jdbc-adapter` reflects the sepa
       |     |
       |     '-- ...
       |
-      '-- jdbc                                 Base implementation for getting metadata from JDBC
+      '-- jdbc                                 Base implementation for getting metadata from JDBC (based in Virtual Schema Common JDBC repository) 
 
 ### Interfaces
 

doc/development/developing-sql-dialect/implementing_additional_dialect_specific_behavior.md

@@ -178,7 +178,7 @@ There are differences in how precise the remote data source can encode integer,
 The best way to find out how good the default mapping works for your source — run a manual [integration test](integration_testing.md) with [remote logging](../remote_logging.md) accessing a table with all data types available in the source.
 If you assume that you don't need to change data type conversion — go to the next checkpoint: [Implementing Query Rewriting](#implementing-query-rewriting)
 
-Let's look at a HIVE dialect example. We only want to change mapping for one data type: DECIMAL.
+Let's look at a HIVE dialect example. We only want to change mapping for two data type: DECIMAL and BINARY.
 
 1. **Create `<Your dialect name>ColumnMetadataReader.java`** class that extends `BaseColumnMetadataReader.java`.
 
@@ -198,8 +198,11 @@ Let's look at a HIVE dialect example. We only want to change mapping for one dat
 
         @Override
         public DataType mapJdbcType(final JdbcTypeDescription jdbcTypeDescription) {
-            if (jdbcTypeDescription.getJdbcType() == Types.DECIMAL) {
+            final int jdbcType = jdbcTypeDescription.getJdbcType();
+            if (jdbcType == Types.DECIMAL) {
                 return mapDecimal(jdbcTypeDescription);
+            } else if (jdbcType == Types.BINARY) {
+                return DataType.createMaximumSizeVarChar(DataType.ExaCharset.UTF8);
             } else {
                 return super.mapJdbcType(jdbcTypeDescription);
             }
@@ -323,7 +326,12 @@ For each of them a base implementation exists which works fine with a number of
     ```java
         @Override
         protected RemoteMetadataReader createRemoteMetadataReader() {
-            return new AthenaMetadataReader(this.connection, this.properties);
+            try {
+                return new AthenaMetadataReader(this.connectionFactory.getConnection(), this.properties);
+            } catch (final SQLException exception) {
+                throw new RemoteMetadataReaderException(
+                        "Unable to create Athena remote metadata reader. Caused by: " + exception.getMessage(), exception);
+            }
         }
     ```
 
@@ -346,4 +354,4 @@ For each of them a base implementation exists which works fine with a number of
            return new BaseTableMetadataReader(this.connection, this.columnMetadataReader, this.properties,
                    this.identifierConverter);
        } 
-   ```
+   ```

doc/development/developing-sql-dialect/implementing_mandatory_sql_dialect_classes.md

@@ -274,18 +274,16 @@ And we also need two corresponding test classes:
 
     Another thing we need to implement in the dialect class is quoting of string literals.
     
-    Athena expects string literals to be wrapped in single quotes and single qoutes inside the literal to be escaped by duplicating each.
+    Athena expects string literals to be wrapped in single quotes and single quotes inside the literal to be escaped by duplicating each.
 
 1. **Create the `testGetLiteralString()` test** method: 
    
     ```java
-    @ValueSource(strings = { "ab:\'ab\'", "a'b:'a''b'", "a''b:'a''''b'", "'ab':'''ab'''" })
+    @ValueSource(strings = { "ab:'ab'", "a'b:'a''b'", "a''b:'a''''b'", "'ab':'''ab'''" })
     @ParameterizedTest
     void testGetLiteralString(final String definition) {
-        final int colonPosition = definition.indexOf(':');
-        final String original = definition.substring(0, colonPosition);
-        final String literal = definition.substring(colonPosition + 1);
-        assertThat(this.dialect.getStringLiteral(original), equalTo(literal));
+        assertThat(this.dialect.getStringLiteral(definition.substring(0, definition.indexOf(':'))),
+                equalTo(definition.substring(definition.indexOf(':') + 1)));
     }
     ```
     
@@ -297,49 +295,69 @@ And we also need two corresponding test classes:
     ```java
     @Override
     public String getStringLiteral(final String value) {
-        final StringBuilder builder = new StringBuilder("'");
-        builder.append(value.replaceAll("'", "''"));
-        builder.append("'");
-        return builder.toString();
+         if (value == null) {
+             return "NULL";
+         } else {
+             return "'" + value.replace("'", "''") + "'";
+         }
     }
     ```
 
     ### Implement the Applying of Quotes
     
 1. The next method to **implement: `applyQuote()`**. It applies quotes to table and schema names.
-    In case of Aurora it's a little bit complicated, so let's see a more generic example:
+    In case of Athena there are two different ways to apply the quotes: "" and ``.
+    If an identifier starts with an underscore, we use the backticks. Otherwise, we use double quotes.  
    
     ```java
-    @Test
-    void testApplyQuote() {
-        assertThat(this.dialect.applyQuote("tableName"), Matchers.equalTo("\"tableName\""));
+    @CsvSource({ "tableName, \"tableName\"", //
+            "table123, \"table123\"", //
+            "_table, `_table`", //
+            "123table, \"123table\"", //
+            "table_name, \"table_name\"" })
+    @ParameterizedTest
+    void testApplyQuote(final String unquoted, final String quoted) {
+        assertThat(this.dialect.applyQuote(unquoted), equalTo(quoted));
     }
     ```
     And implementation:
 
     ```java
     @Override
     public String applyQuote(final String identifier) {
-            return "\"" + identifier + "\"";
-        }   
+       if (this.id.startsWith("_")) {
+            return quoteWithBackticks(this.id);
+        } else {
+            return quoteWithDoubleQuotes(this.id);
+        }
+    }
+
+    private String quoteWithBackticks(final String identifier) {
+        return "`" + identifier + "`";
+    }
+
+    private String quoteWithDoubleQuotes(final String identifier) {
+        return "\"" + identifier + "\"";
+    }   
     ```
     
 1. You have **two unimplemented methods** left: `createQueryRewriter()` and `createRemoteMetadataReader()`.
     Let's use a default implementation for now:
 
     ```java
     @Override
-    protected RemoteMetadataReader createRemoteMetadataReader() {  
+    protected RemoteMetadataReader createRemoteMetadataReader() {
         try {
             return new AthenaMetadataReader(this.connectionFactory.getConnection(), this.properties);
         } catch (final SQLException exception) {
-            throw new RemoteMetadataReaderException("Unable to create Athena remote metadata reader.", exception);
+            throw new RemoteMetadataReaderException(
+                    "Unable to create Athena remote metadata reader. Caused by: " + exception.getMessage(), exception);
         }
     }
 
     @Override
     protected QueryRewriter createQueryRewriter() {
-        return new BaseQueryRewriter(this, this.remoteMetadataReader, this.connectionFactory);
+        return new BaseQueryRewriter(this, createRemoteMetadataReader(), this.connectionFactory);
     }
     ```
 

doc/development/developing-sql-dialect/integration_testing_with_containers.md

@@ -66,9 +66,9 @@ Another way to run integration tests:
 
 List of enabled integration tests:
 
-* ExasolSqlDialectIT
-* PostgreSQLSqlDialectIT
-
+* ExasolSqlDialectIT (in [exasol-virtual-schema](https://github.com/exasol/exasol-virtual-schema) repository)
+* PostgreSQLSqlDialectIT (in [postgresql-virtual-schema](https://github.com/exasol/postgresql-virtual-schema) repository)
+* SqlServerSqlDialectIT
 
 ## Executing Disabled Integration Tests