Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix codestyle check by removing concurrent newlines #2

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
b400210
Add Snowflake JDBC Connector
yuuteng Dec 22, 2021
8700342
Update trino snapshot version to 372
yuuteng Feb 18, 2022
4e08543
Update format of the doc of snowflake
yuuteng Feb 18, 2022
2d84c9c
Update trino jdbc library import
yuuteng Feb 18, 2022
df116e0
Fix date formatter from yyyy to uuuu
yuuteng Feb 21, 2022
911f8dd
Fix date test case
yuuteng Feb 21, 2022
2ace2a8
Remove defunct property allow-drop-table
yuuteng Feb 21, 2022
8b3cfe3
Update trino version to 374
yuuteng Mar 14, 2022
d84861d
Update snowflake config to adapt 374
yuuteng Mar 14, 2022
293895d
Update the range of the test of Date type
yuuteng Mar 14, 2022
c0efb5a
Update to version 375
yuuteng Mar 22, 2022
b26e2b6
Fix snowflake after updating to 375
yuuteng Mar 22, 2022
352b974
Update to 381
yuuteng May 16, 2022
4dcb1fd
Fix mvn pom import
yuuteng May 16, 2022
5059d93
Format snowflake.rst
yuuteng May 16, 2022
fc60485
Reorderd Data tests in type mapping
yuuteng May 16, 2022
8d4dccf
Update function code
yuuteng May 16, 2022
d2fe5e6
Add product test
yuuteng May 16, 2022
1630237
Rename product test tablename
yuuteng May 16, 2022
4379c2f
Add Env, Suite and properties of Snowflake for production test
yuuteng May 16, 2022
9bdc981
Add trinoCreateAndInsert()
yuuteng May 16, 2022
15efc78
Refactor snowflake from single node to multi node
yuuteng May 17, 2022
fa70405
Pass product tests
yuuteng May 18, 2022
b709095
Removed snowflake.properties in trino server dev
yuuteng May 18, 2022
e59ed06
Resolved issues 19 05 2022 and fixed tests
yuuteng May 19, 2022
9476718
Remove Types.VARBINARY
yuuteng May 20, 2022
9f77c49
Add private static SliceWriteFunction charWriteFunction
yuuteng May 31, 2022
6de2b42
Update test case
yuuteng May 31, 2022
187dfe1
Update plugin/trino-snowflake/src/main/java/io/trino/plugin/snowflake…
yuuteng Jun 16, 2022
171c932
Update docs/src/main/sphinx/connector/snowflake.rst
yuuteng Jun 16, 2022
edcdbd7
Update plugin/trino-snowflake/pom.xml
yuuteng Jun 16, 2022
3365a6f
Update plugin/trino-snowflake/src/main/java/io/trino/plugin/snowflake…
yuuteng Jun 16, 2022
322d02e
Resolved review open issues
yuuteng Jun 16, 2022
e735e8c
Disabled JDBC_TREAT_DECIMAL_AS_INT and fixed test case
yuuteng Jun 16, 2022
1fd535a
Updated properties file
yuuteng Jun 17, 2022
fa3347c
Updated properties files
yuuteng Jun 17, 2022
87169cc
Renamed properties in Testing
yuuteng Jun 17, 2022
085b5ae
Revert "Renamed properties in Testing"
yuuteng Jun 17, 2022
4ec8d9a
Renamed properties and fixed tests
yuuteng Jun 17, 2022
1dad198
Update the way to pass ENV values for production test
yuuteng Jun 24, 2022
00a752c
Update trino version to 388
yuuteng Jun 24, 2022
d244d82
Update Trino version 391
yuuteng Jul 18, 2022
a19bf47
Update trino version to 394
yuuteng Aug 26, 2022
3e007a5
Update to 395
yuuteng Aug 31, 2022
463aa95
Update to 411
yuuteng Mar 22, 2023
38539f9
Update and fix errors
yuuteng Mar 27, 2023
dc08bda
Build successfully with 411
yuuteng Mar 27, 2023
2fb9812
Adding Bloomberg Snowflake connector.
dprophet Mar 17, 2023
38cc79f
Adding varchar type to mapping Snowflake.
dprophet Mar 28, 2023
64ad8cd
Fixing some tests.
dprophet Apr 3, 2023
e52bdce
Fix TestSnowflakeConnectorTest.
yuuteng Apr 5, 2023
43b85cd
Fix type mapper
yuuteng Apr 6, 2023
1d2f25f
Fix testconnector
yuuteng Apr 6, 2023
f36ed98
Update version to 413-SNAPSHOT
yuuteng Apr 6, 2023
6493cb6
Add product test
yuuteng May 16, 2022
c2a6fe5
Add Env, Suite and properties of Snowflake for production test
yuuteng May 16, 2022
3b462b3
Refactor snowflake from single node to multi node
yuuteng May 17, 2022
8909bc3
Removed snowflake.properties in trino server dev
yuuteng May 18, 2022
69c9d3b
Renamed properties in Testing
yuuteng Jun 17, 2022
923882a
Revert "Renamed properties in Testing"
yuuteng Jun 17, 2022
643064e
Update and fix errors
yuuteng Mar 27, 2023
a12a5f6
Adding Bloomberg Snowflake connector.
dprophet Mar 17, 2023
6949fd8
Added support for HTTP_PROXY testing.
dprophet Apr 14, 2023
e3f70a5
Connector doesnt support setColumnType. This causes lots of problems …
dprophet Apr 14, 2023
4dc87d5
Fixed and skipped error tests
yuuteng Jun 2, 2023
e7ff72a
Updated to 422
yuuteng Jul 10, 2023
befa8a0
Updated and fixed
yuuteng Jul 10, 2023
5df374d
Fix trino review 13-07-2023
yuuteng Jul 13, 2023
5520b5f
Add timestamp tests
yuuteng Jul 13, 2023
49083ad
Fix codestyle check by removing concurrent newlines
CapCaWork Sep 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions core/trino-server/src/main/provisio/trino.xml
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,12 @@
</artifact>
</artifactSet>

<artifactSet to="plugin/snowflake">
<artifact id="${project.groupId}:trino-snowflake:zip:${project.version}">
<unpack />
</artifact>
</artifactSet>

<artifactSet to="plugin/sqlserver">
<artifact id="${project.groupId}:trino-sqlserver:zip:${project.version}">
<unpack />
Expand Down
107 changes: 107 additions & 0 deletions docs/src/main/sphinx/connector/snowflake.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
===================
Snowflake connector
===================

.. raw:: html

<img src="../_static/img/snowflake.png" class="connector-logo">

The Snowflake connector allows querying and creating tables in an
external `Snowflake <https://www.snowflake.com/>`_ account. This can be used to join data between
different systems like Snowflake and Hive, or between two different
Snowflake accounts.

Configuration
-------------

To configure the Snowflake connector, create a catalog properties file
in ``etc/catalog`` named, for example, ``example.properties``, to
mount the Snowflake connector as the ``snowflake`` catalog.
Create the file with the following contents, replacing the
connection properties as appropriate for your setup:

.. code-block:: none

connector.name=snowflake
connection-url=jdbc:snowflake://<account>.snowflakecomputing.com
connection-user=root
connection-password=secret
snowflake.account=account
snowflake.database=database
snowflake.role=role
snowflake.warehouse=warehouse

Arrow serialization support
^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is an experimental feature which introduces support for using Apache Arrow
as the serialization format when reading from Snowflake. Please note there are
a few caveats:

* Using Apache Arrow serialization is disabled by default. In order to enable
it, add ``--add-opens=java.base/java.nio=ALL-UNNAMED`` to the Trino
:ref:`jvm-config`.


Multiple Snowflake databases or accounts
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The Snowflake connector can only access a single database within
a Snowflake account. Thus, if you have multiple Snowflake databases,
or want to connect to multiple Snowflake accounts, you must configure
multiple instances of the Snowflake connector.

.. snowflake-type-mapping:

Type mapping
------------

Trino supports the following Snowflake data types:

================================== ===============================
Snowflake Type Trino Type
================================== ===============================
``boolean`` ``boolean``
``tinyint`` ``bigint``
``smallint`` ``bigint``
``byteint`` ``bigint``
``int`` ``bigint``
``integer`` ``bigint``
``bigint`` ``bigint``
``float`` ``real``
``real`` ``real``
``double`` ``double``
``decimal`` ``decimal(P,S)``
``varchar(n)`` ``varchar(n)``
``char(n)`` ``varchar(n)``
``binary(n)`` ``varbinary``
``varbinary`` ``varbinary``
``date`` ``date``
``time`` ``time``
``timestampntz`` ``timestamp``
``timestamptz`` ``timestampTZ``
``timestampltz`` ``timestampTZ``
================================== ===============================

Complete list of `Snowflake data types
<https://docs.snowflake.com/en/sql-reference/intro-summary-data-types.html>`_.

.. _snowflake-sql-support:

SQL support
-----------

The connector provides read access and write access to data and metadata in
a Snowflake database. In addition to the :ref:`globally available
<sql-globally-available>` and :ref:`read operation <sql-read-operations>`
statements, the connector supports the following features:

* :doc:`/sql/insert`
* :doc:`/sql/delete`
* :doc:`/sql/truncate`
* :doc:`/sql/create-table`
* :doc:`/sql/create-table-as`
* :doc:`/sql/drop-table`
* :doc:`/sql/alter-table`
* :doc:`/sql/create-schema`
* :doc:`/sql/drop-schema`
Binary file added docs/src/main/sphinx/static/img/snowflake.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
254 changes: 254 additions & 0 deletions plugin/trino-snowflake/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,254 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>io.trino</groupId>
<artifactId>trino-root</artifactId>
<version>422-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>trino-snowflake</artifactId>
<packaging>trino-plugin</packaging>
<description>Trino - Snowflake Connector</description>

<properties>
<air.main.basedir>${project.parent.basedir}</air.main.basedir>
</properties>

<dependencies>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>

<dependency>
<groupId>com.google.inject</groupId>
<artifactId>guice</artifactId>
</dependency>

<dependency>
<groupId>io.airlift</groupId>
<artifactId>configuration</artifactId>
</dependency>

<dependency>
<groupId>io.airlift</groupId>
<artifactId>log</artifactId>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-base-jdbc</artifactId>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-plugin-toolkit</artifactId>
</dependency>

<dependency>
<groupId>net.snowflake</groupId>
<artifactId>snowflake-jdbc</artifactId>
<version>3.13.32</version>
</dependency>

<!-- Trino SPI -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>io.airlift</groupId>
<artifactId>slice</artifactId>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-api</artifactId>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-context</artifactId>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-spi</artifactId>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>org.openjdk.jol</groupId>
<artifactId>jol-core</artifactId>
<scope>provided</scope>
</dependency>

<!-- for testing -->
<dependency>
<groupId>io.airlift</groupId>
<artifactId>testing</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-base-jdbc</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-main</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-main</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-testing</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-testing-services</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-tpch</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.trino.tpch</groupId>
<artifactId>tpch</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>jdbc</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>--add-opens=java.base/java.nio=ALL-UNNAMED</argLine>
</configuration>
</plugin>
</plugins>
</build>

<profiles>
<profile>
<id>default</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/TestSnowflakeClient.java</exclude>
<exclude>**/TestSnowflakeConfig.java</exclude>
<exclude>**/TestSnowflakeConnectorTest.java</exclude>
<exclude>**/TestSnowflakePlugin.java</exclude>
<exclude>**/TestSnowflakeTypeMapping.java</exclude>
<exclude>**/Test*FailureRecoveryTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>

<profile>
<!-- Tests which require third party cloud services are separated from the main test profile -->
<id>cloud-tests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<exclude>**/TestSnowflakeClient.java</exclude>
<exclude>**/TestSnowflakeConfig.java</exclude>
<exclude>**/TestSnowflakeConnectorTest.java</exclude>
<exclude>**/TestSnowflakePlugin.java</exclude>
<exclude>**/TestSnowflakeTypeMapping.java</exclude>
<exclude>**/Test*FailureRecoveryTest.java</exclude>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</profile>

<profile>
<id>fte-tests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/Test*FailureRecoveryTest.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
Loading