-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added initial Teradata JDBC driver implementation #5537
base: master
Are you sure you want to change the base?
Conversation
Introducing the following env vars: CUBEJS_DB_TERADATA_NAME - optional, name of the database to connect to CUBEJS_DB_TERADATA_URL - mandatory, e.g. jdbc:teradata://<teradata_db_ip>/USER=<teradata_user>,PASSWORD=<teradata_user_pwd> CUBEJS_DB_TERADATA_ACCEPT_POLICY - optional, true/false to accept terms and conditions As part of the Docker build we fetch an external dependency, namely the Teradata JDBC jar file into: /cubejs/packages/cubejs-teradata-jdbc-driver/download/terajdbc4.jar More information on setting up a server and using this driver is documented here: packages/cubejs-teradata-jdbc-driver/README.md We tested it with an on-prem Teradata server as well as Teradata Express. Known limitations: 1) Max 15 transactions per cursor https://docs.teradata.com/r/GVKfXcemJFkTJh_89R34UQ/YB_t1rB9boiHUNbSQlvLRA (Error 3130). 2) Cursor limitations After 15 transactions we get the 3130 error from the cursor. Temporary workaround: doing a Cube pre-processing step where we manually generate the schemas outside of Cube using Teradata SQL CLI client (bteq). 3) Only handling table views Handling tables (in addition to table views) Currently we only support table views (tableKind=‘V’) and not tables. Need to implement and test it.
In order to test this patch with Cube you can setup a Teradata Express (Vantage) server on Azure VM (for example) using this guide: You can also setup a local Teradata instance using VirtualBox on x86: |
@ofir-insait I'm not familiar with DB Drivers, so I can't do a full review, but my 2 cents are that drivers usually don't have their own env variable to define DB name: |
|
||
public async getTablesQuery(databaseName: string) { | ||
try { | ||
// Only SELECT VIEWS (tabkeKind=V) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ofir-insait Also, can you provide some insight as to why only Views are supported?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The main reason is time, this is what our clients used and what we needed to support.
We have actually started with schema discovery for tables as well and the delta is not huge to add this support.
Teradata obviously has metadata for tables as well.
Note that this is also stated in the commit message:
Only handling table views
Handling tables (in addition to table views)
Currently we only support table views (tableKind=‘V’) and not tables.
Need to implement and test it.
Hey @pacofvf this is partially true, if you look at DataBricks for example, you would find several db specific env vars. What we could do, though, is:
But we would still need CUBEJS_DB_TERADATA_URL IMHO. |
@ofir-insait Thanks a lot for contributing! An E2E test for Teradata would be very much appreciated. You can use this one as an example: https://github.com/cube-js/cube.js/blob/master/packages/cubejs-testing/test/smoke-materialize.test.ts. |
@ofir-insait While you're working on a test, you might want to consider publishing your driver version as is and adding a backlink in our docs. Please see https://github.com/cube-js/cube.js/blob/master/CONTRIBUTING.md#contributing-database-drivers. |
This patch introduces initial Teradata JDBC driver implementation.
Introducing the following env vars:
CUBEJS_DB_TERADATA_NAME - optional, name of the database to connect to CUBEJS_DB_TERADATA_URL - mandatory, e.g.
jdbc:teradata://<teradata_db_ip>/USER=<teradata_user>,PASSWORD=<teradata_user_pwd>
CUBEJS_DB_TERADATA_ACCEPT_POLICY - optional, true/false to accept terms and conditions
As part of the Docker build we fetch an external dependency, namely the Teradata JDBC jar file into:
/cubejs/packages/cubejs-teradata-jdbc-driver/download/terajdbc4.jar
More information on setting up a server and using this driver is documented here:
packages/cubejs-teradata-jdbc-driver/README.md
We tested it with an on-prem Teradata server as well as Teradata Express.
Known limitations include:
Max 15 transactions per cursor
https://docs.teradata.com/r/GVKfXcemJFkTJh_89R34UQ/YB_t1rB9boiHUNbSQlvLRA (Error 3130).
Cursor limitations
After 15 transactions we get the 3130 error from the cursor.
Temporary workaround: doing a Cube pre-processing step where we manually
generate the schemas outside of Cube using Teradata SQL CLI client (bteq).
Only handling table views
Handling tables (in addition to table views)
Currently we only support table views (tableKind=‘V’) and not tables.
Need to implement and test it.
Check List
Issue Reference this PR resolves
[For example #12]
Description of Changes Made (if issue reference is not provided)
[Description goes here]