🎉 Destination MSSQL: added custom JDBC parameters support. #13054

pcsoni89 · 2022-05-20T13:01:05Z

What

Add support for MSSQL Destinations to take in a JDBC URL Params input to specify any additional JDBC parameters for the connection.

This is a follow up to #10421
This is part of #9501

How

AbstractJdbcDestination can already handle jdbc parameters via the jdbc_url_params field, but the MSSQLDestination spec does not specify that field. We should update the spec to support additional jdbc parameters for MSSQL destination

🚨 User Impact 🚨

For existing destination oracle connections, a new parameter i.e. JDBC URL Params will be displayed.It is an optional field and accepts key=value data. There won't be any impact on existing operations.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub by running the /publish command described here
After the connector is published, connector added to connector index as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

Connector Generator

Issue acceptance criteria met
PR name follows PR naming conventions
If adding a new generator, add it to the list of scaffold modules being tested
The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
Documentation which references the generator is updated as needed

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

alexandertsukanov · 2022-05-23T12:41:24Z

@pcsoni89 I don't see changes on the UI specification regarding additional JDBC params, does this parameter exist?

pcsoni89 · 2022-05-23T12:45:28Z

/test connector=connectors/destination-mssql

🕑 connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/2371357135
❌ connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/2371357135
🐛 https://gradle.com/s/eduyq34awxx6o

alexandertsukanov · 2022-05-23T12:47:25Z

@pcsoni89 cannot approve until the next things won't be done:

Bumping of the version
Update the changelog.md
Tests will successfully pass

Thanks.

pcsoni89 · 2022-05-25T06:51:57Z

@pcsoni89 cannot approve until the next things won't be done:

Bumping of the version

Update the changelog.md

Tests will successfully pass

Thanks.

Added bump files and changelog.
Waiting for integration test issue to be resolved (another issue going on for integration).

pcsoni89 · 2022-05-25T12:56:57Z

/test connector=connectors/destination-mssql

🕑 connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/2384558320
❌ connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/2384558320
🐛 https://gradle.com/s/lduyqkh45tio2

alafanechere · 2022-06-03T09:01:22Z

/test connector=connectors/destination-mssql

🕑 connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/2433619677
✅ connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/2433619677
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 13      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    13      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     155      8    95%
normalization/transform_config/transform.py                         159     31    81%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_catalog/utils.py                             38      9    76%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         61     38    38%
normalization/transform_catalog/stream_processor.py                 543    352    35%
-------------------------------------------------------------------------------------
TOTAL                                                              1333    559    58%

Build Passed

Test summary info:

All Passed

alafanechere · 2022-06-07T17:22:47Z

@alexandertsukanov do you approve now that @pcsoni89 has made the changes?

alexandertsukanov · 2022-06-08T09:02:03Z

@alexandertsukanov do you approve now that @pcsoni89 has made the changes?

@alafanechere Looks like the version in Dockerfile is still not bumped here

alexandertsukanov · 2022-06-08T09:19:23Z

@alexandertsukanov do you approve now that @pcsoni89 has made the changes?

@alafanechere Looks like the version in Dockerfile is still not bumped here

Bumped looks like this could be published and merged. But I want to re-run the tests again.

alexandertsukanov · 2022-06-08T09:19:36Z

/test connector=connectors/destination-mssql

🕑 connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/2460452665
✅ connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/2460452665
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 13      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    13      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     155      8    95%
normalization/transform_config/transform.py                         159     31    81%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_catalog/utils.py                             38      9    76%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         61     38    38%
normalization/transform_catalog/stream_processor.py                 543    352    35%
-------------------------------------------------------------------------------------
TOTAL                                                              1333    559    58%

Build Passed

Test summary info:

All Passed

alexandertsukanov · 2022-06-08T09:19:53Z

/test connector=connectors/destination-mssql-strict-encrypt

🕑 connectors/destination-mssql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/2460454092
✅ connectors/destination-mssql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/2460454092
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 13      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    13      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     155      8    95%
normalization/transform_config/transform.py                         159     31    81%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_catalog/utils.py                             38      9    76%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         61     38    38%
normalization/transform_catalog/stream_processor.py                 543    352    35%
-------------------------------------------------------------------------------------
TOTAL                                                              1333    559    58%

Build Passed

Test summary info:

All Passed

alexandertsukanov · 2022-06-08T10:37:49Z

/publish connector=connectors/destination-mssql

🕑 connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/2460884723
🚀 Successfully published connectors/destination-mssql
🚀 Auto-bumped version for connectors/destination-mssql
✅ connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/2460884723

alexandertsukanov · 2022-06-08T10:37:57Z

/publish connector=connectors/destination-mssql-strict-encrypt

🕑 connectors/destination-mssql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/2460885356
🚀 Successfully published connectors/destination-mssql-strict-encrypt
❌ Couldn't auto-bump version for connectors/destination-mssql-strict-encrypt

🎉 Destination MSSQL: added custom JDBC parameters support.

c02a902

github-actions bot added the area/connectors Connector related issues label May 20, 2022

pcsoni89 requested review from alexandertsukanov and alexandr-shegeda May 20, 2022 13:01

pcsoni89 added 2 commits May 25, 2022 12:12

🎉 Destination MSSQL: added custom JDBC parameters support.

2dc2427

🎉 Destination MSSQL: added custom JDBC parameters support.

02c24ca

github-actions bot added the area/documentation Improvements or additions to documentation label May 25, 2022

pcsoni89 temporarily deployed to more-secrets May 25, 2022 06:50 Inactive

bleonard added bounty community connectors/destination/mssql connectors/destination/mssql-strict-encrypt labels Jun 2, 2022

Merge branch 'master' into prakash_10718_JdbcParam_MSSQL

59071f3

alafanechere self-assigned this Jun 3, 2022

alafanechere added internal and removed bounty labels Jun 7, 2022

alexandertsukanov added 2 commits June 8, 2022 12:05

Merge branch 'master' into prakash_10718_JdbcParam_MSSQL

b6b6365

airbyte-101718: Bumped version.

b0d2413

alexandr-shegeda approved these changes Jun 8, 2022

View reviewed changes

grubberr temporarily deployed to more-secrets June 8, 2022 10:21 Inactive

Merge branch 'master' into prakash_10718_JdbcParam_MSSQL

6be3425

alexandertsukanov force-pushed the prakash_10718_JdbcParam_MSSQL branch from 9695114 to 6be3425 Compare June 8, 2022 10:36

airbytehq deleted a comment from grubberr Jun 8, 2022

auto-bump connector version

ac149b0

alexandertsukanov merged commit b4e43c2 into master Jun 8, 2022

alexandertsukanov deleted the prakash_10718_JdbcParam_MSSQL branch June 8, 2022 11:55

octavia-squidington-iii mentioned this pull request Jun 8, 2022

Bump Airbyte version from 0.39.14-alpha to 0.39.15-alpha #13611

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 Destination MSSQL: added custom JDBC parameters support. #13054

🎉 Destination MSSQL: added custom JDBC parameters support. #13054

pcsoni89 commented May 20, 2022 •

edited

Loading

alexandertsukanov commented May 23, 2022

pcsoni89 commented May 23, 2022 •

edited by github-actions bot

Loading

alexandertsukanov commented May 23, 2022

pcsoni89 commented May 25, 2022

pcsoni89 commented May 25, 2022 •

edited by github-actions bot

Loading

alafanechere commented Jun 3, 2022 •

edited by github-actions bot

Loading

alafanechere commented Jun 7, 2022

alexandertsukanov commented Jun 8, 2022

alexandertsukanov commented Jun 8, 2022

alexandertsukanov commented Jun 8, 2022 •

edited by github-actions bot

Loading

alexandertsukanov commented Jun 8, 2022 •

edited by github-actions bot

Loading

alexandertsukanov commented Jun 8, 2022 •

edited by github-actions bot

Loading

alexandertsukanov commented Jun 8, 2022 •

edited by github-actions bot

Loading

🎉 Destination MSSQL: added custom JDBC parameters support. #13054

🎉 Destination MSSQL: added custom JDBC parameters support. #13054

Conversation

pcsoni89 commented May 20, 2022 • edited Loading

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Airbyter

Community member or Airbyter

Airbyter

Tests

alexandertsukanov commented May 23, 2022

pcsoni89 commented May 23, 2022 • edited by github-actions bot Loading

alexandertsukanov commented May 23, 2022

pcsoni89 commented May 25, 2022

pcsoni89 commented May 25, 2022 • edited by github-actions bot Loading

alafanechere commented Jun 3, 2022 • edited by github-actions bot Loading

Build Passed

alafanechere commented Jun 7, 2022

alexandertsukanov commented Jun 8, 2022

alexandertsukanov commented Jun 8, 2022

alexandertsukanov commented Jun 8, 2022 • edited by github-actions bot Loading

Build Passed

alexandertsukanov commented Jun 8, 2022 • edited by github-actions bot Loading

Build Passed

alexandertsukanov commented Jun 8, 2022 • edited by github-actions bot Loading

alexandertsukanov commented Jun 8, 2022 • edited by github-actions bot Loading

pcsoni89 commented May 20, 2022 •

edited

Loading

pcsoni89 commented May 23, 2022 •

edited by github-actions bot

Loading

pcsoni89 commented May 25, 2022 •

edited by github-actions bot

Loading

alafanechere commented Jun 3, 2022 •

edited by github-actions bot

Loading

alexandertsukanov commented Jun 8, 2022 •

edited by github-actions bot

Loading

alexandertsukanov commented Jun 8, 2022 •

edited by github-actions bot

Loading

alexandertsukanov commented Jun 8, 2022 •

edited by github-actions bot

Loading

alexandertsukanov commented Jun 8, 2022 •

edited by github-actions bot

Loading