diff --git a/.github/workflows/project.yaml b/.github/workflows/project.yaml index a210adc6d52d8..19781cd0ffd76 100644 --- a/.github/workflows/project.yaml +++ b/.github/workflows/project.yaml @@ -28,15 +28,13 @@ jobs: column_name: "Needs Triage" - name: Assign New Community Issues to Github Project uses: srggrs/assign-one-project-github-action@1.2.0 - if: - contains(github.event.issue.labels.*.name, 'community') + if: contains(github.event.issue.labels.*.name, 'community') with: project: "https://github.com/airbytehq/airbyte/projects/16" column_name: "To do" - name: Assign New Community PR to Github Project uses: srggrs/assign-one-project-github-action@1.2.0 - if: - contains(github.event.pull_request.labels.*.name, 'community') + if: contains(github.event.pull_request.labels.*.name, 'community') with: project: "https://github.com/airbytehq/airbyte/projects/15" column_name: "To do" diff --git a/airbyte-integrations/bases/base-java/src/main/resources/ssh-tunnel-spec.json b/airbyte-integrations/bases/base-java/src/main/resources/ssh-tunnel-spec.json index 4eefff75970c0..4597f533a0341 100644 --- a/airbyte-integrations/bases/base-java/src/main/resources/ssh-tunnel-spec.json +++ b/airbyte-integrations/bases/base-java/src/main/resources/ssh-tunnel-spec.json @@ -55,7 +55,7 @@ }, "ssh_key": { "title": "SSH Private Key", - "description": "OS-level user account ssh key credentials for logging into the jump server host.", + "description": "OS-level user account ssh key credentials in RSA PEM format ( created with ssh-keygen -t rsa -m PEM -f myuser_rsa )", "type": "string", "airbyte_secret": true, "multiline": true, diff --git a/docs/integrations/sources/mssql.md b/docs/integrations/sources/mssql.md index ba973e583d9e0..acbe471904104 100644 --- a/docs/integrations/sources/mssql.md +++ b/docs/integrations/sources/mssql.md @@ -190,10 +190,10 @@ configuration means. 1. Configure all fields for the source as you normally would, except `SSH Tunnel Method`. 2. `SSH Tunnel Method` defaults to `No Tunnel` (meaning a direct connection). If you want to use an SSH Tunnel choose `SSH Key Authentication` or `Password Authentication`. -1. Choose `Key Authentication` if you will be using an RSA private key as your secret for - establishing the SSH Tunnel (see below for more information on generating this key). -2. Choose `Password Authentication` if you will be using a password as your secret for establishing - the SSH Tunnel. + 1. Choose `Key Authentication` if you will be using an RSA private key as your secret for + establishing the SSH Tunnel (see below for more information on generating this key). + 2. Choose `Password Authentication` if you will be using a password as your secret for establishing + the SSH Tunnel. 3. `SSH Tunnel Jump Server Host` refers to the intermediate (bastion) server that Airbyte will connect to. This should be a hostname or an IP Address. 4. `SSH Connection Port` is the port on the bastion server with which to make the SSH connection. The default port for @@ -209,6 +209,16 @@ configuration means. the key file starting with `-----BEGIN RSA PRIVATE KEY-----` and ending with `-----END RSA PRIVATE KEY-----`. +### Generating an SSH Key Pair + +The connector expects an RSA key in PEM format. To generate this key: + + ssh-keygen -t rsa -m PEM -f myuser_rsa + +This produces the private key in pem format, and the public key remains in the standard format used by the `authorized_keys` file on +your bastion host. The public key should be added to your bastion host to whichever user you want to use with Airbyte. The private +key is provided via copy-and-paste to the Airbyte connector configuration screen, so it may log in to the bastion. + ## Data type mapping MSSQL data types are mapped to the following data types when synchronizing data. @@ -255,6 +265,7 @@ If you do not see a type in this list, assume that it is coerced into a string. | Version | Date | Pull Request | Subject | | :------ | :-------- | :----- | :------ | +| 0.3.7 | 2021-09-30 | [6585](https://github.com/airbytehq/airbyte/pull/6585) | Improved SSH Tunnel key generation steps | | 0.3.6 | 2021-09-17 | [6318](https://github.com/airbytehq/airbyte/pull/6318) | Added option to connect to DB via SSH | | 0.3.4 | 2021-08-13 | [4699](https://github.com/airbytehq/airbyte/pull/4699) | Added json config validator | | 0.3.3 | 2021-07-05 | [4689](https://github.com/airbytehq/airbyte/pull/4689) | Add CDC support | diff --git a/docs/integrations/sources/mysql.md b/docs/integrations/sources/mysql.md index 5eef076da16f6..853195de31a65 100644 --- a/docs/integrations/sources/mysql.md +++ b/docs/integrations/sources/mysql.md @@ -116,6 +116,7 @@ Airbyte has the ability to connect to a MySQl instance via an SSH Tunnel. The re When using an SSH tunnel, you are configuring Airbyte to connect to an intermediate server (a.k.a. a bastion sever) that _does_ have direct access to the database. Airbyte connects to the bastion and then asks the bastion to connect directly to the server. Using this feature requires additional configuration, when creating the source. We will talk through what each piece of configuration means. + 1. Configure all fields for the source as you normally would, except `SSH Tunnel Method`. 2. `SSH Tunnel Method` defaults to `No Tunnel` (meaning a direct connection). If you want to use an SSH Tunnel choose `SSH Key Authentication` or `Password Authentication`. 1. Choose `Key Authentication` if you will be using an RSA private key as your secret for establishing the SSH Tunnel (see below for more information on generating this key). @@ -126,6 +127,15 @@ Using this feature requires additional configuration, when creating the source. 6. If you are using `Password Authentication`, then `SSH Login Username` should be set to the password of the User from the previous step. If you are using `SSH Key Authentication` leave this blank. Again, this is not the MySQl password, but the password for the OS-user that Airbyte is using to perform commands on the bastion. 7. If you are using `SSH Key Authentication`, then `SSH Private Key` should be set to the RSA Private Key that you are using to create the SSH connection. This should be the full contents of the key file starting with `-----BEGIN RSA PRIVATE KEY-----` and ending with `-----END RSA PRIVATE KEY-----`. +### Generating an SSH Key Pair + +The connector expects an RSA key in PEM format. To generate this key: + + ssh-keygen -t rsa -m PEM -f myuser_rsa + +This produces the private key in pem format, and the public key remains in the standard format used by the `authorized_keys` file on +your bastion host. The public key should be added to your bastion host to whichever user you want to use with Airbyte. The private +key is provided via copy-and-paste to the Airbyte connector configuration screen, so it may log in to the bastion. ## Data Type Mapping @@ -174,6 +184,7 @@ If you do not see a type in this list, assume that it is coerced into a string. | Version | Date | Pull Request | Subject | | :------ | :-------- | :----- | :------ | +| 0.4.7 | 2021-09-30 | [6585](https://github.com/airbytehq/airbyte/pull/6585) | Improved SSH Tunnel key generation steps | | 0.4.6 | 2021-09-29 | [6510](https://github.com/airbytehq/airbyte/pull/6510) | Support SSL connection | | 0.4.5 | 2021-09-17 | [6146](https://github.com/airbytehq/airbyte/pull/6146) | Added option to connect to DB via SSH| | 0.4.1 | 2021-07-23 | [4956](https://github.com/airbytehq/airbyte/pull/4956) | Fix log link | diff --git a/docs/integrations/sources/oracle.md b/docs/integrations/sources/oracle.md index e9a2f42e6fe1e..c29a75c094937 100644 --- a/docs/integrations/sources/oracle.md +++ b/docs/integrations/sources/oracle.md @@ -68,6 +68,7 @@ Airbyte has the ability to connect to a Oracle instance via an SSH Tunnel. The r When using an SSH tunnel, you are configuring Airbyte to connect to an intermediate server (a.k.a. a bastion sever) that _does_ have direct access to the database. Airbyte connects to the bastion and then asks the bastion to connect directly to the server. Using this feature requires additional configuration, when creating the source. We will talk through what each piece of configuration means. + 1. Configure all fields for the source as you normally would, except `SSH Tunnel Method`. 2. `SSH Tunnel Method` defaults to `No Tunnel` (meaning a direct connection). If you want to use an SSH Tunnel choose `SSH Key Authentication` or `Password Authentication`. 1. Choose `Key Authentication` if you will be using an RSA private key as your secret for establishing the SSH Tunnel (see below for more information on generating this key). @@ -78,6 +79,16 @@ Using this feature requires additional configuration, when creating the source. 6. If you are using `Password Authentication`, then `SSH Login Username` should be set to the password of the User from the previous step. If you are using `SSH Key Authentication` leave this blank. Again, this is not the Oracle password, but the password for the OS-user that Airbyte is using to perform commands on the bastion. 7. If you are using `SSH Key Authentication`, then `SSH Private Key` should be set to the RSA Private Key that you are using to create the SSH connection. This should be the full contents of the key file starting with `-----BEGIN RSA PRIVATE KEY-----` and ending with `-----END RSA PRIVATE KEY-----`. +### Generating an SSH Key Pair + +The connector expects an RSA key in PEM format. To generate this key: + + ssh-keygen -t rsa -m PEM -f myuser_rsa + +This produces the private key in pem format, and the public key remains in the standard format used by the `authorized_keys` file on +your bastion host. The public key should be added to your bastion host to whichever user you want to use with Airbyte. The private +key is provided via copy-and-paste to the Airbyte connector configuration screen, so it may log in to the bastion. + ## Data Type Mapping Oracle data types are mapped to the following data types when synchronizing data. @@ -116,6 +127,7 @@ If you do not see a type in this list, assume that it is coerced into a string. | Version | Date | Pull Request | Subject | | :------ | :-------- | :----- | :------ | +| 0.3.6 | 2021-09-30 | [6585](https://github.com/airbytehq/airbyte/pull/6585) | Improved SSH Tunnel key generation steps | | 0.3.5 | 2021-09-22 | [6356](https://github.com/airbytehq/airbyte/pull/6356) | Added option to connect to DB via SSH. | | 0.3.4 | 2021-09-01 | [6038](https://github.com/airbytehq/airbyte/pull/6038) | Remove automatic filtering of system schemas. | | 0.3.3 | 2021-09-01 | [5779](https://github.com/airbytehq/airbyte/pull/5779) | Ability to only discover certain schemas. | diff --git a/docs/integrations/sources/postgres.md b/docs/integrations/sources/postgres.md index 7465272832f94..ca2a9d4a84752 100644 --- a/docs/integrations/sources/postgres.md +++ b/docs/integrations/sources/postgres.md @@ -184,6 +184,7 @@ Airbyte has the ability to connect to a Postgres instance via an SSH Tunnel. The When using an SSH tunnel, you are configuring Airbyte to connect to an intermediate server (a.k.a. a bastion sever) that _does_ have direct access to the database. Airbyte connects to the bastion and then asks the bastion to connect directly to the server. Using this feature requires additional configuration, when creating the source. We will talk through what each piece of configuration means. + 1. Configure all fields for the source as you normally would, except `SSH Tunnel Method`. 2. `SSH Tunnel Method` defaults to `No Tunnel` (meaning a direct connection). If you want to use an SSH Tunnel choose `SSH Key Authentication` or `Password Authentication`. 1. Choose `Key Authentication` if you will be using an RSA Private as your secrets for establishing the SSH Tunnel (see below for more information on generating this key). @@ -194,6 +195,16 @@ Using this feature requires additional configuration, when creating the source. 6. If you are using `Password Authentication`, then `SSH Login Username` should be set to the password of the User from the previous step. If you are using `SSH Key Authentication` leave this blank. Again, this is not the Postgres password, but the password for the OS-user that Airbyte is using to perform commands on the bastion. 7. If you are using `SSH Key Authentication`, then `SSH Private Key` should be set to the RSA Private Key that you are using to create the SSH connection. This should be the full contents of the key file starting with `-----BEGIN RSA PRIVATE KEY-----` and ending with `-----END RSA PRIVATE KEY-----`. +### Generating an RSA Private Key + +The connector expects an RSA key in PEM format. To generate this key: + + ssh-keygen -t rsa -m PEM -f myuser_rsa + +This produces the private key in pem format, and the public key remains in the standard format used by the `authorized_keys` file on +your bastion host. The public key should be added to your bastion host to whichever user you want to use with Airbyte. The private +key is provided via copy-and-paste to the Airbyte connector configuration screen, so it may log in to the bastion. + ## Data type mapping Postgres data types are mapped to the following data types when synchronizing data. @@ -258,6 +269,7 @@ If you can't find the data type you are looking for or have any problems feel fr | Version | Date | Pull Request | Subject | | :------ | :-------- | :----- | :------ | +| 0.3.12 | 2021-09-30 | [6585](https://github.com/airbytehq/airbyte/pull/6585) | Improved SSH Tunnel key generation steps | | 0.3.11 | 2021-09-02 | [5742](https://github.com/airbytehq/airbyte/pull/5742) | Add SSH Tunnel support | | 0.3.9 | 2021-08-17 | [5304](https://github.com/airbytehq/airbyte/pull/5304) | Fix CDC OOM issue | | 0.3.8 | 2021-08-13 | [4699](https://github.com/airbytehq/airbyte/pull/4699) | Added json config validator |