Clarify ssh private key format for ssh tunnels (#6585)

* Clarify ssh private key format for ssh tunnels * Improved SSH Tunnel key generation steps, fixed formatting * Modified wording 'app' to 'connector' for consistency * Ran format
airbytehq · Oct 4, 2021 · 2e5fbba · 2e5fbba
1 parent 34941a8
commit 2e5fbba
Show file tree

Hide file tree

Showing 6 changed files with 53 additions and 9 deletions.
diff --git a/.github/workflows/project.yaml b/.github/workflows/project.yaml
@@ -28,15 +28,13 @@ jobs:
           column_name: "Needs Triage"
       - name: Assign New Community Issues to Github Project
         uses: srggrs/assign-one-project-github-action@1.2.0
-        if: 
-          contains(github.event.issue.labels.*.name, 'community')
+        if: contains(github.event.issue.labels.*.name, 'community')
         with:
           project: "https://github.com/airbytehq/airbyte/projects/16"
           column_name: "To do"
       - name: Assign New Community PR to Github Project
         uses: srggrs/assign-one-project-github-action@1.2.0
-        if: 
-          contains(github.event.pull_request.labels.*.name, 'community')
+        if: contains(github.event.pull_request.labels.*.name, 'community')
         with:
           project: "https://github.com/airbytehq/airbyte/projects/15"
           column_name: "To do"
diff --git a/airbyte-integrations/bases/base-java/src/main/resources/ssh-tunnel-spec.json b/airbyte-integrations/bases/base-java/src/main/resources/ssh-tunnel-spec.json
@@ -55,7 +55,7 @@
         },
         "ssh_key": {
           "title": "SSH Private Key",
-          "description": "OS-level user account ssh key credentials for logging into the jump server host.",
+          "description": "OS-level user account ssh key credentials in RSA PEM format ( created with ssh-keygen -t rsa -m PEM -f myuser_rsa )",
           "type": "string",
           "airbyte_secret": true,
           "multiline": true,

diff --git a/docs/integrations/sources/mssql.md b/docs/integrations/sources/mssql.md
@@ -190,10 +190,10 @@ configuration means.
 1. Configure all fields for the source as you normally would, except `SSH Tunnel Method`.
 2. `SSH Tunnel Method` defaults to `No Tunnel` (meaning a direct connection). If you want to use an
    SSH Tunnel choose `SSH Key Authentication` or `Password Authentication`.
-1. Choose `Key Authentication` if you will be using an RSA private key as your secret for
-   establishing the SSH Tunnel (see below for more information on generating this key).
-2. Choose `Password Authentication` if you will be using a password as your secret for establishing
-   the SSH Tunnel.
+    1. Choose `Key Authentication` if you will be using an RSA private key as your secret for
+       establishing the SSH Tunnel (see below for more information on generating this key).
+    2. Choose `Password Authentication` if you will be using a password as your secret for establishing
+       the SSH Tunnel.
 3. `SSH Tunnel Jump Server Host` refers to the intermediate (bastion) server that Airbyte will connect to. This should
    be a hostname or an IP Address.
 4. `SSH Connection Port` is the port on the bastion server with which to make the SSH connection. The default port for
@@ -209,6 +209,16 @@ configuration means.
    the key file starting with `-----BEGIN RSA PRIVATE KEY-----` and ending
    with `-----END RSA PRIVATE KEY-----`.
 
+### Generating an SSH Key Pair
+
+The connector expects an RSA key in PEM format.  To generate this key:
+
+    ssh-keygen -t rsa -m PEM -f myuser_rsa
+
+This produces the private key in pem format, and the public key remains in the standard format used by the `authorized_keys` file on 
+your bastion host.  The public key should be added to your bastion host to whichever user you want to use with Airbyte.  The private
+key is provided via copy-and-paste to the Airbyte connector configuration screen, so it may log in to the bastion.
+
 ## Data type mapping
 
 MSSQL data types are mapped to the following data types when synchronizing data.
@@ -255,6 +265,7 @@ If you do not see a type in this list, assume that it is coerced into a string.
 
 | Version | Date       | Pull Request | Subject |
 | :------ | :--------  | :-----       | :------ |
+| 0.3.7   | 2021-09-30 | [6585](https://github.com/airbytehq/airbyte/pull/6585) | Improved SSH Tunnel key generation steps |
 | 0.3.6   | 2021-09-17 | [6318](https://github.com/airbytehq/airbyte/pull/6318) | Added option to connect to DB via SSH |
 | 0.3.4   | 2021-08-13 | [4699](https://github.com/airbytehq/airbyte/pull/4699) | Added json config validator | 
 | 0.3.3   | 2021-07-05 | [4689](https://github.com/airbytehq/airbyte/pull/4689) | Add CDC support |

diff --git a/docs/integrations/sources/mysql.md b/docs/integrations/sources/mysql.md
@@ -116,6 +116,7 @@ Airbyte has the ability to connect to a MySQl instance via an SSH Tunnel. The re
 When using an SSH tunnel, you are configuring Airbyte to connect to an intermediate server (a.k.a. a bastion sever) that _does_ have direct access to the database. Airbyte connects to the bastion and then asks the bastion to connect directly to the server.
 
 Using this feature requires additional configuration, when creating the source. We will talk through what each piece of configuration means.
+
 1. Configure all fields for the source as you normally would, except `SSH Tunnel Method`.
 2. `SSH Tunnel Method` defaults to `No Tunnel` (meaning a direct connection). If you want to use an SSH Tunnel choose `SSH Key Authentication` or `Password Authentication`.
    1. Choose `Key Authentication` if you will be using an RSA private key as your secret for establishing the SSH Tunnel (see below for more information on generating this key).
@@ -126,6 +127,15 @@ Using this feature requires additional configuration, when creating the source.
 6. If you are using `Password Authentication`, then `SSH Login Username` should be set to the password of the User from the previous step. If you are using `SSH Key Authentication` leave this blank. Again, this is not the MySQl password, but the password for the OS-user that Airbyte is using to perform commands on the bastion.
 7. If you are using `SSH Key Authentication`, then `SSH Private Key` should be set to the RSA Private Key that you are using to create the SSH connection. This should be the full contents of the key file starting with `-----BEGIN RSA PRIVATE KEY-----` and ending with `-----END RSA PRIVATE KEY-----`.
 
+### Generating an SSH Key Pair
+
+The connector expects an RSA key in PEM format.  To generate this key:
+
+    ssh-keygen -t rsa -m PEM -f myuser_rsa
+
+This produces the private key in pem format, and the public key remains in the standard format used by the `authorized_keys` file on 
+your bastion host.  The public key should be added to your bastion host to whichever user you want to use with Airbyte.  The private
+key is provided via copy-and-paste to the Airbyte connector configuration screen, so it may log in to the bastion.
 
 ## Data Type Mapping
 
@@ -174,6 +184,7 @@ If you do not see a type in this list, assume that it is coerced into a string.
 
 | Version | Date       | Pull Request | Subject |
 | :------ | :--------  | :-----       | :------ |
+| 0.4.7   | 2021-09-30 | [6585](https://github.com/airbytehq/airbyte/pull/6585) | Improved SSH Tunnel key generation steps |
 | 0.4.6   | 2021-09-29 | [6510](https://github.com/airbytehq/airbyte/pull/6510) | Support SSL connection |
 | 0.4.5   | 2021-09-17 | [6146](https://github.com/airbytehq/airbyte/pull/6146) | Added option to connect to DB via SSH|
 | 0.4.1   | 2021-07-23 | [4956](https://github.com/airbytehq/airbyte/pull/4956) | Fix log link |

diff --git a/docs/integrations/sources/oracle.md b/docs/integrations/sources/oracle.md
@@ -68,6 +68,7 @@ Airbyte has the ability to connect to a Oracle instance via an SSH Tunnel. The r
 When using an SSH tunnel, you are configuring Airbyte to connect to an intermediate server (a.k.a. a bastion sever) that _does_ have direct access to the database. Airbyte connects to the bastion and then asks the bastion to connect directly to the server.
 
 Using this feature requires additional configuration, when creating the source. We will talk through what each piece of configuration means.
+
 1. Configure all fields for the source as you normally would, except `SSH Tunnel Method`.
 2. `SSH Tunnel Method` defaults to `No Tunnel` (meaning a direct connection). If you want to use an SSH Tunnel choose `SSH Key Authentication` or `Password Authentication`.
    1. Choose `Key Authentication` if you will be using an RSA private key as your secret for establishing the SSH Tunnel (see below for more information on generating this key).
@@ -78,6 +79,16 @@ Using this feature requires additional configuration, when creating the source.
 6. If you are using `Password Authentication`, then `SSH Login Username` should be set to the password of the User from the previous step. If you are using `SSH Key Authentication` leave this blank. Again, this is not the Oracle password, but the password for the OS-user that Airbyte is using to perform commands on the bastion.
 7. If you are using `SSH Key Authentication`, then `SSH Private Key` should be set to the RSA Private Key that you are using to create the SSH connection. This should be the full contents of the key file starting with `-----BEGIN RSA PRIVATE KEY-----` and ending with `-----END RSA PRIVATE KEY-----`.
 
+### Generating an SSH Key Pair
+
+The connector expects an RSA key in PEM format.  To generate this key:
+
+    ssh-keygen -t rsa -m PEM -f myuser_rsa
+
+This produces the private key in pem format, and the public key remains in the standard format used by the `authorized_keys` file on 
+your bastion host.  The public key should be added to your bastion host to whichever user you want to use with Airbyte.  The private
+key is provided via copy-and-paste to the Airbyte connector configuration screen, so it may log in to the bastion.
+
 ## Data Type Mapping
 
 Oracle data types are mapped to the following data types when synchronizing data.
@@ -116,6 +127,7 @@ If you do not see a type in this list, assume that it is coerced into a string.
 
 | Version | Date       | Pull Request | Subject |
 | :------ | :--------  | :-----       | :------ |
+| 0.3.6   | 2021-09-30 | [6585](https://github.com/airbytehq/airbyte/pull/6585) | Improved SSH Tunnel key generation steps |
 | 0.3.5   | 2021-09-22 | [6356](https://github.com/airbytehq/airbyte/pull/6356) | Added option to connect to DB via SSH. |
 | 0.3.4   | 2021-09-01 | [6038](https://github.com/airbytehq/airbyte/pull/6038) | Remove automatic filtering of system schemas. |
 | 0.3.3   | 2021-09-01 | [5779](https://github.com/airbytehq/airbyte/pull/5779) | Ability to only discover certain schemas. |

diff --git a/docs/integrations/sources/postgres.md b/docs/integrations/sources/postgres.md
@@ -184,6 +184,7 @@ Airbyte has the ability to connect to a Postgres instance via an SSH Tunnel. The
 When using an SSH tunnel, you are configuring Airbyte to connect to an intermediate server (a.k.a. a bastion sever) that _does_ have direct access to the database. Airbyte connects to the bastion and then asks the bastion to connect directly to the server.
 
 Using this feature requires additional configuration, when creating the source. We will talk through what each piece of configuration means.
+
 1. Configure all fields for the source as you normally would, except `SSH Tunnel Method`.
 2. `SSH Tunnel Method` defaults to `No Tunnel` (meaning a direct connection). If you want to use an SSH Tunnel choose `SSH Key Authentication` or `Password Authentication`.
    1. Choose `Key Authentication` if you will be using an RSA Private as your secrets for establishing the SSH Tunnel (see below for more information on generating this key).
@@ -194,6 +195,16 @@ Using this feature requires additional configuration, when creating the source.
 6. If you are using `Password Authentication`, then `SSH Login Username` should be set to the password of the User from the previous step. If you are using `SSH Key Authentication` leave this blank. Again, this is not the Postgres password, but the password for the OS-user that Airbyte is using to perform commands on the bastion.
 7. If you are using `SSH Key Authentication`, then `SSH Private Key` should be set to the RSA Private Key that you are using to create the SSH connection. This should be the full contents of the key file starting with `-----BEGIN RSA PRIVATE KEY-----` and ending with `-----END RSA PRIVATE KEY-----`.
 
+### Generating an RSA Private Key
+
+The connector expects an RSA key in PEM format.  To generate this key:
+
+    ssh-keygen -t rsa -m PEM -f myuser_rsa
+
+This produces the private key in pem format, and the public key remains in the standard format used by the `authorized_keys` file on 
+your bastion host.  The public key should be added to your bastion host to whichever user you want to use with Airbyte.  The private
+key is provided via copy-and-paste to the Airbyte connector configuration screen, so it may log in to the bastion.
+
 ## Data type mapping
 
 Postgres data types are mapped to the following data types when synchronizing data.
@@ -258,6 +269,7 @@ If you can't find the data type you are looking for or have any problems feel fr
 
 | Version | Date       | Pull Request | Subject |
 | :------ | :--------  | :-----       | :------ |
+| 0.3.12   | 2021-09-30 | [6585](https://github.com/airbytehq/airbyte/pull/6585) | Improved SSH Tunnel key generation steps |
 | 0.3.11   | 2021-09-02 | [5742](https://github.com/airbytehq/airbyte/pull/5742) | Add SSH Tunnel support |
 | 0.3.9   | 2021-08-17 | [5304](https://github.com/airbytehq/airbyte/pull/5304) | Fix CDC OOM issue |
 | 0.3.8   | 2021-08-13 | [4699](https://github.com/airbytehq/airbyte/pull/4699) | Added json config validator |