Skip to content

Commit

Permalink
Updated postgres doc and fixed other minor nits (#15297)
Browse files Browse the repository at this point in the history
* Updated postgres doc and other minor nits

* fixed formatting

* added note about connectTimeout

* added limits to the initial wait time values
  • Loading branch information
Amruta-Ranade authored Aug 5, 2022
1 parent 1115637 commit b9b064f
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 6 deletions.
1 change: 1 addition & 0 deletions docs/integrations/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ For more information about the grading system, see [Product Release Stages](http
| [Oracle PeopleSoft](sources/oracle-peoplesoft.md) | Alpha | No |
| [Oracle Siebel CRM](sources/oracle-siebel-crm.md) | Alpha | No |
| [Orb](sources/orb.md) | Alpha | Yes |
| [Orbit](sources/orbit.md) | Alpha | No |
| [Outreach](./sources/outreach.md) | Alpha | No |
| [PagerDuty](sources/pagerduty.md) | Alpha | No |
| [PayPal Transaction](sources/paypal-transaction.md) | Alpha | Yes |
Expand Down
5 changes: 1 addition & 4 deletions docs/integrations/sources/glassfrog.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Shortio
# Glassfrog

## Sync overview

Expand All @@ -20,9 +20,6 @@ This Source is capable of syncing the following Streams:
* [Roles](https://documenter.getpostman.com/view/1014385/glassfrog-api-v3/2SJViY#d1f31f7a-1d42-8c86-be1d-a36e640bf993)





### Data type mapping

| Integration Type | Airbyte Type | Notes |
Expand Down
21 changes: 19 additions & 2 deletions docs/integrations/sources/postgres.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,13 +102,15 @@ This issue is tracked in [#9771](https://github.com/airbytehq/airbyte/issues/977

These parameters will be added at the end of the JDBC URL that the AirByte will use to connect to your Postgres database.

The connector now supports `connectTimeout` and defaults to 60 seconds. Setting connectTimeout to 0 seconds will set the timeout to the longest time available.

**Note:** Do not use the following keys in JDBC URL Params field as they will be overwritten by Airbyte:
`currentSchema`, `user`, `password`, `ssl`, and `sslmode`.

:::warning
This is an advanced configuration option. Users are advised to use it with caution.
:::

9. For Airbyte OSS, toggle the switch to connect using SSL. Airbyte Cloud uses SSL by default.
10. For Replication Method, select Standard or [Logical CDC](https://www.postgresql.org/docs/10/logical-replication.html) from the dropdown. Refer to [Configuring Postgres connector with Change Data Capture (CDC)](#configuring-postgres-connector-with-change-data-capture-cdc) for more information.
11. For SSH Tunnel Method, select:
Expand Down Expand Up @@ -238,7 +240,22 @@ Also, the publication should include all the tables and only the tables that nee
The Airbyte UI currently allows selecting any tables for CDC. If a table is selected that is not part of the publication, it will not be replicated even though it is selected. If a table is part of the publication but does not have a replication identity, that replication identity will be created automatically on the first run if the Airbyte user has the necessary permissions.
:::

#### Step 5: Set up the Postgres source connector
#### Step 5: [Optional] Set up initial waiting time

:::warning
This is an advanced feature. Use it if absolutely necessary.
:::

The Postgres connector may need some time to start processing the data in the CDC mode in the following scenarios:

- When the connection is set up for the first time and a snapshot is needed
- When the connector has a lot of change logs to process

The connector waits for the default initial wait time of 5 minutes (300 seconds). Setting the parameter to a longer duration will result in slower syncs, while setting it to a shorter duration may cause the connector to not have enough time to create the initial snapshot or read through the change logs. The valid range is 120 seconds to 1200 seconds.

If you know there are database changes to be synced, but the connector cannot read those changes, the root cause may be insufficient waiting time. In that case, you can increase the waiting time (example: set to 600 seconds) to test if it is indeed the root cause. On the other hand, if you know there are no database changes, you can decrease the wait time to speed up the zero record syncs.

#### Step 6: Set up the Postgres source connector

In [Step 2](#step-2-set-up-the-postgres-connector-in-airbyte) of the connector setup guide, enter the replication slot and publication you just created.

Expand Down

0 comments on commit b9b064f

Please sign in to comment.