diff --git a/docs/cloud/managing-airbyte-cloud.md b/docs/cloud/managing-airbyte-cloud.md index ceb6d5d4d402a..a443764dd3425 100644 --- a/docs/cloud/managing-airbyte-cloud.md +++ b/docs/cloud/managing-airbyte-cloud.md @@ -171,6 +171,7 @@ To view the sync summary: 3. To view the full sync log, click the sync summary dropdown. ### Sync summary + | Data | Description | |--------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------| | x GB (also measured in KB, MB) | Amount of data moved during the sync. If basic normalization is on, the amount of data would not change since normalization occurs in the destination. | @@ -186,17 +187,18 @@ In a successful sync, the number of emitted records and committed records should ## Edit stream configuration -1. On the [Airbyte Cloud](http://cloud.airbyte.io) dashboard, click **Connections** and then click a connection in the list you want to change. +1. On the [Airbyte Cloud](http://cloud.airbyte.io) dashboard, click **Connections** and then click the connection you want to change. 2. Click the **Replication** tab. The **Transfer** and **Streams** settings include the following parameters: -| Parameter | Description | -|---------------------------|-------------------------------------------------------------------------------------------------------------------------------------------| -| Replication Frequency | How often the data syncs | -| [Destination Namespace](https://docs.airbyte.com/understanding-airbyte/namespaces/) | Where the replicated data is written | -| Destination Stream Prefix | Helps you identify streams from different connectors | +| Parameter | Description | +|--------------------------------------|-------------------------------------------------------------------------------------| +| Replication frequency | How often the data syncs | +| [Non-breaking schema updates](#review-non-breaking-schema-changes) detected | How Airbyte handles syncs when it detects non-breaking schema changes in the source | +| Destination Namespace | Where the replicated data is written | +| Destination Stream Prefix | Helps you identify streams from different connectors | :::note @@ -274,9 +276,79 @@ Airbyte recommends that you reset streams. A reset will delete data in the desti To refresh the source schema: 1. Click **Refresh source schema** to fetch the schema of your data source. -2. If there are changes to the schema, the **Refreshed source schema** dialog displays them. +2. If the schema has changed, the **Refreshed source schema** dialog displays them. + +## Manage schema changes + +Once every 24 hours, Airbyte checks for changes in your source schema and allows you to review the changes and fix breaking changes. + +:::note + +Schema changes are flagged in your connection but are not propagated to your destination. + +::: + +### Review non-breaking schema changes + +To review non-breaking schema changes: +1. On the [Airbyte Cloud](http://cloud.airbyte.com/) dashboard, click **Connections** and select the connection with non-breaking changes (indicated by a **yellow exclamation mark** icon). + +2. Click **Review changes**. + +3. The **Refreshed source schema** dialog displays the changes. + +4. Review the changes and click **OK** to close the dialog. + +5. Scroll to the bottom of the page and click **Save changes**. + +:::note + + By default, Airbyte ignores non-breaking changes and continues syncing. You can configure how Airbyte handles syncs when it detects non-breaking changes by [editing the stream configuration](#edit-stream-configuration). + +::: + +### Fix breaking schema changes + +:::note + +Breaking changes can only occur in the **Cursor** or **Primary key** fields. + +::: + +To review and fix breaking schema changes: +1. On the [Airbyte Cloud](http://cloud.airbyte.com/) dashboard, click **Connections** and select the connection with breaking changes (indicated by a **red exclamation mark** icon). + +2. Click **Review changes**. + +3. The **Refreshed source schema** dialog displays the changes. + +4. Review the changes and click **OK** to close the dialog. + +5. In the streams table, the stream with a breaking change is highlighted. + +6. Fix the breaking change by selecting a new **Cursor** or **Primary key**. + +7. Scroll to the bottom of the page and click **Save changes**. + +:::note + +If a connection’s source schema has breaking changes, it will stop syncing. You must review and fix the changes before editing the connection or resuming syncs. + +::: + +### Enable schema update notifications + +To get notified when your source schema changes: +1. Make sure you have [webhook notifications](https://docs.airbyte.com/cloud/managing-airbyte-cloud#manage-airbyte-cloud-notifications) set up. + +2. On the [Airbyte Cloud](http://cloud.airbyte.com/) dashboard, click **Connections** and select the connection you want to receive notifications for. + +3. Click the **Settings** tab on the Connection page. + +4. Toggle **Schema update notifications**. ## Display Connection State + **Connection State** provides additional information about incremental syncs. It includes the most recent values for the global or stream-level cursors, which can aid in debugging or determining which data will be included in the next syncs. To display **Connection State**: @@ -291,7 +363,7 @@ To display **Connection State**: 5. Click the **Settings** tab on the Connection page. - The **Connection State** displays. + The **Connection State** displays. ## Choose the data residency for a connection You can choose the data residency for your connection in the connection settings. You can also choose data residency when creating a [new connection](https://docs.airbyte.com/cloud/getting-started-with-airbyte-cloud#set-up-a-connection), or you can set the [default data residency](#choose-your-default-data-residency) for your workspace.