From 3b4b458e0c652020ba298485cc8c9404b9430781 Mon Sep 17 00:00:00 2001 From: Edward Gao Date: Thu, 15 Feb 2024 14:35:25 -0800 Subject: [PATCH] =?UTF-8?q?=F0=9F=93=9D=20add=20instructions=20for=20soft?= =?UTF-8?q?=20reset=20(#35335)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../core-concepts/typing-deduping.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/docs/using-airbyte/core-concepts/typing-deduping.md b/docs/using-airbyte/core-concepts/typing-deduping.md index d0493b2866535..eb606fc3e0965 100644 --- a/docs/using-airbyte/core-concepts/typing-deduping.md +++ b/docs/using-airbyte/core-concepts/typing-deduping.md @@ -105,9 +105,9 @@ You also now see the following changes in Airbyte-provided columns: ## On final table re-creation -From time to time, Airbyte will drop and re-create the final table produced by a sync. This is done -as transitionally as possible, and should be invisible to most observers. This is done for a number -of reasons, including: +From time to time, Airbyte will drop and re-create the final table produced by a sync (sometimes +called a "soft reset"). This is done as transactionally as possible, and should be invisible to most +observers. This is done for a number of reasons, including: - **Schema Migrations** - Many destinations lack the ability to control column order, or cannot alter one data type to another. Re-creating the table allows Airbyte to strictly control the @@ -123,6 +123,18 @@ table outside of Airbyte could be lost during a sync. Many destinations provide wildcards to grant permissions to tables, which are better suited for this ELT process. We do not recommend altering the final tables (e.g. adding constraints) as it may cause issues with the sync. +### Manually triggering a final table re-creation + +In some cases, you need to manually run a soft reset - for example, if you accidentally delete some +records from the final table and want to repopulate them from the raw data. This can be done by: +1. Dropping the final table entirely (`DROP TABLE `) +1. Unsetting the raw table's `_airbyte_loaded_at` column + (`UPDATE airbyte_internal. SET _airbyte_loaded_at = NULL`) + 1. If you are using a nonstandard raw table schema, replace `airbyte_internal` with that schema. +1. And then running a sync. + +After the sync completes, your final table will be restored to its correct state. + ## Loading Data Incrementally to Final Tables :::note