Full refresh overwrites existing data even on failure

[cgardens]

I got the same issue. We are using "full refresh | overwrite" sync mode, and every time Airbyte cannot connect to source database, normalization still run, clear all table data. Hope this change will occur asap.
-- @tq-tien

This was originally mentioned in a comment in this issue: #9672 (comment). I think it is getting at a separate topic so I'm splitting it into a separate issue.

The described behavior doesn't match my understanding of how things should work. Normalization should never delete data, it should just pull from whatever is in the raw table. If for some reason a the replication activity is overwriting existing data but not adding new data that could cause this behavior. Just one thesis. Could be something else, but probably separate from the issue to run normalization when it is not needed.

@tq-tien is this still an issue you are seeing?

[andreytyu]

Hey @cgardens, @evantahler, is there any progress with this issue? We are running into it as well. Connection to a source fails with a JSONDecodeError and then all destination tables that were not replicated are cleared from all of the data

airbyte_logs_error_20230104.txt

[grishick]

I think the solution is described here: #9663

[killthekitten]

@grishick until the solution from #9663 is implemented, are there any known workarounds for this?