Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[source-postgres] support cdc against a read-replica (continuation) #46322

Merged
merged 3 commits into from
Oct 2, 2024

Conversation

theyueli
Copy link
Contributor

@theyueli theyueli commented Oct 2, 2024

What

Fixes #45396

In a previous community PR , we added support to query LSN against a read replica. However, this is still not sufficient as we also need to retrieve current transaction id to create Debezium state together with the LSN.

However, we currently retrieve transaction id using select * from txid_current() which can only be performed against a primary. We will approximate this in this patch if we read against a replica:

SELECT 
    CASE 
        WHEN pg_is_in_recovery() THEN txid_snapshot_xmin(txid_current_snapshot())
        ELSE txid_current()
    END AS current_txid;

Here, txid_snapshot_xmin(txid_current_snapshot()) provides a lower bound to txid_current(), and can be executed on the replica.

How

Review guide

User Impact

Can this PR be safely reverted and rolled back?

  • [X ] YES 💚
  • NO ❌

@theyueli theyueli added the team/db-dw-sources Backlog for Database and Data Warehouse Sources team label Oct 2, 2024
@theyueli theyueli self-assigned this Oct 2, 2024
@theyueli theyueli requested a review from a team as a code owner October 2, 2024 22:53
Copy link

vercel bot commented Oct 2, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Oct 2, 2024 11:07pm

@theyueli
Copy link
Contributor Author

theyueli commented Oct 2, 2024

@assaadhjb we need at least another change I think.

@theyueli
Copy link
Contributor Author

theyueli commented Oct 2, 2024

/bump-version type=patch changelog="Support CDC against a read-replica (continuation)"

Bump Version job started... Check job output.

✅ Changes applied successfully. (34fa335)

@octavia-squidington-iii octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Oct 2, 2024
@theyueli theyueli enabled auto-merge (squash) October 2, 2024 23:11
@theyueli theyueli merged commit f9aedb2 into master Oct 2, 2024
38 checks passed
@theyueli theyueli deleted the yue/cdc-pg-replica branch October 2, 2024 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/postgres team/db-dw-sources Backlog for Database and Data Warehouse Sources team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[source-postgres] Logical replication not working on read replicas
3 participants