Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

java CDK, source-postgres: speed up tests involving debezium #31976

Merged
merged 17 commits into from
Oct 31, 2023
Merged

java CDK, source-postgres: speed up tests involving debezium #31976

Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
cb371a0
source-postgres: more frequent heartbeats in tests
Oct 27, 2023
5f852e7
java CDK: less polling in DebeziumShutdownProcedure
Oct 27, 2023
5468077
java CDK, source-postgres: short-circuit debezium when WAL interval h…
Oct 27, 2023
691b3a0
source-postgres: use local CDK
Oct 27, 2023
406c1bc
source-postgres: fix test flakiness due to unintended `sync_checkpoin…
Oct 27, 2023
54888f9
java CDK: add javadoc
Oct 27, 2023
be30e0e
reformat javadoc
Oct 27, 2023
dfe63cf
undo many changes in 5f852e7
Oct 27, 2023
4c78b2d
add extra assertions in LsnCommitTest
Oct 27, 2023
1c67c6c
Merge branch 'master' into dbz-postgres-test-speedup
Oct 30, 2023
fb7f057
Merge branch 'master' into dbz-postgres-test-speedup
Oct 30, 2023
daf6dea
only short-circuit in tests
Oct 30, 2023
d347b32
make cdkVersionShouldMatch test pass
alafanechere Oct 31, 2023
d83189f
bump source-postgres version
alafanechere Oct 31, 2023
4cfeb92
Revert "make cdkVersionShouldMatch test pass"
Oct 31, 2023
eb7913f
Merge branch 'master' into dbz-postgres-test-speedup
Oct 31, 2023
4e4041e
set java CDK version to 0.2.2, update changelog, declare dep in sourc…
Oct 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ...src/main/java/io/airbyte/integrations/source/postgres/cdc/PostgresCdcCtidInitializer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,8 @@ public static List<AutoCloseableIterator<AirbyteMessage>> cdcCtidIteratorsCombin
final PostgresCdcStateHandler postgresCdcStateHandler = new PostgresCdcStateHandler(stateManager);

final boolean canShortCircuitDebeziumEngine = savedOffset.isPresent() &&
// Until the need presents itself in production, short-circuiting should only be done in tests.
sourceConfig.has("is_test") && sourceConfig.get("is_test").asBoolean() &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

!postgresDebeziumStateUtil.maybeReplicationStreamIntervalHasRecords(
database.getDatabaseConfig(),
sourceConfig.get("replication_method").get("replication_slot").asText(),
Expand Down