🐛Source MSSQL: Fix CDC with sync mode is New changes only #19372

quannh-uet · 2022-11-14T07:36:29Z

What

Fix: 🐛 #18860

How

After #14910
Replication in source spec change from

"replication": {
    "replication_type": "CDC",
    "data_to_sync": "New Changes Only",
    "snapshot_isolation": "Snapshot"
  }

to

  "replication_method": {
    "method": "CDC",
    "data_to_sync": "New Changes Only",
    "snapshot_isolation": "Snapshot"
  }

And following guideline for upgrading from version 0.4.17 and older versions to 0.4.18 and newer version legacy config will change from

"replication_method": "STANDARD"
or 
"replication_method": "CDC"

to

"replication_method": {
    "method": "STANDARD" 
  }
or
"replication_method": {
    "method": "CDC"
  }

We need support both newer version (after 0.4.18) and migration from legacy to newer.

🚨 User Impact 🚨

User only need following migration guideline in source documentation page.

Pre-merge Checklist

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Tests

Unit

> Task :airbyte-integrations:connectors:source-mssql:test

MssqlJdbcSourceAcceptanceTest > testReadMultipleTables() PASSED

MssqlJdbcSourceAcceptanceTest > testDiscoverWithNonCursorFields() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalDateCheckCursor() PASSED

MssqlJdbcSourceAcceptanceTest > testReadMultipleTablesIncrementally() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalCursorChanges() PASSED

MssqlJdbcSourceAcceptanceTest > testDiscoverWithNullableCursorFields() PASSED

MssqlJdbcSourceAcceptanceTest > testSpec() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckSuccess() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalIntCheckCursor() PASSED

MssqlJdbcSourceAcceptanceTest > testDiscoverWithMultipleSchemas() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalWithConcurrentInsertion() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalNoPreviousState() PASSED

MssqlJdbcSourceAcceptanceTest > testDiscover() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalStringCheckCursor() PASSED

MssqlJdbcSourceAcceptanceTest > testReadSuccess() PASSED

MssqlJdbcSourceAcceptanceTest > testReadOneColumn() PASSED

MssqlJdbcSourceAcceptanceTest > testReadOneTableIncrementallyTwice() PASSED

MssqlJdbcSourceAcceptanceTest > testReadFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalStringCheckCursorSpaceInColumnName() PASSED

MssqlJdbcSourceAcceptanceTest > testTablesWithQuoting() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectDataBaseFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectPasswordFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testUserHasNoPermissionToDataBase() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectHostFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectUsernameFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectPortFailure() PASSED

CdcMssqlSourceTest > Verify that when data is inserted into the database while a sync is happening and after the first sync, it all gets replicated. PASSED

CdcMssqlSourceTest > When no changes have been made to the database since the previous sync, no records are returned. PASSED

CdcMssqlSourceTest > testCheck() PASSED

CdcMssqlSourceTest > When both incremental CDC and full refresh are configured for different streams in a sync, the data is replicated as expected. PASSED

CdcMssqlSourceTest > testDiscover() PASSED

CdcMssqlSourceTest > When a record is deleted, produces a deletion record. PASSED

CdcMssqlSourceTest > When no records exist, no records are returned. PASSED

CdcMssqlSourceTest > On the first sync, produce returns records that exist in the database. PASSED

CdcMssqlSourceTest > When a record is updated, produces an update record. PASSED

CdcMssqlSourceTest > testAssertSnapshotIsolationAllowed() PASSED

CdcMssqlSourceTest > testAssertSnapshotIsolationDisabled() PASSED

CdcMssqlSourceTest > testCdcCheckOperationsWithDot() PASSED

CdcMssqlSourceTest > testAssertCdcSchemaQueryable() PASSED

CdcMssqlSourceTest > testGetTargetPosition() PASSED

CdcMssqlSourceTest > testAssertSqlServerAgentRunning() PASSED

CdcMssqlSourceTest > testCdcCheckOperations() PASSED

CdcMssqlSourceTest > testAssertCdcEnabledInDb() PASSED

MssqlCdcHelperTest > testIsCdc() PASSED

MssqlCdcHelperTest > testGetDataToSyncConfig() PASSED

MssqlCdcHelperTest > testGetSnapshotIsolation() PASSED

MssqlSourceTest > testDiscoverWithPk() PASSED

MssqlStressTest > stressTest() SKIPPED

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 15m 1s
81 actionable tasks: 8 executed, 73 up-to-date
S3 cache 4s wasted on misses, reads: 4, elapsed: 4s

Integration

Acceptance

> Task :airbyte-integrations:connectors:source-mssql:integrationTestJava

MssqlRdsSourceAcceptanceTest > testIdenticalFullRefreshes() STARTED

MssqlRdsSourceAcceptanceTest > testIdenticalFullRefreshes() FAILED
    java.lang.RuntimeException: java.nio.file.NoSuchFileException: secrets/config.json
        at io.airbyte.commons.io.IOs.readFile(IOs.java:74)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.getStaticConfig(MssqlRdsSourceAcceptanceTest.java:47)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.setupEnvironment(MssqlRdsSourceAcceptanceTest.java:27)

        Caused by:
        java.nio.file.NoSuchFileException: secrets/config.json
            at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
            at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
            at java.base/java.nio.file.Files.readAllBytes(Files.java:3288)
            at java.base/java.nio.file.Files.readString(Files.java:3366)
            at io.airbyte.commons.io.IOs.readFile(IOs.java:72)
            ... 2 more

MssqlRdsSourceAcceptanceTest > testEntrypointEnvVar() STARTED

MssqlRdsSourceAcceptanceTest > testEntrypointEnvVar() FAILED
    java.lang.RuntimeException: java.nio.file.NoSuchFileException: secrets/config.json
        at io.airbyte.commons.io.IOs.readFile(IOs.java:74)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.getStaticConfig(MssqlRdsSourceAcceptanceTest.java:47)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.setupEnvironment(MssqlRdsSourceAcceptanceTest.java:27)

        Caused by:
        java.nio.file.NoSuchFileException: secrets/config.json
            at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
            at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
            at java.base/java.nio.file.Files.readAllBytes(Files.java:3288)
            at java.base/java.nio.file.Files.readString(Files.java:3366)
            at io.airbyte.commons.io.IOs.readFile(IOs.java:72)
            ... 2 more

MssqlRdsSourceAcceptanceTest > testIncrementalSyncWithState() STARTED

MssqlRdsSourceAcceptanceTest > testIncrementalSyncWithState() FAILED
    java.lang.RuntimeException: java.nio.file.NoSuchFileException: secrets/config.json
        at io.airbyte.commons.io.IOs.readFile(IOs.java:74)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.getStaticConfig(MssqlRdsSourceAcceptanceTest.java:47)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.setupEnvironment(MssqlRdsSourceAcceptanceTest.java:27)

        Caused by:
        java.nio.file.NoSuchFileException: secrets/config.json
            at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
            at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
            at java.base/java.nio.file.Files.readAllBytes(Files.java:3288)
            at java.base/java.nio.file.Files.readString(Files.java:3366)
            at io.airbyte.commons.io.IOs.readFile(IOs.java:72)
            ... 2 more

MssqlRdsSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() STARTED

MssqlRdsSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() FAILED
    java.lang.RuntimeException: java.nio.file.NoSuchFileException: secrets/config.json
        at io.airbyte.commons.io.IOs.readFile(IOs.java:74)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.getStaticConfig(MssqlRdsSourceAcceptanceTest.java:47)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.setupEnvironment(MssqlRdsSourceAcceptanceTest.java:27)

        Caused by:
        java.nio.file.NoSuchFileException: secrets/config.json
            at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
            at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
            at java.base/java.nio.file.Files.readAllBytes(Files.java:3288)
            at java.base/java.nio.file.Files.readString(Files.java:3366)
            at io.airbyte.commons.io.IOs.readFile(IOs.java:72)
            ... 2 more

MssqlRdsSourceAcceptanceTest > testGetSpec() STARTED

MssqlRdsSourceAcceptanceTest > testGetSpec() FAILED
    java.lang.RuntimeException: java.nio.file.NoSuchFileException: secrets/config.json
        at io.airbyte.commons.io.IOs.readFile(IOs.java:74)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.getStaticConfig(MssqlRdsSourceAcceptanceTest.java:47)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.setupEnvironment(MssqlRdsSourceAcceptanceTest.java:27)

        Caused by:
        java.nio.file.NoSuchFileException: secrets/config.json
            at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
            at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
            at java.base/java.nio.file.Files.readAllBytes(Files.java:3288)
            at java.base/java.nio.file.Files.readString(Files.java:3366)
            at io.airbyte.commons.io.IOs.readFile(IOs.java:72)
            ... 2 more

MssqlRdsSourceAcceptanceTest > testFullRefreshRead() STARTED

MssqlRdsSourceAcceptanceTest > testFullRefreshRead() FAILED
    java.lang.RuntimeException: java.nio.file.NoSuchFileException: secrets/config.json
        at io.airbyte.commons.io.IOs.readFile(IOs.java:74)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.getStaticConfig(MssqlRdsSourceAcceptanceTest.java:47)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.setupEnvironment(MssqlRdsSourceAcceptanceTest.java:27)

        Caused by:
        java.nio.file.NoSuchFileException: secrets/config.json
            at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
            at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
            at java.base/java.nio.file.Files.readAllBytes(Files.java:3288)
            at java.base/java.nio.file.Files.readString(Files.java:3366)
            at io.airbyte.commons.io.IOs.readFile(IOs.java:72)
            ... 2 more

MssqlRdsSourceAcceptanceTest > testCheckConnection() STARTED

MssqlRdsSourceAcceptanceTest > testCheckConnection() FAILED
    java.lang.RuntimeException: java.nio.file.NoSuchFileException: secrets/config.json
        at io.airbyte.commons.io.IOs.readFile(IOs.java:74)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.getStaticConfig(MssqlRdsSourceAcceptanceTest.java:47)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.setupEnvironment(MssqlRdsSourceAcceptanceTest.java:27)

        Caused by:
        java.nio.file.NoSuchFileException: secrets/config.json
            at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
            at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
            at java.base/java.nio.file.Files.readAllBytes(Files.java:3288)
            at java.base/java.nio.file.Files.readString(Files.java:3366)
            at io.airbyte.commons.io.IOs.readFile(IOs.java:72)
            ... 2 more

MssqlRdsSourceAcceptanceTest > testDiscover() STARTED

MssqlRdsSourceAcceptanceTest > testDiscover() FAILED
    java.lang.RuntimeException: java.nio.file.NoSuchFileException: secrets/config.json
        at io.airbyte.commons.io.IOs.readFile(IOs.java:74)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.getStaticConfig(MssqlRdsSourceAcceptanceTest.java:47)
        at io.airbyte.integrations.source.mssql.MssqlRdsSourceAcceptanceTest.setupEnvironment(MssqlRdsSourceAcceptanceTest.java:27)

        Caused by:
        java.nio.file.NoSuchFileException: secrets/config.json
            at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
            at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
            at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
            at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
            at java.base/java.nio.file.Files.readAllBytes(Files.java:3288)
            at java.base/java.nio.file.Files.readString(Files.java:3366)
            at io.airbyte.commons.io.IOs.readFile(IOs.java:72)
            ... 2 more

CdcMssqlSourceTest > Verify that when data is inserted into the database while a sync is happening and after the first sync, it all gets replicated. STARTED

CdcMssqlSourceTest > Verify that when data is inserted into the database while a sync is happening and after the first sync, it all gets replicated. PASSED

CdcMssqlSourceTest > When no changes have been made to the database since the previous sync, no records are returned. STARTED

CdcMssqlSourceTest > When no changes have been made to the database since the previous sync, no records are returned. PASSED

CdcMssqlSourceTest > testCheck() STARTED

CdcMssqlSourceTest > testCheck() PASSED

CdcMssqlSourceTest > When both incremental CDC and full refresh are configured for different streams in a sync, the data is replicated as expected. STARTED

CdcMssqlSourceTest > When both incremental CDC and full refresh are configured for different streams in a sync, the data is replicated as expected. PASSED

CdcMssqlSourceTest > testDiscover() STARTED

CdcMssqlSourceTest > testDiscover() PASSED

CdcMssqlSourceTest > When a record is deleted, produces a deletion record. STARTED

CdcMssqlSourceTest > When a record is deleted, produces a deletion record. PASSED

CdcMssqlSourceTest > When no records exist, no records are returned. STARTED

CdcMssqlSourceTest > When no records exist, no records are returned. PASSED

CdcMssqlSourceTest > On the first sync, produce returns records that exist in the database. STARTED

CdcMssqlSourceTest > On the first sync, produce returns records that exist in the database. PASSED

CdcMssqlSourceTest > When a record is updated, produces an update record. STARTED

CdcMssqlSourceTest > When a record is updated, produces an update record. PASSED

CdcMssqlSourceTest > testAssertSnapshotIsolationAllowed() STARTED

CdcMssqlSourceTest > testAssertSnapshotIsolationAllowed() PASSED

CdcMssqlSourceTest > testAssertSnapshotIsolationDisabled() STARTED

CdcMssqlSourceTest > testAssertSnapshotIsolationDisabled() PASSED

CdcMssqlSourceTest > testCdcCheckOperationsWithDot() STARTED

CdcMssqlSourceTest > testCdcCheckOperationsWithDot() PASSED

CdcMssqlSourceTest > testAssertCdcSchemaQueryable() STARTED

CdcMssqlSourceTest > testAssertCdcSchemaQueryable() PASSED

CdcMssqlSourceTest > testGetTargetPosition() STARTED

CdcMssqlSourceTest > testGetTargetPosition() PASSED

CdcMssqlSourceTest > testAssertSqlServerAgentRunning() STARTED

CdcMssqlSourceTest > testAssertSqlServerAgentRunning() PASSED

CdcMssqlSourceTest > testCdcCheckOperations() STARTED

CdcMssqlSourceTest > testCdcCheckOperations() PASSED

CdcMssqlSourceTest > testAssertCdcEnabledInDb() STARTED

CdcMssqlSourceTest > testAssertCdcEnabledInDb() PASSED

MssqlCdcHelperTest > testIsCdc() STARTED

MssqlCdcHelperTest > testIsCdc() PASSED

MssqlCdcHelperTest > testGetDataToSyncConfig() STARTED

MssqlCdcHelperTest > testGetDataToSyncConfig() PASSED

MssqlCdcHelperTest > testGetSnapshotIsolation() STARTED

MssqlCdcHelperTest > testGetSnapshotIsolation() PASSED

MssqlJdbcSourceAcceptanceTest > testReadMultipleTables() STARTED

MssqlJdbcSourceAcceptanceTest > testReadMultipleTables() PASSED

MssqlJdbcSourceAcceptanceTest > testDiscoverWithNonCursorFields() STARTED

MssqlJdbcSourceAcceptanceTest > testDiscoverWithNonCursorFields() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalDateCheckCursor() STARTED

MssqlJdbcSourceAcceptanceTest > testIncrementalDateCheckCursor() PASSED

MssqlJdbcSourceAcceptanceTest > testReadMultipleTablesIncrementally() STARTED

MssqlJdbcSourceAcceptanceTest > testReadMultipleTablesIncrementally() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalCursorChanges() STARTED

MssqlJdbcSourceAcceptanceTest > testIncrementalCursorChanges() PASSED

MssqlJdbcSourceAcceptanceTest > testDiscoverWithNullableCursorFields() STARTED

MssqlJdbcSourceAcceptanceTest > testDiscoverWithNullableCursorFields() PASSED

MssqlJdbcSourceAcceptanceTest > testSpec() STARTED

MssqlJdbcSourceAcceptanceTest > testSpec() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckSuccess() STARTED

MssqlJdbcSourceAcceptanceTest > testCheckSuccess() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalIntCheckCursor() STARTED

MssqlJdbcSourceAcceptanceTest > testIncrementalIntCheckCursor() PASSED

MssqlJdbcSourceAcceptanceTest > testDiscoverWithMultipleSchemas() STARTED

MssqlJdbcSourceAcceptanceTest > testDiscoverWithMultipleSchemas() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckFailure() STARTED

MssqlJdbcSourceAcceptanceTest > testCheckFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalWithConcurrentInsertion() STARTED

MssqlJdbcSourceAcceptanceTest > testIncrementalWithConcurrentInsertion() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalNoPreviousState() STARTED

MssqlJdbcSourceAcceptanceTest > testIncrementalNoPreviousState() PASSED

MssqlJdbcSourceAcceptanceTest > testDiscover() STARTED

MssqlJdbcSourceAcceptanceTest > testDiscover() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalStringCheckCursor() STARTED

MssqlJdbcSourceAcceptanceTest > testIncrementalStringCheckCursor() PASSED

MssqlJdbcSourceAcceptanceTest > testReadSuccess() STARTED

MssqlJdbcSourceAcceptanceTest > testReadSuccess() PASSED

MssqlJdbcSourceAcceptanceTest > testReadOneColumn() STARTED

MssqlJdbcSourceAcceptanceTest > testReadOneColumn() PASSED

MssqlJdbcSourceAcceptanceTest > testReadOneTableIncrementallyTwice() STARTED

MssqlJdbcSourceAcceptanceTest > testReadOneTableIncrementallyTwice() PASSED

MssqlJdbcSourceAcceptanceTest > testReadFailure() STARTED

MssqlJdbcSourceAcceptanceTest > testReadFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testIncrementalStringCheckCursorSpaceInColumnName() STARTED

MssqlJdbcSourceAcceptanceTest > testIncrementalStringCheckCursorSpaceInColumnName() PASSED

MssqlJdbcSourceAcceptanceTest > testTablesWithQuoting() STARTED

MssqlJdbcSourceAcceptanceTest > testTablesWithQuoting() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectDataBaseFailure() STARTED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectDataBaseFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectPasswordFailure() STARTED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectPasswordFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testUserHasNoPermissionToDataBase() STARTED

MssqlJdbcSourceAcceptanceTest > testUserHasNoPermissionToDataBase() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectHostFailure() STARTED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectHostFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectUsernameFailure() STARTED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectUsernameFailure() PASSED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectPortFailure() STARTED

MssqlJdbcSourceAcceptanceTest > testCheckIncorrectPortFailure() PASSED

MssqlSourceTest > testDiscoverWithPk() STARTED

MssqlSourceTest > testDiscoverWithPk() PASSED

MssqlStressTest > stressTest() STARTED

CdcMssqlSourceAcceptanceTest > testIdenticalFullRefreshes() STARTED

CdcMssqlSourceAcceptanceTest > testIdenticalFullRefreshes() PASSED

CdcMssqlSourceAcceptanceTest > testEntrypointEnvVar() STARTED

CdcMssqlSourceAcceptanceTest > testEntrypointEnvVar() PASSED

CdcMssqlSourceAcceptanceTest > testIncrementalSyncWithState() STARTED

CdcMssqlSourceAcceptanceTest > testIncrementalSyncWithState() PASSED

CdcMssqlSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() STARTED

CdcMssqlSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() PASSED

CdcMssqlSourceAcceptanceTest > testGetSpec() STARTED

CdcMssqlSourceAcceptanceTest > testGetSpec() PASSED

CdcMssqlSourceAcceptanceTest > testFullRefreshRead() STARTED

CdcMssqlSourceAcceptanceTest > testFullRefreshRead() PASSED

CdcMssqlSourceAcceptanceTest > testCheckConnection() STARTED

CdcMssqlSourceAcceptanceTest > testCheckConnection() PASSED

CdcMssqlSourceAcceptanceTest > testDiscover() STARTED

CdcMssqlSourceAcceptanceTest > testDiscover() PASSED

CdcMssqlSourceDatatypeTest > testDataTypes() STARTED

CdcMssqlSourceDatatypeTest > testDataTypes() PASSED

MssqlSourceAcceptanceTest > testIdenticalFullRefreshes() STARTED

MssqlSourceAcceptanceTest > testIdenticalFullRefreshes() PASSED

MssqlSourceAcceptanceTest > testEntrypointEnvVar() STARTED

MssqlSourceAcceptanceTest > testEntrypointEnvVar() PASSED

MssqlSourceAcceptanceTest > testIncrementalSyncWithState() STARTED

MssqlSourceAcceptanceTest > testIncrementalSyncWithState() PASSED

MssqlSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() STARTED

MssqlSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() PASSED

MssqlSourceAcceptanceTest > testGetSpec() STARTED

MssqlSourceAcceptanceTest > testGetSpec() PASSED

MssqlSourceAcceptanceTest > testFullRefreshRead() STARTED

MssqlSourceAcceptanceTest > testFullRefreshRead() PASSED

MssqlSourceAcceptanceTest > testCheckConnection() STARTED

MssqlSourceAcceptanceTest > testCheckConnection() PASSED

MssqlSourceAcceptanceTest > testDiscover() STARTED

MssqlSourceAcceptanceTest > testDiscover() PASSED

MssqlSourceDatatypeTest > testDataTypes() STARTED

MssqlSourceDatatypeTest > testDataTypes() PASSED

SshKeyMssqlSourceAcceptanceTest > testIdenticalFullRefreshes() STARTED

SshKeyMssqlSourceAcceptanceTest > testIdenticalFullRefreshes() PASSED

SshKeyMssqlSourceAcceptanceTest > testEntrypointEnvVar() STARTED

SshKeyMssqlSourceAcceptanceTest > testEntrypointEnvVar() PASSED

SshKeyMssqlSourceAcceptanceTest > testIncrementalSyncWithState() STARTED

SshKeyMssqlSourceAcceptanceTest > testIncrementalSyncWithState() PASSED

SshKeyMssqlSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() STARTED

SshKeyMssqlSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() PASSED

SshKeyMssqlSourceAcceptanceTest > testGetSpec() STARTED

SshKeyMssqlSourceAcceptanceTest > testGetSpec() PASSED

SshKeyMssqlSourceAcceptanceTest > testFullRefreshRead() STARTED

SshKeyMssqlSourceAcceptanceTest > testFullRefreshRead() PASSED

SshKeyMssqlSourceAcceptanceTest > testCheckConnection() STARTED

SshKeyMssqlSourceAcceptanceTest > testCheckConnection() PASSED

SshKeyMssqlSourceAcceptanceTest > testDiscover() STARTED

SshKeyMssqlSourceAcceptanceTest > testDiscover() PASSED

SshPasswordMssqlSourceAcceptanceTest > testIdenticalFullRefreshes() STARTED

SshPasswordMssqlSourceAcceptanceTest > testIdenticalFullRefreshes() PASSED

SshPasswordMssqlSourceAcceptanceTest > testEntrypointEnvVar() STARTED

SshPasswordMssqlSourceAcceptanceTest > testEntrypointEnvVar() PASSED

SshPasswordMssqlSourceAcceptanceTest > testIncrementalSyncWithState() STARTED

SshPasswordMssqlSourceAcceptanceTest > testIncrementalSyncWithState() PASSED

SshPasswordMssqlSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() STARTED

SshPasswordMssqlSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() PASSED

SshPasswordMssqlSourceAcceptanceTest > testGetSpec() STARTED

SshPasswordMssqlSourceAcceptanceTest > testGetSpec() PASSED

SshPasswordMssqlSourceAcceptanceTest > testFullRefreshRead() STARTED

SshPasswordMssqlSourceAcceptanceTest > testFullRefreshRead() PASSED

SshPasswordMssqlSourceAcceptanceTest > testCheckConnection() STARTED

SshPasswordMssqlSourceAcceptanceTest > testCheckConnection() PASSED

SshPasswordMssqlSourceAcceptanceTest > testDiscover() STARTED

SshPasswordMssqlSourceAcceptanceTest > testDiscover() PASSED

SslEnabledMssqlSourceAcceptanceTest > testIdenticalFullRefreshes() STARTED

SslEnabledMssqlSourceAcceptanceTest > testIdenticalFullRefreshes() PASSED

SslEnabledMssqlSourceAcceptanceTest > testEntrypointEnvVar() STARTED

SslEnabledMssqlSourceAcceptanceTest > testEntrypointEnvVar() PASSED

SslEnabledMssqlSourceAcceptanceTest > testIncrementalSyncWithState() STARTED

SslEnabledMssqlSourceAcceptanceTest > testIncrementalSyncWithState() PASSED

SslEnabledMssqlSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() STARTED

SslEnabledMssqlSourceAcceptanceTest > testEmptyStateIncrementalIdenticalToFullRefresh() PASSED

SslEnabledMssqlSourceAcceptanceTest > testGetSpec() STARTED

SslEnabledMssqlSourceAcceptanceTest > testGetSpec() PASSED

SslEnabledMssqlSourceAcceptanceTest > testFullRefreshRead() STARTED

SslEnabledMssqlSourceAcceptanceTest > testFullRefreshRead() PASSED

SslEnabledMssqlSourceAcceptanceTest > testCheckConnection() STARTED

SslEnabledMssqlSourceAcceptanceTest > testCheckConnection() PASSED

SslEnabledMssqlSourceAcceptanceTest > testDiscover() STARTED

SslEnabledMssqlSourceAcceptanceTest > testDiscover() PASSED

99 tests completed, 8 failed, 1 skipped

> Task :airbyte-integrations:connectors:source-mssql:integrationTestJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':airbyte-integrations:connectors:source-mssql:integrationTestJava'.

CLAassistant · 2022-11-14T07:58:29Z

All committers have signed the CLA.

marcosmarxm · 2022-11-22T20:25:30Z

Hello 👋, first thank you for this amazing contribution.

We really appreciate the effort you've made to improve the project.
We ask you patience for the code review. Last month our team was focused on Hacktoberfest event and that probably left some PR without the proper feedback. And this week, due to the Thanksgiving US Holiday, most our team is out of office with their families. Another important piece of information why code won't be merge this week is: as a safety measure the core team has decided to freeze merging code to main branch to keep the release stable. Next week we'll return to you with the proper code review and update the status of your contribution.

If you have any questions feel free to send me a message in Slack!
Thanks!

sivankumar86 · 2022-11-28T02:23:21Z

it is impacted me as well. @quannh-uet Could you change connector version in docker file ?

quannh-uet · 2022-11-28T03:16:31Z

it is impacted me as well. @quannh-uet Could you change connector version in docker file ?

@sivankumar86 It's okay. I rebased to latest main branch and changed connector version in Dockerfile.

marcosmarxm · 2022-12-08T19:59:44Z

/test connector=connectors/source-mssql

🕑 connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/3651661922
❌ connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/3651661922
🐛

marcosmarxm · 2022-12-20T19:51:54Z

Hello 👋:skin-tone-2: and thank you for your contribution!

Airbyte has instituted a code freeze between 19 and 30 December, to make sure there are no disruptions during the holidays.
Because of this, reviewing and merging your contribution may take longer than usual.
We apologize for the delay, but we want everyone to have a quiet and happy holiday.

If you have any questions or need further clarification, please don't hesitate to ping via Slack.

sivankumar86 · 2023-03-01T22:38:35Z

Is there any update on this?

marcosmarxm · 2023-03-17T00:01:51Z

@grishick can you add this to source-dw team backlog? Look it is impacting some users.

marcosmarxm · 2023-05-30T13:58:20Z

Close due inactivity. @quannh-uet can you submit the contribution again

quannh-uet requested a review from a team as a code owner November 14, 2022 07:36

octavia-squidington-iv added area/connectors Connector related issues bounty community connectors/source/mssql labels Nov 14, 2022

quannh-uet changed the title ~~Source mssql fix cdc newchanges only~~ WIP: Source mssql fix cdc newchanges only Nov 14, 2022

quannh-uet force-pushed the source_mssql_fix_cdc_newchanges_only branch from f2e1165 to 0ac9c01 Compare November 15, 2022 01:37

quannh-uet changed the title ~~WIP: Source mssql fix cdc newchanges only~~ 🐛Source MSSQL: Fix CDC with sync mode is New changes only Nov 15, 2022

quannh-uet added 4 commits November 28, 2022 10:11

Change test for MSSQL source

cdd70c2

MSSQL Source: Fix CDC helper new spec config after version 0.4.18

eeaf68d

Change integration test check messages in incorrect port

fb1701d

Change Dockerfile connector version

2fdf0f1

quannh-uet force-pushed the source_mssql_fix_cdc_newchanges_only branch from b580332 to 2fdf0f1 Compare November 28, 2022 03:14

sajarin added the bounty-M Maintainer program: claimable medium bounty PR label Dec 5, 2022

sajarin added internal and removed bounty labels Jan 2, 2023

marcosmarxm assigned grishick Mar 17, 2023

marcosmarxm unassigned grishick Apr 7, 2023

marcosmarxm added the pr-alpha-beta label Apr 14, 2023

Merge branch 'master' into source_mssql_fix_cdc_newchanges_only

f530087

quannh-uet requested a review from a team as a code owner April 20, 2023 07:39

marcosmarxm added the community-legacy label May 15, 2023

marcosmarxm removed the community label May 15, 2023

marcosmarxm closed this May 30, 2023

marcosmarxm added community and removed community-legacy labels Aug 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛Source MSSQL: Fix CDC with sync mode is New changes only #19372

🐛Source MSSQL: Fix CDC with sync mode is New changes only #19372

quannh-uet commented Nov 14, 2022 •

edited

Loading

CLAassistant commented Nov 14, 2022 •

edited

Loading

marcosmarxm commented Nov 22, 2022

sivankumar86 commented Nov 28, 2022

quannh-uet commented Nov 28, 2022

marcosmarxm commented Dec 8, 2022 •

edited by github-actions bot

Loading

marcosmarxm commented Dec 20, 2022

sivankumar86 commented Mar 1, 2023

marcosmarxm commented Mar 17, 2023

marcosmarxm commented May 30, 2023

🐛Source MSSQL: Fix CDC with sync mode is New changes only #19372

🐛Source MSSQL: Fix CDC with sync mode is New changes only #19372

Conversation

quannh-uet commented Nov 14, 2022 • edited Loading

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Tests

CLAassistant commented Nov 14, 2022 • edited Loading

marcosmarxm commented Nov 22, 2022

sivankumar86 commented Nov 28, 2022

quannh-uet commented Nov 28, 2022

marcosmarxm commented Dec 8, 2022 • edited by github-actions bot Loading

marcosmarxm commented Dec 20, 2022

sivankumar86 commented Mar 1, 2023

marcosmarxm commented Mar 17, 2023

marcosmarxm commented May 30, 2023

quannh-uet commented Nov 14, 2022 •

edited

Loading

CLAassistant commented Nov 14, 2022 •

edited

Loading

marcosmarxm commented Dec 8, 2022 •

edited by github-actions bot

Loading