Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support display_type property for oneOf fields and adjust CDC connectors to use it #29821

Merged
merged 22 commits into from
Aug 31, 2023

Conversation

lmossman
Copy link
Contributor

@lmossman lmossman commented Aug 25, 2023

What

Relates to #29399

We want to apply the same replication method UX improvements that were made to MySQL in #28882 to the other database connectors that support CDC: Postgres and MSSQL.

How

To accomplish the above, the following changes are made:

  • Add support for a display_type property that can be set on oneOf fields
    • This can be set to dropdown to get the current oneOf UX, or radio to get the radio-button card UX
    • FE changes to support this can be found here: https://github.com/airbytehq/airbyte-platform-internal/pull/8493
    • This PR adds CAT tests for that property, updates the connector spec docs, updates the 3 applicable connectors' specs to use it
  • Update the MySQL source connector to set display_type: radio on the replication_method oneOf
  • Update the Postgres source connector:
    • Set display_type: radio on the replication_method oneOf
    • Set CDC to be the default choice for new connectors
    • Adjust the titles and descriptions of the choices to more clearly explain their effects, and match the style of the MySQL changes that were made in 🎨 Improve replication method selection UX #28882
    • See the new postgres configuration UX in the screenshot below:
      Screenshot 2023-08-24 at 6 42 03 PM
  • Update the MSSQL source connector:
    • Set display_type: radio on the replication_method oneOf
    • Set CDC to be the default choice for new connectors
    • Adjust the titles and descriptions of the choices to more clearly explain their effects, and match the style of the MySQL changes that were made in 🎨 Improve replication method selection UX #28882
    • See the new mssql configuration UX in the screenshot below:
      Screenshot 2023-08-24 at 6 43 55 PM

🚨 User Impact 🚨

No breaking changes - these are only visual UI changes

@lmossman lmossman changed the title Support display_type property for oneOf fields Support display_type property for oneOf fields and adjust CDC connectors to use it Aug 25, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Aug 25, 2023

Coverage report for source-postgres

There is no coverage information present for the Files changed

Total Project Coverage 70.21% 🍏

@lmossman lmossman requested review from girarda and flash1293 August 25, 2023 02:01
@lmossman lmossman marked this pull request as ready for review August 25, 2023 02:01
@lmossman lmossman requested a review from a team as a code owner August 25, 2023 02:01
@github-actions
Copy link
Contributor

source-postgres test report (commit 2fe237984c) - ✅

⏲️ Total pipeline duration: 34mn24s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-postgres docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-postgres/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres test

@github-actions
Copy link
Contributor

source-postgres-strict-encrypt test report (commit 2fe237984c) - ✅

⏲️ Total pipeline duration: 11mn19s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-postgres-strict-encrypt docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-postgres-strict-encrypt/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres-strict-encrypt test

@github-actions
Copy link
Contributor

source-mysql-strict-encrypt test report (commit 2fe237984c) - ❌

⏲️ Total pipeline duration: 41mn30s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-mysql-strict-encrypt docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mysql-strict-encrypt test

@github-actions
Copy link
Contributor

source-mssql-strict-encrypt test report (commit 2fe237984c) - ❌

⏲️ Total pipeline duration: 315mn13s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-mssql-strict-encrypt docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-mssql-strict-encrypt/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mssql-strict-encrypt test

@github-actions
Copy link
Contributor

source-postgres-strict-encrypt test report (commit f922b16c1e) - ✅

⏲️ Total pipeline duration: 13mn03s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-postgres-strict-encrypt docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-postgres-strict-encrypt/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres-strict-encrypt test

@github-actions
Copy link
Contributor

source-postgres test report (commit f922b16c1e) - ✅

⏲️ Total pipeline duration: 32mn20s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-postgres docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-postgres/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres test

@github-actions
Copy link
Contributor

source-mysql test report (commit f922b16c1e) - ❌

⏲️ Total pipeline duration: 32mn57s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-mysql docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mysql test

@github-actions
Copy link
Contributor

source-mssql test report (commit f922b16c1e) - ❌

⏲️ Total pipeline duration: 8.06s

Step Result

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mssql test

@github-actions
Copy link
Contributor

source-postgres-strict-encrypt test report (commit d1193be2cf) - ✅

⏲️ Total pipeline duration: 14mn22s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-postgres-strict-encrypt docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-postgres-strict-encrypt/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres-strict-encrypt test

@github-actions
Copy link
Contributor

source-mysql test report (commit d1193be2cf) - ❌

⏲️ Total pipeline duration: 16mn52s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-mysql docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mysql test

@github-actions
Copy link
Contributor

source-postgres test report (commit d1193be2cf) - ✅

⏲️ Total pipeline duration: 32mn13s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-postgres docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-postgres/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres test

@lmossman
Copy link
Contributor Author

/approve-and-merge "just visual changes to the 3 connectors"

@octavia-approvington
Copy link
Contributor

Trying to merge your PR using a bot?
Comment this on the PR:

/approve-and-merge reason="<STATE YOUR REASON HERE>"

Make sure you've reviewed the docs!

@lmossman
Copy link
Contributor Author

/approve-and-merge reason="just visual changes to the 3 connectors"

@octavia-approvington
Copy link
Contributor

Our work here is done
done

@octavia-approvington octavia-approvington merged commit b19ed6d into master Aug 31, 2023
@octavia-approvington octavia-approvington deleted the lmossman/oneof-display-type branch August 31, 2023 17:19
@github-actions
Copy link
Contributor

source-mysql-strict-encrypt test report (commit d1193be2cf) - ❌

⏲️ Total pipeline duration: 38mn40s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-mysql-strict-encrypt docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mysql-strict-encrypt test

@github-actions
Copy link
Contributor

source-mssql test report (commit d1193be2cf) - ❌

⏲️ Total pipeline duration: 7.20s

Step Result

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mssql test

@github-actions
Copy link
Contributor

source-mssql-strict-encrypt test report (commit d1193be2cf) - ❌

⏲️ Total pipeline duration: 315mn07s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-mssql-strict-encrypt docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-mssql-strict-encrypt/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mssql-strict-encrypt test

@github-actions
Copy link
Contributor

source-openweather test report (commit c799a68b9a) - ❌

⏲️ Total pipeline duration: 02mn35s

Step Result
Connector package install
Build source-openweather docker image for platform linux/x86_64
Acceptance tests
Code format checks
Validate airbyte-integrations/connectors/source-openweather/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-openweather test

@github-actions
Copy link
Contributor

source-clockify test report (commit c799a68b9a) - ❌

⏲️ Total pipeline duration: 03mn13s

Step Result
Connector package install
Build source-clockify docker image for platform linux/x86_64
Acceptance tests
Code format checks
Validate airbyte-integrations/connectors/source-clockify/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-clockify test

@github-actions
Copy link
Contributor

source-mysql test report (commit c799a68b9a) - ❌

⏲️ Total pipeline duration: 36mn28s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-mysql docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mysql test

@github-actions
Copy link
Contributor

source-postgres-strict-encrypt test report (commit c799a68b9a) - ❌

⏲️ Total pipeline duration: 03mn32s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-postgres-strict-encrypt docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-postgres-strict-encrypt/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres-strict-encrypt test

@github-actions
Copy link
Contributor

source-mssql test report (commit c799a68b9a) - ❌

⏲️ Total pipeline duration: 5.83s

Step Result

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mssql test

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2023

source-mysql-strict-encrypt test report (commit c799a68b9a) - ❌

⏲️ Total pipeline duration: 40mn06s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-mysql-strict-encrypt docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mysql-strict-encrypt test

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2023

source-mssql-strict-encrypt test report (commit c799a68b9a) - ❌

⏲️ Total pipeline duration: 316mn51s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-mssql-strict-encrypt docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-mssql-strict-encrypt/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mssql-strict-encrypt test

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2023

source-postgres test report (commit c799a68b9a) - ❌

⏲️ Total pipeline duration: 34mn15s

Step Result
Java Connector Unit Tests
Build connector tar
Build source-postgres docker image for platform linux/x86_64
Java Connector Integration Tests
Acceptance tests
Validate airbyte-integrations/connectors/source-postgres/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres test

harrytou pushed a commit to KYVENetwork/airbyte that referenced this pull request Sep 1, 2023
…ors to use it (airbytehq#29821)

Co-authored-by: lmossman <lmossman@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants