Quote cursor name to support uppercase letters (required by postgres) #28892

rodireich · 2023-08-01T00:43:41Z

What

Quote cursor field in getting max value query.
Postgres requires quotes to preserve case sensitivity in queries

closes #28879

github-actions · 2023-08-01T00:44:01Z

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

PR name follows PR naming conventions
Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
Secrets in the connector's spec are annotated with airbyte_secret
All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

Check for hidden checklists in your PR description
Toggle the github label checklist-action-run on/off to re-run the checklist CI.

nguyenaiden · 2023-08-01T00:46:06Z

...ource-postgres/src/main/java/io/airbyte/integrations/source/postgres/PostgresQueryUtils.java

        final String cursorField = cursorInfoOptional.get().getCursorField();
        final String cursorBasedSyncStatusQuery = String.format(MAX_CURSOR_VALUE_QUERY,
            cursorField,
            name,
            cursorField,
            cursorField,
            name);
+        LOGGER.info("*** {}", cursorBasedSyncStatusQuery);


Remove if not needed. Same with L154

removed. thanks

nguyenaiden

Awesome catch. Thanks for this!

octavia-squidington-iii · 2023-08-01T00:48:17Z

source-alloydb test report (commit `520138bcc9`) - ❌

⏲️ Total pipeline duration: 02mn29s

Step	Result
Validate airbyte-integrations/connectors/source-alloydb/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Build connector tar	✅
Build source-alloydb docker image for platform linux/x86_64	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb test

octavia-squidington-iii · 2023-08-01T00:51:02Z

source-postgres-strict-encrypt test report (commit `520138bcc9`) - ✅

⏲️ Total pipeline duration: 02mn33s

Step	Result

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres-strict-encrypt test

octavia-squidington-iii · 2023-08-01T00:53:50Z

source-postgres test report (commit `520138bcc9`) - ✅

⏲️ Total pipeline duration: 02mn33s

Step	Result

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres test

octavia-squidington-iii · 2023-08-01T00:56:33Z

source-alloydb-strict-encrypt test report (commit `520138bcc9`) - ❌

⏲️ Total pipeline duration: 02mn31s

Step	Result
Validate airbyte-integrations/connectors/source-alloydb-strict-encrypt/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Build connector tar	✅
Build source-alloydb-strict-encrypt docker image for platform linux/x86_64	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb-strict-encrypt test

octavia-squidington-iii · 2023-08-01T01:01:15Z

source-alloydb-strict-encrypt test report (commit `ff5d523184`) - ✅

⏲️ Total pipeline duration: 02mn31s

Step	Result

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb-strict-encrypt test

octavia-squidington-iii · 2023-08-01T01:03:56Z

source-alloydb test report (commit `ff5d523184`) - ❌

⏲️ Total pipeline duration: 02mn29s

Step	Result
Validate airbyte-integrations/connectors/source-alloydb/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Build connector tar	✅
Build source-alloydb docker image for platform linux/x86_64	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb test

octavia-squidington-iii · 2023-08-01T01:06:37Z

source-postgres-strict-encrypt test report (commit `ff5d523184`) - ❌

⏲️ Total pipeline duration: 02mn27s

Step	Result
Validate airbyte-integrations/connectors/source-postgres-strict-encrypt/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Build connector tar	✅
Build source-postgres-strict-encrypt docker image for platform linux/x86_64	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres-strict-encrypt test

github-actions · 2023-08-01T01:08:48Z

Coverage report for source-postgres

File	Coverage [82.66%]	🍏
PostgresQueryUtils.java	82.66%	🍏

Total Project Coverage	70.27%	🍏

octavia-squidington-iii · 2023-08-01T01:09:23Z

source-postgres test report (commit `ff5d523184`) - ❌

⏲️ Total pipeline duration: 02mn34s

Step	Result
Validate airbyte-integrations/connectors/source-postgres/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Build connector tar	✅
Build source-postgres docker image for platform linux/x86_64	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres test

octavia-squidington-iii · 2023-08-01T01:11:45Z

source-alloydb-strict-encrypt test report (commit `74de8f1b62`) - ❌

⏲️ Total pipeline duration: 6.03s

Step	Result
Validate airbyte-integrations/connectors/source-alloydb-strict-encrypt/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Build connector tar	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb-strict-encrypt test

octavia-squidington-iii · 2023-08-01T01:14:25Z

source-alloydb test report (commit `74de8f1b62`) - ❌

⏲️ Total pipeline duration: 02mn28s

Step	Result
Validate airbyte-integrations/connectors/source-alloydb/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Build connector tar	✅
Build source-alloydb docker image for platform linux/x86_64	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb test

octavia-squidington-iii · 2023-08-01T01:17:09Z

source-postgres-strict-encrypt test report (commit `74de8f1b62`) - ❌

⏲️ Total pipeline duration: 02mn31s

Step	Result
Validate airbyte-integrations/connectors/source-postgres-strict-encrypt/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Build connector tar	✅
Build source-postgres-strict-encrypt docker image for platform linux/x86_64	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres-strict-encrypt test

octavia-squidington-iii · 2023-08-01T01:20:06Z

source-postgres test report (commit `74de8f1b62`) - ❌

⏲️ Total pipeline duration: 02mn43s

Step	Result
Validate airbyte-integrations/connectors/source-postgres/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Build connector tar	✅
Build source-postgres docker image for platform linux/x86_64	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres test

prateekmukhedkar · 2023-08-01T01:36:28Z

...ource-postgres/src/main/java/io/airbyte/integrations/source/postgres/PostgresQueryUtils.java

@@ -73,7 +73,7 @@ public record ResultWithFailed<T>(T result, List<io.airbyte.protocol.models.v0.A
      """;
  public static final String MAX_CURSOR_VALUE_QUERY =
      """
-        SELECT %s FROM %s WHERE %s = (SELECT MAX(%s) FROM %s);
+        SELECT "%s" FROM %s WHERE "%s" = (SELECT MAX("%s") FROM %s);


verified by looking at NULL_CURSOR_VALUE_WITH_SCHEMA_QUERY and NULL_CURSOR_VALUE_NO_SCHEMA_QUERY that column names are enclosed in double quotes.

rodireich · 2023-08-01T03:16:16Z

/approve-and-merge reason="tested locally. ci pipeline issue"

octavia-approvington · 2023-08-01T03:16:50Z

Approve this

quote cursor name to support uppercase letters (required by postgres)

520138b

rodireich requested a review from a team as a code owner August 1, 2023 00:43

octavia-squidington-iii added the area/connectors Connector related issues label Aug 1, 2023

rodireich changed the title ~~quote cursor name to support uppercase letters (required by postgres)~~ Quote cursor name to support uppercase letters (required by postgres) Aug 1, 2023

octavia-squidington-iii added the connectors/source/postgres label Aug 1, 2023

nguyenaiden reviewed Aug 1, 2023

View reviewed changes

sanity

7d139a9

nguyenaiden approved these changes Aug 1, 2023

View reviewed changes

sanity

ff5d523

octavia-squidington-iii added area/documentation Improvements or additions to documentation connectors/source/postgres-strict-encrypt connectors/source/alloydb connectors/source/alloydb-strict-encrypt labels Aug 1, 2023

Merge branch 'master' into rodi/uppercase-cursor

74de8f1

prateekmukhedkar reviewed Aug 1, 2023

View reviewed changes

prateekmukhedkar approved these changes Aug 1, 2023

View reviewed changes

octavia-approvington approved these changes Aug 1, 2023

View reviewed changes

octavia-approvington merged commit d37d2e2 into master Aug 1, 2023

octavia-approvington deleted the rodi/uppercase-cursor branch August 1, 2023 03:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Quote cursor name to support uppercase letters (required by postgres) #28892

Quote cursor name to support uppercase letters (required by postgres) #28892

rodireich commented Aug 1, 2023 •

edited

Loading

github-actions bot commented Aug 1, 2023

nguyenaiden Aug 1, 2023

rodireich Aug 1, 2023

nguyenaiden left a comment

octavia-squidington-iii commented Aug 1, 2023

octavia-squidington-iii commented Aug 1, 2023

octavia-squidington-iii commented Aug 1, 2023

octavia-squidington-iii commented Aug 1, 2023

octavia-squidington-iii commented Aug 1, 2023

octavia-squidington-iii commented Aug 1, 2023

octavia-squidington-iii commented Aug 1, 2023

github-actions bot commented Aug 1, 2023 •

edited

Loading

octavia-squidington-iii commented Aug 1, 2023

octavia-squidington-iii commented Aug 1, 2023

octavia-squidington-iii commented Aug 1, 2023

octavia-squidington-iii commented Aug 1, 2023

octavia-squidington-iii commented Aug 1, 2023

prateekmukhedkar Aug 1, 2023 •

edited

Loading

rodireich commented Aug 1, 2023

octavia-approvington commented Aug 1, 2023

Quote cursor name to support uppercase letters (required by postgres) #28892

Quote cursor name to support uppercase letters (required by postgres) #28892

Conversation

rodireich commented Aug 1, 2023 • edited Loading

What

github-actions bot commented Aug 1, 2023

Before Merging a Connector Pull Request

nguyenaiden Aug 1, 2023

Choose a reason for hiding this comment

rodireich Aug 1, 2023

Choose a reason for hiding this comment

nguyenaiden left a comment

Choose a reason for hiding this comment

octavia-squidington-iii commented Aug 1, 2023

source-alloydb test report (commit 520138bcc9) - ❌

octavia-squidington-iii commented Aug 1, 2023

source-postgres-strict-encrypt test report (commit 520138bcc9) - ✅

octavia-squidington-iii commented Aug 1, 2023

source-postgres test report (commit 520138bcc9) - ✅

octavia-squidington-iii commented Aug 1, 2023

source-alloydb-strict-encrypt test report (commit 520138bcc9) - ❌

octavia-squidington-iii commented Aug 1, 2023

source-alloydb-strict-encrypt test report (commit ff5d523184) - ✅

octavia-squidington-iii commented Aug 1, 2023

source-alloydb test report (commit ff5d523184) - ❌

octavia-squidington-iii commented Aug 1, 2023

source-postgres-strict-encrypt test report (commit ff5d523184) - ❌

github-actions bot commented Aug 1, 2023 • edited Loading

Coverage report for source-postgres

octavia-squidington-iii commented Aug 1, 2023

source-postgres test report (commit ff5d523184) - ❌

octavia-squidington-iii commented Aug 1, 2023

source-alloydb-strict-encrypt test report (commit 74de8f1b62) - ❌

octavia-squidington-iii commented Aug 1, 2023

source-alloydb test report (commit 74de8f1b62) - ❌

octavia-squidington-iii commented Aug 1, 2023

source-postgres-strict-encrypt test report (commit 74de8f1b62) - ❌

octavia-squidington-iii commented Aug 1, 2023

source-postgres test report (commit 74de8f1b62) - ❌

prateekmukhedkar Aug 1, 2023 • edited Loading

Choose a reason for hiding this comment

rodireich commented Aug 1, 2023

octavia-approvington commented Aug 1, 2023

rodireich commented Aug 1, 2023 •

edited

Loading

source-alloydb test report (commit `520138bcc9`) - ❌

source-postgres-strict-encrypt test report (commit `520138bcc9`) - ✅

source-postgres test report (commit `520138bcc9`) - ✅

source-alloydb-strict-encrypt test report (commit `520138bcc9`) - ❌

source-alloydb-strict-encrypt test report (commit `ff5d523184`) - ✅

source-alloydb test report (commit `ff5d523184`) - ❌

source-postgres-strict-encrypt test report (commit `ff5d523184`) - ❌

github-actions bot commented Aug 1, 2023 •

edited

Loading

source-postgres test report (commit `ff5d523184`) - ❌

source-alloydb-strict-encrypt test report (commit `74de8f1b62`) - ❌

source-alloydb test report (commit `74de8f1b62`) - ❌

source-postgres-strict-encrypt test report (commit `74de8f1b62`) - ❌

source-postgres test report (commit `74de8f1b62`) - ❌

prateekmukhedkar Aug 1, 2023 •

edited

Loading