🐛 Source Zendesk Support: Fix wrong types for schemas, add Transformer #8050

yevhenii-ldv · 2021-11-17T13:20:10Z

What

resolves #7441.
The problem was that the types were not specified for some of the fields in the schemas.

How

Describe the solution

🚨 User Impact 🚨

Are there any breaking changes? If yes, please make sure to include it here and in any changelogs with the 🚨🚨 emoji
What is the end result perceived by the user?

Pre-merge Checklist

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here
After the new connector version is published, connector version bumped in the seed directory as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

yevhenii-ldv · 2021-11-17T13:20:33Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1471764479
❌ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1471764479
🐛 https://gradle.com/s/taryx4npw2h2a

yevhenii-ldv · 2021-11-17T13:37:12Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1471853659
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1471853659
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        75      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       38     27    29%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 62     25    60%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  896    440    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                 Stmts   Miss  Cover
	 --------------------------------------------------------
	 source_zendesk_support/__init__.py       2      0   100%
	 source_zendesk_support/source.py        38     15    61%
	 source_zendesk_support/streams.py      277     78    72%
	 --------------------------------------------------------
	 TOTAL                                  317     93    71%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                 Stmts   Miss  Cover
	 --------------------------------------------------------
	 source_zendesk_support/__init__.py       2      0   100%
	 source_zendesk_support/source.py        38     15    61%
	 source_zendesk_support/streams.py      277     87    69%
	 --------------------------------------------------------
	 TOTAL                                  317    102    68%

avida · 2021-11-17T13:59:38Z

airbyte-integrations/connectors/source-zendesk-support/unit_tests/unit_test.py

+)
+def test_transform_for_tickets_stream(prepare_stream_args, input_data, expected_data):
+    """Checks Transform in case when records come with invalid fields data types"""
+    stream = Tickets(**prepare_stream_args)


What the point of testing it on stream level and manualy calling transorm method? Please test it on Source level like here connectors/source-iterable/unit_tests/test_source.py

avida

Please update CDK version to 0.1.35 where false warning were fixed

yevhenii-ldv · 2021-11-17T15:05:38Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1472289443
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1472289443
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        75      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       38     27    29%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 62     25    60%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  896    440    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                 Stmts   Miss  Cover
	 --------------------------------------------------------
	 source_zendesk_support/__init__.py       2      0   100%
	 source_zendesk_support/source.py        38     15    61%
	 source_zendesk_support/streams.py      277     77    72%
	 --------------------------------------------------------
	 TOTAL                                  317     92    71%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                 Stmts   Miss  Cover
	 --------------------------------------------------------
	 source_zendesk_support/__init__.py       2      0   100%
	 source_zendesk_support/source.py        38     13    66%
	 source_zendesk_support/streams.py      277     86    69%
	 --------------------------------------------------------
	 TOTAL                                  317     99    69%

avida · 2021-11-17T15:58:58Z

docs/integrations/sources/zendesk-support.md

@@ -97,6 +97,7 @@ We recommend creating a restricted, read-only key specifically for Airbyte acces

 | Version | Date | Pull Request | Subject |
 | :------ | :--------  | :-----       | :------ |
+| `0.1.5` | 2021-11-?? | [????](https://github.com/airbytehq/airbyte/pull/????) | Fix wrong types for schemas, add Transformer |


forgot to update this

I am going to request review from Airbyte team for this PR, and update the changeogs before publishing.

yevhenii-ldv · 2021-11-18T13:47:16Z

/publish connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1476678415
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1476678415

airbytehq#8050) * Fix wrong types for schemas, add Transformer

Fix wrong types for schemas, add Transformer

1355d5f

github-actions bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Nov 17, 2021

jrhizor temporarily deployed to more-secrets November 17, 2021 13:23 Inactive

jrhizor temporarily deployed to more-secrets November 17, 2021 13:39 Inactive

yevhenii-ldv requested review from avida, htrueman, sergei-solonitcyn and Zirochkaa November 17, 2021 13:52

avida reviewed Nov 17, 2021

View reviewed changes

avida suggested changes Nov 17, 2021

View reviewed changes

Zirochkaa approved these changes Nov 17, 2021

View reviewed changes

update after review

f03145e

yevhenii-ldv temporarily deployed to more-secrets November 17, 2021 15:07 Inactive

jrhizor temporarily deployed to more-secrets November 17, 2021 15:08 Inactive

yevhenii-ldv requested a review from avida November 17, 2021 15:29

avida approved these changes Nov 17, 2021

View reviewed changes

avida reviewed Nov 17, 2021

View reviewed changes

yevhenii-ldv requested review from sherifnada and Phlair November 17, 2021 16:19

sherifnada removed the request for review from Phlair November 18, 2021 00:34

sherifnada approved these changes Nov 18, 2021

View reviewed changes

yevhenii-ldv added 2 commits November 18, 2021 15:45

merge master

650800c

bump version

5ae44c4

yevhenii-ldv temporarily deployed to more-secrets November 18, 2021 13:48 Inactive

jrhizor temporarily deployed to more-secrets November 18, 2021 13:49 Inactive

update version

4761ee0

yevhenii-ldv merged commit 006c34c into master Nov 18, 2021

yevhenii-ldv deleted the ykurochkin/custom-fields-not-extracted-properly-after-normalization branch November 18, 2021 14:07

yevhenii-ldv temporarily deployed to more-secrets November 18, 2021 14:08 Inactive

This was referenced Nov 18, 2021

Bump Airbyte version from 0.32.2-alpha to 0.32.3-alpha #8105

Merged

Bump Airbyte version from 0.32.2-alpha to 0.32.3-alpha #8109

Closed

schlattk pushed a commit to schlattk/airbyte that referenced this pull request Jan 4, 2022

🐛 Source Zendesk Support: Fix wrong types for schemas, add Transformer (

3a80e47

airbytehq#8050) * Fix wrong types for schemas, add Transformer

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 Source Zendesk Support: Fix wrong types for schemas, add Transformer #8050

🐛 Source Zendesk Support: Fix wrong types for schemas, add Transformer #8050

yevhenii-ldv commented Nov 17, 2021

yevhenii-ldv commented Nov 17, 2021 •

edited by github-actions bot

Loading

yevhenii-ldv commented Nov 17, 2021 •

edited by github-actions bot

Loading

avida Nov 17, 2021

avida left a comment

yevhenii-ldv commented Nov 17, 2021 •

edited by github-actions bot

Loading

avida Nov 17, 2021

yevhenii-ldv Nov 17, 2021

yevhenii-ldv commented Nov 18, 2021 •

edited by github-actions bot

Loading

🐛 Source Zendesk Support: Fix wrong types for schemas, add Transformer #8050

🐛 Source Zendesk Support: Fix wrong types for schemas, add Transformer #8050

Conversation

yevhenii-ldv commented Nov 17, 2021

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

yevhenii-ldv commented Nov 17, 2021 • edited by github-actions bot Loading

yevhenii-ldv commented Nov 17, 2021 • edited by github-actions bot Loading

avida Nov 17, 2021

Choose a reason for hiding this comment

avida left a comment

Choose a reason for hiding this comment

yevhenii-ldv commented Nov 17, 2021 • edited by github-actions bot Loading

avida Nov 17, 2021

Choose a reason for hiding this comment

yevhenii-ldv Nov 17, 2021

Choose a reason for hiding this comment

yevhenii-ldv commented Nov 18, 2021 • edited by github-actions bot Loading

yevhenii-ldv commented Nov 17, 2021 •

edited by github-actions bot

Loading

yevhenii-ldv commented Nov 17, 2021 •

edited by github-actions bot

Loading

yevhenii-ldv commented Nov 17, 2021 •

edited by github-actions bot

Loading

yevhenii-ldv commented Nov 18, 2021 •

edited by github-actions bot

Loading