🎉 New Source: Pinterest #7493

lazebnyi · 2021-10-29T14:31:31Z

What

#3484 - New Source: Pinterest

How

Using CDK and custom methods implemented the connector logic for SearchMetrics as a source for Airbyte.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

#### Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here

antixar

Looks good! Some minor comments.
And did you implement some rate-limit logic? https://developers.pinterest.com/docs/redoc/pinner_app/#tag/Rate-Limits

airbyte-integrations/connectors/source-pinterest/source_pinterest/source.py

airbyte-integrations/connectors/source-pinterest/acceptance-test-config.yml

lazebnyi · 2021-11-05T16:02:20Z

/test connector=connectors/source-pinterest

🕑 connectors/source-pinterest https://github.com/airbytehq/airbyte/actions/runs/1426441994
✅ connectors/source-pinterest https://github.com/airbytehq/airbyte/actions/runs/1426441994
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                       Stmts   Miss  Cover
	 --------------------------------------------------------------
	 base_python/__init__.py                       13      0   100%
	 base_python/catalog_helpers.py                10      6    40%
	 base_python/cdk/__init__.py                    0      0   100%
	 base_python/cdk/abstract_source.py            83     59    29%
	 base_python/cdk/streams/__init__.py            0      0   100%
	 base_python/cdk/streams/auth/__init__.py       0      0   100%
	 base_python/cdk/streams/auth/core.py           8      1    88%
	 base_python/cdk/streams/auth/jwt.py            5      5     0%
	 base_python/cdk/streams/auth/oauth.py         37     26    30%
	 base_python/cdk/streams/auth/token.py          9      4    56%
	 base_python/cdk/streams/core.py               63     32    49%
	 base_python/cdk/streams/exceptions.py         10      2    80%
	 base_python/cdk/streams/http.py               67     33    51%
	 base_python/cdk/streams/rate_limiting.py      30     14    53%
	 base_python/cdk/utils/__init__.py              0      0   100%
	 base_python/cdk/utils/casing.py                4      0   100%
	 base_python/client.py                         56     33    41%
	 base_python/entrypoint.py                     70     56    20%
	 base_python/integration.py                    52     25    52%
	 base_python/logger.py                         33     19    42%
	 base_python/schema_helpers.py                 56     41    27%
	 base_python/source.py                         51     34    33%
	 main_dev.py                                    3      3     0%
	 --------------------------------------------------------------
	 TOTAL                                        660    393    40%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        75      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       38     27    29%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 62     25    60%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  896    440    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                           Stmts   Miss  Cover
	 --------------------------------------------------
	 source_pinterest/__init__.py       2      0   100%
	 source_pinterest/source.py       172     35    80%
	 source_pinterest/utils.py          4      0   100%
	 --------------------------------------------------
	 TOTAL                            178     35    80%

airbyte-integrations/connectors/source-pinterest/bootstrap.md

airbyte-integrations/connectors/source-pinterest/build.gradle

...te-integrations/connectors/source-pinterest/source_pinterest/schemas/ad_group_analytics.json

airbyte-integrations/connectors/source-pinterest/source_pinterest/spec.json

docs/integrations/sources/pinterest.md

…est/spec.json Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

…est/spec.json Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

lazebnyi · 2021-11-08T20:37:53Z

/test connector=connectors/source-pinterest

🕑 connectors/source-pinterest https://github.com/airbytehq/airbyte/actions/runs/1436609998
✅ connectors/source-pinterest https://github.com/airbytehq/airbyte/actions/runs/1436609998
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        75      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       38     27    29%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 62     25    60%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  896    440    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                           Stmts   Miss  Cover
	 --------------------------------------------------
	 source_pinterest/__init__.py       2      0   100%
	 source_pinterest/source.py       175     36    79%
	 source_pinterest/utils.py          4      0   100%
	 --------------------------------------------------
	 TOTAL                            181     36    80%

airbyte-integrations/connectors/source-pinterest/source_pinterest/source.py

lazebnyi · 2021-11-09T10:33:30Z

/publish connector=connectors/source-pinterest

🕑 connectors/source-pinterest https://github.com/airbytehq/airbyte/actions/runs/1439035166
❌ connectors/source-pinterest https://github.com/airbytehq/airbyte/actions/runs/1439035166

lazebnyi · 2021-11-09T10:43:57Z

/publish connector=connectors/source-pinterest

🕑 connectors/source-pinterest https://github.com/airbytehq/airbyte/actions/runs/1439075058
✅ connectors/source-pinterest https://github.com/airbytehq/airbyte/actions/runs/1439075058

* Add Pinterest implementation * Updated to review * Added rate limits and PR number to doc * Fixed unit tests and data slicing bug * Update airbyte-integrations/connectors/source-pinterest/source_pinterest/spec.json Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * Update airbyte-integrations/connectors/source-pinterest/build.gradle Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * Update airbyte-integrations/connectors/source-pinterest/source_pinterest/spec.json Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * Updated to review * Updated window in days * Fixed unit test Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

Add Pinterest implementation

af30288

lazebnyi linked an issue Oct 29, 2021 that may be closed by this pull request

Support Pinterest Marketing API #3484

Closed

github-actions bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Oct 29, 2021

lazebnyi requested review from annalvova05 and antixar November 1, 2021 13:18

antixar suggested changes Nov 3, 2021

View reviewed changes

Updated to review

57322b7

lazebnyi temporarily deployed to more-secrets November 3, 2021 11:13 Inactive

Added rate limits and PR number to doc

8e66d79

lazebnyi requested a review from antixar November 3, 2021 13:57

lazebnyi temporarily deployed to more-secrets November 3, 2021 13:59 Inactive

antixar approved these changes Nov 3, 2021

View reviewed changes

annalvova05 approved these changes Nov 4, 2021

View reviewed changes

Fixed unit tests and data slicing bug

eabcd8a

lazebnyi temporarily deployed to more-secrets November 5, 2021 15:29 Inactive

Merge branch 'master' into lazebnyi/3484-pinterest

83338ef

lazebnyi temporarily deployed to more-secrets November 5, 2021 15:40 Inactive

jrhizor temporarily deployed to more-secrets November 5, 2021 16:04 Inactive

lazebnyi requested a review from sherifnada November 5, 2021 16:10

lazebnyi marked this pull request as ready for review November 5, 2021 16:15

sherifnada reviewed Nov 8, 2021

View reviewed changes

lazebnyi and others added 2 commits November 8, 2021 15:11

Update airbyte-integrations/connectors/source-pinterest/source_pinter…

7d7980e

…est/spec.json Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

Update airbyte-integrations/connectors/source-pinterest/build.gradle

4d6d01a

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

lazebnyi temporarily deployed to more-secrets November 8, 2021 13:12 Inactive

Update airbyte-integrations/connectors/source-pinterest/source_pinter…

ac837b8

…est/spec.json Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

lazebnyi temporarily deployed to more-secrets November 8, 2021 13:15 Inactive

Updated to review

d5271fd

lazebnyi temporarily deployed to more-secrets November 8, 2021 20:34 Inactive

jrhizor temporarily deployed to more-secrets November 8, 2021 20:40 Inactive

lazebnyi requested a review from sherifnada November 8, 2021 21:03

sherifnada approved these changes Nov 9, 2021

View reviewed changes

airbyte-integrations/connectors/source-pinterest/source_pinterest/source.py Outdated Show resolved Hide resolved

Updated window in days

e17b5bd

lazebnyi temporarily deployed to more-secrets November 9, 2021 10:34 Inactive

jrhizor temporarily deployed to more-secrets November 9, 2021 10:35 Inactive

Fixed unit test

531f1f9

lazebnyi temporarily deployed to more-secrets November 9, 2021 10:45 Inactive

jrhizor temporarily deployed to more-secrets November 9, 2021 10:46 Inactive

lazebnyi merged commit 2d2965b into master Nov 9, 2021

lazebnyi deleted the lazebnyi/3484-pinterest branch November 9, 2021 10:54

jrhizor mentioned this pull request Nov 9, 2021

Bump Airbyte version from 0.30.36-alpha to 0.30.37-alpha #7800

Merged

lazebnyi mentioned this pull request Nov 30, 2021

🐛 Source Marketo: Fix timestamp value format issue #8298

Merged

16 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 New Source: Pinterest #7493

🎉 New Source: Pinterest #7493

lazebnyi commented Oct 29, 2021 •

edited

Loading

antixar left a comment

lazebnyi commented Nov 5, 2021 •

edited by github-actions bot

Loading

lazebnyi commented Nov 8, 2021 •

edited by github-actions bot

Loading

lazebnyi commented Nov 9, 2021 •

edited by github-actions bot

Loading

lazebnyi commented Nov 9, 2021 •

edited by github-actions bot

Loading

🎉 New Source: Pinterest #7493

🎉 New Source: Pinterest #7493

Conversation

lazebnyi commented Oct 29, 2021 • edited Loading

What

How

Recommended reading order

Pre-merge Checklist

Airbyter

antixar left a comment

Choose a reason for hiding this comment

lazebnyi commented Nov 5, 2021 • edited by github-actions bot Loading

lazebnyi commented Nov 8, 2021 • edited by github-actions bot Loading

lazebnyi commented Nov 9, 2021 • edited by github-actions bot Loading

lazebnyi commented Nov 9, 2021 • edited by github-actions bot Loading

lazebnyi commented Oct 29, 2021 •

edited

Loading

lazebnyi commented Nov 5, 2021 •

edited by github-actions bot

Loading

lazebnyi commented Nov 8, 2021 •

edited by github-actions bot

Loading

lazebnyi commented Nov 9, 2021 •

edited by github-actions bot

Loading

lazebnyi commented Nov 9, 2021 •

edited by github-actions bot

Loading