🎉 New Source: ClickUp #17770

josix · 2022-10-09T15:35:20Z

What

Adding a source connector for ingesting data from the ClickUp by using low-code configuration. (ref)

How

Following the instruction mentioned in tutorials to update clickup_api.yaml, spec.yaml, schemas/*.json, and configured_catalog.json.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub by running the /publish command described here
After the connector is published, connector added to connector index as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

CLAassistant · 2022-10-09T15:58:00Z

All committers have signed the CLA.

marcosmarxm

@josix do you need any further assistance here?

marcosmarxm · 2022-10-18T17:00:21Z

airbyte-integrations/connectors/source-clickup-api/integration_tests/catalog.json

+{
+  "streams": [
+    {
+      "name": "TODO fix this file",
+      "supported_sync_modes": ["full_refresh", "incremental"],
+      "source_defined_cursor": true,
+      "default_cursor_field": "column1",


Remove this file.

Removed at 8a2d8e7ef

marcosmarxm · 2022-10-18T17:00:46Z

airbyte-integrations/connectors/source-clickup-api/source_clickup_api/schemas/TODO.md

@@ -0,0 +1,16 @@
+# TODO: Define your stream schemas


Remove this file.

Removed at 8a2d8e7ef

marcosmarxm · 2022-10-18T17:01:29Z

airbyte-integrations/connectors/source-clickup-api/source_clickup_api/spec.yaml

+  additionalProperties: true
+  properties:
+    # 'TODO: This schema defines the configuration required for the source. This usually involves metadata such as database and/or authentication information.':
+    Authorization:


Suggested change

Authorization:

api_token:

Please rename to better naming.

Thanks for pointing this out. I've updated this.

josix · 2022-10-19T08:46:00Z

Hi @marcosmarxm thank you for the reminder and reviewing, I'm still working on that. But I guess I will be off here for around 2-3 days since I'm working on my master's degree oral exam at the same time. I'll continue to create some endpoints for sub-stream and incremental stream. Do you have some documents or examples about these? I think I could have a look before I start to implement. That would be helpful. Thanks a lot.

marcosmarxm · 2022-11-01T01:10:43Z

@josix I lost the notification of your message. Do you still need assistance here?

josix · 2022-11-01T06:58:10Z

@josix I lost the notification of your message. Do you still need assistance here?

Yes, please. I would start to continue the work this week although the hacktoberfest is gone lol.

marcosmarxm · 2022-11-01T16:22:38Z

Because this was submitted before 2-november the PR is eligible to receive the prize.

marcosmarxm · 2022-11-01T17:18:42Z

@josix there are some tutorials here: https://docs.airbyte.com/connector-development/config-based/tutorial/getting-started besides that please point what is your question to help your, or enter our Slack and ask for help in #hacktoberfest-2022 channel.

marcosmarxm · 2022-11-04T08:54:44Z

Hello! I'm going to be out of the office this Friday and won't be able to review your contribution again today, I return next Monday. So far, most contributions look solid and are almost done to be approved. As said in Chris' comment all contributions made before 2-November are eligible to receive the prize and have 2 weeks to merge the contributions. But I ensure next week we're going to have your contribution merged. If you have questions about the implementation you can send them in #hacktoberfest-2022 in Airbyte's Slack.

Sorry the inconvenience and see you again next week, thank you so much for your contribution!

…date/integration-source-clickup

josix · 2022-11-06T19:39:50Z

Hello @marcosmarxm, thank you for letting me know. I've added docs and removed some redundant files following your comments. Please have a look, thank you!

marcosmarxm

Some considerations

marcosmarxm · 2022-11-07T17:50:40Z

airbyte-integrations/connectors/source-clickup-api/source_clickup_api/clickup_api.yaml

+streams:
+  - type: DeclarativeStream


@josix are you planning to add more streams? (Tasks, Lists, etc?)

Yeah, I'll add Tasks stream these days.

marcosmarxm · 2022-11-07T17:52:09Z

airbyte-integrations/connectors/source-clickup-api/source_clickup_api/clickup_api.yaml

+    paginator:
+      type: NoPagination


Please add Pagination: https://clickup.com/api/clickupreference/operation/GetTasks/

marcosmarxm · 2022-11-10T13:26:33Z

@josix let me know if you need any assistance to finish the changes

josix · 2022-11-11T09:38:28Z

@marcosmarxm sorry for late reply, I've added list and task stream, please have a look thanks!

…date/integration-source-clickup

https://github.com/airbytehq/airbyte/tree/459856b73cfebe659741cccdd83590ba0dff2493/airbyte-integrations/bases/source-acceptance-test#migrating-acceptance-test-configyml-to-latest-configuration-format

…ceptance-tests

marcosmarxm · 2022-11-11T21:31:06Z

/test connector=connectors/source-clickup-api

🕑 connectors/source-clickup-api https://github.com/airbytehq/airbyte/actions/runs/3448097622
✅ connectors/source-clickup-api https://github.com/airbytehq/airbyte/actions/runs/3448097622
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       139      5    96%   87, 93, 235, 239-240
	 source_acceptance_test/conftest.py                     196     92    53%   35, 41-43, 48, 54, 60, 66, 72-74, 93, 98-100, 106-108, 114-115, 120-121, 126, 132, 141-150, 156-161, 176, 200, 231, 237, 243-248, 256-261, 269-282, 287-293, 300-311, 318-334
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              394    111    72%   53, 58, 87-95, 100-107, 111-112, 116-117, 293, 331-348, 357-365, 369-374, 380, 413-418, 456-463, 506-508, 511, 576-584, 596-599, 604, 660-661, 667, 670, 706-716, 729-754
	 source_acceptance_test/tests/test_incremental.py       158     14    91%   52-59, 64-77, 240
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1557    349    78%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: Incremental syncs are not supported on this connector.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:88: The previous connector image could not be retrieved.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:358: The previous connector image could not be retrieved.
======================== 26 passed, 3 skipped in 51.30s ========================

marcosmarxm · 2022-11-11T21:45:02Z

/publish connector=connectors/source-clickup-api

🕑 Publishing the following connectors:
connectors/source-clickup-api
https://github.com/airbytehq/airbyte/actions/runs/3448162962

Connector	Did it publish?	Were definitions generated?
connectors/source-clickup-api	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

* feat(connectors): add source clickup api template * feat(connectors/clickup-api): add /user and /team endpoint configs * docs(clickup-connectors): add clickup source doc * feat: add tasks stream and list stream to ClickUp API * feat: add space stream and folder stream to clikcup api * feat: update acceptance-test-config.yml w/ version in 0.2.12 https://github.com/airbytehq/airbyte/tree/459856b73cfebe659741cccdd83590ba0dff2493/airbyte-integrations/bases/source-acceptance-test#migrating-acceptance-test-configyml-to-latest-configuration-format * docs: add descripiton for optional configs fields * fix: fix list.json schema * fix: add ignore_fields to ignore changing value in full_refresh of acceptance-tests * add clickup to source def * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

feat(connectors): add source clickup api template

bcd48c9

github-actions bot added the area/connectors Connector related issues label Oct 9, 2022

octavia-squidington-iv added community connectors/source/clickup-api labels Oct 9, 2022

josix force-pushed the update/integration-source-clickup branch from d9ceb5d to e01b314 Compare October 10, 2022 17:20

josix mentioned this pull request Oct 11, 2022

Collection fails when running acceptance tests in python 3.10 env of low-code configuration source #17855

Closed

marcosmarxm requested changes Oct 18, 2022

View reviewed changes

marcosmarxm added the hacktober label Nov 1, 2022

feat(connectors/clickup-api): add /user and /team endpoint configs

8a2d8e7

josix force-pushed the update/integration-source-clickup branch from e01b314 to 8a2d8e7 Compare November 6, 2022 17:18

josix marked this pull request as ready for review November 6, 2022 18:15

octavia-squidington-iv added the bounty label Nov 6, 2022

josix added 2 commits November 7, 2022 02:56

Merge branch 'master' of https://github.com/airbytehq/airbyte into up…

f8560f7

…date/integration-source-clickup

docs(clickup-connectors): add clickup source doc

113bc64

github-actions bot added the area/documentation Improvements or additions to documentation label Nov 6, 2022

josix requested a review from marcosmarxm November 6, 2022 19:33

marcosmarxm requested changes Nov 7, 2022

View reviewed changes

feat: add tasks stream and list stream to ClickUp API

b34ab0f

feat: add space stream and folder stream to clikcup api

f7db99d

josix requested a review from marcosmarxm November 11, 2022 12:32

josix and others added 6 commits November 11, 2022 20:35

Merge branch 'master' of https://github.com/airbytehq/airbyte into up…

3591c47

…date/integration-source-clickup

feat: update acceptance-test-config.yml w/ version in 0.2.12

9879768

https://github.com/airbytehq/airbyte/tree/459856b73cfebe659741cccdd83590ba0dff2493/airbyte-integrations/bases/source-acceptance-test#migrating-acceptance-test-configyml-to-latest-configuration-format

docs: add descripiton for optional configs fields

7a4561d

fix: fix list.json schema

1948fc1

fix: add ignore_fields to ignore changing value in full_refresh of ac…

b2ba612

…ceptance-tests

add clickup to source def

6bc1394

octavia-squidington-iii and others added 3 commits November 11, 2022 22:04

auto-bump connector version

d794214

Merge branch 'master' into update/integration-source-clickup

34b5bb7

Merge branch 'master' into update/integration-source-clickup

bc216b1

marcosmarxm approved these changes Nov 13, 2022

View reviewed changes

marcosmarxm merged commit 07af2ff into airbytehq:master Nov 13, 2022

RealChrisSean added the low-code label Nov 15, 2022

github-actions bot mentioned this pull request Nov 15, 2022

Bump helm chart version reference to 0.40.43 #19453

Closed

octavia-squidington-iii mentioned this pull request Nov 18, 2022

Bump Airbyte version from 0.40.18 to 0.40.19 #19579

Merged

sajarin added internal and removed bounty labels Dec 5, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 New Source: ClickUp #17770

🎉 New Source: ClickUp #17770

josix commented Oct 9, 2022 •

edited

Loading

CLAassistant commented Oct 9, 2022 •

edited

Loading

marcosmarxm left a comment

marcosmarxm Oct 18, 2022

josix Nov 6, 2022

marcosmarxm Oct 18, 2022

josix Nov 6, 2022

marcosmarxm Oct 18, 2022

josix Nov 6, 2022

josix commented Oct 19, 2022

marcosmarxm commented Nov 1, 2022

josix commented Nov 1, 2022

marcosmarxm commented Nov 1, 2022

marcosmarxm commented Nov 1, 2022

marcosmarxm commented Nov 4, 2022

josix commented Nov 6, 2022

marcosmarxm left a comment

marcosmarxm Nov 7, 2022

josix Nov 8, 2022

marcosmarxm Nov 7, 2022

marcosmarxm commented Nov 10, 2022

josix commented Nov 11, 2022

marcosmarxm commented Nov 11, 2022 •

edited by github-actions bot

Loading

marcosmarxm commented Nov 11, 2022 •

edited by github-actions bot

Loading

🎉 New Source: ClickUp #17770

🎉 New Source: ClickUp #17770

Conversation

josix commented Oct 9, 2022 • edited Loading

What

How

Recommended reading order

Pre-merge Checklist

Community member or Airbyter

Airbyter

Tests

CLAassistant commented Oct 9, 2022 • edited Loading

marcosmarxm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josix commented Oct 19, 2022

marcosmarxm commented Nov 1, 2022

josix commented Nov 1, 2022

marcosmarxm commented Nov 1, 2022

marcosmarxm commented Nov 1, 2022

marcosmarxm commented Nov 4, 2022

josix commented Nov 6, 2022

marcosmarxm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcosmarxm commented Nov 10, 2022

josix commented Nov 11, 2022

marcosmarxm commented Nov 11, 2022 • edited by github-actions bot Loading

Build Passed

marcosmarxm commented Nov 11, 2022 • edited by github-actions bot Loading

josix commented Oct 9, 2022 •

edited

Loading

CLAassistant commented Oct 9, 2022 •

edited

Loading

marcosmarxm commented Nov 11, 2022 •

edited by github-actions bot

Loading

marcosmarxm commented Nov 11, 2022 •

edited by github-actions bot

Loading