Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Source ZenDesk Support: fix 404 responses for the ticket_comments stream #6513

Merged
merged 15 commits into from
Oct 15, 2021
Merged

🐛 Source ZenDesk Support: fix 404 responses for the ticket_comments stream #6513

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
87d02f5
fix 404 responses for the ticket_comments stream
gl-pix Sep 28, 2021
a873a6d
add unit test
gl-pix Sep 29, 2021
274a195
add unit test
gl-pix Sep 29, 2021
72108cf
Merge remote-tracking branch 'origin' into antixar/6285-zendesk-suppo…
gl-pix Sep 30, 2021
d8d6058
add oauth2 access token
gl-pix Sep 30, 2021
9f98748
Update airbyte-integrations/connectors/source-zendesk-support/source_…
antixar Oct 8, 2021
7df812c
switching among auth methods
gl-pix Oct 8, 2021
01f8a1d
merge with master
gl-pix Oct 11, 2021
d455f30
Update docs/integrations/sources/zendesk-support.md
antixar Oct 12, 2021
4a831f9
Merge branch 'antixar/6249-zendesk-support-not-found-comments' of git…
gl-pix Oct 13, 2021
2da950d
add log message for 404 errors
gl-pix Oct 13, 2021
716e8c1
Update docs/integrations/sources/zendesk-support.md
antixar Oct 15, 2021
0f25474
Update airbyte-integrations/connectors/source-zendesk-support/source_…
antixar Oct 15, 2021
f0bcd10
remove oauth logic from this PR
gl-pix Oct 15, 2021
18903ef
remove oauth logic from this PR
gl-pix Oct 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/publish-command.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ jobs:
ZENDESK_SUNSHINE_TEST_CREDS: ${{ secrets.ZENDESK_SUNSHINE_TEST_CREDS }}
ZENDESK_TALK_TEST_CREDS: ${{ secrets.ZENDESK_TALK_TEST_CREDS }}
ZENDESK_SUPPORT_TEST_CREDS: ${{ secrets.ZENDESK_SUPPORT_TEST_CREDS }}
ZENDESK_SUPPORT_OAUTH_TEST_CREDS: ${{ secrets.ZENDESK_SUPPORT_OAUTH_TEST_CREDS }}
ZOOM_INTEGRATION_TEST_CREDS: ${{ secrets.ZOOM_INTEGRATION_TEST_CREDS }}
PLAID_INTEGRATION_TEST_CREDS: ${{ secrets.PLAID_INTEGRATION_TEST_CREDS }}
DESTINATION_S3_INTEGRATION_TEST_CREDS: ${{ secrets.DESTINATION_S3_INTEGRATION_TEST_CREDS }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test-command.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ jobs:
ZENDESK_SUNSHINE_TEST_CREDS: ${{ secrets.ZENDESK_SUNSHINE_TEST_CREDS }}
ZENDESK_TALK_TEST_CREDS: ${{ secrets.ZENDESK_TALK_TEST_CREDS }}
ZENDESK_SUPPORT_TEST_CREDS: ${{ secrets.ZENDESK_SUPPORT_TEST_CREDS }}
ZENDESK_SUPPORT_OAUTH_TEST_CREDS: ${{ secrets.ZENDESK_SUPPORT_OAUTH_TEST_CREDS }}
ZOOM_INTEGRATION_TEST_CREDS: ${{ secrets.ZOOM_INTEGRATION_TEST_CREDS }}
PLAID_INTEGRATION_TEST_CREDS: ${{ secrets.PLAID_INTEGRATION_TEST_CREDS }}
DESTINATION_S3_INTEGRATION_TEST_CREDS: ${{ secrets.DESTINATION_S3_INTEGRATION_TEST_CREDS }}
Expand Down
2 changes: 1 addition & 1 deletion ...ain/resources/config/STANDARD_SOURCE_DEFINITION/79c1aa37-dae3-42ae-b333-d1c105477715.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"sourceDefinitionId": "79c1aa37-dae3-42ae-b333-d1c105477715",
"name": "Zendesk Support",
"dockerRepository": "airbyte/source-zendesk-support",
"dockerImageTag": "0.1.1",
"dockerImageTag": "0.1.2",
"documentationUrl": "https://docs.airbyte.io/integrations/sources/zendesk-support",
"icon": "zendesk.svg"
}
2 changes: 1 addition & 1 deletion airbyte-config/init/src/main/resources/seed/source_definitions.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@
- sourceDefinitionId: 79c1aa37-dae3-42ae-b333-d1c105477715
name: Zendesk Support
dockerRepository: airbyte/source-zendesk-support
dockerImageTag: 0.1.1
dockerImageTag: 0.1.2
documentationUrl: https://docs.airbyte.io/integrations/sources/zendesk-support
icon: zendesk.svg
sourceType: api
Expand Down
8 changes: 6 additions & 2 deletions airbyte-integrations/connectors/source-zendesk-support/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ FROM base as builder


RUN apk --no-cache upgrade \
&& pip install --upgrade pip
&& pip install --upgrade pip \
&& apk --no-cache add tzdata build-base

WORKDIR /airbyte/integration_code
COPY setup.py ./
Expand All @@ -12,6 +13,9 @@ RUN pip install --prefix=/install .

FROM base
COPY --from=builder /install /usr/local
# add default timezone settings
COPY --from=builder /usr/share/zoneinfo/Etc/UTC /etc/localtime
RUN echo "Etc/UTC" > /etc/timezone

WORKDIR /airbyte/integration_code
COPY main.py ./
Expand All @@ -21,5 +25,5 @@ COPY source_zendesk_support ./source_zendesk_support
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.1
LABEL io.airbyte.version=0.1.2
LABEL io.airbyte.name=airbyte/source-zendesk-support
8 changes: 4 additions & 4 deletions airbyte-integrations/connectors/source-zendesk-support/integration_tests/integration_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import pendulum
import requests_mock
from source_zendesk_support import SourceZendeskSupport
from source_zendesk_support.streams import Macros, TicketAudits, TicketMetrics, Tickets, Users
from source_zendesk_support.streams import LAST_END_TIME_KEY, Macros, TicketAudits, TicketMetrics, Tickets, Users

CONFIG_FILE = "secrets/config.json"

Expand All @@ -32,12 +32,12 @@ def _test_export_stream(self, stream_cls: type):
# save the first 5 records
if len(record_timestamps) > 5:
break
if stream._last_end_time not in record_timestamps.values():
record_timestamps[record["id"]] = stream._last_end_time
if stream.last_end_time not in record_timestamps.values():
record_timestamps[record["id"]] = stream.last_end_time

stream.page_size = 10
for record_id, timestamp in record_timestamps.items():
state = {"_last_end_time": timestamp}
state = {LAST_END_TIME_KEY: timestamp}
for record in stream.read_records(sync_mode=None, stream_state=state):
assert record["id"] != record_id
break
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-zendesk-support/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from setuptools import find_packages, setup

MAIN_REQUIREMENTS = ["airbyte-cdk", "pytz"]
MAIN_REQUIREMENTS = ["airbyte-cdk~=0.1.23", "pytz"]

TEST_REQUIREMENTS = ["pytest~=6.1", "source-acceptance-test", "requests-mock==1.9.3"]

Expand Down
6 changes: 4 additions & 2 deletions airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/source.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
import requests
from airbyte_cdk.sources import AbstractSource
from airbyte_cdk.sources.streams import Stream
from airbyte_cdk.sources.streams.http.auth import TokenAuthenticator
from airbyte_cdk.sources.streams.http.requests_native_auth import TokenAuthenticator

from .streams import (
GroupMemberships,
Expand Down Expand Up @@ -47,7 +47,9 @@ class SourceZendeskSupport(AbstractSource):

@classmethod
def get_authenticator(cls, config: Mapping[str, Any]) -> BasicApiTokenAuthenticator:
if config["auth_method"].get("email") and config["auth_method"].get("api_token"):
if config["auth_method"]["auth_method"] == "access_token":
return TokenAuthenticator(token=config["auth_method"]["access_token"])
elif config["auth_method"]["auth_method"] == "api_token":
return BasicApiTokenAuthenticator(config["auth_method"]["email"], config["auth_method"]["api_token"])
raise SourceZendeskException(f"Not implemented authorization method: {config['auth_method']}")

Expand Down
11 changes: 11 additions & 0 deletions airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/spec.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,17 @@
"default": "api_token",
"description": "Zendesk service provides 2 auth method: API token and oAuth2. Now only the first one is available. Another one will be added in the future",
"oneOf": [
{
"type": "object",
"title": "OAuth2.0 authorization (not implemented)",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we adding this if it's not implemented? A user who sees this will lose confidence in the product

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was planned as temporary gap for the oauth future functionality. I remember to CI tests checked that oneOf items have to include more than 1 subitem. But now I see CI passes this test without problems.
You're right to it will be able to confuse users.
I've added a PR with correction.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah got it. thanks for making the change!

"properties": {
"auth_method": {
"type": "string",
"const": "access_token"
}
},
"additionalProperties": false
},
{
"title": "API Token",
"type": "object",
Expand Down
Loading