Skip to content

Commit

Permalink
šŸ› Source Freshdesk: Add mising / that's causing issues when buildinā€¦
Browse files Browse the repository at this point in the history 
ā€¦g URL (#18397)

* Add mising / that's causing issues when building URL

* attempt to align changes with tests

* Update test_300_page.py

* Update Dockerfile

* Update freshdesk.md

* auto-bump connector version

Co-authored-by: Vincent Koc <koconder@users.noreply.github.com>
Co-authored-by: Vincent Koc <koconder@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
  • Loading branch information
@ireyna-modo @vincentkoc @octavia-squidington-iii
4 people authored Nov 7, 2022
1 parent 91bb1bf commit 5846c65
Show file tree
Hide file tree
Showing 7 changed files with 35 additions and 34 deletions.
2 changes: 1 addition & 1 deletion airbyte-config/init/src/main/resources/seed/source_definitions.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -401,7 +401,7 @@
- name: Freshdesk
sourceDefinitionId: ec4b9503-13cb-48ab-a4ab-6ade4be46567
dockerRepository: airbyte/source-freshdesk
dockerImageTag: 0.3.6
dockerImageTag: 0.3.7
documentationUrl: https://docs.airbyte.com/integrations/sources/freshdesk
icon: freshdesk.svg
sourceType: api
Expand Down
2 changes: 1 addition & 1 deletion airbyte-config/init/src/main/resources/seed/source_specs.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3831,7 +3831,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-freshdesk:0.3.6"
- dockerImage: "airbyte/source-freshdesk:0.3.7"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/freshdesk"
connectionSpecification:
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-freshdesk/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,5 @@ COPY source_freshdesk ./source_freshdesk
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.3.6
LABEL io.airbyte.version=0.3.7
LABEL io.airbyte.name=airbyte/source-freshdesk
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-freshdesk/source_freshdesk/streams.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def __init__(self, authenticator: AuthBase, config: Mapping[str, Any], *args, **

@property
def url_base(self) -> str:
return parse.urljoin(f"https://{self.domain.rstrip('/')}", "/api/v2")
return parse.urljoin(f"https://{self.domain.rstrip('/')}", "/api/v2/")

def backoff_time(self, response: requests.Response) -> Optional[float]:
if response.status_code == requests.codes.too_many_requests:
Expand Down
32 changes: 16 additions & 16 deletions airbyte-integrations/connectors/source-freshdesk/unit_tests/test_300_page.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,70 +11,70 @@
def responses_fixtures():
return [
{
"url": "/api/tickets?per_page=1&updated_since=2002-02-10T22%3A21%3A44Z",
"url": "/api/v2/tickets?per_page=1&updated_since=2002-02-10T22%3A21%3A44Z",
"json": [{"id": 1, "updated_at": "2018-01-02T00:00:00Z"}],
"headers": {
"Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2002-02-10T22%3A21%3A44Z>; rel="next"'
"Link": '<https://test.freshdesk.com/api/v2/tickets?per_page=1&page=2&updated_since=2002-02-10T22%3A21%3A44Z>; rel="next"'
},
},
{
"url": "/api/tickets?per_page=1&page=2&updated_since=2002-02-10T22%3A21%3A44Z",
"url": "/api/v2/tickets?per_page=1&page=2&updated_since=2002-02-10T22%3A21%3A44Z",
"json": [{"id": 2, "updated_at": "2018-02-02T00:00:00Z"}],
"headers": {
"Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2002-02-10T22%3A21%3A44Z>; rel="next"'
"Link": '<https://test.freshdesk.com/api/v2/tickets?per_page=1&page=3&updated_since=2002-02-10T22%3A21%3A44Z>; rel="next"'
},
},
{
"url": "/api/tickets?per_page=1&updated_since=2018-02-02T00%3A00%3A00Z",
"url": "/api/v2/tickets?per_page=1&updated_since=2018-02-02T00%3A00%3A00Z",
"json": [{"id": 2, "updated_at": "2018-02-02T00:00:00Z"}],
"headers": {
"Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2018-02-02T00%3A00%3A00Z>; rel="next"'
"Link": '<https://test.freshdesk.com/api/v2/tickets?per_page=1&page=2&updated_since=2018-02-02T00%3A00%3A00Z>; rel="next"'
},
},
{
"url": "/api/tickets?per_page=1&page=2&updated_since=2018-02-02T00%3A00%3A00Z",
"url": "/api/v2/tickets?per_page=1&page=2&updated_since=2018-02-02T00%3A00%3A00Z",
"json": [{"id": 3, "updated_at": "2018-03-02T00:00:00Z"}],
"headers": {
"Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2018-02-02T00%3A00%3A00Z>; rel="next"'
"Link": '<https://test.freshdesk.com/api/v2/tickets?per_page=1&page=3&updated_since=2018-02-02T00%3A00%3A00Z>; rel="next"'
},
},
{
"url": "/api/tickets?per_page=1&updated_since=2018-03-02T00%3A00%3A00Z",
"url": "/api/v2/tickets?per_page=1&updated_since=2018-03-02T00%3A00%3A00Z",
"json": [{"id": 3, "updated_at": "2018-03-02T00:00:00Z"}],
"headers": {
"Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2018-03-02T00%3A00%3A00Z>; rel="next"'
"Link": '<https://test.freshdesk.com/api/v2/tickets?per_page=1&page=2&updated_since=2018-03-02T00%3A00%3A00Z>; rel="next"'
},
},
{
"url": "/api/tickets?per_page=1&page=2&updated_since=2018-03-02T00%3A00%3A00Z",
"url": "/api/v2/tickets?per_page=1&page=2&updated_since=2018-03-02T00%3A00%3A00Z",
"json": [{"id": 4, "updated_at": "2019-01-03T00:00:00Z"}],
"headers": {
"Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2018-03-02T00%3A00%3A00Z>; rel="next"'
"Link": '<https://test.freshdesk.com/api/v2/tickets?per_page=1&page=3&updated_since=2018-03-02T00%3A00%3A00Z>; rel="next"'
},
},
{
"url": "/api/tickets?per_page=1&updated_since=2019-01-03T00%3A00%3A00Z",
"url": "/api/v2/tickets?per_page=1&updated_since=2019-01-03T00%3A00%3A00Z",
"json": [{"id": 4, "updated_at": "2019-01-03T00:00:00Z"}],
"headers": {
"Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2019-01-03T00%3A00%3A00Z>; rel="next"'
},
},
{
"url": "/api/tickets?per_page=1&page=2&updated_since=2019-01-03T00%3A00%3A00Z",
"url": "/api/v2/tickets?per_page=1&page=2&updated_since=2019-01-03T00%3A00%3A00Z",
"json": [{"id": 5, "updated_at": "2019-02-03T00:00:00Z"}],
"headers": {
"Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2019-01-03T00%3A00%3A00Z>; rel="next"'
},
},
{
"url": "/api/tickets?per_page=1&updated_since=2019-02-03T00%3A00%3A00Z",
"url": "/api/v2/tickets?per_page=1&updated_since=2019-02-03T00%3A00%3A00Z",
"json": [{"id": 5, "updated_at": "2019-02-03T00:00:00Z"}],
"headers": {
"Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2019-02-03T00%3A00%3A00Z>; rel="next"'
},
},
{
"url": "/api/tickets?per_page=1&page=2&updated_since=2019-02-03T00%3A00%3A00Z",
"url": "/api/v2/tickets?per_page=1&page=2&updated_since=2019-02-03T00%3A00%3A00Z",
"json": [{"id": 6, "updated_at": "2019-03-03T00:00:00Z"}],
},
]
Expand Down
28 changes: 14 additions & 14 deletions airbyte-integrations/connectors/source-freshdesk/unit_tests/test_streams.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ def _read_incremental(stream_instance: Stream, stream_state: MutableMapping[str,
],
)
def test_full_refresh(stream, resource, authenticator, config, requests_mock):
requests_mock.register_uri("GET", f"/api/{resource}", json=[{"id": x, "updated_at": "2022-05-05T00:00:00Z"} for x in range(25)])
requests_mock.register_uri("GET", f"/api/v2/{resource}", json=[{"id": x, "updated_at": "2022-05-05T00:00:00Z"} for x in range(25)])

stream = stream(authenticator=authenticator, config=config)
records = _read_full_refresh(stream)
Expand All @@ -93,9 +93,9 @@ def test_full_refresh(stream, resource, authenticator, config, requests_mock):


def test_full_refresh_conversations(authenticator, config, requests_mock):
requests_mock.register_uri("GET", "/api/tickets", json=[{"id": x, "updated_at": "2022-05-05T00:00:00Z"} for x in range(5)])
requests_mock.register_uri("GET", "/api/v2/tickets", json=[{"id": x, "updated_at": "2022-05-05T00:00:00Z"} for x in range(5)])
for i in range(5):
requests_mock.register_uri("GET", f"/api/tickets/{i}/conversations", json=[{"id": x} for x in range(10)])
requests_mock.register_uri("GET", f"/api/v2/tickets/{i}/conversations", json=[{"id": x} for x in range(10)])

stream = Conversations(authenticator=authenticator, config=config)
records = _read_full_refresh(stream)
Expand All @@ -105,7 +105,7 @@ def test_full_refresh_conversations(authenticator, config, requests_mock):

def test_full_refresh_settings(authenticator, config, requests_mock):
json_resp = {"primary_language": "en", "supported_languages": [], "portal_languages": []}
requests_mock.register_uri("GET", "/api/settings/helpdesk", json=json_resp)
requests_mock.register_uri("GET", "/api/v2/settings/helpdesk", json=json_resp)

stream = Settings(authenticator=authenticator, config=config)
records = _read_full_refresh(stream)
Expand All @@ -129,7 +129,7 @@ def test_incremental(stream, resource, authenticator, config, requests_mock):
with patch(f"source_freshdesk.streams.{stream.__name__}.use_cache", new_callable=PropertyMock, return_value=False):
requests_mock.register_uri(
"GET",
f"/api/{resource}",
f"/api/v2/{resource}",
json=[{"id": x, "updated_at": highest_updated_at if x == highest_index else other_updated_at} for x in range(25)],
)

Expand All @@ -151,9 +151,9 @@ def test_incremental(stream, resource, authenticator, config, requests_mock):
],
)
def test_substream_full_refresh(requests_mock, stream_class, parent_path, sub_paths, authenticator, config):
requests_mock.register_uri("GET", "/api/" + parent_path, json=[{"id": x, "updated_at": "2022-05-05T00:00:00Z"} for x in range(5)])
requests_mock.register_uri("GET", "/api/v2/" + parent_path, json=[{"id": x, "updated_at": "2022-05-05T00:00:00Z"} for x in range(5)])
for sub_path in sub_paths:
requests_mock.register_uri("GET", "/api/" + sub_path, json=[{"id": x, "updated_at": "2022-05-05T00:00:00Z"} for x in range(10)])
requests_mock.register_uri("GET", "/api/v2/" + sub_path, json=[{"id": x, "updated_at": "2022-05-05T00:00:00Z"} for x in range(10)])

stream = stream_class(authenticator=authenticator, config=config)
records = _read_full_refresh(stream)
Expand All @@ -179,11 +179,11 @@ def test_substream_full_refresh(requests_mock, stream_class, parent_path, sub_pa
],
)
def test_full_refresh_with_two_sub_levels(requests_mock, stream_class, parent_path, sub_paths, sub_sub_paths, authenticator, config):
requests_mock.register_uri("GET", f"/api/{parent_path}", json=[{"id": x} for x in range(5)])
requests_mock.register_uri("GET", f"/api/v2/{parent_path}", json=[{"id": x} for x in range(5)])
for sub_path in sub_paths:
requests_mock.register_uri("GET", f"/api/{sub_path}", json=[{"id": x} for x in range(5)])
requests_mock.register_uri("GET", f"/api/v2/{sub_path}", json=[{"id": x} for x in range(5)])
for sub_sub_path in sub_sub_paths:
requests_mock.register_uri("GET", f"/api/{sub_sub_path}", json=[{"id": x} for x in range(10)])
requests_mock.register_uri("GET", f"/api/v2/{sub_sub_path}", json=[{"id": x} for x in range(10)])

stream = stream_class(authenticator=authenticator, config=config)
records = _read_full_refresh(stream)
Expand All @@ -192,13 +192,13 @@ def test_full_refresh_with_two_sub_levels(requests_mock, stream_class, parent_pa


def test_full_refresh_discussion_comments(requests_mock, authenticator, config):
requests_mock.register_uri("GET", "/api/discussions/categories", json=[{"id": x} for x in range(2)])
requests_mock.register_uri("GET", "/api/v2/discussions/categories", json=[{"id": x} for x in range(2)])
for i in range(2):
requests_mock.register_uri("GET", f"/api/discussions/categories/{i}/forums", json=[{"id": x} for x in range(3)])
requests_mock.register_uri("GET", f"/api/v2/discussions/categories/{i}/forums", json=[{"id": x} for x in range(3)])
for j in range(3):
requests_mock.register_uri("GET", f"/api/discussions/forums/{j}/topics", json=[{"id": x} for x in range(4)])
requests_mock.register_uri("GET", f"/api/v2/discussions/forums/{j}/topics", json=[{"id": x} for x in range(4)])
for k in range(4):
requests_mock.register_uri("GET", f"/api/discussions/topics/{k}/comments", json=[{"id": x} for x in range(5)])
requests_mock.register_uri("GET", f"/api/v2/discussions/topics/{k}/comments", json=[{"id": x} for x in range(5)])

stream = DiscussionComments(authenticator=authenticator, config=config)
records = _read_full_refresh(stream)
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/freshdesk.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ The Freshdesk connector should not run into Freshdesk API limitations under norm

| Version | Date | Pull Request | Subject |
| :------ | :--------- | :------------------------------------------------------- | :------------------------------------------------------------------------------------ |
| 0.3.7 | 2022-11-03 | [18397](https://github.com/airbytehq/airbyte/pull/18397) | Fix base url for v2 API. |
| 0.3.6 | 2022-09-29 | [17410](https://github.com/airbytehq/airbyte/pull/17410) | Migrate to per-stream states. |
| 0.3.5 | 2022-09-27 | [17249](https://github.com/airbytehq/airbyte/pull/17249) | Added nullable to all stream schemas, added transformation into declared schema types |
| 0.3.4 | 2022-09-27 | [17243](https://github.com/airbytehq/airbyte/pull/17243) | Fixed the issue, when selected stream is not available due to Subscription Plan |
Expand Down

0 comments on commit 5846c65

Please sign in to comment.