Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Sendgrid: Update manifest for adapting changes with AsyncRetriever #55185

Merged
merged 12 commits into from
Mar 11, 2025
Merged

Source Sendgrid: Update manifest for adapting changes with AsyncRetriever #55185

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
4924bcb
Update manifest according to the description
btkcodedev Mar 4, 2025
b799c36
chore: bump-version
octavia-squidington-iii Mar 4, 2025
d73ef27
update version
btkcodedev Mar 4, 2025
ba87070
update cdk version and manifest
btkcodedev Mar 5, 2025
f1eef63
update download_decoder
btkcodedev Mar 10, 2025
9bf9d1a
update CDK
btkcodedev Mar 10, 2025
e8bcc17
update url_base
btkcodedev Mar 10, 2025
0f07a38
update alternate emails as string
btkcodedev Mar 10, 2025
66834d6
update cdk
btkcodedev Mar 10, 2025
1f9556b
Merge branch 'master' into btkcodedev/sendgridUpdate
btkcodedev Mar 10, 2025
bc859f1
add bounces for no records
btkcodedev Mar 10, 2025
28bb72d
remove expected records
btkcodedev Mar 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions airbyte-integrations/connectors/source-sendgrid/acceptance-test-config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ acceptance_tests:
bypass_reason: "can not populate"
- name: blocks
bypass_reason: "can not populate"
- name: bounces
bypass_reason: "can not populate"
fail_on_extra_columns: false
incremental:
tests:
Expand Down
6 changes: 0 additions & 6 deletions airbyte-integrations/connectors/source-sendgrid/integration_tests/expected_records.jsonl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,12 +184,6 @@
{"stream": "suppression_group_members", "data": {"email": "test-forsuppressiongroup number8@example.com", "group_id": 14772, "group_name": "Test Suggestions Group 12", "created_at": 1612363238}, "emitted_at": 1631093393000}
{"stream": "suppression_group_members", "data": {"email": "test-forsuppressiongroup number9@example.com", "group_id": 14772, "group_name": "Test Suggestions Group 12", "created_at": 1612363238}, "emitted_at": 1631093393000}
{"stream": "suppression_group_members", "data": {"email": "avida.d3@gmail.com", "group_id": 14780, "group_name": "Test Suggestions Group 20", "created_at": 1631093329}, "emitted_at": 1631093393000}
{"stream": "bounces", "data": {"status": "", "reason": "Invalid Domain", "email": "vadym.hevlich@zazmic_com", "created": 1621439283}, "emitted_at": 1708535996116}
{"stream": "bounces", "data": {"status": "", "reason": "Invalid Domain", "email": "vadym.hevlich@zazmicinvalid", "created": 1621439221}, "emitted_at": 1708535996116}
{"stream": "bounces", "data": {"status": "", "reason": "Invalid Domain", "email": "vadym.hevlich@zazmicio", "created": 1621439211}, "emitted_at": 1708535996116}
{"stream": "bounces", "data": {"status": "", "reason": "Invalid Domain", "email": "vadym.hevlich@zazmiccom2", "created": 1621437507}, "emitted_at": 1708535996117}
{"stream": "bounces", "data": {"status": "", "reason": "Invalid Domain", "email": "vadym.hevlich@zazmiccom1", "created": 1621437504}, "emitted_at": 1708535996117}
{"stream": "bounces", "data": {"status": "", "reason": "Invalid Domain", "email": "vadym.hevlich@zazmiccom", "created": 1621426437}, "emitted_at": 1708535996117}
{"stream": "campaigns", "data": {"created_at": "2021-09-08T09:07:48Z", "id": "3c5a9fa6-1084-11ec-ac32-4228d699bad5", "name": "Untitled Single Send", "status": "triggered", "updated_at": "2021-09-08T09:11:08Z", "is_abtest": false, "channels": ["email"]}, "emitted_at": 1678791750589}
{"stream": "campaigns", "data": {"created_at": "2021-09-08T09:04:36Z", "id": "c9f286fb-1083-11ec-ae03-ca0fc7f28419", "name": "Copy of Untitled Single Send", "status": "triggered", "updated_at": "2021-09-08T09:09:08Z", "is_abtest": false, "channels": ["email"]}, "emitted_at": 1678791750589}
{"stream": "campaigns", "data": {"created_at": "2021-09-08T08:53:59Z", "id": "4e5be6a3-1082-11ec-8512-9afd40c324e6", "name": "Untitled Single Send", "status": "triggered", "updated_at": "2021-09-08T08:57:08Z", "is_abtest": false, "channels": ["email"]}, "emitted_at": 1678791750590}
Expand Down
16 changes: 9 additions & 7 deletions airbyte-integrations/connectors/source-sendgrid/manifest.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
version: 5.13.0
version: 6.38.3

type: DeclarativeSource

Expand Down Expand Up @@ -604,7 +604,7 @@ definitions:
- timeout
completed:
- ready
urls_extractor:
download_target_extractor:
type: DpathExtractor
field_path:
- urls
Expand All @@ -620,7 +620,7 @@ definitions:
polling_requester:
type: HttpRequester
path: >-
/v3/marketing/contacts/exports/{{stream_slice['create_job_response'].json()['id']
/v3/marketing/contacts/exports/{{creation_response['id']
}}
url_base: https://api.sendgrid.com
http_method: GET
Expand All @@ -637,11 +637,12 @@ definitions:
api_token: "{{ config['api_key'] }}"
download_requester:
type: HttpRequester
path: "{{stream_slice.extra_fields['url']}}"
url_base: ""
url_base: "{{download_target}}"
http_method: GET
download_extractor:
type: ResponseToFileExtractor
download_decoder:
type: GzipDecoder
decoder:
type: CsvDecoder
schema_loader:
type: InlineSchemaLoader
schema:
Expand Down Expand Up @@ -1066,6 +1067,7 @@ schemas:
type:
- 'null'
- array
- string
city:
type:
- string
Expand Down
4 changes: 2 additions & 2 deletions airbyte-integrations/connectors/source-sendgrid/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ data:
hosts:
- api.sendgrid.com
connectorBuildOptions:
baseImage: docker.io/airbyte/source-declarative-manifest:6.36.1@sha256:52ad641b57995fbd43975d64f9a68fac15fcb80687df0f7acc27c4942a3b6bbb
baseImage: docker.io/airbyte/source-declarative-manifest:6.38.3@sha256:fcba02266f262aabc2f37e4f14574aa1c8c5cffd018504bab28803e405c93afe
connectorSubtype: api
connectorType: source
definitionId: fbb5fbe2-16ad-4cf4-af7d-ff9d9c316c87
dockerImageTag: 1.2.9
dockerImageTag: 1.3.0
releases:
breakingChanges:
1.0.0:
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/sendgrid.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ The connector is restricted by normal Sendgrid [requests limitation](https://doc

| Version | Date | Pull Request | Subject |
|:--------|:-----------| :------------------------------------------------------- |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1.3.0 | 2025-03-04 | [55185](https://github.com/airbytehq/airbyte/pull/55185) | Update manifest for adapting changes with AsyncRetriever |
| 1.2.9 | 2025-02-23 | [54625](https://github.com/airbytehq/airbyte/pull/54625) | Update dependencies |
| 1.2.8 | 2025-02-15 | [54013](https://github.com/airbytehq/airbyte/pull/54013) | Update dependencies |
| 1.2.7 | 2025-02-08 | [53508](https://github.com/airbytehq/airbyte/pull/53508) | Update dependencies |
Expand Down
Loading