🎉 New Source Braze: new source implemented in low-code #20520

roman-yermilov-gl · 2022-12-15T15:54:32Z

What

New Braze connector

How

Low-code connector

Recommended reading order

braze.yaml
y.python

roman-yermilov-gl · 2022-12-16T15:24:44Z

/test connector=connectors/source-braze

🕑 connectors/source-braze https://github.com/airbytehq/airbyte/actions/runs/3714358672
✅ connectors/source-braze https://github.com/airbytehq/airbyte/actions/runs/3714358672
Python tests coverage:

Name                                     Stmts   Miss  Cover
------------------------------------------------------------
source_braze/transformations.py             14      0   100%
source_braze/__init__.py                     4      0   100%
source_braze/datetime_stream_slicer.py      32      5    84%
source_braze/source.py                       4      1    75%
------------------------------------------------------------
TOTAL                                       54      6    89%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       140      5    96%   87, 93, 238, 242-243
	 source_acceptance_test/conftest.py                     208     92    56%   36, 42-44, 49, 54, 77, 83, 89-91, 110, 115-117, 123-125, 131-132, 137-138, 143, 149, 158-167, 173-178, 193, 217, 248, 254, 262-267, 275-280, 288-301, 306-312, 319-330, 337-353
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              398    111    72%   53, 58, 87-95, 100-107, 111-112, 116-117, 299, 337-354, 363-371, 375-380, 386, 419-424, 462-469, 512-514, 517, 582-590, 602-605, 610, 666-667, 673, 676, 712-722, 735-760
	 source_acceptance_test/tests/test_incremental.py       158     14    91%   52-59, 64-77, 240
	 source_acceptance_test/utils/asserts.py                 39      2    95%   62-63
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       133     33    75%   24-27, 46-47, 50-54, 57-58, 73-75, 78-80, 83-85, 88-90, 93-95, 124-125, 159-161, 208
	 source_acceptance_test/utils/json_schema_helper.py     107     13    88%   30-31, 38, 41, 65-68, 96, 120, 192-194
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1599    332    79%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:88: The previous connector image could not be retrieved.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:364: The previous connector image could not be retrieved.
================== 29 passed, 2 skipped in 212.24s (0:03:32) ===================

sherifnada · 2022-12-16T22:32:42Z

airbyte-integrations/connectors/source-braze/source_braze/datetime_stream_slicer.py

+
+
+@dataclass
+class DatetimeStreamSlicerComponent(DatetimeStreamSlicer):


can you create an issue requesting that this is added to the datetimestreamslicer like described here and clarifying what the new component should do, and add the issue to the docstring?

I don't understand why this is different than the datetimestreamslicer. Can you clarify in the docstring? doesn't datetime stream slicer already have step size?

Step exists in basic DatetimeStreamSlicer but I need it to be an option which will be passed as a request parameter and to be calculated based on difference between start/end dates. I need the step amount to be more than actual size of the difference by 1, but not for fist slice. I also do not need last slice if start/end dates are equal. And it is important thing that step amount is dynamic because last slice must not be equal to the step amount if the difference between start/end dates lesser than in step. All of this is not a standard thing and I am not sure we need it in DatetimeStreamSlicer.
Why I need it:
Braze API provides us with ending_at and length query params, so we can to specify latest point and then move back by length until reach start_date(initially specified by user), which is ignored by API (even if we specify starting_at we still getting records older than starting_at, if step is bigger than difference of ending_at - starting_at. And starting_at is available for only few endponts). Date specified in ending_at is an exclusive property which means we are getting length records strictly less than ending_at, but DatetimeStreamSlicer adds 1 day to previous end date to calculate next start date for next slice, that is why we loose 1 record between each slice. Having step query parameter amount one higher than actual difference lets us avoid the problem with gaps in slice windows

sherifnada

LGTM for this PR but please create two github issues to track the custom components in this PR

sherifnada · 2022-12-16T23:40:15Z

airbyte-integrations/connectors/source-braze/source_braze/transformations.py

+
+
+@dataclass
+class TransformToRecordComponent(AddFields):


please create a Github issue for this custom component as well so we can keep track of which components we need to add to the CDK

roman-yermilov-gl · 2022-12-17T09:29:12Z

/publish connector=connectors/source-braze

🕑 Publishing the following connectors:
connectors/source-braze
https://github.com/airbytehq/airbyte/actions/runs/3719436851

Connector	Did it publish?	Were definitions generated?
connectors/source-braze	✅	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

roman-yermilov-gl · 2022-12-17T13:18:01Z

/publish connector=connectors/source-braze

🕑 Publishing the following connectors:
connectors/source-braze
https://github.com/airbytehq/airbyte/actions/runs/3720220038

Connector	Did it publish?	Were definitions generated?
connectors/source-braze	✅	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

roman-yermilov-gl · 2022-12-17T13:51:40Z

/publish connector=connectors/source-braze

🕑 Publishing the following connectors:
connectors/source-braze
https://github.com/airbytehq/airbyte/actions/runs/3720329423

Connector	Did it publish?	Were definitions generated?
connectors/source-braze	✅	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

roman-yermilov-gl · 2022-12-17T14:20:04Z

/publish connector=connectors/source-braze

🕑 Publishing the following connectors:
connectors/source-braze
https://github.com/airbytehq/airbyte/actions/runs/3720432920

Connector	Did it publish?	Were definitions generated?
connectors/source-braze	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

ni-aschuler · 2022-12-28T13:43:05Z

Thank you for this connector, very much appreciated!
Any plans on supporting additional "details" endpoints? especially:
https://www.braze.com/docs/api/endpoints/export/canvas/get_canvas_details/
https://www.braze.com/docs/api/endpoints/export/campaigns/get_campaign_details/
https://www.braze.com/docs/api/endpoints/export/segments/get_segment_details/
Thank you, Cheers

🎉 New Source Braze: new source implemented in low-code

e4cb12d

roman-yermilov-gl requested review from grubberr, midavadim, davydov-d, arsenlosenko, bazarnov, artem1205, lazebnyi and darynaishchenko December 15, 2022 15:54

roman-yermilov-gl self-assigned this Dec 15, 2022

octavia-squidington-iv added connectors/source/braze area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Dec 15, 2022

roman-yermilov-gl added 2 commits December 15, 2022 19:57

New Source Braze: changelog PR id update

04103e9

New Source Braze: unittests for custom components

409dd74

roman-yermilov-gl force-pushed the ryermilov/new-source-braze branch 2 times, most recently from fa4a8de to 409dd74 Compare December 15, 2022 17:46

roman-yermilov-gl added 2 commits December 16, 2022 00:45

New Source Braze: remove unused definition in low-code config

8515d85

New Source Braze: fix expected records

eae1369

roman-yermilov-gl requested review from brianjlai and girarda and removed request for grubberr, midavadim, arsenlosenko, bazarnov and darynaishchenko December 16, 2022 15:57

New Source Braze: remove cards_analytics expected records

f6f1cb2

lazebnyi approved these changes Dec 16, 2022

View reviewed changes

sherifnada suggested changes Dec 16, 2022

View reviewed changes

sherifnada approved these changes Dec 16, 2022

View reviewed changes

bazarnov approved these changes Dec 17, 2022

View reviewed changes

New Source Braze: bump version, add source definition

79a76f9

roman-yermilov-gl temporarily deployed to more-secrets December 17, 2022 13:18 — with GitHub Actions Inactive

roman-yermilov-gl temporarily deployed to more-secrets December 17, 2022 13:19 — with GitHub Actions Inactive

New Source Braze: bump version

750d3a0

roman-yermilov-gl temporarily deployed to more-secrets December 17, 2022 13:45 — with GitHub Actions Inactive

roman-yermilov-gl temporarily deployed to more-secrets December 17, 2022 13:46 — with GitHub Actions Inactive

New Source Braze: bump version, fix connector name in definitions

d863008

roman-yermilov-gl temporarily deployed to more-secrets December 17, 2022 14:19 — with GitHub Actions Inactive

roman-yermilov-gl temporarily deployed to more-secrets December 17, 2022 14:20 — with GitHub Actions Inactive

auto-bump connector version

a01ebc7

octavia-squidington-iii temporarily deployed to more-secrets December 17, 2022 14:44 — with GitHub Actions Inactive

octavia-squidington-iii temporarily deployed to more-secrets December 17, 2022 14:45 — with GitHub Actions Inactive

roman-yermilov-gl merged commit 06e1717 into master Dec 17, 2022

roman-yermilov-gl deleted the ryermilov/new-source-braze branch December 17, 2022 15:16

octavia-squidington-iii mentioned this pull request Dec 18, 2022

Bump Airbyte version from 0.40.25 to 0.40.26 #20630

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 New Source Braze: new source implemented in low-code #20520

🎉 New Source Braze: new source implemented in low-code #20520

roman-yermilov-gl commented Dec 15, 2022

roman-yermilov-gl commented Dec 16, 2022 •

edited by github-actions bot

Loading

sherifnada Dec 16, 2022

sherifnada Dec 16, 2022

roman-yermilov-gl Dec 16, 2022 •

edited

Loading

sherifnada left a comment

sherifnada Dec 16, 2022

roman-yermilov-gl commented Dec 17, 2022 •

edited by github-actions bot

Loading

roman-yermilov-gl commented Dec 17, 2022 •

edited by github-actions bot

Loading

roman-yermilov-gl commented Dec 17, 2022 •

edited by github-actions bot

Loading

roman-yermilov-gl commented Dec 17, 2022 •

edited by github-actions bot

Loading

ni-aschuler commented Dec 28, 2022



		@dataclass
		class DatetimeStreamSlicerComponent(DatetimeStreamSlicer):

🎉 New Source Braze: new source implemented in low-code #20520

🎉 New Source Braze: new source implemented in low-code #20520

Conversation

roman-yermilov-gl commented Dec 15, 2022

What

How

Recommended reading order

roman-yermilov-gl commented Dec 16, 2022 • edited by github-actions bot Loading

Build Passed

sherifnada Dec 16, 2022

Choose a reason for hiding this comment

sherifnada Dec 16, 2022

Choose a reason for hiding this comment

roman-yermilov-gl Dec 16, 2022 • edited Loading

Choose a reason for hiding this comment

sherifnada left a comment

Choose a reason for hiding this comment

sherifnada Dec 16, 2022

Choose a reason for hiding this comment

roman-yermilov-gl commented Dec 17, 2022 • edited by github-actions bot Loading

roman-yermilov-gl commented Dec 17, 2022 • edited by github-actions bot Loading

roman-yermilov-gl commented Dec 17, 2022 • edited by github-actions bot Loading

roman-yermilov-gl commented Dec 17, 2022 • edited by github-actions bot Loading

ni-aschuler commented Dec 28, 2022

roman-yermilov-gl commented Dec 16, 2022 •

edited by github-actions bot

Loading

roman-yermilov-gl Dec 16, 2022 •

edited

Loading

roman-yermilov-gl commented Dec 17, 2022 •

edited by github-actions bot

Loading

roman-yermilov-gl commented Dec 17, 2022 •

edited by github-actions bot

Loading

roman-yermilov-gl commented Dec 17, 2022 •

edited by github-actions bot

Loading

roman-yermilov-gl commented Dec 17, 2022 •

edited by github-actions bot

Loading