Source Prestashop: Migrate connector from Alpha (Python) to Beta (YAML) #18927

grubberr · 2022-11-03T18:16:16Z

Signed-off-by: Sergey Chvalyuk grubberr@gmail.com

What

Beta certification checklist https://docs.google.com/spreadsheets/d/1FegkY5Uz2GIJC7TJ3zWGDD2y0zZsTJHv9GJP4XSVQvY/edit#gid=1288588270

How

Describe the solution

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr · 2022-11-07T10:58:22Z

/test connector=connectors/source-prestashop

🕑 connectors/source-prestashop https://github.com/airbytehq/airbyte/actions/runs/3409722244
✅ connectors/source-prestashop https://github.com/airbytehq/airbyte/actions/runs/3409722244
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       133      3    98%   87, 93, 230
	 source_acceptance_test/conftest.py                     196     97    51%   35, 41-43, 48, 54, 60, 66, 72-74, 80-95, 100, 105-107, 113-115, 121-122, 127-128, 133, 139, 148-157, 163-168, 232, 238, 244-250, 258-263, 271-284, 289-295, 302-313, 320-336
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              329    106    68%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 377-379, 382, 447-455, 484-485, 491, 494, 530-540, 553-578
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     10    87%   15-16, 24-30, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/config_migration.py        23     23     0%   5-37
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1479    376    75%

Build Passed

Test summary info:

All Passed

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr · 2022-11-07T11:24:32Z

/test connector=connectors/source-prestashop

🕑 connectors/source-prestashop https://github.com/airbytehq/airbyte/actions/runs/3409910859
✅ connectors/source-prestashop https://github.com/airbytehq/airbyte/actions/runs/3409910859
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       133      3    98%   87, 93, 230
	 source_acceptance_test/conftest.py                     196     97    51%   35, 41-43, 48, 54, 60, 66, 72-74, 80-95, 100, 105-107, 113-115, 121-122, 127-128, 133, 139, 148-157, 163-168, 232, 238, 244-250, 258-263, 271-284, 289-295, 302-313, 320-336
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              329    106    68%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 377-379, 382, 447-455, 484-485, 491, 494, 530-540, 553-578
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     10    87%   15-16, 24-30, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/config_migration.py        23     23     0%   5-37
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1479    376    75%

Build Passed

Test summary info:

All Passed

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr · 2022-11-14T20:22:28Z

/test connector=connectors/source-prestashop

🕑 connectors/source-prestashop https://github.com/airbytehq/airbyte/actions/runs/3464902349
❌ connectors/source-prestashop https://github.com/airbytehq/airbyte/actions/runs/3464902349
🐛 https://gradle.com/s/p3p6624ncxtw4

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - Failed: Stream suppl...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:94: Backward compatibility tests are disabled for version 0.2.0.
============= 1 failed, 29 passed, 1 skipped in 214.51s (0:03:34) ==============

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr · 2022-11-14T21:03:33Z

/test connector=connectors/source-prestashop

🕑 connectors/source-prestashop https://github.com/airbytehq/airbyte/actions/runs/3465156659

This reverts commit 0997f4c.

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr · 2022-11-14T22:46:28Z

/test connector=connectors/source-prestashop

🕑 connectors/source-prestashop https://github.com/airbytehq/airbyte/actions/runs/3465794636
❌ connectors/source-prestashop https://github.com/airbytehq/airbyte/actions/runs/3465794636
🐛 https://gradle.com/s/bpolf3r2ue4my

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - Failed: Stream suppl...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:94: Backward compatibility tests are disabled for version 0.2.0.
============= 1 failed, 29 passed, 1 skipped in 211.45s (0:03:31) ==============

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr · 2022-11-14T23:17:33Z

/test connector=connectors/source-prestashop

🕑 connectors/source-prestashop https://github.com/airbytehq/airbyte/actions/runs/3465979880
✅ connectors/source-prestashop https://github.com/airbytehq/airbyte/actions/runs/3465979880
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       139      5    96%   87, 93, 235, 239-240
	 source_acceptance_test/conftest.py                     196     92    53%   35, 41-43, 48, 54, 60, 66, 72-74, 93, 98-100, 106-108, 114-115, 120-121, 126, 132, 141-150, 156-161, 176, 200, 231, 237, 243-248, 256-261, 269-282, 287-293, 300-311, 318-334
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              394    111    72%   53, 58, 87-95, 100-107, 111-112, 116-117, 293, 331-348, 357-365, 369-374, 380, 413-418, 456-463, 506-508, 511, 576-584, 596-599, 604, 660-661, 667, 670, 706-716, 729-754
	 source_acceptance_test/tests/test_incremental.py       158     14    91%   52-59, 64-77, 240
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1557    349    78%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:94: Backward compatibility tests are disabled for version 0.2.0.
================== 30 passed, 1 skipped in 212.20s (0:03:32) ===================

girarda · 2022-11-15T02:07:16Z

airbyte-integrations/connectors/source-prestashop/source_prestashop/prestashop.yaml

+            $ref: "*ref(definitions.requester.request_options_provider.request_parameters)"
+            date: "1"
+            sort: "[{{ options['cursor_field'] }}_ASC,{{ options['primary_key'] }}_ASC]"
+            "filter[{{ options['cursor_field'] }}]": "[{{ stream_slice['start_time'] }},{{ stream_slice['end_time'] }}]"


quite a scary line but I'm glad we're able to support these advanced features

grubberr · 2022-11-15T06:11:06Z

/publish connector=connectors/source-prestashop

🕑 Publishing the following connectors:
connectors/source-prestashop
https://github.com/airbytehq/airbyte/actions/runs/3467936566

Connector	Did it publish?	Were definitions generated?
connectors/source-prestashop	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

…L) (#18927) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

bump 0.2.1

f3ae381

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr self-assigned this Nov 3, 2022

github-actions bot added the area/connectors Connector related issues label Nov 3, 2022

octavia-squidington-iv added the connectors/source/prestashop label Nov 3, 2022

grubberr added 15 commits November 3, 2022 21:14

prestashop.yaml added

337bb50

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

all streams added

fe34587

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

bump 0.3.0

b12bf9a

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Merge branch 'master' into grubberr/18710-source-prestashop

da45248

data_key added

8c22b76

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Merge branch 'master' into grubberr/18710-source-prestashop

5e4f99c

Merge branch 'master' into grubberr/18710-source-prestashop

dad71fa

moved primary_key to base_stream

7aa5cfc

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Merge branch 'master' into grubberr/18710-source-prestashop

cbe04b4

LimitOffsetPaginator added

2bed075

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Merge branch 'master' into grubberr/18710-source-prestashop

780e3d4

NoSlicer added

7c965e1

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

stream_cursor_field added

f16a8bd

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

_filter_params added

5cc8e8c

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

abnormal_state.json added

8f83178

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

releaseStage alpha -> beta

06cd8d0

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr temporarily deployed to more-secrets November 7, 2022 11:05 Inactive

migrated acceptance-test-config.yml

d8d08dd

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr temporarily deployed to more-secrets November 7, 2022 11:25 Inactive

grubberr linked an issue Nov 7, 2022 that may be closed by this pull request

Source Prestashop: Migrate connector from Alpha (Python) to Beta (YAML) #18710

Closed

grubberr added 2 commits November 8, 2022 10:12

Merge branch 'master' into grubberr/18710-source-prestashop

d0a12f3

presta-shop.md updated

fa6eb15

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

github-actions bot added the area/documentation Improvements or additions to documentation label Nov 8, 2022

grubberr temporarily deployed to more-secrets November 8, 2022 08:16 Inactive

integration_tests/catalog.json removed

f46b862

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr temporarily deployed to more-secrets November 14, 2022 19:00 Inactive

Merge branch 'master' into grubberr/18710-source-prestashop

e836e61

grubberr temporarily deployed to more-secrets November 14, 2022 20:12 Inactive

disable_for_version: 0.1.0 -> 0.2.0

73f10bd

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr temporarily deployed to more-secrets November 14, 2022 20:25 Inactive

revert me

0997f4c

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr temporarily deployed to more-secrets November 14, 2022 21:05 Inactive

Revert "revert me"

20c5045

This reverts commit 0997f4c.

grubberr temporarily deployed to more-secrets November 14, 2022 22:45 Inactive

fix bad symbols

e4f4a0f

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr temporarily deployed to more-secrets November 14, 2022 22:47 Inactive

expected_records.txt fixed

6f4431e

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr temporarily deployed to more-secrets November 14, 2022 23:19 Inactive

grubberr requested a review from girarda November 14, 2022 23:35

girarda approved these changes Nov 15, 2022

View reviewed changes

Merge branch 'master' into grubberr/18710-source-prestashop

cf8ae2b

grubberr temporarily deployed to more-secrets November 15, 2022 06:11 Inactive

auto-bump connector version

0392abc

grubberr merged commit a3e83c3 into master Nov 15, 2022

grubberr deleted the grubberr/18710-source-prestashop branch November 15, 2022 06:39

octavia-squidington-iii temporarily deployed to more-secrets November 15, 2022 06:40 Inactive

github-actions bot mentioned this pull request Nov 15, 2022

Bump helm chart version reference to 0.40.43 #19453

Closed

octavia-squidington-iii mentioned this pull request Nov 18, 2022

Bump Airbyte version from 0.40.18 to 0.40.19 #19579

Merged

akashkulk pushed a commit that referenced this pull request Dec 2, 2022

Source Prestashop: Migrate connector from Alpha (Python) to Beta (YAM…

a03789e

…L) (#18927) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Source Prestashop: Migrate connector from Alpha (Python) to Beta (YAML) #18927

Source Prestashop: Migrate connector from Alpha (Python) to Beta (YAML) #18927

grubberr commented Nov 3, 2022 •

edited

Loading

grubberr commented Nov 7, 2022 •

edited by github-actions bot

Loading

grubberr commented Nov 7, 2022 •

edited by github-actions bot

Loading

grubberr commented Nov 14, 2022 •

edited by github-actions bot

Loading

grubberr commented Nov 14, 2022 •

edited by github-actions bot

Loading

grubberr commented Nov 14, 2022 •

edited by github-actions bot

Loading

grubberr commented Nov 14, 2022 •

edited by github-actions bot

Loading

girarda Nov 15, 2022

grubberr commented Nov 15, 2022 •

edited by github-actions bot

Loading

Source Prestashop: Migrate connector from Alpha (Python) to Beta (YAML) #18927

Source Prestashop: Migrate connector from Alpha (Python) to Beta (YAML) #18927

Conversation

grubberr commented Nov 3, 2022 • edited Loading

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

Tests

grubberr commented Nov 7, 2022 • edited by github-actions bot Loading

Build Passed

grubberr commented Nov 7, 2022 • edited by github-actions bot Loading

Build Passed

grubberr commented Nov 14, 2022 • edited by github-actions bot Loading

Build Failed

grubberr commented Nov 14, 2022 • edited by github-actions bot Loading

grubberr commented Nov 14, 2022 • edited by github-actions bot Loading

Build Failed

grubberr commented Nov 14, 2022 • edited by github-actions bot Loading

Build Passed

girarda Nov 15, 2022

Choose a reason for hiding this comment

grubberr commented Nov 15, 2022 • edited by github-actions bot Loading

grubberr commented Nov 3, 2022 •

edited

Loading

grubberr commented Nov 7, 2022 •

edited by github-actions bot

Loading

grubberr commented Nov 7, 2022 •

edited by github-actions bot

Loading

grubberr commented Nov 14, 2022 •

edited by github-actions bot

Loading

grubberr commented Nov 14, 2022 •

edited by github-actions bot

Loading

grubberr commented Nov 14, 2022 •

edited by github-actions bot

Loading

grubberr commented Nov 14, 2022 •

edited by github-actions bot

Loading

grubberr commented Nov 15, 2022 •

edited by github-actions bot

Loading