🎉 New Source: Pocket [low-code CDK] #18655
Conversation
github-actions
bot
added
area/connectors
|
{
"consumer_key": "REPLACE-ME",
"access_token": "REPLACE-ME"
}Instructions on how to get them in |
|
/test connector=connectors/source-pocket
Build FailedTest summary info: |
It seems like it's missing the |
|
@Xabilahu this is normal as there is no test config file for this connection yet on airbyte side. Please hang tight as i review. |
There was a problem hiding this comment.
Hello @Xabilahu, Marcos from Airbyte here 👋 . We received more than 25 new contributions along the weekend. One is yours 🎉 thank so much for! Our team is limited and maybe the review process can take longer than expected. As described in the Airbyte's Hacktoberfest your contribution was submitted before November 2nd and it is eligible to win the prize. The review process will validate other requirements. I ask to you patience until someone from the team review it.
Because I reviewed some contributions for Hacktoberfest so far I saw some common patterns you can check in advance:
- Make sure you have added connector documentation to
/docs/integrations/ - Remove the file
catalogfrom/integration_tests - Edit the
sample_config.jsoninside/integration_tests - For the
configured_catalogyou can use onlyjson_schema: {} - Add title to all properties in the
spec.yaml - Make sure the
documentationUrlin thespec.yamlredirect to Airbyte's future connector page, eg: connector Airtable thedocumentationUrl: https://docs.airbyte.com/integrations/sources/airtable - Review now new line at EOF (end-of-file) for all files.
If possible send to me a DM in Slack with the tests credentials, this process will make easier to us run integration tests and publish your connector. If you only has production keys, make sure to create a bootstrap.md explaining how to get the keys.
|
Hi @marcosmarxm, thanks for the update! Below the status of the checklist:
Also created the |
There was a problem hiding this comment.
@Xabilahu looks good, i made a few small changes. I have my API keys and will test locally shortly.
|
@Xabilahu @marcosmarxm all local tests passing, will get this finalised soon |
|
FYI @Xabilahu Unit Test Acceptance Test: |
|
Looking at the code from @dataclass
class DeclarativeStream(Stream, JsonSchemaMixin):
...
def __post_init__(self, options: Mapping[str, Any]):
self.stream_cursor_field = self.stream_cursor_field or []
self.transformations = self.transformations or []
self._schema_loader = self.schema_loader if self.schema_loader else EmptySchemaLoader(config=self.config, options=options)I tested to create an environment from scracth with the following commands,: python -m venv .venv
source .venv/bin/activate
pip install airbyte-cdk~=0.2 --no-cacheAnd, in fact, the code is the same as above. This seems like a breaking change in |
|
@Xabilahu let me know if you need any assistance and able to work-around this or this is a bug elsewhere. |
@koconder I added |
|
@Xabilahu still an issue, see below. Are you able to run gradle command i gave you? One of the issues, not sure where/what/why |
|
@koconder, as I told you in a previous comment, I'm not able to replicate those failures that you show. For me, the build passes successfully: Can you please show your |
|
@Xabilahu can you contact me on Airbyte slack, I'll share my test credentials there with you. |
|
The issue is that the config used for acceptance tests tries to sync a huge amount of data, which doesn't fit into the 300s limit for tests. |
|
As mentioned, i will see if we can adjust the tests before I finalize this PR to save the issue from re-occurring with others that will test again in future. |
|
@koconder any updates? Do you need help? |
|
@Xabilahu we will add the config to the Github Actions and attempt to finalise the acceptance tests. Once done we can merge and publish the connector :) |
|
/test connector=connectors/source-pocket
Build PassedTest summary info: |
vincentkoc
dismissed
marcosmarxm’s stale review
maintainer will review those changes
|
/publish connector=connectors/source-pocket
if you have connectors that successfully published but failed definition generation, follow step 4 here |
* Pocket Source Connector initial implementation * Update changelog with PR id * Style fixes & bootstrap.md * Delete abnormal_state.json * Update setup.py * Update invalid_config.json * Delete sample_state.json * Update retrieve.json * Update acceptance.py * Rename Pocket Extractor test * Add `schema_loader` as a workaround to prevent `EmptySchemaLoader` * auto-bump connector version Co-authored-by: Vincent Koc <koconder@users.noreply.github.com> Co-authored-by: Vincent Koc <koconder@gmail.com> Co-authored-by: Vincent Koc <vincentkoc@ieee.org> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
What
New Source: Pocket. https://getpocket.com
🚨 User Impact 🚨
Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.
Pre-merge Checklist
Expand the relevant checklist and delete the others.
New Connector
Community member or Airbyter
airbyte_secret./gradlew :airbyte-integrations:connectors:<name>:integrationTest.README.mdbootstrap.md. See description and examplesdocs/integrations/<source or destination>/<name>.mdincluding changelog. See changelog exampledocs/integrations/README.mdairbyte-integrations/builds.mdAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>command is passing/publishcommand described hereUpdating a connector
Community member or Airbyter
airbyte_secret./gradlew :airbyte-integrations:connectors:<name>:integrationTest.README.mdbootstrap.md. See description and examplesdocs/integrations/<source or destination>/<name>.mdincluding changelog. See changelog exampleAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>command is passing/publishcommand described hereConnector Generator
-scaffoldin their name) have been updated with the latest scaffold by running./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplatesthen checking in your changesTests
Unit
Integration
Put your integration tests output here.
Acceptance