Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Source Pinterest: Add Account ID parameter and stream filter #53135

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

karolur
Copy link

@karolur karolur commented Feb 3, 2025

What

This PR introduces an Account Id filter for the Pinterest source connector.

  • Users can now specify an Account Id to fetch data only for that particular account.
  • Some organizations have access restrictions that limit data retrieval to a single ad account, making it necessary to filter at the source.
  • Previously, the connector always fetched data for all accessible accounts, which was not always desirable for teams with stricter data permissions.

🔗 Related Discussion: Airbyte Discussion #52036

How

  1. Modified the connector's stream logic to accept and process an account_id if provided.
  2. Introduced AdAccountValidationStream to validate the provided account_id before proceeding with data retrieval.
  3. Implemented AdAccountRecordExtractor to handle both scenarios—fetching data for a single account or all accounts—ensuring correct processing based on user input.

Review guide

  1. source_pinterest/source.pyUpdated SourcePinterest to validate account_id in _validate_and_transform.
  2. source_pinterest/streams.pyAdded AdAccountValidationStream to verify the existence of the specified account_id.
  3. source_pinterest/components/components.pyAdded AdAccountRecordExtractor to handle data extraction for single vs. multiple accounts.
  4. source_pinterest/manifest.yamlUpdated connector specification to include account_id as an optional configuration parameter.
  5. source_pinterest/components/error_mapping.pyExtended error handling to return meaningful errors for invalid account_id values.

User Impact

New Feature: Users can now filter Pinterest ad data by a specific account_id, ensuring more precise data retrieval.
Compliance-Friendly: Helps organizations align with data access policies by restricting API calls to only authorized accounts.
No Breaking Changes: Users who do not specify an Account Id will continue fetching all available data as before.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

- Implement AdAccountValidationStream for validating ad account IDs
- Introduce AdAccountRecordExtractor for better record processing
- Improve error handling and implement structured error mapping
- Update manifest.yaml to reflect new stream additions
- Ensure graceful handling of invalid accounts without breaking execution
- Added new invalid_account_config.json for acceptance tests
- Updated acceptance-test-config.yml to include invalid account test
- Added new invalid_account_config.json for acceptance tests
- Updated acceptance-test-config.yml to include invalid account test
Copy link

vercel bot commented Feb 3, 2025

@karolur is attempting to deploy a commit to the Airbyte Growth Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant
Copy link

CLAassistant commented Feb 3, 2025

CLA assistant check
All committers have signed the CLA.

@octavia-squidington-iii octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Feb 3, 2025
@karolur karolur temporarily deployed to community-ci-auto February 3, 2025 20:45 — with GitHub Actions Inactive
@marcosmarxm
Copy link
Member

marcosmarxm commented Feb 4, 2025

/format-fix

Format-fix job started... Check job output.

✅ Changes applied successfully. (56a39da)

@marcosmarxm
Copy link
Member

Hello 👋, I apologize if your contribution hasn't been reviewed or updated yet. I'm working to address the community contribution backlog. I created this GitHub project to help you understand when to expect a review. This week, my goal is to complete all contributions related to Marketplace and Manifest-only connectors. Next week, I will focus on API Certified Connectors, followed by DB Java Connector contributions. If you have any questions about your contribution, please feel free to send me a direct message in the Airbyte Community Slack (and include the link to your pull request).

Copy link

vercel bot commented Feb 18, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 18, 2025 11:55pm

Copy link
Member

@marcosmarxm marcosmarxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good to me. I'm running integration tests and need to wait until their done and all green. After need final approval from connector team to introduce this feature to Pinterest connector.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation community connectors/source/pinterest
Projects
Status: Ready to Ship
Development

Successfully merging this pull request may close these issues.

5 participants