Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 Source Salesforce: Support oauth'ing into sandbox environment #7877

Closed

Conversation

vitaliizazmic
Copy link
Contributor

@vitaliizazmic vitaliizazmic commented Nov 11, 2021

What

When oauth'ing a salesforce user into a sandbox environment, we need to send them to test.salesforce.com rather than login.salesforce.com Our current server implementation of salesforce oauth always assumes they are not trying to use a sandbox environment and redirects them to login.salesforce.com.

Closes #7722

How

Temporary solution was created according to proposed algorithm into issue #7722

  1. Disabled the sandbox option of the Salesforce connector by remove it from the spec but not from the code, and allow additional properties = true. Set by default is_sandbox = false in code.
  2. Created a fork of the salesforce connector source-salesforce-sandbox, which always assumes we are using the salesforce sandbox environment. Set by default is_sandbox = true in code.
  3. Created a new oauth flow for sandbox test.salesforce.com.
  4. Generated credentials using client id and secret of Airbyte connected app from from not sandbox account.
  5. Tested existing and new connector. In sandbox account only AppDefenition stream is enabled and it is used in SAT.

Pre-merge Checklist

  • Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions

@vitaliizazmic vitaliizazmic self-assigned this Nov 11, 2021
@github-actions github-actions bot added the area/connectors Connector related issues label Nov 11, 2021
@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Nov 11, 2021

/test connector=source-salesforce

🕑 source-salesforce https://github.com/airbytehq/airbyte/actions/runs/1450250703
✅ source-salesforce https://github.com/airbytehq/airbyte/actions/runs/1450250703
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        75      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       38     27    29%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 62     25    60%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  896    440    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                 Stmts   Miss  Cover
	 Name                                 Stmts   Miss  Cover/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-salesforce/.venv/lib/python3.8/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
	   self._warn("No data was collected.", slug="no-data-collected")
	 --------------------------------------------------------
	 source_salesforce/__init__.py            2      2     0%
	 source_salesforce/api.py               115    115     0%
	 source_salesforce/exceptions.py          1      1     0%
	 source_salesforce/rate_limiting.py      22     22     0%
	 source_salesforce/source.py             52     52     0%
	 source_salesforce/streams.py           201    201     0%
	 --------------------------------------------------------
	 TOTAL                                  393    393     0%

@jrhizor jrhizor temporarily deployed to more-secrets November 11, 2021 19:28 Inactive
@vitaliizazmic vitaliizazmic temporarily deployed to more-secrets November 12, 2021 13:36 Inactive
@vitaliizazmic vitaliizazmic temporarily deployed to more-secrets November 16, 2021 21:08 Inactive
@vitaliizazmic vitaliizazmic temporarily deployed to more-secrets November 16, 2021 21:18 Inactive
@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Nov 17, 2021

/test connector=source-salesforce-sandbox

🕑 source-salesforce-sandbox https://github.com/airbytehq/airbyte/actions/runs/1473651004
✅ source-salesforce-sandbox https://github.com/airbytehq/airbyte/actions/runs/1473651004
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        75      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       38     27    29%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 62     25    60%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  896    440    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                         Stmts   Miss  Cover
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-salesforce-sandbox/.venv/lib/python3.8/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
	   self._warn("No data was collected.", slug="no-data-collected")
	 ----------------------------------------------------------------
	 source_salesforce_sandbox/__init__.py            2      2     0%
	 source_salesforce_sandbox/api.py               114    114     0%
	 source_salesforce_sandbox/exceptions.py          1      1     0%
	 source_salesforce_sandbox/rate_limiting.py      22     22     0%
	 source_salesforce_sandbox/source.py             52     52     0%
	 source_salesforce_sandbox/streams.py           201    201     0%
	 ----------------------------------------------------------------
	 TOTAL                                          392    392     0%

@jrhizor jrhizor temporarily deployed to more-secrets November 17, 2021 21:08 Inactive
@vitaliizazmic vitaliizazmic temporarily deployed to more-secrets November 17, 2021 21:19 Inactive
@vitaliizazmic vitaliizazmic temporarily deployed to more-secrets November 17, 2021 21:25 Inactive
Copy link
Contributor

@sherifnada sherifnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vitaliizazmic is it possible instead of completely replicating the connector, to only inherit from the salesforce docker image, similar to source-file-secure?

@@ -496,6 +496,13 @@
documentationUrl: https://docs.airbyte.io/integrations/sources/salesforce
icon: salesforce.svg
sourceType: api
- name: SalesforceSandbox
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- name: SalesforceSandbox
- name: Salesforce (Sandbox)

@sherifnada sherifnada closed this Jan 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Source Salesforce: Support oauth'ing into sandbox environment
3 participants