Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(source-google-sheets): migrate low code #50843

Merged
merged 75 commits into from
Jan 30, 2025
Merged
Changes from 1 commit
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
c9faf8b
source-google-sheets: adding dymamic streams and dynamic schemas for …
aldogonzalez8 Dec 12, 2024
b4f8deb
source-google-sheets: add custom retriever with custom slicer for ranges
aldogonzalez8 Dec 14, 2024
86f4cc9
source-google-sheets: fix nested records in value field with custom e…
aldogonzalez8 Dec 16, 2024
777daa5
source-google-sheets:" -m " - Add transformations to record selector…
aldogonzalez8 Dec 18, 2024
9435c86
source-google-sheets: remove teste file
aldogonzalez8 Dec 18, 2024
624d671
source-google-sheets:
aldogonzalez8 Dec 18, 2024
4d87723
source-google-sheets: initial changes for integrations tests passing.
aldogonzalez8 Dec 19, 2024
5b61aed
source-google-sheets: - all check operations (excelpt service auth fl…
aldogonzalez8 Dec 21, 2024
e1f6a75
source-google-sheets: all discovery operations passing
aldogonzalez8 Dec 23, 2024
30b6673
source-google-sheets: all original read integration tests passing
aldogonzalez8 Dec 24, 2024
510e074
source-google-sheets: test for read with conversion ready and also ch…
aldogonzalez8 Dec 24, 2024
76def28
source-google-sheets: add support for empty cells
aldogonzalez8 Dec 24, 2024
f9f658b
source-google-sheets: add tests for empty columns
aldogonzalez8 Dec 24, 2024
4b6636b
source-google-sheets: complete all read protocol tests
aldogonzalez8 Dec 31, 2024
6f24fac
source-google-sheets: merge from master
aldogonzalez8 Jan 1, 2025
6c53e9e
source-google-sheets: reduce unnecesary response filters
aldogonzalez8 Jan 1, 2025
041280a
source-google-sheets: fix styles
aldogonzalez8 Jan 1, 2025
00edfb4
source-google-sheets: split tests in different files per type
aldogonzalez8 Jan 1, 2025
5dd67d0
source-google-sheets: ruff-format
aldogonzalez8 Jan 1, 2025
6021253
source-google-sheets: update release information
aldogonzalez8 Jan 2, 2025
5876dc9
chore: auto-fix lint and format issues
octavia-squidington-iii Jan 2, 2025
874c191
source-google-sheets: add tests for components and urls
aldogonzalez8 Jan 3, 2025
c0e9e6f
source-google-sheets: ruff fix format
aldogonzalez8 Jan 3, 2025
ba3620b
source-google-sheets: add tests for parsing spreadsheet_id in config …
aldogonzalez8 Jan 3, 2025
a35f07f
source-google-sheets: ruff-format
aldogonzalez8 Jan 3, 2025
7ed534a
source-google-sheets: add manager for to increase batch size when 429…
aldogonzalez8 Jan 6, 2025
6ff3244
source-google-sheets: ruff-format
aldogonzalez8 Jan 6, 2025
e625be6
source-google-sheets: minor changes to batch size manager to handle t…
aldogonzalez8 Jan 6, 2025
6b487ea
source-google-sheets: ruff-format
aldogonzalez8 Jan 6, 2025
fc59b0d
source-google-sheets: add sheet id to range logging and remove old files
aldogonzalez8 Jan 6, 2025
1803ba8
source-google-sheets: ruff-format
aldogonzalez8 Jan 6, 2025
1dc50a5
source-google-sheets: Update lock file with latest CDK
aldogonzalez8 Jan 9, 2025
2b2f970
source-google-sheets: initial changes for jwt_requester
aldogonzalez8 Jan 10, 2025
0eef551
source-google-sheets: ruff format
aldogonzalez8 Jan 10, 2025
75ec92b
Merge branch 'master' into aldogonzalez8/source-google-sheets/migrate…
lazebnyi Jan 21, 2025
9dace30
Update authentication, add config migration, and fix backoff
lazebnyi Jan 23, 2025
91f4ff6
Merge branch 'aldogonzalez8/source-google-sheets/migrate-low-code' of…
lazebnyi Jan 23, 2025
7f052d9
chore: auto-fix lint and format issues
octavia-squidington-iii Jan 23, 2025
65b09ce
Fix some checks
lazebnyi Jan 23, 2025
02a140a
Merge branch 'aldogonzalez8/source-google-sheets/migrate-low-code' of…
lazebnyi Jan 23, 2025
1a19868
chore: auto-fix lint and format issues
octavia-squidington-iii Jan 23, 2025
8838242
Bump CDK version
lazebnyi Jan 23, 2025
18be3a4
Merge branch 'aldogonzalez8/source-google-sheets/migrate-low-code' of…
lazebnyi Jan 23, 2025
fee240e
Merge branch 'master' into aldogonzalez8/source-google-sheets/migrate…
lazebnyi Jan 24, 2025
4e2c3fd
Removed google libs
lazebnyi Jan 24, 2025
d5fad2d
Merge branch 'master' into aldogonzalez8/source-google-sheets/migrate…
lazebnyi Jan 29, 2025
78daf19
Merge branch 'aldogonzalez8/source-google-sheets/migrate-low-code' of…
lazebnyi Jan 29, 2025
fe14f92
Fix unit tests
lazebnyi Jan 29, 2025
8014fca
chore: auto-fix lint and format issues
octavia-squidington-iii Jan 29, 2025
3331a9b
Update configs name
lazebnyi Jan 29, 2025
9419696
Merge branch 'aldogonzalez8/source-google-sheets/migrate-low-code' of…
lazebnyi Jan 29, 2025
bf396e0
Merge branch 'master' into aldogonzalez8/source-google-sheets/migrate…
lazebnyi Jan 29, 2025
90eb46d
Update test secrets names
lazebnyi Jan 29, 2025
d3cb431
Fix coverage
lazebnyi Jan 29, 2025
3d724ba
chore: auto-fix lint and format issues
octavia-squidington-iii Jan 29, 2025
3f2b754
Fix unit tests
lazebnyi Jan 29, 2025
58d2686
Fix unit tests
lazebnyi Jan 29, 2025
be9b645
chore: auto-fix lint and format issues
octavia-squidington-iii Jan 29, 2025
2d76e00
Fix backoff policy
lazebnyi Jan 29, 2025
d9ef054
Merge branch 'aldogonzalez8/source-google-sheets/migrate-low-code' of…
lazebnyi Jan 29, 2025
17d7781
Speedup conenctor
lazebnyi Jan 29, 2025
23c494d
chore: auto-fix lint and format issues
octavia-squidington-iii Jan 29, 2025
37724b8
Remove unnecessary imports
lazebnyi Jan 29, 2025
46981a1
Add coverage ignore file
lazebnyi Jan 29, 2025
caafa61
disable http proxy
lazebnyi Jan 30, 2025
8996ce3
rollback http proxy
lazebnyi Jan 30, 2025
0b684e3
Update docs and add rc version
lazebnyi Jan 30, 2025
de3c0aa
Add enableProgressiveRollout to metadata
lazebnyi Jan 30, 2025
5e2ecd2
Update version in pyproject
lazebnyi Jan 30, 2025
a009326
Bump CDK version to latest
lazebnyi Jan 30, 2025
dfa5dae
Merge branch 'master' into aldogonzalez8/source-google-sheets/migrate…
lazebnyi Jan 30, 2025
bcec7e9
Remove config migration
lazebnyi Jan 30, 2025
48ea89d
Merge branch 'aldogonzalez8/source-google-sheets/migrate-low-code' of…
lazebnyi Jan 30, 2025
bf1ecac
Move to CDK last relese version
lazebnyi Jan 30, 2025
b656c05
Fix jinja macros
lazebnyi Jan 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Remove config migration
  • Loading branch information
lazebnyi committed Jan 30, 2025
commit bcec7e9e609f0143c86cdc9fd92e38cacdcaee27
186 changes: 93 additions & 93 deletions airbyte-integrations/connectors/source-google-sheets/poetry.lock

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -17,7 +17,7 @@ include = "source_google_sheets"

[tool.poetry.dependencies]
python = ">=3.10,<3.13"
airbyte-cdk = "^6"
airbyte-cdk = { version = "6.30.0.dev0", allow-prereleases = true }

[tool.poetry.scripts]
source-google-sheets = "source_google_sheets.run:run"

This file was deleted.

Original file line number Diff line number Diff line change
@@ -334,12 +334,12 @@ definitions:
- values
jwt_authenticator:
type: JwtAuthenticator
secret_key: "{{ config['credentials']['service_account']['private_key'] }}"
secret_key: "{{ json_load(config['credentials']['service_account_info'])['private_key'] }}"
algorithm: "RS256"
token_duration: 3600
jwt_payload:
aud: "{{ config['credentials']['service_account']['token_uri'] }}"
iss: "{{ config['credentials']['service_account']['client_email'] }}"
aud: "{{ json_load(config['credentials']['service_account_info'])['token_uri'] }}"
iss: "{{ json_load(config['credentials']['service_account_info'])['client_email'] }}"
additional_jwt_payload:
scope: "https://www.googleapis.com/auth/spreadsheets.readonly https://www.googleapis.com/auth/drive.readonly"
oauth_authenticator:
Original file line number Diff line number Diff line change
@@ -12,7 +12,6 @@

from airbyte_cdk.entrypoint import AirbyteEntrypoint, launch
from airbyte_cdk.models import AirbyteErrorTraceMessage, AirbyteMessage, AirbyteMessageSerializer, AirbyteTraceMessage, TraceType, Type
from source_google_sheets.config_migrations import MigrateServiceAccountInfo
from source_google_sheets.source import SourceGoogleSheets


@@ -50,6 +49,5 @@ def _get_source(args: List[str]):
def run():
_args = sys.argv[1:]
source = _get_source(_args)
MigrateServiceAccountInfo.migrate(sys.argv[1:], source)
if source:
launch(source, _args)

This file was deleted.

This file was deleted.

This file was deleted.