Greg/guykoh update dbt clickhouse #14897

grishick · 2022-07-20T19:41:35Z

What

Update dbt-clickhouse version to support Airbyte on ClickHouse cloud
This is an internal PR to run CI for this contributor PR

edgao

fyi we can actually run CI against community branches directly now. E.g. #14485

grishick · 2022-07-20T20:29:42Z

/test connector=connectors/destination-clickhouse

🕑 connectors/destination-clickhouse https://github.com/airbytehq/airbyte/actions/runs/2707483954
✅ connectors/destination-clickhouse https://github.com/airbytehq/airbyte/actions/runs/2707483954
No Python unittests run

Build Passed

Test summary info:

All Passed

grishick · 2022-07-20T21:44:15Z

/test connector=connectors/destination-snowflake

🕑 connectors/destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/2707860973
✅ connectors/destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/2707860973
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 13      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    13      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     155      8    95%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_config/transform.py                         162     34    79%
normalization/transform_catalog/utils.py                             38      9    76%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         61     38    38%
normalization/transform_catalog/stream_processor.py                 570    377    34%
-------------------------------------------------------------------------------------
TOTAL                                                              1363    587    57%

Build Passed

Test summary info:

All Passed

grishick · 2022-07-20T21:50:41Z

/publish connector=bases/base-normalization

🕑 Publishing the following connectors:
bases/base-normalization
https://github.com/airbytehq/airbyte/actions/runs/2707890780

Connector	Did it publish?	Were definitions generated?
bases/base-normalization	❌	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

grishick · 2022-07-20T21:52:30Z

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2707898882
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2707898882
🐛 https://gradle.com/s/7pcpwxnphpaxa

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 SKIPPED [1] integration_tests/test_ephemeral.py:61: Destinations DestinationType.MYSQL is not in NORMALIZATION_TEST_TARGET env variable (MYSQL is also skipped)
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.CLICKHOUSE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:146: DestinationType.MSSQL is disabled as it doesnt fully support schema change in incremental yet
	 SKIPPED [2] integration_tests/test_normalization.py:134: DestinationType.MYSQL does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.ORACLE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:143: DestinationType.SNOWFLAKE is disabled as it doesnt support schema change in incremental yet (column type changes)
	 ERROR integration_tests/test_ephemeral.py::test_stream_with_1_airbyte_column
	 ERROR integration_tests/test_normalization.py::test_redshift_normalization_migration
	 FAILED integration_tests/test_ephemeral.py::test_destination_supported_limits[DestinationType.CLICKHOUSE-1000]
	 FAILED integration_tests/test_normalization.py::test_normalization[DestinationType.CLICKHOUSE-test_simple_streams]
	 FAILED integration_tests/test_normalization.py::test_normalization[DestinationType.ORACLE-test_simple_streams]
	 �[31m======== �[31m�[1m3 failed�[0m, �[32m24 passed�[0m, �[33m7 skipped�[0m, �[31m�[1m2 errors�[0m�[31m in 3317.37s (0:55:17)�[0m�[31m ========�[0m

grishick · 2022-07-20T23:40:42Z

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2708345828
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2708345828
🐛 https://gradle.com/s/jxlbg2w7vxitu

Build Failed

Test summary info:

Could not find result summary

grishick · 2022-07-20T23:57:13Z

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2708406705
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2708406705
🐛 https://gradle.com/s/odbzwbz3pclja

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 SKIPPED [1] integration_tests/test_ephemeral.py:99: ephemeral materialization isn't supported in ClickHouse yet
	 SKIPPED [1] integration_tests/test_ephemeral.py:61: Destinations DestinationType.MYSQL is not in NORMALIZATION_TEST_TARGET env variable (MYSQL is also skipped)
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.CLICKHOUSE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:146: DestinationType.MSSQL is disabled as it doesnt fully support schema change in incremental yet
	 SKIPPED [2] integration_tests/test_normalization.py:134: DestinationType.MYSQL does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:134: DestinationType.ORACLE does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.ORACLE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:143: DestinationType.SNOWFLAKE is disabled as it doesnt support schema change in incremental yet (column type changes)
	 ERROR integration_tests/test_normalization.py::test_redshift_normalization_migration
	 FAILED integration_tests/test_normalization.py::test_normalization[DestinationType.CLICKHOUSE-test_simple_streams]
	 �[31m======== �[31m�[1m1 failed�[0m, �[32m24 passed�[0m, �[33m9 skipped�[0m, �[31m�[1m1 error�[0m�[31m in 2974.20s (0:49:34)�[0m�[31m =========�[0m

edgao · 2022-07-21T02:49:12Z

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2708972761
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2708972761
🐛 https://gradle.com/s/ce3unllmi4bfw

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 FAILED unit_tests/test_transform_config.py::TestTransformConfig::test_transform_clickhouse
	 �[31m======================== �[31m�[1m1 failed�[0m, �[32m164 passed�[0m�[31m in 2.36s�[0m�[31m =========================�[0m

edgao · 2022-07-21T03:33:44Z

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2709129312
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2709129312
🐛 https://gradle.com/s/746k2p5ny2mem

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 SKIPPED [1] integration_tests/test_ephemeral.py:99: ephemeral materialization isn't supported in ClickHouse yet
	 SKIPPED [1] integration_tests/test_ephemeral.py:61: Destinations DestinationType.MYSQL is not in NORMALIZATION_TEST_TARGET env variable (MYSQL is also skipped)
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.CLICKHOUSE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:146: DestinationType.MSSQL is disabled as it doesnt fully support schema change in incremental yet
	 SKIPPED [2] integration_tests/test_normalization.py:134: DestinationType.MYSQL does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:134: DestinationType.ORACLE does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.ORACLE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:143: DestinationType.SNOWFLAKE is disabled as it doesnt support schema change in incremental yet (column type changes)
	 ERROR integration_tests/test_ephemeral.py::test_stream_with_1_airbyte_column
	 ERROR integration_tests/test_normalization.py::test_redshift_normalization_migration
	 FAILED integration_tests/test_ephemeral.py::test_destination_supported_limits[DestinationType.POSTGRES-1000]
	 FAILED integration_tests/test_ephemeral.py::test_destination_failure_over_limits[Postgres-1665-target lists can have at most 1664 entries]
	 FAILED integration_tests/test_ephemeral.py::test_destination_failure_over_limits[MySQL-250-Row size too large]
	 FAILED integration_tests/test_ephemeral.py::test_empty_streams - AssertionErr...
	 FAILED integration_tests/test_ephemeral.py::test_stream_with_1_airbyte_column
	 FAILED integration_tests/test_normalization.py::test_normalization[DestinationType.CLICKHOUSE-test_simple_streams]
	 �[31m======== �[31m�[1m6 failed�[0m, �[32m19 passed�[0m, �[33m9 skipped�[0m, �[31m�[1m2 errors�[0m�[31m in 3494.29s (0:58:14)�[0m�[31m ========�[0m

guykoh · 2022-07-21T13:09:39Z

airbyte-integrations/bases/base-normalization/normalization/transform_config/transform.py

@@ -293,6 +293,7 @@ def transform_clickhouse(config: Dict[str, Any]):
        # https://docs.getdbt.com/reference/warehouse-profiles/clickhouse-profile
        dbt_config = {
            "type": "clickhouse",
+            "driver": "clickhouse-connect",


This is not a valid value for 'driver' field - use 'native'. In fact, you don't need that field at all, the connector will derive the driver protocol from the port (9440).

hrm. will try setting it explicitly - integration tests were failing because driver was being set to None:

1 check failed: dbt was unable to connect to the specified database. The database returned the following error: >Database Error Library for ClickHouse driver type None not found

Ran the tests locally and ran into this error:

14:29:05 Compilation Error in model dedup_exchange_rate_ab1 (models/generated/airbyte_ctes/test_normalization/dedup_exchange_rate_ab1.sql) 14:29:05 No materialization 'ignore' was found for adapter clickhouse! (searched types 'default' and 'clickhouse')

Maybe the approach here is be to use either view materialization, or materialize as tables and then drop the tables afterward, unless I'm misunderstanding the intent?

Sure, let's use view materialization instead.

edgao · 2022-07-21T14:33:46Z

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2712544184
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2712544184
🐛 https://gradle.com/s/g5est7cbdrad4

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 SKIPPED [1] integration_tests/test_ephemeral.py:99: ephemeral materialization isn't supported in ClickHouse yet
	 SKIPPED [1] integration_tests/test_ephemeral.py:61: Destinations DestinationType.MYSQL is not in NORMALIZATION_TEST_TARGET env variable (MYSQL is also skipped)
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.CLICKHOUSE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:146: DestinationType.MSSQL is disabled as it doesnt fully support schema change in incremental yet
	 SKIPPED [2] integration_tests/test_normalization.py:134: DestinationType.MYSQL does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:134: DestinationType.ORACLE does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.ORACLE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:143: DestinationType.SNOWFLAKE is disabled as it doesnt support schema change in incremental yet (column type changes)
	 FAILED integration_tests/test_ephemeral.py::test_destination_supported_limits[DestinationType.REDSHIFT-1000]
	 FAILED integration_tests/test_normalization.py::test_normalization[DestinationType.CLICKHOUSE-test_simple_streams]
	 �[31m============= �[31m�[1m2 failed�[0m, �[32m23 passed�[0m, �[33m9 skipped�[0m�[31m in 3220.25s (0:53:40)�[0m�[31m =============�[0m

grishick · 2022-07-21T21:05:34Z

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2714699723
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2714699723
🐛 https://gradle.com/s/wfe5fxuyb43pc

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 SKIPPED [1] integration_tests/test_ephemeral.py:99: ephemeral materialization isn't supported in ClickHouse yet
	 SKIPPED [1] integration_tests/test_ephemeral.py:61: Destinations DestinationType.MYSQL is not in NORMALIZATION_TEST_TARGET env variable (MYSQL is also skipped)
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.CLICKHOUSE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:146: DestinationType.MSSQL is disabled as it doesnt fully support schema change in incremental yet
	 SKIPPED [2] integration_tests/test_normalization.py:134: DestinationType.MYSQL does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.ORACLE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:134: DestinationType.ORACLE does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:143: DestinationType.SNOWFLAKE is disabled as it doesnt support schema change in incremental yet (column type changes)
	 FAILED integration_tests/test_normalization.py::test_normalization[DestinationType.CLICKHOUSE-test_simple_streams]
	 �[31m============= �[31m�[1m1 failed�[0m, �[32m24 passed�[0m, �[33m9 skipped�[0m�[31m in 3239.61s (0:53:59)�[0m�[31m =============�[0m

grishick · 2022-08-21T04:15:03Z

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2897150817
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2897150817
🐛 https://gradle.com/s/4cdrsvocmdggi

Build Failed

Test summary info:

Could not find result summary

grishick · 2022-08-21T19:19:01Z

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2899631566
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2899631566
🐛 https://gradle.com/s/ajbaeqx3vths4

Build Failed

Test summary info:

Could not find result summary

grishick · 2022-08-22T00:10:50Z

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2900442858
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2900442858
🐛 https://gradle.com/s/bf3ufuj2d7k4y

Build Failed

Test summary info:

Could not find result summary

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2900795875
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2900795875
🐛 https://gradle.com/s/vzq62rtcq6cnq

Build Failed

Test summary info:

Could not find result summary

grishick · 2022-08-22T16:42:33Z

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2905592574
✅ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/2905592574
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 13      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    13      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     161      8    95%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_config/transform.py                         186     49    74%
normalization/transform_catalog/utils.py                             51     14    73%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         61     38    38%
normalization/transform_catalog/stream_processor.py                 589    394    33%
-------------------------------------------------------------------------------------
TOTAL                                                              1425    624    56%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       121     25    79%   21-23, 29-31, 36-43, 48-61, 208-216
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       110     48    56%   23-26, 32, 36, 39-64, 67-69, 72-74, 77-79, 82-84, 87-89, 92-110, 144-146
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1322    463    65%
Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 13      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    13      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     161      8    95%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_config/transform.py                         186     49    74%
normalization/transform_catalog/utils.py                             51     14    73%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         61     38    38%
normalization/transform_catalog/stream_processor.py                 589    394    33%
-------------------------------------------------------------------------------------
TOTAL                                                              1425    624    56%
Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 13      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    13      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/utils.py                             51      1    98%
normalization/transform_catalog/destination_name_transformer.py     161      5    97%
normalization/transform_catalog/stream_processor.py                 589     35    94%
normalization/transform_catalog/catalog_processor.py                147     12    92%
normalization/transform_catalog/dbt_macro.py                         22      3    86%
normalization/transform_catalog/table_name_registry.py              174     51    71%
normalization/transform_catalog/transform.py                         61     22    64%
normalization/transform_config/transform.py                         186     77    59%
-------------------------------------------------------------------------------------
TOTAL                                                              1425    206    86%

Build Passed

Test summary info:

	 =========================== short test summary info ============================
	 SKIPPED [1] integration_tests/test_ephemeral.py:99: ephemeral materialization isn't supported in ClickHouse yet
	 SKIPPED [1] integration_tests/test_ephemeral.py:61: Destinations DestinationType.MYSQL is not in NORMALIZATION_TEST_TARGET env variable (MYSQL is also skipped)
	 SKIPPED [1] integration_tests/test_normalization.py:143: DestinationType.CLICKHOUSE is disabled as it doesnt support schema change in incremental yet (column type changes)
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.CLICKHOUSE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:146: DestinationType.MSSQL is disabled as it doesnt fully support schema change in incremental yet
	 SKIPPED [2] integration_tests/test_normalization.py:134: DestinationType.MYSQL does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:134: DestinationType.ORACLE does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.ORACLE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:143: DestinationType.SNOWFLAKE is disabled as it doesnt support schema change in incremental yet (column type changes)
	 �[32m================= �[32m�[1m24 passed�[0m, �[33m10 skipped�[0m�[32m in 3873.74s (1:04:33)�[0m�[32m ==================�[0m
	 =========================== short test summary info ============================
	 SKIPPED [1] integration_tests/test_ephemeral.py:99: ephemeral materialization isn't supported in ClickHouse yet
	 SKIPPED [1] integration_tests/test_ephemeral.py:61: Destinations DestinationType.MYSQL is not in NORMALIZATION_TEST_TARGET env variable (MYSQL is also skipped)
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.CLICKHOUSE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:143: DestinationType.CLICKHOUSE is disabled as it doesnt support schema change in incremental yet (column type changes)
	 SKIPPED [1] integration_tests/test_normalization.py:146: DestinationType.MSSQL is disabled as it doesnt fully support schema change in incremental yet
	 SKIPPED [2] integration_tests/test_normalization.py:134: DestinationType.MYSQL does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:81: Destinations DestinationType.ORACLE does not support nested streams
	 SKIPPED [1] integration_tests/test_normalization.py:134: DestinationType.ORACLE does not support incremental yet
	 SKIPPED [1] integration_tests/test_normalization.py:143: DestinationType.SNOWFLAKE is disabled as it doesnt support schema change in incremental yet (column type changes)
	 �[32m================= �[32m�[1m24 passed�[0m, �[33m10 skipped�[0m�[32m in 3773.60s (1:02:53)�[0m�[32m ==================�[0m

grishick · 2022-08-22T20:02:27Z

/test connector=connectors/destination-clickhouse

🕑 connectors/destination-clickhouse https://github.com/airbytehq/airbyte/actions/runs/2906733727
✅ connectors/destination-clickhouse https://github.com/airbytehq/airbyte/actions/runs/2906733727
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 13      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    13      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     161      8    95%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_config/transform.py                         186     49    74%
normalization/transform_catalog/utils.py                             51     14    73%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         61     38    38%
normalization/transform_catalog/stream_processor.py                 589    394    33%
-------------------------------------------------------------------------------------
TOTAL                                                              1425    624    56%

Build Passed

Test summary info:

All Passed

…se cloud

grishick · 2022-08-22T23:35:34Z

/publish connector=bases/base-normalization

🕑 Publishing the following connectors:
bases/base-normalization
https://github.com/airbytehq/airbyte/actions/runs/2907813926

Connector	Did it publish?	Were definitions generated?
bases/base-normalization	✅	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

* Update dbt-clickhouse version to 1.1.7 to support AirByte on ClickHouse cloud * Fix quote handling in Clickhouse normalization tests * Update test output for Clickhouse * Bump version and update changelog Co-authored-by: guykohen <guy@clickhouse.com>

grishick requested a review from a team as a code owner July 20, 2022 19:41

github-actions bot added area/documentation Improvements or additions to documentation area/platform issues related to the platform area/worker Related to worker normalization labels Jul 20, 2022

grishick temporarily deployed to more-secrets July 20, 2022 19:43 Inactive

edgao approved these changes Jul 20, 2022

View reviewed changes

grishick force-pushed the greg/guykoh_update_dbt_clickhouse branch from 7289356 to dbb93a6 Compare July 20, 2022 23:38

github-actions bot added area/documentation Improvements or additions to documentation and removed area/documentation Improvements or additions to documentation labels Jul 20, 2022

grishick temporarily deployed to more-secrets July 20, 2022 23:42 Inactive

grishick temporarily deployed to more-secrets July 20, 2022 23:59 Inactive

grishick mentioned this pull request Jul 21, 2022

Update dbt clickhouse #14833

Closed

14 tasks

edgao temporarily deployed to more-secrets July 21, 2022 02:51 Inactive

edgao temporarily deployed to more-secrets July 21, 2022 03:35 Inactive

guykoh reviewed Jul 21, 2022

View reviewed changes

edgao temporarily deployed to more-secrets July 21, 2022 14:32 Inactive

grishick temporarily deployed to more-secrets July 21, 2022 21:07 Inactive

grishick force-pushed the greg/guykoh_update_dbt_clickhouse branch from 372f97b to 3eb3b7a Compare July 21, 2022 22:17

grishick temporarily deployed to more-secrets July 21, 2022 22:19 Inactive

grishick force-pushed the greg/guykoh_update_dbt_clickhouse branch 2 times, most recently from ccb430c to 9d818b0 Compare August 21, 2022 23:52

grishick temporarily deployed to more-secrets August 22, 2022 02:06 Inactive

grishick force-pushed the greg/guykoh_update_dbt_clickhouse branch from b3cdbf9 to e4964b9 Compare August 22, 2022 15:44

grishick force-pushed the greg/guykoh_update_dbt_clickhouse branch from e4964b9 to 6fe668d Compare August 22, 2022 20:28

guykoh and others added 12 commits August 22, 2022 16:13

Update dbt-clickhouse version to 1.1.7 to support AirByte on ClickHou…

46a4d09

…se cloud

Rebase and resolve conflicts

36c83fd

fixing ClickhouseDestinationStrictEncryptAcceptanceTest

7f6e3d5

add quote: true to dbt config

bc202ab

replace special characters for clickhouse

832fa85

add replacement for column with quotes

e5f06cb

Fix quote handling in Clickhouse normalization tests

1380600

Fix quotes in normalization tests for clickhouse

1a1a1cc

Quote identifiers for Clickhouse with quote() statement

65cbdbf

Update test output for clickhouse

fd96f3c

Remove unnecessary changes

5a936e3

Bump version and update changelog

bd2dd4a

grishick force-pushed the greg/guykoh_update_dbt_clickhouse branch from 6fe668d to bd2dd4a Compare August 22, 2022 23:13

grishick merged commit 5819733 into master Aug 23, 2022

grishick deleted the greg/guykoh_update_dbt_clickhouse branch August 23, 2022 04:53

This was referenced Aug 23, 2022

Bump Airbyte version from 0.40.1 to 0.40.2 #15909

Closed

Bump Airbyte version from 0.40.1 to 0.40.2 #15931

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Greg/guykoh update dbt clickhouse #14897

Greg/guykoh update dbt clickhouse #14897

grishick commented Jul 20, 2022

edgao left a comment

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

edgao commented Jul 21, 2022 •

edited by github-actions bot

Loading

edgao commented Jul 21, 2022 •

edited by github-actions bot

Loading

guykoh Jul 21, 2022

edgao Jul 21, 2022

edgao Jul 21, 2022

guykoh Jul 21, 2022

edgao commented Jul 21, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 21, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 21, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 21, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 22, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 22, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 22, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 22, 2022 •

edited by github-actions bot

Loading

Greg/guykoh update dbt clickhouse #14897

Greg/guykoh update dbt clickhouse #14897

Conversation

grishick commented Jul 20, 2022

What

edgao left a comment

Choose a reason for hiding this comment

grishick commented Jul 20, 2022 • edited by github-actions bot Loading

Build Passed

grishick commented Jul 20, 2022 • edited by github-actions bot Loading

Build Passed

grishick commented Jul 20, 2022 • edited by github-actions bot Loading

grishick commented Jul 20, 2022 • edited by github-actions bot Loading

Build Failed

grishick commented Jul 20, 2022 • edited by github-actions bot Loading

Build Failed

grishick commented Jul 20, 2022 • edited by github-actions bot Loading

Build Failed

edgao commented Jul 21, 2022 • edited by github-actions bot Loading

Build Failed

edgao commented Jul 21, 2022 • edited by github-actions bot Loading

Build Failed

guykoh Jul 21, 2022

Choose a reason for hiding this comment

edgao Jul 21, 2022

Choose a reason for hiding this comment

edgao Jul 21, 2022

Choose a reason for hiding this comment

guykoh Jul 21, 2022

Choose a reason for hiding this comment

edgao commented Jul 21, 2022 • edited by github-actions bot Loading

Build Failed

grishick commented Jul 21, 2022 • edited by github-actions bot Loading

Build Failed

grishick commented Aug 21, 2022 • edited by github-actions bot Loading

Build Failed

grishick commented Aug 21, 2022 • edited by github-actions bot Loading

Build Failed

grishick commented Aug 22, 2022 • edited by github-actions bot Loading

Build Failed

Build Failed

grishick commented Aug 22, 2022 • edited by github-actions bot Loading

Build Passed

grishick commented Aug 22, 2022 • edited by github-actions bot Loading

Build Passed

grishick commented Aug 22, 2022 • edited by github-actions bot Loading

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 20, 2022 •

edited by github-actions bot

Loading

edgao commented Jul 21, 2022 •

edited by github-actions bot

Loading

edgao commented Jul 21, 2022 •

edited by github-actions bot

Loading

edgao commented Jul 21, 2022 •

edited by github-actions bot

Loading

grishick commented Jul 21, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 21, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 21, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 22, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 22, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 22, 2022 •

edited by github-actions bot

Loading

grishick commented Aug 22, 2022 •

edited by github-actions bot

Loading