Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add normalization-clickhouse docker build step #9029

Closed
wants to merge 2 commits into from

Conversation

jzcruiser
Copy link
Contributor

What

mysql to clickhouse normalization task fails because docker image airbyte/normalization-clickhouse:0.1.61 is missing:

Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.35.0-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.0-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.0-alpha.jar:?]
	... 1 more
Caused by: io.airbyte.workers.WorkerException: Could not find image: airbyte/normalization-clickhouse:0.1.61
	at io.airbyte.workers.process.DockerProcessFactory.create(DockerProcessFactory.java:97) ~[io.airbyte-airbyte-workers-0.35.0-alpha.jar:?]
	at io.airbyte.workers.normalization.DefaultNormalizationRunner.runProcess(DefaultNormalizationRunner.java:123) ~[io.airbyte-airbyte-workers-0.35.0-alpha.jar:?]
	at io.airbyte.workers.normalization.DefaultNormalizationRunner.normalize(DefaultNormalizationRunner.java:108) ~[io.airbyte-airbyte-workers-0.35.0-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:55) ~[io.airbyte-airbyte-workers-0.35.0-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.0-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.0-alpha.jar:?]

How

Add clickhouse related configurations to base-normalization's build script

Recommended reading order

  1. airbyte-integrations/bases/base-normalization/build.gradle
  2. airbyte-integrations/bases/base-normalization/docker-compose.yaml

@marcosmarxm marcosmarxm self-assigned this Dec 22, 2021
@marcosmarxm
Copy link
Member

thanks @jzcruiser I'm having problems publishing the code, but soon I'll release the new version.

@jzcruiser
Copy link
Contributor Author

@marcosmarxm thanks, really appreciate it.

@jzcruiser
Copy link
Contributor Author

Hi @marcosmarxm I just patched settings.gradle to tackle the integration test failure. Please kindly review.

@marcosmarxm
Copy link
Member

@jzcruiser I'm correcting some tests in #9072 so after that we can merge your contribution.

@marcosmarxm
Copy link
Member

Thanks @jzcruiser I published the code in #9072. Because it needed more changes I merged that one. I'll close this one.

@marcosmarxm marcosmarxm closed this Jan 5, 2022
@jzcruiser
Copy link
Contributor Author

@marcosmarxm it's ok, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants