Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[helm] Deployment with external minio: Cannot create enum from ${env:S3_LOG_BUCKET_REGION} #34539

Open
leonardo-lages-hyperflow opened this issue Jan 25, 2024 · 10 comments
Open

[helm] Deployment with external minio: Cannot create enum from ${env:S3_LOG_BUCKET_REGION} #34539

leonardo-lages-hyperflow opened this issue Jan 25, 2024 · 10 comments
Labels
area/platform issues related to the platform community helm kubernetes team/deployments type/bug Something isn't working

Comments

@leonardo-lages-hyperflow
Copy link

Platform Version

0.50.14

What step the error happened?

On deploy

Revelant information

When the worker start happens an error which blocks the use of connectors.

The documentation tells to disable minio, enable externalMinio and then put the s3 region empty. But when we do this occurs the error showed in the log.

Relevant log output

Unsetting empty environment variable 'ACTIVITY_MAX_DELAY_BETWEEN_ATTEMPTS_SECONDS'
Unsetting empty environment variable 'GOOGLE_APPLICATION_CREDENTIALS'
Unsetting empty environment variable 'S3_LOG_BUCKET_REGION'
Unsetting empty environment variable 'OTEL_COLLECTOR_ENDPOINT'
Unsetting empty environment variable 'JOB_MAIN_CONTAINER_CPU_REQUEST'
Unsetting empty environment variable 'JOB_MAIN_CONTAINER_MEMORY_REQUEST'
Unsetting empty environment variable 'ACTIVITY_MAX_ATTEMPT'
Unsetting empty environment variable 'JOB_MAIN_CONTAINER_CPU_LIMIT'
Unsetting empty environment variable 'VERSION'
Unsetting empty environment variable 'WORKFLOW_FAILURE_RESTART_DELAY_SECONDS'
Unsetting empty environment variable 'METRIC_CLIENT'
Unsetting empty environment variable 'JOB_MAIN_CONTAINER_MEMORY_LIMIT'
Unsetting empty environment variable 'GCS_LOG_BUCKET'
Unsetting empty environment variable 'ACTIVITY_INITIAL_DELAY_BETWEEN_ATTEMPTS_SECONDS'
INFO StatusConsoleListener Loading mask data from '/seed/specs_secrets_mask.yaml

    ___    _      __          __
   /   |  (_)____/ /_  __  __/ /____
  / /| | / / ___/ __ \/ / / / __/ _ \
 / ___ |/ / /  / /_/ / /_/ / /_/  __/
/_/  |_/_/_/  /_.___/\__, /\__/\___/
                    /____/
        : airbyte-workers :

  Micronaut (v3.10.1)

2024-01-25 19:26:22 �[32mINFO�[m i.m.c.e.DefaultEnvironment(<init>):167 - Established active environments: [k8s, cloud, ec2, control-plane]
2024-01-25 19:26:23 �[32mINFO�[m i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):111 - Setting log level 'DEBUG' for logger: 'io.airbyte.bootloader'
2024-01-25 19:26:23 �[32mINFO�[m i.a.f.ConfigFileClient(<init>):105 - path /flags does not exist, will return default flag values
2024-01-25 19:26:26 �[32mINFO�[m i.a.c.EnvConfigs(getEnvOrDefault):1149 - Using default value for environment variable METRIC_CLIENT: ''
2024-01-25 19:26:26 �[32mINFO�[m i.a.c.EnvConfigs(getEnvOrDefault):1149 - Using default value for environment variable METRIC_CLIENT: ''
2024-01-25 19:26:26 �[33mWARN�[m i.a.m.l.MetricClientFactory(initialize):74 - MetricClient was not recognized or not provided. Accepted values are `datadog` or `otel`. 
2024-01-25 19:26:26 �[32mINFO�[m i.a.c.EnvConfigs(getEnvOrDefault):1149 - Using default value for environment variable FEATURE_FLAG_CLIENT: ''
2024-01-25 19:26:26 �[32mINFO�[m i.a.c.EnvConfigs(getEnvOrDefault):1149 - Using default value for environment variable LAUNCHDARKLY_KEY: ''
2024-01-25 19:26:26 �[32mINFO�[m i.a.c.EnvConfigs(getEnvOrDefault):1149 - Using default value for environment variable OTEL_COLLECTOR_ENDPOINT: ''
2024-01-25 19:26:26 �[32mINFO�[m i.a.c.EnvConfigs(getEnvOrDefault):1149 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2024-01-25 19:26:26 �[32mINFO�[m i.a.c.EnvConfigs(getEnvOrDefault):1149 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2024-01-25 19:26:26 �[32mINFO�[m i.a.c.EnvConfigs(getEnvOrDefault):1149 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2024-01-25 19:26:26 �[32mINFO�[m i.a.c.EnvConfigs(getEnvOrDefault):1149 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2024-01-25 19:26:26 �[32mINFO�[m i.a.c.t.TemporalUtils(getTemporalClientWhenConnected):148 - Waiting for temporal server...
2024-01-25 19:26:26 �[33mWARN�[m i.a.c.t.TemporalUtils(getTemporalClientWhenConnected):159 - Waiting for namespace default to be initialized in temporal...
2024-01-25 19:26:28 �[32mINFO�[m i.t.s.WorkflowServiceStubsImpl(<init>):86 - Created WorkflowServiceStubs for channel: ManagedChannelOrphanWrapper{delegate=ManagedChannelImpl{logId=1, target=airbyte-temporal:7233}}
2024-01-25 19:26:34 �[32mINFO�[m i.a.c.t.TemporalUtils(getTemporalClientWhenConnected):176 - Temporal namespace default initialized!
2024-01-25 19:26:34 �[32mINFO�[m i.a.w.ApplicationInitializer(initializeCommonDependencies):167 - Initializing common worker dependencies.
2024-01-25 19:26:34 �[32mINFO�[m i.a.w.ApplicationInitializer(configureTemporal):322 - Configuring Temporal....
ERROR StatusConsoleListener Could not create plugin of type class com.van.logging.log4j2.Log4j2Appender for element Log4j2Appender: java.lang.RuntimeException: Cannot build appender due to errors
 java.lang.RuntimeException: Cannot build appender due to errors
	at com.van.logging.log4j2.Log4j2AppenderBuilder.build(Log4j2AppenderBuilder.java:147)
	at com.van.logging.log4j2.Log4j2AppenderBuilder.build(Log4j2AppenderBuilder.java:25)
	at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:124)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:1162)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:1083)
	at org.apache.logging.log4j.core.appender.routing.RoutingAppender.createAppender(RoutingAppender.java:323)
	at org.apache.logging.log4j.core.appender.routing.RoutingAppender.getControl(RoutingAppender.java:295)
	at org.apache.logging.log4j.core.appender.routing.RoutingAppender.append(RoutingAppender.java:253)
	at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:160)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88)
	at org.apache.logging.log4j.core.appender.rewrite.RewriteAppender.append(RewriteAppender.java:89)
	at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:160)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88)
	at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:705)
	at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:663)
	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:639)
	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:575)
	at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:92)
	at org.apache.logging.log4j.core.Logger.log(Logger.java:169)
	at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2933)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2886)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2868)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2652)
	at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:2400)
	at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:179)
	at io.airbyte.workers.ApplicationInitializer.configureTemporal(ApplicationInitializer.java:322)
	at io.airbyte.workers.ApplicationInitializer.initializeCommonDependencies(ApplicationInitializer.java:177)
	at io.airbyte.workers.ApplicationInitializer.onApplicationEvent(ApplicationInitializer.java:143)
	at io.airbyte.workers.ApplicationInitializer.onApplicationEvent(ApplicationInitializer.java:57)
	at io.micronaut.context.event.ApplicationEventPublisherFactory.notifyEventListeners(ApplicationEventPublisherFactory.java:262)
	at io.micronaut.context.event.ApplicationEventPublisherFactory.access$200(ApplicationEventPublisherFactory.java:60)
	at io.micronaut.context.event.ApplicationEventPublisherFactory$2.publishEvent(ApplicationEventPublisherFactory.java:229)
	at io.micronaut.http.server.netty.NettyHttpServer.lambda$fireStartupEvents$15(NettyHttpServer.java:600)
	at java.base/java.util.Optional.ifPresent(Optional.java:178)
	at io.micronaut.http.server.netty.NettyHttpServer.fireStartupEvents(NettyHttpServer.java:594)
	at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:298)
	at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:104)
	at io.micronaut.runtime.Micronaut.lambda$start$2(Micronaut.java:81)
	at java.base/java.util.Optional.ifPresent(Optional.java:178)
	at io.micronaut.runtime.Micronaut.start(Micronaut.java:79)
	at io.micronaut.runtime.Micronaut.run(Micronaut.java:323)
	at io.micronaut.runtime.Micronaut.run(Micronaut.java:309)
	at io.airbyte.workers.Application.main(Application.java:15)
Caused by: java.lang.IllegalArgumentException: Cannot create enum from ${env:S3_LOG_BUCKET_REGION} value!
	at com.amazonaws.regions.Regions.fromName(Regions.java:105)
	at com.van.logging.aws.S3Configuration.resolveRegion(S3Configuration.java:204)
	at com.van.logging.aws.S3Configuration.setRegion(S3Configuration.java:125)
	at com.van.logging.log4j2.Log4j2AppenderBuilder.getS3ConfigIfEnabled(Log4j2AppenderBuilder.java:172)
	at com.van.logging.log4j2.Log4j2AppenderBuilder.createCachePublisher(Log4j2AppenderBuilder.java:271)
	at com.van.logging.log4j2.Log4j2AppenderBuilder.build(Log4j2AppenderBuilder.java:140)
	... 46 more
ERROR StatusConsoleListener Unable to invoke factory method in class com.van.logging.log4j2.Log4j2Appender for element Log4j2Appender: java.lang.IllegalStateException: No factory method found for class com.van.logging.log4j2.Log4j2Appender
 java.lang.IllegalStateException: No factory method found for class com.van.logging.log4j2.Log4j2Appender
	at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.findFactoryMethod(PluginBuilder.java:268)
	at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:140)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:1162)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:1083)
	at org.apache.logging.log4j.core.appender.routing.RoutingAppender.createAppender(RoutingAppender.java:323)
	at org.apache.logging.log4j.core.appender.routing.RoutingAppender.getControl(RoutingAppender.java:295)
	at org.apache.logging.log4j.core.appender.routing.RoutingAppender.append(RoutingAppender.java:253)
	at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:160)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88)
	at org.apache.logging.log4j.core.appender.rewrite.RewriteAppender.append(RewriteAppender.java:89)
	at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:160)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88)
	at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:705)
	at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:663)
	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:639)
	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:575)
	at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:92)
	at org.apache.logging.log4j.core.Logger.log(Logger.java:169)
	at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2933)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2886)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2868)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2652)
	at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:2400)
	at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:179)
	at io.airbyte.workers.ApplicationInitializer.configureTemporal(ApplicationInitializer.java:322)
	at io.airbyte.workers.ApplicationInitializer.initializeCommonDependencies(ApplicationInitializer.java:177)
	at io.airbyte.workers.ApplicationInitializer.onApplicationEvent(ApplicationInitializer.java:143)
	at io.airbyte.workers.ApplicationInitializer.onApplicationEvent(ApplicationInitializer.java:57)
	at io.micronaut.context.event.ApplicationEventPublisherFactory.notifyEventListeners(ApplicationEventPublisherFactory.java:262)
	at io.micronaut.context.event.ApplicationEventPublisherFactory.access$200(ApplicationEventPublisherFactory.java:60)
	at io.micronaut.context.event.ApplicationEventPublisherFactory$2.publishEvent(ApplicationEventPublisherFactory.java:229)
	at io.micronaut.http.server.netty.NettyHttpServer.lambda$fireStartupEvents$15(NettyHttpServer.java:600)
	at java.base/java.util.Optional.ifPresent(Optional.java:178)
	at io.micronaut.http.server.netty.NettyHttpServer.fireStartupEvents(NettyHttpServer.java:594)
	at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:298)
	at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:104)
	at io.micronaut.runtime.Micronaut.lambda$start$2(Micronaut.java:81)
	at java.base/java.util.Optional.ifPresent(Optional.java:178)
	at io.micronaut.runtime.Micronaut.start(Micronaut.java:79)
	at io.micronaut.runtime.Micronaut.run(Micronaut.java:323)
	at io.micronaut.runtime.Micronaut.run(Micronaut.java:309)
	at io.airbyte.workers.Application.main(Application.java:15)
ERROR StatusConsoleListener Unable to create Appender of type Log4j2Appender
2024-01-25 19:26:34 �[32mINFO�[m i.a.c.t.TemporalUtils(configureTemporalNamespace):123 - Workflow execution TTL already set for namespace default. Remains unchanged as: 30 days
ERROR StatusConsoleListener Could not create plugin of type class com.van.logging.log4j2.Log4j2Appender for element Log4j2Appender: java.lang.RuntimeException: Cannot build appender due to errors
 java.lang.RuntimeException: Cannot build appender due to errors
	at com.van.logging.log4j2.Log4j2AppenderBuilder.build(Log4j2AppenderBuilder.java:147)
	at com.van.logging.log4j2.Log4j2AppenderBuilder.build(Log4j2AppenderBuilder.java:25)
	at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:124)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:1162)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:1083)
	at org.apache.logging.log4j.core.appender.routing.RoutingAppender.createAppender(RoutingAppender.java:323)
	at org.apache.logging.log4j.core.appender.routing.RoutingAppender.getControl(RoutingAppender.java:295)
	at org.apache.logging.log4j.core.appender.routing.RoutingAppender.append(RoutingAppender.java:253)
	at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:160)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88)
	at org.apache.logging.log4j.core.appender.rewrite.RewriteAppender.append(RewriteAppender.java:89)
	at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:160)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124)
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88)
	at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:705)
	at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:663)
	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:639)
	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:575)
	at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:92)
	at org.apache.logging.log4j.core.Logger.log(Logger.java:169)
	at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2933)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2886)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2868)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2652)
	at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:2400)
	at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:179)
	at io.airbyte.commons.temporal.TemporalUtils.configureTemporalNamespace(TemporalUtils.java:123)
	at io.airbyte.workers.ApplicationInitializer.configureTemporal(ApplicationInitializer.java:324)
	at io.airbyte.workers.ApplicationInitializer.initializeCommonDependencies(ApplicationInitializer.java:177)
	at io.airbyte.workers.ApplicationInitializer.onApplicationEvent(ApplicationInitializer.java:143)
	at io.airbyte.workers.ApplicationInitializer.onApplicationEvent(ApplicationInitializer.java:57)
	at io.micronaut.context.event.ApplicationEventPublisherFactory.notifyEventListeners(ApplicationEventPublisherFactory.java:262)
	at io.micronaut.context.event.ApplicationEventPublisherFactory.access$200(ApplicationEventPublisherFactory.java:60)
	at io.micronaut.context.event.ApplicationEventPublisherFactory$2.publishEvent(ApplicationEventPublisherFactory.java:229)
	at io.micronaut.http.server.netty.NettyHttpServer.lambda$fireStartupEvents$15(NettyHttpServer.java:600)
	at java.base/java.util.Optional.ifPresent(Optional.java:178)
	at io.micronaut.http.server.netty.NettyHttpServer.fireStartupEvents(NettyHttpServer.java:594)
	at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:298)
	at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:104)
	at io.micronaut.runtime.Micronaut.lambda$start$2(Micronaut.java:81)
	at java.base/java.util.Optional.ifPresent(Optional.java:178)
	at io.micronaut.runtime.Micronaut.start(Micronaut.java:79)
	at io.micronaut.runtime.Micronaut.run(Micronaut.java:323)
	at io.micronaut.runtime.Micronaut.run(Micronaut.java:309)
	at io.airbyte.workers.Application.main(Application.java:15)
Caused by: java.lang.IllegalArgumentException: Cannot create enum from ${env:S3_LOG_BUCKET_REGION} value!
	at com.amazonaws.regions.Regions.fromName(Regions.java:105)
	at com.van.logging.aws.S3Configuration.resolveRegion(S3Configuration.java:204)
	at com.van.logging.aws.S3Configuration.setRegion(S3Configuration.java:125)
	at com.van.logging.log4j2.Log4j2AppenderBuilder.getS3ConfigIfEnabled(Log4j2AppenderBuilder.java:172)
	at com.van.logging.log4j2.Log4j2AppenderBuilder.createCachePublisher(Log4j2AppenderBuilder.java:271)
	at com.van.logging.log4j2.Log4j2AppenderBuilder.build(Log4j2AppenderBuilder.java:140)
@leonardo-lages-hyperflow leonardo-lages-hyperflow added area/platform issues related to the platform docker needs-triage type/bug Something isn't working labels Jan 25, 2024
@leonardo-lages-hyperflow
Copy link
Author

If we put the region occurs another error that showed in the log below:

Caused by: java.lang.IllegalStateException: Only one of Region or EndpointConfiguration may be set.
at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450)
at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at com.van.logging.aws.AwsClientHelpers.buildClient(AwsClientHelpers.java:88)
at com.van.logging.aws.S3PublishHelper.(S3PublishHelper.java:52)
at com.van.logging.log4j2.Log4j2AppenderBuilder.lambda$createCachePublisher$0(Log4j2AppenderBuilder.java:276)
at java.base/java.util.Optional.ifPresent(Optional.java:178)
at com.van.logging.log4j2.Log4j2AppenderBuilder.createCachePublisher(Log4j2AppenderBuilder.java:271)
at com.van.logging.log4j2.Log4j2AppenderBuilder.build(Log4j2AppenderBuilder.java:140)
... 52 more

@marcosmarxm
Copy link
Member

@jdpgrailsdev is this the problem you had with the latest version of Airbyte and Log4j?

@marcosmarxm
Copy link
Member

I think the solution was done by airbytehq/airbyte-platform@28e4bfc probably need wait a new release of Helm Chart.

@marcosmarxm
Copy link
Member

@leonardo-lages-hyperflow can you share the Chart App version you're using?

@marcosmarxm marcosmarxm changed the title Helm Chart Deployment with external MINIO causing problem [helm] Deployment with external minio: Cannot create enum from ${env:S3_LOG_BUCKET_REGION} Jan 26, 2024
This was referenced Jan 26, 2024
Closed
Closed
@marcosmarxm
Copy link
Member

@leonardo-lages-hyperflow there is a new release platform 0.50.45 and chart 0.50.115 please run some tests with the new version

@marcosmarxm
Copy link
Member

Confirmed the logs are showing in latest Helm version
image

@leonardo-lages-hyperflow
Copy link
Author

leonardo-lages-hyperflow commented Jan 27, 2024 •
edited
Loading

Hi @marcosmarxm that problem was solved Thank you!

But now I'm getting another problem, the worker can't connect to the external minio, as you can see in the log bellow:

2024-01-27 15:56:38 �[32mINFO�[m i.a.w.ApplicationInitializer(registerWorkerFactory):182 - Registering worker factories....
2024-01-27 15:56:39 �[32mINFO�[m i.a.w.ApplicationInitializer(registerGetSpec):278 - Get Spec Workflow registered.
2024-01-27 15:56:39 �[32mINFO�[m i.a.w.ApplicationInitializer(registerCheckConnection):231 - Check Connection Workflow registered.
2024-01-27 15:56:39 �[32mINFO�[m i.a.w.ApplicationInitializer(registerDiscover):264 - Discover Workflow registered.
2024-01-27 15:56:39 �[32mINFO�[m i.a.w.ApplicationInitializer(registerSync):291 - Registering sync workflow for task queue 'SYNC'...
2024-01-27 15:56:39 �[32mINFO�[m i.a.w.ApplicationInitializer(registerSync):301 - Sync Workflow registered.
2024-01-27 15:56:39 �[32mINFO�[m i.a.w.ApplicationInitializer(registerConnectionManager):247 - Connection Manager Workflow registered.
2024-01-27 15:56:39 �[32mINFO�[m i.a.w.ApplicationInitializer(registerNotification):209 - registering new notification workflow
2024-01-27 15:56:39 �[32mINFO�[m i.a.w.ApplicationInitializer(onApplicationEvent):149 - Starting worker factory...
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Workflow Poller taskQueue="DISCOVER_SCHEMA", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Activity Poller taskQueue="DISCOVER_SCHEMA", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Workflow Poller taskQueue="NOTIFY", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Activity Poller taskQueue="NOTIFY", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Workflow Poller taskQueue="CONNECTION_UPDATER", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Activity Poller taskQueue="CONNECTION_UPDATER", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Workflow Poller taskQueue="CHECK_CONNECTION", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Activity Poller taskQueue="CHECK_CONNECTION", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Workflow Poller taskQueue="SYNC", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Activity Poller taskQueue="SYNC", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Workflow Poller taskQueue="GET_SPEC", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.t.i.w.Poller(start):91 - start: Poller{name=Activity Poller taskQueue="GET_SPEC", namespace="default", identity=1@airbyte-worker-69cbb669fc-wdsq2}
2024-01-27 15:56:39 �[32mINFO�[m i.a.w.ApplicationInitializer(onApplicationEvent):152 - Application initialized.
2024-01-27 15:56:39 �[32mINFO�[m i.m.r.Micronaut(lambda$start$2):98 - Startup completed in 17084ms. Server Running: http://airbyte-worker-69cbb669fc-wdsq2:9000
2024-01-27 15:57:33 �[1;31mERROR�[m c.v.l.BufferPublisher(startPublish):38 - Cannot start publish with com.van.logging.aws.S3PublishHelper@fd56c64 due to error
java.lang.RuntimeException: Cannot start publishing: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: null; S3 Extended Request ID: null; Proxy: null)
at com.van.logging.aws.S3PublishHelper.start(S3PublishHelper.java:80) ~[appender-core-5.0.0.jar:?]
at com.van.logging.BufferPublisher.startPublish(BufferPublisher.java:36) ~[appender-core-5.0.0.jar:?]
at com.van.logging.LoggingEventCache.publishEventsFromFile(LoggingEventCache.java:199) ~[appender-core-5.0.0.jar:?]
at com.van.logging.LoggingEventCache.lambda$publishCache$0(LoggingEventCache.java:255) ~[appender-core-5.0.0.jar:?]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: null; S3 Extended Request ID: null; Proxy: null)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5437) ~[aws-java-sdk-s3-1.12.6.jar:?]
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5384) ~[aws-java-sdk-s3-1.12.6.jar:?]
at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1445) ~[aws-java-sdk-s3-1.12.6.jar:?]
at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1381) ~[aws-java-sdk-s3-1.12.6.jar:?]
at com.van.logging.aws.S3PublishHelper.start(S3PublishHelper.java:73) ~[appender-core-5.0.0.jar:?]
... 8 more
2024-01-27 15:57:33 �[1;31mERROR�[m c.v.l.BufferPublisher(endPublish):69 - Cannot end publish with com.van.logging.aws.S3PublishHelper@fd56c64 due to error
java.lang.RuntimeException: Cannot end publishing: Cannot publish to S3: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: null; S3 Extended Request ID: null; Proxy: null)
at com.van.logging.AbstractFilePublishHelper.end(AbstractFilePublishHelper.java:66) ~[appender-core-5.0.0.jar:?]
at com.van.logging.BufferPublisher.endPublish(BufferPublisher.java:67) ~[appender-core-5.0.0.jar:?]
at com.van.logging.LoggingEventCache.publishEventsFromFile(LoggingEventCache.java:210) ~[appender-core-5.0.0.jar:?]
at com.van.logging.LoggingEventCache.lambda$publishCache$0(LoggingEventCache.java:255) ~[appender-core-5.0.0.jar:?]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: java.lang.RuntimeException: Cannot publish to S3: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: null; S3 Extended Request ID: null; Proxy: null)
at com.van.logging.aws.S3PublishHelper.publishFile(S3PublishHelper.java:131) ~[appender-core-5.0.0.jar:?]
at com.van.logging.AbstractFilePublishHelper.end(AbstractFilePublishHelper.java:61) ~[appender-core-5.0.0.jar:?]
... 8 more
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: null; S3 Extended Request ID: null; Proxy: null)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541) ~[aws-java-sdk-core-1.12.510.jar:?]
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5437) ~[aws-java-sdk-s3-1.12.6.jar:?]
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5384) ~[aws-java-sdk-s3-1.12.6.jar:?]
at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:421) ~[aws-java-sdk-s3-1.12.6.jar:?]
at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6508) ~[aws-java-sdk-s3-1.12.6.jar:?]
at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1856) ~[aws-java-sdk-s3-1.12.6.jar:?]
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1816) ~[aws-java-sdk-s3-1.12.6.jar:?]
at com.van.logging.aws.S3PublishHelper.publishFile(S3PublishHelper.java:126) ~[appender-core-5.0.0.jar:?]
at com.van.logging.AbstractFilePublishHelper.end(AbstractFilePublishHelper.java:61) ~[appender-core-5.0.0.jar:?]
... 8 more

@marcosmarxm
Copy link
Member

I recommend to use the default minio for now. There are some work being done to improve the existing chart.

@raphaelauv
Copy link

raphaelauv commented Feb 5, 2024 •
edited
Loading

I make it work airbyte/worker:0.50.47 and helm chart 0.53.52

-> #31988 (comment)

@justenwalker justenwalker mentioned this issue Feb 16, 2024
Closed
1 task
@kelkawi-a kelkawi-a mentioned this issue May 28, 2024
Merged
@octavia-squidington-iii
Copy link
Collaborator

At Airbyte, we seek to be clear about the project priorities and roadmap. This issue has not had any activity for 180 days, suggesting that it's not as critical as others. It's possible it has already been fixed. It is being marked as stale and will be closed in 20 days if there is no activity. To keep it open, please comment to let us know why it is important to you and if it is still reproducible on recent versions of Airbyte.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/platform issues related to the platform community helm kubernetes team/deployments type/bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@timroes @marcosmarxm @davinchia @raphaelauv @octavia-squidington-iii