2021-10-29 06:00:29 INFO () WorkerRun(call):42 - Executing worker wrapper. Airbyte version: 0.30.18-alpha 2021-10-29 06:00:29 INFO () TemporalAttemptExecution(get):94 - Executing worker wrapper. Airbyte version: 0.30.18-alpha 2021-10-29 06:00:29 WARN () Databases(createPostgresDatabaseWithRetry):38 - Waiting for database to become available... 2021-10-29 06:00:29 INFO () JobsDatabaseInstance(lambda$static$2):25 - Testing if jobs database is ready... 2021-10-29 06:00:29 INFO () Databases(createPostgresDatabaseWithRetry):55 - Database available! 2021-10-29 06:00:29 INFO () DefaultReplicationWorker(run):82 - start sync worker. job id: 1186 attempt id: 0 2021-10-29 06:00:29 INFO () DefaultReplicationWorker(run):91 - configured sync modes: {null.email_send=full_refresh - append} 2021-10-29 06:00:29 INFO () DefaultAirbyteDestination(start):58 - Running destination... 2021-10-29 06:00:29 INFO () LineGobbler(voidCall):65 - Checking if airbyte/destination-******** exists... 2021-10-29 06:00:29 INFO () LineGobbler(voidCall):65 - airbyte/destination-******** was found locally. 2021-10-29 06:00:29 INFO () DockerProcessFactory(create):127 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/1186/0 --network host --log-driver none airbyte/destination-******** write --config destination_config.json --catalog destination_catalog.json 2021-10-29 06:00:29 INFO () LineGobbler(voidCall):65 - Checking if airbyte/source-******** exists... 2021-10-29 06:00:29 INFO () LineGobbler(voidCall):65 - airbyte/source-******** was found locally. 2021-10-29 06:00:29 INFO () DockerProcessFactory(create):127 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/1186/0 --network host --log-driver none airbyte/source-******** read --config source_config.json --catalog source_catalog.json --state input_state.json 2021-10-29 06:00:29 INFO () DefaultReplicationWorker(run):119 - Waiting for source thread to join. 2021-10-29 06:00:29 INFO () DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):226 - Destination output thread started. 2021-10-29 06:00:29 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):190 - Replication thread started. 2021-10-29 06:00:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:00:31 INFO i.a.i.b.IntegrationRunner(run):96 - {} - Running integration: io.airbyte.integrations.destination.s3.S3Destination 2021-10-29 06:00:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:00:31 INFO i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2021-10-29 06:00:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:00:31 INFO i.a.i.b.IntegrationRunner(run):100 - {} - Command: WRITE 2021-10-29 06:00:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:00:31 INFO i.a.i.b.IntegrationRunner(run):101 - {} - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2021-10-29 06:00:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:00:31 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2021-10-29 06:00:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:00:31 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2021-10-29 06:00:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:00:31 INFO i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):42 - {} - S3 format config: {"format_type":"JSONL"} 2021-10-29 06:00:32 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:00:32 INFO i.a.i.d.s.j.S3JsonlWriter():70 - {} - Full S3 path for stream 'email_send': *********** 2021-10-29 06:00:32 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:00:32 INFO a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 - {} - Initiated multipart upload to *********** with full ID ************* 2021-10-29 06:52:17 INFO () DefaultAirbyteStreamFactory(internalLog):90 - Starting syncing SourceIterable 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - Traceback (most recent call last): 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "*********", line 33, in 2021-10-29 06:52:17 INFO () DefaultAirbyteStreamFactory(internalLog):90 - Syncing stream: email_send 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - launch(source, sys.argv[1:]) 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 117, in launch 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - for message in source_entrypoint.run(parsed_args): 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 108, in run 2021-10-29 06:52:17 ERROR () DefaultAirbyteStreamFactory(internalLog):88 - Encountered an exception while reading stream SourceIterable Traceback (most recent call last): File "**********", line 110, in read logger=logger, stream_instance=stream_instance, configured_stream=configured_stream, connector_state=connector_state File "**********", line 135, in _read_stream for record in record_iterator: File "**********", line 181, in _read_full_refresh for record in records: File "**********", line 228, in read_records response = self._send_request(request) File "**********", line 94, in retry ret = target(*args, **kwargs) File "**********", line 94, in retry ret = target(*args, **kwargs) File "**********", line 193, in _send_request response: requests.Response = self._session.send(request) File "**********", line 697, in send r.content File "**********", line 831, in content self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b'' MemoryError 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - for message in generator: 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 114, in read 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - raise e 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 110, in read 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - logger=logger, stream_instance=stream_instance, configured_stream=configured_stream, connector_state=connector_state 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 135, in _read_stream 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - for record in record_iterator: 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 181, in _read_full_refresh 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - for record in records: 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 228, in read_records 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - response = self._send_request(request) 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 94, in retry 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - ret = target(*args, **kwargs) 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 94, in retry 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - ret = target(*args, **kwargs) 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 193, in _send_request 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - response: requests.Response = self._session.send(request) 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 697, in send 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - r.content 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - File "**********", line 831, in content 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b'' 2021-10-29 06:52:17 ERROR () LineGobbler(voidCall):65 - MemoryError 2021-10-29 06:52:18 INFO () DefaultReplicationWorker(run):121 - Source thread complete. 2021-10-29 06:52:18 INFO () DefaultReplicationWorker(run):122 - Waiting for destination thread to join. 2021-10-29 06:52:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:52:18 INFO i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):80 - {} - Airbyte message consumer: succeeded. 2021-10-29 06:52:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:52:18 INFO i.a.i.d.s.w.BaseS3Writer(close):118 - {} - Uploading remaining data for stream 'email_send'. 2021-10-29 06:52:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:52:18 INFO a.m.s.MultiPartOutputStream(close):158 - {} - Called close() on [MultipartOutputStream for parts 1 - 10000] 2021-10-29 06:52:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:52:18 INFO a.m.s.MultiPartOutputStream(close):158 - {} - Called close() on [MultipartOutputStream for parts 1 - 10000] 2021-10-29 06:52:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:52:18 WARN a.m.s.MultiPartOutputStream(close):160 - {} - [MultipartOutputStream for parts 1 - 10000] is already closed 2021-10-29 06:52:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:52:18 INFO a.m.s.StreamTransferManager(complete):395 - {} - [Manager uploading to *********** with id *********]: Completed 2021-10-29 06:52:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:52:18 INFO i.a.i.d.s.w.BaseS3Writer(close):120 - {} - Upload completed for stream 'email_send'. 2021-10-29 06:52:18 INFO () JsonSchemaValidator(test):56 - JSON schema validation failed. errors: $: null found, object expected 2021-10-29 06:52:18 ERROR () DefaultAirbyteStreamFactory(lambda$create$1):63 - Validation failed: null 2021-10-29 06:52:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):53 - 2021-10-29 06:52:18 INFO i.a.i.b.IntegrationRunner(run):153 - {} - Completed integration: io.airbyte.integrations.destination.s3.S3Destination 2021-10-29 06:52:18 INFO () DefaultReplicationWorker(run):124 - Destination thread complete. 2021-10-29 06:52:18 ERROR () DefaultReplicationWorker(run):128 - Sync worker failed. io.airbyte.workers.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled. at io.airbyte.workers.protocols.airbyte.DefaultAirbyteSource.close(DefaultAirbyteSource.java:115) ~[io.airbyte-airbyte-workers-0.30.18-alpha.jar:?] at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:126) ~[io.airbyte-airbyte-workers-0.30.18-alpha.jar:?] at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:32) ~[io.airbyte-airbyte-workers-0.30.18-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$1(TemporalAttemptExecution.java:145) ~[io.airbyte-airbyte-workers-0.30.18-alpha.jar:?] at java.lang.Thread.run(Thread.java:832) [?:?] 2021-10-29 06:52:18 INFO () DefaultReplicationWorker(run):152 - sync summary: io.airbyte.config.ReplicationAttemptSummary@6ff70434[status=failed,recordsSynced=0,bytesSynced=0,startTime=1635487229532,endTime=1635490338698] 2021-10-29 06:52:18 INFO () DefaultReplicationWorker(run):161 - Source did not output any state messages 2021-10-29 06:52:18 WARN () DefaultReplicationWorker(run):169 - State capture: No new state, falling back on input state: io.airbyte.config.State@380a8029[state={}] 2021-10-29 06:52:18 INFO () TemporalAttemptExecution(get):115 - Stopping cancellation check scheduling... 2021-10-29 06:52:18 INFO () SyncWorkflow$ReplicationActivityImpl(replicate):177 - sync summary: io.airbyte.config.StandardSyncOutput@15016098[standardSyncSummary=io.airbyte.config.StandardSyncSummary@523c37ff[status=failed,recordsSynced=0,bytesSynced=0,startTime=1635487229532,endTime=1635490338698],state=io.airbyte.config.State@380a8029[state={}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@1f3b546a[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@f54adfa[stream=io.airbyte.protocol.models.AirbyteStream@114f1928[name=email_send,jsonSchema={"type":["null","object"],"properties":{"email":{"type":["null","string"]},"channelId":{"type":["null","integer"]},"contentId":{"type":["null","integer"]},"createdAt":{"type":["null","string"],"format":"date-time"},"messageId":{"type":["null","string"]},"campaignId":{"type":["null","integer"]},"templateId":{"type":["null","integer"]},"itblInternal":{"type":["null","object"],"properties":{"documentCreatedAt":{"type":["null","string"],"format":"date-time"},"documentUpdatedAt":{"type":["null","string"],"format":"date-time"}}},"messageBusId":{"type":["null","string"]},"messageTypeId":{"type":["null","integer"]},"transactionalData":{"type":["null","object"],"properties":{"id":{"type":["null","string"]},"sku":{"type":["null","string"]},"url":{"type":["null","string"]},"name":{"type":["null","string"]},"email":{"type":["null","string"]},"price":{"type":["null","integer"]},"handle":{"type":["null","string"]},"vendor":{"type":["null","string"]},"discount":{"type":["null","integer"]},"imageUrl":{"type":["null","string"]},"createdAt":{"type":["null","string"],"format":"date-time"},"eventName":{"type":["null","string"]},"inventory":{"type":["null","integer"]},"campaignId":{"type":["null","integer"]},"categories":{"type":["null","array"],"items":{}},"product_id":{"type":["null","string"]},"templateId":{"type":["null","integer"]},"description":{"type":["null","string"]},"itblInternal":{"type":["null","object"],"properties":{"documentCreatedAt":{"type":["null","string"],"format":"date-time"},"documentUpdatedAt":{"type":["null","string"],"format":"date-time"}}},"product_type":{"type":["null","string"]},"eventUpdatedAt":{"type":["null","string"],"format":"date-time"},"compare_at_price":{"type":["null","number"]}}}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[createdAt],sourceDefinedPrimaryKey=[[id]],namespace=,additionalProperties={}],syncMode=full_refresh,cursorField=[createdAt],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}]],additionalProperties={}]]