source-mixpanel sync error (datatypes)

[xtreding]

Enviroment

• Airbyte version: 0.30.20-alpha
• OS Version / Instance: macOS
• Deployment: Docker compose
• Source Connector and version: Mixpanel 0.1.1
• Destination Connector and version: Postgres 0.3.10
• Severity: High
• Step where error happened: Sync job

Current Behavior

Mixpanel synchronization failed at the Basic normalization step. Streams cohort_members and engage end with errors e.g. invalid input syntax for type double precision: "89.0.4389.114".

Expected Behavior

All streams should be synced successfully.

Logs

logs-62-0.txt

Steps to Reproduce

1. Create MixPanel source (with cohort_members and engage data)
2. Connect to Postgres
3. Start sync

It seems that some fields in the catalog have different data types in Mixpanel data. e.g. browser_version. There might be more fields types differences. It should be checked.

[marcosmarxm]

Thanks for reporting this @xtreding I added this to our connector roadmap to check it

[midavadim]

@xtreding

Hello,

I have tried to sync mixpanel to postgres and everything went well on my side.

According to the schema for 'engage' stream, the expected type for 'browser_version' field is a string.
And I was able to see entries for 'browser_version' field like '84.0.4147.105' in Postgres db.

As for the error message 'invalid input syntax for type double precision: "89.0.4389.114"' - it means that it tries to insert string value into postgres numeric field (double precision)

The only idea about the reason for this problem is the so-called 'custom' fields in 'engage' stream (which customers could create manually).

What to do:

1. Update source schema and try to sync again:
   in UI: Sync --> Settings --> Press "Update latest source schema"

2. If error still happens, please provide us with a screenshot of 'engage' schema (with a list of all fields)
   in UI: Sync -> Settings --> unfold 'engage' schema

   additional question: Have you had successful mixpanel-postgres sync before?

[sherifnada]

@xtreding could you share the catalog/schema of your data as displayed in Airbyte in as much detail as you can so we can see the individual column types? it should be available from connections --> --> settings, for example:

[xtreding]

Hi @sherifnada!

1. cohort_members:

2. engage:

[midavadim]

Hi @sherifnada!

1. cohort_members:

2. engage:

Hi @xtreding ,

I have additional questions:

1. Have you tried to "Update latest source schema"

2. was it ever working before?

What is interesting, I have tried to add 'custom' properties and its not possible to specify property type and by it sets "string" type by default