[EPIC] Report failure stack traces via sentry

[sherifnada]

Description

The goal of this ticket is to ensure that we are able to receive stack traces from failing connectors (OSS and Cloud) in Sentry.

TODOs

• [P0] Integrate Sentry to GA Java destination connectors.
  • Experiment with Sentry performance monitoring for BigQuery destination #9328
  • Integrate Sentry to Snowflake destination #9533
• [P0] Integrate Sentry to GA Python source connectors.
  • This is partially completed. But not every Python connector exception is sent to Sentry yet.
• [P1] Inject any needed env variables to the connector.
  • Move the environment variable to .env.
  • Propagate the variable to worker and the connector.
• [P1] Enable suspect commits feature on Sentry.
  • Add Sentry release when publishing a connector
• [P2] Set an environment to distinguish production from dev
• [P2] Integrate Sentry to all other destination connectors.
  • [JAVA][SENTRY] Integrate all Java-Based connectors with Sentry #9867
• [P2] Integrate Sentry to all other source connectors.
• [P0] Publish new connector versions with Sentry enabled.
  • destination-e2e-test Add sentry release for connectors #9940
  • destination-dev-null Add sentry release for connectors #9940
  • source-e2e-test
  • source-e2e-test-cloud
  • destination-bigquery 🎉 Destination snowflake & bigquery: integrate with sentry #9945
  • destination-bigquery-denormalized 🎉 Destination snowflake & bigquery: integrate with sentry #9945
  • destination-snowflake 🎉 Destination snowflake & bigquery: integrate with sentry #9945

Acceptance criteria

• All GA connector exceptions, from both OSS and Cloud, will be sent to Sentry.

Timeline

• P0 issues: Jan 28th.

Reference

• Related tickets can be found here

[sherifnada]

@augan-rymkhan @VasylLazebnyk apologies for the confusion -- this should be a platform issue

[tuliren]

The project to integrate Sentry to Java connectors has been delegated to GL. It is already tracked here: #9328

[tuliren]

Sprint Summary

• Made solid progress on integrating Sentry with Java destination.
• MVP should be ready by the end of this week.

[tuliren]

Sprint Summary

• Sentry integration is not completely done as planned.
• Local test of Sentry integration works as expected.
• Still working on deploying it to cloud.

[tuliren]

Sprint summary

• Stack trace and performance monitoring is working for Snowflake and BigQuery on Sentry now.

Next step

• [P0] Activate Sentry for Python GA connectors.
• [P2] Propagate env variable properly. The current approach is hacky. But this is working, so the priority is low.

[tuliren]

Close because we will integration with Datadog instead of Sentry.