CI connector ops: introduce QA checks in /test

[alafanechere]

What

Closes #21694
I want to run QA checks during the integration test workflow run.
These checks will run on /test and nightly builds.
Their failure will lead to a build failure in the CI.
They are not meant to be run manually by connector developers to test actual connector behavior.
They are run in the CI to asses whether a connector meets all the QA requirements to join our connector catalog.

How

• Introduce the run-qa-checks commands in ci-connector-ops python package
• Create empty functions returning True for all the planned checks
• Run the run-qa-checks command in

  airbyte/tools/bin/ci_integration_test.sh

  Line 35 in 7f5e5d9

  run-qa-checks $connector_name

• Discard the use of virtualenv in

  airbyte/.github/workflows/test-command.yml

  Line 1 in 7f5e5d9

  name: Run Integration Test

  
  They are not required and their absence simplifies the availability of run-qa-checks command in path.

🚨 User Impact 🚨

None: all the checks are forced to successful and yet to be implemented in future PRs.

[alafanechere]

An example of a /test execution on this branch can be found here. Check the head of the Test connectors/source-facebook-marketing logs.

[github-actions]

Airbyte Code Coverage

There is no coverage information present for the Files changed

Total Project Coverage	26.72%	🍏

[erohmensing]

These checks will run on /test and nightly builds.
Their failure will lead to a build failure in the CI.
They are not meant to be run manually by connector developers to test actual connector behavior.
They are run in the CI to asses whether a connector meets all the QA requirements to join our connector catalog.

Just wondering where we plan to run/consume this info in the future - since we're tying it to a /test failure; if we're using it to remove connectors from cloud, does that also then mean we'd remove them if their integration tests fail, even if QA checks are passing?

This is my only open question re: approval

[erohmensing]

Non-blocking, but ere, I find the naming of successful_check a bit confusing, and we're only storing the boolean result, so we have to rely on the print statements to interpret which one failed. WDYT about a dictionary of check_results? {"check_name": bool} e.g. {"check_connector_icon_is_available": True, "check_changelog_entry_is_updated": False}

[alafanechere]

I improved it as you suggested 😄

[alafanechere]

Just wondering where we plan to run/consume this info in the future - since we're tying it to a /test failure; if we're using it to remove connectors from cloud, does that also then mean we'd remove them if their integration tests fail, even if QA checks are passing?

These new checks will be run on each /test command dispatch. This slash command is run manually from PR or during the nightly builds.
These new checks are not meant to decide if a connector gets added or removed from Cloud (this will be a different action based on these test results and other logic).
The role of this PR is only to add a new set of QA checks that are run on connectors built in the CI that will make this build fail if the QA requirements are not met.

[erohmensing]

These new checks are not meant to decide if a connector gets added or removed from Cloud (this will be a different action based on these test results and other logic).

Got it - that was the clarification I needed!