Slowly migrate to Github actions

[binarylogic]

This has been a long-running discussion. We've never been completely happy with CircleCI, but it's never felt like a good use of time to migrate to something else. Primarily because other platforms felt just as risky, it was likely we'd have similar complaints with whatever we moved to. That said, we've been slowly adopting Github actions due to its convenience and it appears to solve some of the frustrations we have with CircleCI:

1. Github Actions allows us to run our own custom runners on our own hardware (such as EC2 instances). This resolves the variety of resource contention issues we've had on CircleCI. The latest being chore(testing): Disable kuberenetes tests in CI temporarily #1629.
2. Github Actions has a marketplace of helpful actions that will reduce our own internal engineering time. We've made use of this through enforcing conventional commit semantics, running cargo audit, etc.
3. We're planning on using Github actions for Automate test harness and integrate into build process #1416.
4. We can compose actions and isolate them, making it easier to manage. This is much better than a single circleci/config.yaml file which is daunting to look at and frustrating to test.
5. It's tightly integrated into our Github workflow.

I'd like to open this issue for discussion. Assuming we all agree we should create a checklist of Github actions we'd like to create over time.

Requirements

• Preserve the existing Dockerized approach we have with Circle right now so that we do not baloon scope.
• Remove the existing Kubernetes tests since we are changing this in chore: Kubernetes Integration RFC #2222.
• Strive to build Vector once for all tests.
• Only run CI if files in relevant directories change (/src, /tests, /proto).

Outstanding Questions

1. How are we handling Windows testing and releasing?
2. Do we need to consider self-hosted runners for larger machines?

[Hoverbear]

I'd love to be part of this task!

[MOZGIII]

I'm doing some work with Github Actions to automate the test harness and make it invocable via the specially formatted PR comment (i.e. something like /test) - see #1416.

I've described the design here, and one of the points there is building vector as the Github Action step. This seems relevant, and we could probably reuse either build step scripts, or the artifacts from the previous builds if those exist.
It's just something to consider. For now I'll go with the simplest way to make it work initially - building the requested commit from scratch every time.

[binarylogic]

Yes, I'd like to also note that #1663 will greatly simplify how we migrate since we'll be mostly decoupling from Circle with the tasks outlined in that issue. That said, this issue is blocked until that work is done.

[MOZGIII]

Found this: https://github.com/nektos/act

[MOZGIII]

Since I linked this act above, it has changed a lot. In short, it works now.

[Hoverbear]

How are we handling Windows testing and releasing?

GA supports windows, this shouldn't be an issue.

[binarylogic]

Noting, we still have the following workflows to transition:

• Installation script checking
• Installation script syncing
• Releasing

[binarylogic]

Closing this since it's mostly done. I will be opening a new issue that is more accurate.