Switch from CircleCI to GitHub Actions

[MattiSG]

RFC

This RFC suggests switching the default CI provider for OpenFisca codebases from CircleCI to GitHub Actions, enabling a 5x performance gain.

Context

Currently, OpenFisca tests in continuous integration run with CircleCI’s free plan. This is the case for OpenFisca Core, but also for the Country Template, and for most known country packages (since they derive from the Country Template).

With the main aim to decrease the duration of France's longer test runs (between 10 and 12 minutes per build), an investigation was led to identify ways to speed up test that could benefit the entire community in #1027.

Some minor ways for improvement were identified and implemented (openfisca/openfisca-france#1649, openfisca/openfisca-france#1650) and relevant ones are or soon will be shared back in the Country Template (openfisca/country-template#113).

However, by far the most effective way to speed up the tests that was identified is to parallelise them.

There is some exploratory work to do this for local tests in #1025. In CI, however, parallelisation has been effective for 4 years (since #516).

Current situation

With CircleCI's free plan for open-source, we can run builds with up to 4x parallelism.

Expected situation

With GitHub Action's free plan for opens-source, we would run builds with up to 20x parallelism.

Expected gains

• 5 times faster test runs, as measured and evidenced in Improve tests performance #1027 (comment).
• Deeper integration within GitHub.

Expected costs

• Writing a new CI setup for both Core and the Country Template. This has already been covered a lot in Improve tests performance #1027’s exploration, and has been volunteered by @openfisca/france-contrib-dinsic (in particular @HAEKADI).
• Setting up the new CI pipeline. This has already been covered a lot in Improve tests performance #1027’s exploration, and has been volunteered by @openfisca/international-maintainers (in particular @sandcha).

Expected losses

• Detailed logs from test runs won't be retained after 90 days.

Workarounds

• Rely on worflow-level metadata (which is kept indefinitely) for performance decisions made on test runs, and learn to copy-paste in the issue body the subset of the logs when we make a decision based on them.

More information

• A detailed comparison table is available in Improve tests performance #1027 (comment).
• The CircleCI account and pipeline will stay active for the foreseeable future, letting each country package migrate on its own schedule.

Decision-making process

• Please agree with (👍 reaction) or reject (👎 reaction) this proposal.
• If rejecting, please add a comment explaining your position.
• This RFC will stay open for 7 days (7 ⨉ 24 hours) after the last rejection comment (or after this initial post if no rejection comments are added).
• When the RFC closes, if there is no consensus, votes will be taken as emoji reactions on this first post.
• If there is a consensus or, if not, the majority votes for this change, a PR will be opened with code enabling the switch.

[benoit-cty]

Good idea, in addition Github Action has a local runner : https://github.com/nektos/act Maybe it could also speed up local test.

[MattiSG]

This RFC will close in 53 hours. For the moment, the consensus is in favour of it, with no change.

[MaxGhenis]

We've moved everything to GH Actions at both the Policy Simulation Library and the UBI Center, very happy with it!

[bonjourmauko]

I think this will be a great improvement! Thanks @MattiSG , @HAEKADI , et al. 👍

[MattiSG]

This RFC has been adopted, thanks to everyone who participated! 😃
Locking this thread to make sure the votes stay still 🙂

[MattiSG]

This was implemented in the Core ecosystem with #1057 on Core, and openfisca/country-template#115 on Country Template.

Extension Template, doc and openfisca.org are still on CircleCI. Moving them to GitHub Actions would be welcome for consistency, but this is not a major issue, as the main aim of this change was to increase tests performance. This has been delivered to country packages with #1057. This RFC has thus been implemented, and it will stay here for future reference.

• France has implemented this improvement in Passage de CircleCI vers GitHub Actions openfisca-france#1663.
• UK also uses GitHub Actions.