GitHub workflows exhaust worker disk space

[ashleythedeveloper]

The workflow failures in our GitHub Actions are occurring due to disk space limitations. GitHub only guarantees 14 GB of free disk space on their standard runners, with actual availability varying between workers.

Raw logs

2025-03-06T11:23:26.4867661Z #30 [app dependencies 2/2] RUN yarn install && yarn cache clean
2025-03-06T11:23:41.2914017Z #30 116.4 [5/5] Building fresh packages...
2025-03-06T11:23:57.7509248Z ##[warning]You are running out of disk space. The runner will stop working when the machine runs out of disk space. Free space left: 0 MB
2025-03-06T11:44:56.0239438Z ##[error]The operation was canceled.

Affected workflows

• build_test.yml
• release.yml

Potential Solutions

1. Upgrade GitHub-Hosted Runner

GitHub offers enhanced runner specifications that would resolve this issue, but this feature is only available for Teams and Enterprise accounts.

2. Self-Host the GitHub Runner

We could set up our own runners on cloud infrastructure (AWS, GCP, etc.), giving us complete control over hardware specifications.

3. Free Up Disk Space (not viable)

Implement third-party actions to clear disk space and remove node_modules directories/Docker images after each step. This approach would be somewhat cumbersome as three separate steps require these assets (application build/unit tests, end-to-end tests, and documentation build). The end-to-end test step appears to be the primary trigger for disk space exhaustion.

4. Separate Steps into Different Workflows (not viable)

Split high disk-consuming steps into separate workflows, each using a fresh runner. Initial testing indicates this would work, but it introduces complexity for the release process. Currently, a release is blocked if end-to-end tests fail, but this dependency would be lost with separate workflows. I'm unaware of the ability to depend on the status of another workflow without including it within the same runner.

Awaiting @kshychko's guidance on the preferred approach.

[kshychko]

@ashleythedeveloper, I found this workaround - actions/runner-images#2840 (comment)
Seems like it helps with the space, but all issue_workflow_test tests are failing.