Make our pre-release CI job run on a CRON and open an issue instead of run on each pull request

[choldgraf]

Context

Right now we have a prerelease job that runs in our CI to catch any bugs that are coming in upcoming releases of things:

pydata-sphinx-theme/.github/workflows/tests.yml

Lines 75 to 113 in 2856ae4

	prerelease:
	runs-on: ${{ matrix.os }}
	strategy:
	matrix:
	os: [ubuntu-latest]
	python-version: ["3.8", "3.9", "3.10"]
	steps:
	- uses: actions/checkout@v3
	- name: Set up Python ${{ matrix.python-version }}
	uses: actions/setup-python@v4
	with:
	python-version: ${{ matrix.python-version }}
	cache: "pip"
	cache-dependency-path: "pyproject.toml"
	- name: Install (prerelease) dependencies
	run: |
	python -m pip install --upgrade pip
	python -m pip install --upgrade --pre -e .[test]
	- name: Build docs to store
	run: |
	sphinx-build -b html docs/ docs/_build/html --keep-going -w warnings.txt
	- name: Check that there are no unexpected Sphinx warnings
	if: matrix.python-version == 3.9
	shell: python
	run: |
	from pathlib import Path
	text = Path("./warnings.txt").read_text().strip()
	print("\n=== Sphinx Warnings ===\n\n" + text) # Print just for reference so we can look at the logs
	unexpected = [ii for ii in text.split("\n") if all(i not in ii for i in ["kitchen-sink", "_static", "parse.py"])]
	print("\n=== Unexpected Warnings ===\n\n" + "\n".join(unexpected))
	assert len(unexpected) == 0
	- name: Run the tests
	run: pytest --color=yes

However, it runs in every pull request, and when it fails it makes it look like our CI is failing even though there may not be anything actionable on our end just yet. In this case, we want to know that a pre-release job will fail, but don't necessarily want to block all PRs on it.

Proposal

Let's move our prerelease job to run on a CRON job (say, every week). If it fails, then it should open an issue that says The pre-release jobs are failing along with the error output. We can then use that issue to track changes we need to make for pre-releases, rather than cluttering up our PR CI jobs with a failing test.

To do this, we could do this:

• Put the pre-release job in its own GitHub workflow
• In that workflow, make the trigger a CRON job (or a workflow-dispatch)
• Run the workflow and if unexpected errors happen, use the create an issue github action to generate an issue from the failing job.