Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consistent python version checks and troubleshooting #42944

Merged
merged 1 commit into from
Oct 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions dev/breeze/doc/04_troubleshooting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,38 @@ describe your problem.
stated in `This comment <https://github.com/moby/moby/issues/43361#issuecomment-1227617516>`_ and allows to
run Breeze with no problems.

Cannot import name 'cache' or Python >=3.9 required
---------------------------------------------------

When you see this error:

.. code-block::

ImportError: cannot import name 'cache' from 'functools' (/Users/jarek/Library/Application Support/hatch/pythons/3.8/python/lib/python3.8/functools.py)

or

.. code-block::

ERROR: Package 'blacken-docs' requires a different Python: 3.8.18 not in '>=3.9'


It means that your pre-commit hook is installed with (already End-Of-Life) Python 3.8 and you should reinstall
it and clean pre-commit cache.

This can be done (if you use ``pipx`` to install ``pre-commit``):

.. code-block:: bash

pipx uninstall pre-commit
pipx install pre-commit --python $(which python3.9) --force
pre-commit clean
pre-commit install

If you installed ``pre-commit`` differently, you should remove and reinstall
it (and clean cache) in the way you installed it.


Bad Interpreter Error
---------------------

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -797,7 +797,7 @@ def prepare_provider_packages(
skip_tag_check: bool,
version_suffix_for_pypi: str,
):
check_python_version()
check_python_version(release_provider_packages=True)
perform_environment_checks()
fix_ownership_using_docker()
cleanup_python_generated_files()
Expand Down
12 changes: 5 additions & 7 deletions dev/breeze/src/airflow_breeze/utils/python_versions.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,12 @@ def get_python_version_list(python_versions: str) -> list[str]:
return python_version_list


def check_python_version():
error = False
if not sys.version_info >= (3, 9):
get_console().print("[error]At least Python 3.9 is required to prepare reproducible archives.\n")
error = True
if error:
def check_python_version(release_provider_packages: bool = False):
if not sys.version_info < (3, 12) and release_provider_packages:
get_console().print("[error]Python 3.12 is not supported.\n")
get_console().print(
"[warning]Please reinstall Breeze using Python 3.9 - 3.11 environment.[/]\n\n"
"[warning]Please reinstall Breeze using Python 3.9 - 3.11 environment because not all "
"provider packages support Python 3.12 yet.[/]\n\n"
"For example:\n\n"
"pipx uninstall apache-airflow-breeze\n"
"pipx install --python $(which python3.9) -e ./dev/breeze --force\n"
Expand Down
5 changes: 4 additions & 1 deletion providers/src/airflow/providers/apache/beam/provider.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,10 @@ additional-extras:
- apache-beam[gcp]

# Apache Beam currently does not support Python 3.12
# There is an issue tracking it https://github.com/apache/beam/issues/29149
# There is an issue tracking it https://github.com/apache/beam/issues/29149.
# While the original issue above is closed, Apache Beam still does not support Python 3.12
# because the dill version used by them break our PythonVirtualenvOperator when dill is enabled
# See https://github.com/apache/beam/issues/32617
excluded-python-versions: ['3.12']

integrations:
Expand Down