Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uv should exclude abi3 wheels in free-threaded Python 3.13+ #2429

Closed
colesbury opened this issue Mar 13, 2024 · 5 comments · Fixed by #2805
Closed

uv should exclude abi3 wheels in free-threaded Python 3.13+ #2429

colesbury opened this issue Mar 13, 2024 · 5 comments · Fixed by #2805
Assignees
Labels
bug Something isn't working

Comments

@colesbury
Copy link

Starting with the version 3.13 alpha releases, CPython can optionally be configured to run without the global interpreter lock. This build configuration ("free-threaded") is not compatible with abi3 wheels.

Additionally, the version specific wheels require the "t" suffix in the ABI tag. For example, cp313-cp313t-manylinux_2_35_x86_64 (free-threaded build) instead of cp313-cp313-manylinux_2_35_x86_64 (default 3.13 build).

Steps to reproduce issue

  1. Build python from source with --disable-gil
  2. uv venv, source .venv/bin/activate
  3. uv pip install psutil
  4. python -c "import psutil" Segmentation fault (core dumped)

Identifying --disable-gil CPython builds

I think the best way is to check if sysconfig.get_config_var("Py_GIL_DISABLED") returns True.

packaging.tags instead checks for a "t" in sys.abiflags, but this is because of constraints on the public API exposed by the package. I think using sysconfig is likely a better option for uv.

See also:

@charliermarsh charliermarsh added the bug Something isn't working label Mar 13, 2024
@charliermarsh
Copy link
Member

Awesome, thank you.

@charliermarsh
Copy link
Member

@konstin - Interested in this one?

@konstin konstin self-assigned this Mar 14, 2024
konstin added a commit that referenced this issue Apr 3, 2024
No-gil/gil-disabled/free-threaded python reintroduces abiflags since it is incompatible with regular native modules and abi3.

Tests: None yet! We're lacking cpython 3.13 no-gil builds we can use in ci.

My test setup:

```
PYTHON_CONFIGURE_OPTS="--enable-shared --disable-gil" pyenv install 3.13.0a5
cargo run -q -- venv -q -p python3.13 .venv3.13 --no-cache-dir && cargo run -q -- pip install -v psutil --no-cache-dir && .venv3.13/bin/python -c "import psutil"
```

Fixes #2429
@konstin
Copy link
Member

konstin commented Apr 3, 2024

@colesbury What is the canonical name for this feature, "no-gil python", "gil disabled" or "free-threaded python"?

As a more technical question, is psutil expected to still error when built from source with python 3.13.0a5? (I can built from main too if that's better)

uv

$ uv venv -q -p python3.13 .venv3.13 --no-cache-dir && uv pip install psutil --no-cache-dir && .venv3.13/bin/python -c "import psutil"
Resolved 1 package in 206ms
Downloaded 1 package in 90ms
Installed 1 package in 1ms
 + psutil==5.9.8
Segmentation fault (core dumped)

pip

$ cargo run -q -- venv -q -p python3.13 .venv3.13 --no-cache-dir && cargo run -q -- pip install psutil --no-cache-dir && .venv3.13/bin/python -c "import psutil"
Resolved 1 package in 225ms
   Built psutil==5.9.8                                                                                                                                                                                                                                                                                                                                                                                                           Downloaded 1 package in 1.51s
Installed 1 package in 0.78ms
 + psutil==5.9.8
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import psutil
  File "/home/konsti/projects/uv/.venv3.13/lib/python3.13/site-packages/psutil/__init__.py", line 102, in <module>
    from . import _pslinux as _psplatform
  File "/home/konsti/projects/uv/.venv3.13/lib/python3.13/site-packages/psutil/_pslinux.py", line 25, in <module>
    from . import _psutil_linux as cext
ImportError: /home/konsti/projects/uv/.venv3.13/lib/python3.13/site-packages/psutil/_psutil_linux.abi3.so: undefined symbol: _Py_atomic_add_ssize

If yes, here's a PR implementing the t abiflag: #2805

@AlexWaygood
Copy link
Member

@colesbury What is the canonical name for this feature, "no-gil python", "gil disabled" or "free-threaded python"?

Not Sam, but as per the Steering Council decision, the canonical name is "free-threaded Python": python/steering-council#221 (comment).

@colesbury
Copy link
Author

As a more technical question, is psutil expected to still error when built from source with python 3.13.0a5...

Oof... yeah I think that's something we need to fix either in CPython or in psutil

konstin added a commit that referenced this issue Apr 12, 2024
No-gil/gil-disabled/free-threaded python reintroduces abiflags since it is incompatible with regular native modules and abi3.

Tests: None yet! We're lacking cpython 3.13 no-gil builds we can use in ci.

My test setup:

```
PYTHON_CONFIGURE_OPTS="--enable-shared --disable-gil" pyenv install 3.13.0a5
cargo run -q -- venv -q -p python3.13 .venv3.13 --no-cache-dir && cargo run -q -- pip install -v psutil --no-cache-dir && .venv3.13/bin/python -c "import psutil"
```

Fixes #2429
konstin added a commit that referenced this issue Apr 12, 2024
freethreaded python reintroduces abiflags since it is incompatible with
regular native modules and abi3.

Tests: None yet! We're lacking cpython 3.13 no-gil builds we can use in
ci.

My test setup:

```
PYTHON_CONFIGURE_OPTS="--enable-shared --disable-gil" pyenv install 3.13.0a5
cargo run -q -- venv -q -p python3.13 .venv3.13 --no-cache-dir && cargo run -q -- pip install -v psutil --no-cache-dir && .venv3.13/bin/python -c "import psutil"
```

Fixes #2429
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants