fix 'uv pip install' handling of gzip'd response and PEP 691

[thundergolfer]

Thank you for writing uv! We're already using it internally on some container image builds and finding that it's noticeably faster 💯

Summary

I was attempting to use uv alongside modal's internal PyPi mirror and ran into some issues. The first issue was the following error:

error: Failed to download: nltk==3.8.1
  Caused by: content-length header is missing from response

This error was coming from within RegistryClient::wheel_metadata_no_pep658. By logging requests on the client (uv) and server (internal mirror) sides I've concluded that it's occurring because uv is sending a header suggesting that it can accept a gzip'd response, but decompressing the gzip'd response strips the content-length header: seanmonstar/reqwest#294.

Logged request, client-side:

0.981664s   0ms  INFO uv_client::registry_client JONO, REQ: Request { method: HEAD, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv4(172.21.0.1)), port: Some(5555), path: "/simple/joblib/joblib-1.3.2-py3-none-any.whl", query: None, fragment: None }, headers: {} }

No headers set explicitly by uv.

Logged request, server-side:

2024-02-26T03:45:08.598272Z DEBUG pypi_mirror: origin request = Request { method: HEAD, uri: /simple/joblib/joblib-1.3.2-py3-none-any.whl, version: HTTP/1.1, headers: {"accept": "*/*", "user-agent": "uv", "accept-encoding": "gzip, br", "host": "172.21.0.1:5555"}, body: Body(Empty) }

Server receives "accept-encoding": "gzip, br",.

My change adding the header to the request fixed this issue. But our internal mirror is just passing through PyPI responses and PyPI responses do contain PEP 658 data, and so wheel_metadata_no_pep658 shouldn't execute.

The issue there is that the PyPi response field has dashes not underscores (https://peps.python.org/pep-0691/).

After changing the alias the PEP 658 codepath now runs correctly :)

Test Plan

I tested by installing against both our mirror and against PyPi:

RUST_LOG="uv=trace" UV_NO_CACHE=true UV_INDEX_URL="http://172.21.0.1:5555/simple" target/release/uv pip install -v nltk
RUST_LOG="uv=trace" UV_NO_CACHE=true UV_INDEX_URL="http://172.21.0.1:5555/simple" target/release/uv pip uninstall -v nltk

target/release/uv pip install -v nltk
target/release/uv pip uninstall -v nltk

[charliermarsh]

Iiinteresting, so when the server returned a gzipped response here, we effectively lost the headers that are required for the range request?

[charliermarsh]

I wonder if this is related: #1754.

[charliermarsh]

Eh, that issue suggests that Content-Type is preserved, nevermind.

[thundergolfer]

Yep they get stripped because they're not accurate after decompression seanmonstar/reqwest#294

[charliermarsh]

Wow that's a huge oversight, thank you! This could make a big difference in cold resolution time actually.

[zanieb]

👀

[charliermarsh]

I'm gonna benchmark this real quick.

[charliermarsh]

The bad news is that we missed that typo and it's embarrassing. The good news is that it makes cold resolution like 30-40% faster for free:

❯ python -m scripts.bench \
        --uv-path ./target/release/uv \
        --uv-path ./target/release/main \
        ./scripts/requirements/jupyter.in --benchmark resolve-cold --min-runs 20
Benchmark 1: ./target/release/uv (resolve-cold)
  Time (mean ± σ):     627.2 ms ±  33.8 ms    [User: 215.1 ms, System: 97.3 ms]
  Range (min … max):   567.0 ms … 719.3 ms    20 runs

Benchmark 2: ./target/release/main (resolve-cold)
  Time (mean ± σ):     838.0 ms ±  38.9 ms    [User: 228.0 ms, System: 110.9 ms]
  Range (min … max):   775.9 ms … 897.9 ms    20 runs

Summary
  './target/release/uv (resolve-cold)' ran
    1.34 ± 0.10 times faster than './target/release/main (resolve-cold)'

[charliermarsh]

Thanks for the great find! I'll release this in the morning.

[zanieb]

Thank you! This is great work