Replace custom Finder with Resolver in pip sync
#2694
Assignees
Labels
internal
A refactor or improvement that is not user-facing
Comments
charliermarsh
added
the
internal
charliermarsh
added a commit
that referenced
this issue
Mar 27, 2024
## Summary This PR removes the custom `DistFinder` that we use in `pip sync`. This originally existed because `VersionMap` wasn't lazy, and so we saved a lot of time in `DistFinder` by reading distribution data lazily. But now, AFAICT, there's really no benefit. Maintaining `DistFinder` means we effectively have to maintain two resolvers, and end up fixing bugs in `DistFinder` that don't exist in the `Resolver` (like #2688. Closes #2694. Closes #2443. ## Test Plan I ran this benchmark a bunch. It's basically a wash. Sometimes one is faster than the other. ``` ❯ python -m scripts.bench \ --uv-path ./target/release/main \ --uv-path ./target/release/uv \ scripts/requirements/compiled/trio.txt --min-runs 50 --benchmark install-warm --warmup 25 Benchmark 1: ./target/release/main (install-warm) Time (mean ± σ): 54.0 ms ± 10.6 ms [User: 8.7 ms, System: 98.1 ms] Range (min … max): 45.5 ms … 94.3 ms 50 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. Benchmark 2: ./target/release/uv (install-warm) Time (mean ± σ): 50.7 ms ± 9.2 ms [User: 8.7 ms, System: 98.6 ms] Range (min … max): 44.0 ms … 98.6 ms 50 runs Warning: The first benchmarking run for this command was significantly slower than the rest (77.6 ms). This could be caused by (filesystem) caches that were not filled until after the first run. You should consider using the '--warmup' option to fill those caches before the actual benchmark. Alternatively, use the '--prepare' option to clear the caches before each timing run. Summary './target/release/uv (install-warm)' ran 1.06 ± 0.29 times faster than './target/release/main (install-warm)' ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We see bug from time-to-time that only manifest themselves in
pip sync. The issue is thatpip syncuses a custom mini-resolver to take advantage of the fact that we expect versions to be pinned.I think the resolver is now optimized enough that this shouldn't really matter. Let's try removing the
Finderand doing some basic benchmarking.The text was updated successfully, but these errors were encountered: