add pip-compatible --group flag to uv pip install and uv pip compile

[Gankra]

This is a minimal redux of #10861 to be compatible with uv pip.

This implements the interface described in: pypa/pip#13065 (comment) for uv pip install and uv pip compile. Namely --group <[path:]name>, where path when not defined defaults to pyproject.toml.

In that interface they add --group to pip install, pip download, and pip wheel. Notably we do not define uv pip download and uv pip wheel, so for parity we only need to implement uv pip install. However, we also support uv pip compile which is not part of pip itself, and --group makes sense there too.

---

The behaviour of --group for uv pip commands makes sense for the cases upstream pip supports, but has confusing meanings in cases that only we support (because reading pyproject.tomls is New Tech to them but heavily supported by us). Specifically case (h) below is a concerning footgun, and case (e) below may get complaints from people who aren't well-versed in dependency-groups-as-they-pertain-to-wheels.

Only Group Flags

Group flags on their own work reasonably and uncontroversially, except perhaps that they don't do very clever automatic project discovery.

a) uv pip install --group path/to/pyproject.toml:mygroup pulls up path/to/project.toml and installs all the packages listed by its mygroup dependency-group (essentially treating it like another kind of requirements.txt). In this regard it functions similarly to --only-group in the rest of uv's interface.

b) uv pip install --group mygroup is just sugar for uv pip install --group pyproject.toml:mygroup (note that no project discovery occurs, upstream pip simply hardcodes the path "pyproject.toml" here and we reproduce that.)

c) uv pip install --group a/pyproject.toml:groupx --group b/pyproject.toml:groupy, and any other instance of multiple --group flags, can be understood as completely independent requests for the given groups at the given files.

Groups With Named Packages

Groups being mixed with named packages also work in a fairly unsurprising way, especially if you understand that things like dependency-groups are not really supposed to exist on pypi, they're just for local development.

d) uv pip install mypackage --group path/to/pyproject.toml:mygroup much like multiple instances of --group the two requests here are essentially completely independent: pleases install mypackage, and please also install path/to/pyproject.toml:mygroup.

e) uv pip install mypackage --group mygroup is exactly the same, but this is where it becomes possible for someone to be a little confused, as you might think mygroup is supposed to refer to mypackage in some way (it can't). But no, it's sourcing pyproject.toml:mygroup from the current working directory.

Groups With Requirements/Sourcetrees/Editables

Requirements and sourcetrees are where I expect users to get confused. It behaves exactly the same as it does in the previous sections but you would absolutely be forgiven for expecting a different behaviour. Especially because --group with the rest of uv does do something different.

f) uv pip install -r a/pyproject.toml --group b/pyproject.toml:mygroup is again just two independent requests (install a/pyproject.toml's dependencies, and b/pyproject.toml's mygroup).

g) uv pip install -r pyproject.toml --group mygroup is exactly like the previous case but incidentally the two requests refer to the same file. What the user wanted to happen is almost certainly happening, but they are likely getting "lucky" here that they're requesting something simple.

h) uv pip install -r a/pyproject.toml --group mygroup is again exactly the same but the user is likely to get surprised and upset as this invocation actually sources two different files (install a/pyproject.toml's dependencies, and pyproject.toml's mygroup)! I would expect most people to assume the --group flag here is covering all applicable requirements/sourcetrees/editables, but no, it continues to be a totally independent reference to a file with a hardcoded relative path.

---

Fixes #8590
Fixes #8969

[Gankra]

I started doing a cleanup pass to compute a BTreeMap<PathBuf, Vec<GroupName>> that would be used by subsequent steps but it was just more code and value threading for no real benefit over the current impl that just does a filter(group.path == path) later.

[henryiii]

FYI, pip support was merged.

[zanieb]

Are these conflicts present in the pip implementation?

[zanieb]

(it doesn't really look like it?)

[Gankra]

I thought -r was exclusive to our CLI (I didn't check the others though).

[zanieb]

Yeah -r is exclusive to us, but I'd expect pip install -e . --group foo and pip install anyio --group foo to work unless they explicitly choose for it not to upstream.

[zanieb]

(Whether -r should have the same semantics as these other options, I'm not sure)

[zanieb]

One problem is that uv pip install -r pyproject.toml --extra foo is the "correct" use of --extra but uv pip install -r pyproject.toml --group foo would be invalid? Even more confusing, if we allow it, uv pip install -r foo/pyproject.toml --group bar would install bar from ./pyproject.toml instead of foo/pyproject.toml.

[Gankra]

Yeah that's part of why I opted to disallow it. The upstream pip semantics are really confusing to pair with anything else.

[zanieb]

I think it's important that we're compatible with pip on whether --group can be used with <package> and --editable though, which introduces all the same complexities.

[Gankra]

I implemented/pushed the following changes, because I needed to wrap my head around them in code/tests to think them through semantically:

• totally unrestrict --group from all other sources in uv-pip-install
  • semantically i still have it that --group just installs that group in a pyproject.toml (still hardcoded ./pyproject.toml default for now, see below). All the following cases are accepted:
    • "unrelated": uv pip install somepackage --group bar (essentially sugar for two separate uv pip install invocations as it is in upstream pip install)
    • "nice": uv pip install -r pyproject.toml --group bar (installs both the package's deps and its group bar) (but this is the user of the cli potentially getting "lucky")
    • "confusing": uv pip install -r foo/project.toml --group bar (installs foo/pyproject.toml's deps and ./pyproject.toml:bar)
  • implementation-wise this was a bit hairy as we have to kind of detect if we're doing --group or --only-group by looking at all the other inputs, but it ends up making sense (really it's always --only-group-but-additive-only... it's just an artifact of the internal source_tree stuff and wanting to dedupe files that we need to do anything complex).
• enable the same semantics for uv-pip-compile
  • ...this one I "implemented" and then I went to test it and realized that the semantics for pip-install are absolute nonsense here, i think? i think the old-old-old impl from last month where the --group just applies to every selected source is what you want here..? I'm not totally sure, I don't have a good intuition of what uses of this would look like... but I get the impression that uv pip compile --group pyproject.toml:foo is essentially gibberish? Or am I wrong?
• did not add project discovery in lieu of hardcoding pyproject.toml -- I was going to do this but then I realized that nowhere in the pip interface do we ever seem to invoke project discovery? Which was surprising. This seems like a big semantic departure if we start doing that? At very least needs to be discussed more.

[Gankra]

Zanie convinced me that pip-compile is fine actually, so I just loosened the CLI restriction to make it accepted.

I'm just adding more tests now.