Add --check option to pip-compile, to check if output file is a valid compilation of input files

[khgitting]

What's the problem this feature will solve?

I've been in a number of contexts where we require developers to add requirements to a setup.py or .in file(s), pip-compile, and then commit the changes to both the input and the output files.

In these contexts, it is possible for a human developer to make erroneously make changes to only the input or directly in the output, without pip-compiling. (Or they could do everything correctly locally, pip-compile, and then accidentally exclude one of the files in their commits.)

Most of the time, a reviewer will be able to catch an error like this, but ideally the check would be automated.

Describe the solution you'd like

A --check option for the pip-compile command would, instead of compiling the input requirement files to the output, exit non-zero if one of those human errors has occurred.

eg

$ pip-compile reqs.in -o reqs.txt --check

If reqs.txt is a valid output of compiling reqs.in, exit 0 (and maybe log an affirming message). If reqs.txt is not a valid output of compiling reqs.in, exit 1 (and maybe log a "that's not right" message).

With that option available, developers could include a pip-compile --check command in their CI/CD pipeline, to eliminate the possibility of human errors related to the pip-compile process from being merged into a master branch.

Alternative Solutions

I've been able to add a basic version of this check to CI/CD pipelines. It's just a script called pip-compile-check, with the same interface as pip-compile; it compiles both the input AND the output files together into a new, temporary file, and inspects the annotations therein.

For example, if the expected compilation was pip-compile reqs1.in reqs2.in --output-file reqs.txt, then the check would run pip-compile reqs.txt reqs1.in reqs2.in --output-file /tmp/somefile.txt and inspect that temporary output file.

Since the expected output (reqs.txt) is now an input, it should fully constrain the compilation process, and the # via x, y, z for every requirement in /tmp/somefile.txt should include both reqs.txt AND at least one additional additional source (potentially a secondary/etc source rather than directly from an input file).

If instead a requirement in /tmp/somefile.txt only has # via reqs.txt that means a developer has added a requirement manually to reqs.txt, rather than adding it to an input file and then pip-compiling; similarly, if a requirement in /tmp/somefile.txt only has # via reqs1.in (but not reqs.txt), that means a developer added it to that input file, but did not pip-compile.

Additional context

The solution I describe above works for my purposes (ensuring that inadvertent dependencies additions/subtractions or version changes don't creep in), but it doesn't handle all edge cases.

For example, it's easy to check that compiled requirement come from some input file; it's a bit more work to check if it came from the right file or files. Most of my dependency management flows involve only one input file so this hasn't been an issue for me.

Additionally, I don't know how tracking hashes or other options would come into play in the check feature.

--

Is there interest in this feature? If so, I'd be happy to make a pull request with the basic implementation as I describe above, but I probably don't have the bandwidth to work out all the edge cases. IMO the basic implementation has value on its own, and its limitations could be explained in a help string; but it is incomplete, and I could see y'all having a preference for more complete enhancements.

[atugushev]

Hello @khgitting,

Thanks for the detailed issue! It duplicates #882.