how does mach-nix resolve urls and hashes

[milahu]

there is mach-nix#how-does-mach-nix-work

As core for the resolving resolvelib is used: https://github.com/sarugaku/resolvelib

Mach-nix supports multiple providers to retrieve python packages from. The user can specify which providers should be preferred. Packages from different providers can be mixed.

but it does not explain how mach-nix resolves urls and hashes of the sdist/wheel/conda files

from the nix point of view, that is the magic part
(assuming that fetchurl is only allowed if we know the hash before downloading)

[FRidh]

A database is downloaded first that contains the hashes of all files on PyPI https://github.com/DavHau/pypi-deps-db. Furthermore, that metadata also contains the dependencies for at least some packages, but I don't know whether mach-nix uses that or not.

[milahu]

that's what i thought too, but pypi-deps-db does not have the hashes

manually reading through pypi-deps-db, i see only dependencies like

  "arithmat": {
    "py2": {
      "1.0.0": {
        "arithmat-1.0.0-py2-none-any.whl": {
          "requires_dist": [
            "sage-package",
            "networkx (~=2.2)"
          ],
          "requires_python": ">=2.7, <3"
        }
      },

grep -r sha256 gives only false positives

$ grep -r sha256
...
sdist/dd.json:            "http-parser@ https://github.com/adamnovak/http-parser/archive/190a17839ba229c635b59d960579451a81fe73f3.zip#sha256=3d30c84a426627e468657c44de199daee9d3210a48e392d4ad2e7497c5010949",
sdist/dd.json:            "http-parser@ https://github.com/adamnovak/http-parser/archive/190a17839ba229c635b59d960579451a81fe73f3.zip#sha256=3d30c84a426627e468657c44de199daee9d3210a48e392d4ad2e7497c5010949",
sdist/ee.json:          "apsw@ https://github.com/rogerbinns/apsw/releases/download/3.31.1-r1/apsw-3.31.1-r1.zip#sha256=dd7913cc6e547ac584aee369af3c76a0993892c9dced6fc908771d66322d57bd"
sdist/ee.json:          "apsw@ https://github.com/rogerbinns/apsw/releases/download/3.31.1-r1/apsw-3.31.1-r1.zip#sha256=dd7913cc6e547ac584aee369af3c76a0993892c9dced6fc908771d66322d57bd"
sdist/ee.json:          "apsw@ https://github.com/rogerbinns/apsw/releases/download/3.31.1-r1/apsw-3.31.1-r1.zip#sha256=dd7913cc6e547ac584aee369af3c76a0993892c9dced6fc908771d66322d57bd"

sorry im too lazy to "just read the source" ^^
but also, i feel this is worth documenting

[DavHau]

The hashes are coming from https://github.com/DavHau/nix-pypi-fetcher
Each pypi-deps-db references a specific version of nix-pypi-fetcher via ./https://github.com/DavHau/pypi-deps-db/blob/master/PYPI_FETCHER_COMMIT and https://github.com/DavHau/pypi-deps-db/blob/master/PYPI_FETCHER_SHA256

[DavHau]

Documentation PRs are very welcome.