Generate //cargo:cargo.bzl during WORKSPACE

[mattmoor]

Having both Cargo.toml and //cargo:crates.bzl checked in forks the source of truth in undesirable ways. There is a trick that rules_python is using to avoid this split, which would enable Bazel to directly consume Cargo.toml (or preferably Cargo.lock).

The pattern is effectively a two-phase load:

load("@somewhere//something:cargo.bzl", "cargo_import")

# This would run cargo-raze to produce @the_repo_name//:crates.bzl
cargo_import(
    name = "the_repo_name",
    file = "Cargo.lock",
)

# Actually evaluate the generated skylark
load("@the_repo_name//:crates.bzl", "raze_fetch_remote_crates")
raze_fetch_remote_crates()

This is basically identical to what you are doing, but avoids checking in crates.bzl by running it during WORKSPACE iff Cargo.lock changes (no more forking the source of truth!).

I also notice that this generates repository names containing exact versions, which could lead to significant churn when dependencies update (e.g. update toml => lock). The way rules_python addresses this is by generating a function in the_repo_name called requirement() that maps the requirement's name to the py_library we generate for it. This enables BUILD files to avoid the versioned dependency names (see example).

[acmcarther]

Having both Cargo.toml and //cargo:crates.bzl checked in forks the source of truth in undesirable ways. There is a trick that rules_python is using to avoid this split, which would enable Bazel to directly consume Cargo.toml (or preferably Cargo.lock).

I had considered this possibility, but wasn't satisfied that it was easy enough to make the cargo-raze binary available without cargo install, as it needs a whole bunch of dependencies of its own in order to be built. Furthermore, I don't think this approach is possible in the vendored case, which most of the people interested in tool have been preferring so far. I'd be interested in looking into this if there was more interest in the non-vendored case though.

I also notice that this generates repository names containing exact versions, which could lead to significant churn when dependencies update (e.g. update toml => lock). The way rules_python addresses this is by generating a function in the_repo_name called requirement() that maps the requirement's name to the py_library we generate for it. This enables BUILD files to avoid the versioned dependency names (see example).

Actually, the exact named directories are not visible outside of the //cargo directory. Users are expected to access their external Cargo dependencies via the aliases declared in //cargo:BUILD, which only export declared dependencies (not transitives) similar to how Cargo does this. Those labels are not explicitly versioned.

That said, this mechanism is very interesting, and might enable per-target dependency specification which is interesting in its own right. I'll read through how this is being done in rules_python in greater detail.

[acmcarther]

Ah one more nuance:

If i understand the rules_python implementation, multiple versions of transitive dependencies are not supported. This is suboptimal for the cargo usecase though, as Cargo explicitly supports bringing in more than one version of a given dependency if it makes the dependency graph easier to resolve. That is to say, something like requirement("some-crate") would probably not be sufficient internally, as a given crate could depend on one version of a dependency, while one of its dependencies or their transitives could depend on some other version of that same dependency. Both versions would need to be brought in.

EDIT: Added "internally" after "sufficient" as in "not sufficient internally", to distinguish from what is exported in the alias file.

[mattmoor]

That's interesting, I hadn't realized that any language but Node.js was doing that.

TIL :)