Switching between "cargo check" and "cargo build <with flags>" keeps rebuilding dependencies

[RalfJung]

Problem

In Miri, switching between debug-building the project and checking it rebuilds the dependencies each time. The two builds use different flags, and I got used to different flags meaning that the final binary gets rebuilt all the time, but at least the dependencies usually got properly cached -- so it feels like there is some kind of bug here.

Steps

1. Clone https://github.com/rust-lang/miri/
2. Run ./miri build-debug
3. Run cargo check (usually this happens because I save a file in vscode)
4. Run ./miri build-debug again

Expected behavior: the build is a NOP as everything was already built in step 2. At most, the final binary should be rebuilt (not great, but cargo currently cannot do better I think).

Actual behavior: it rebuilds most of the dependencies.

Work-around: do a release build instead. That seems to not have its cache mixed up with check builds. But release builds also take longer, so this is not a great solution.

Output of cargo version:
cargo 1.46.0-nightly (c26576f 2020-06-23)

[ehuss]

Hm, following the steps doesn't seem to rebuild for me. I also tried ./miri check in step 2, and that also seemed ok. Do you maybe run cargo check --release in step 2? Otherwise, from reading the miri script, it should be in the release directory and cargo check should be in the debug directory, there should be no overlap.

You can run with the CARGO_LOG=cargo::core::compiler::fingerprint=trace environment variable to get an explanation of why cargo thinks it needs to rebuild.

If it is cargo check --release you are running, this is because RUSTFLAGS is not incorporated in the filename hash. (It did for a brief time, but was reverted in #7417.) Build scripts are shared between check and build, so they need to be rebuilt (which is a separate issue of #3739 where build scripts share RUSTFLAGS with the target).

If you cannot unify RUSTFLAGS between the environments, I would recommend using separate target directories. Alternatively, use the --target flag so that RUSTFLAGS is not shared with build scripts.

Also, looking it its use of RUSTFLAGS, I would consider investigating using cargo profiles instead. It looks like all those flags are part of the profile. Using profiles should work better than RUSTFLAGS.

[RalfJung]

Hm, following the steps doesn't seem to rebuild for me. I also tried ./miri check in step 2, and that also seemed ok.

Yes that is less surprising as it also uses the same flags.

Do you maybe run cargo check --release in step 2? Otherwise, from reading the miri script, it should be in the release directory and cargo check should be in the debug directory, there should be no overlap.

Oh sorry, my instructions were wrong... I meant build-debug instead of build. I will update the OP.

[RalfJung]

If it is cargo check --release you are running, this is because RUSTFLAGS is not incorporated in the filename hash. (It did for a brief time, but was reverted in #7417.) Build scripts are shared between check and build, so they need to be rebuilt (which is a separate issue of #3739 where build scripts share RUSTFLAGS with the target).

Oh I see, that would explain the problem then, thanks.

Also, looking it its use of RUSTFLAGS, I would consider investigating using cargo profiles instead. It looks like all those flags are part of the profile. Using profiles should work better than RUSTFLAGS.

The problem is that we do not want those flags to be used when the same project, included as a submodule in the rustc tree, is built for rustup. I will look into what I can do with profiles.

[RalfJung]

The one thing I think I cannot set with a profile is -C link-args=-Wl,-rpath,$LIBDIR. At least I found nothing searching for "link-args" on https://doc.rust-lang.org/cargo/reference/profiles.html. So looks like I do need to use $RUSTFLAGS?

In particular, LIBDIR is computed at LIBDIR=$SYSROOT/lib/rustlib/$TARGET/lib. This is needed to work around rust-lang/rust#58343.

[RalfJung]

But it looks like adding --target helped indeed. :-)