Nightly Rust using cargo 0.25.0-nightly (930f9d949 2017-12-05) fails to respect --frozen

[carols10cents]

Previous discussion at the end of #4563 (comment) as that issue was suspected at first:

Hi!

We (The Tor Project) appear to be having the same bug as @DanielKeep (and reasoning for backwards compatibility with the lockfiles which are now in our older, maintenance-only branches). However, instead of with --locked, we're specifying --frozen, and newer cargo is desperately trying to update the lockfile, then bailing because it can't. Our bug for tracking this is https://bugs.torproject.org/24608, the description of which includes the logs of a TravisCI build which is failing (we suspect) for this reason.

/cc @matklad @isislovecruft @alexcrichton @nrc

[carols10cents]

Ok, hypothesizing that the problem is that checksums of dependencies would be different under latest rustc stable and latest rustc nightly, but I was unable to reproduce the failure tor is seeing in travis using this hypothesis :-/

What I did:

$ cargo new frozen
$ cd frozen
$ rustup default stable
$ rustc --version
rustc 1.22.1 (05e2e1c41 2017-11-22)
$ cargo --version
cargo 0.23.0 (61fa02415 2017-11-22)

I added libc = "*" as a dependency

$ cargo build
    Updating registry `https://github.com/rust-lang/crates.io-index`
   Compiling libc v0.2.34
   Compiling frozen v0.1.0 (file:///Users/carolnichols/rust/frozen)
    Finished dev [unoptimized + debuginfo] target(s) in 0.96 secs

Cargo.lock now contains:

[[package]]
name = "frozen"
version = "0.1.0"
dependencies = [
 "libc 0.2.34 (registry+https://github.com/rust-lang/crates.io-index)",
]

[[package]]
name = "libc"
version = "0.2.34"
source = "registry+https://github.com/rust-lang/crates.io-index"

[metadata]
"checksum libc 0.2.34 (registry+https://github.com/rust-lang/crates.io-index)" = "36fbc8a8929c632868295d0178dd8f63fc423fd7537ad0738372bd010b3ac9b0"

I checked that I could successfully use --frozen on stable:

$ cargo build --frozen
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs

I switched to nightly:

$ rustc --version
rustc 1.24.0-nightly (6a3601944 2017-12-12)
11:42:48 (master) ~/rust/frozen $ cargo --version
cargo 0.25.0-nightly (930f9d949 2017-12-05)

And I was able to successfully use --frozen:

11:42:50 (master) ~/rust/frozen $ cargo build --frozen
   Compiling libc v0.2.34
   Compiling frozen v0.1.0 (file:///Users/carolnichols/rust/frozen)
    Finished dev [unoptimized + debuginfo] target(s) in 0.72 secs

So I don't think it's the checksum. Talking with Alex at lunch, he says the checksum commit that updated the test should only apply to cargo's tests. He also can't think of anything recently that could have broken --frozen.

My next step is going to be figuring out what cargo was before it was updated a week ago and start looking at the commits in between to make a new hypothesis :-/

[carols10cents]

Last passing tor build

• https://travis-ci.org/tlyu/tor/jobs/312629401#L543
• rustc 1.24.0-nightly (cfba0d446 2017-12-05)
• cargo 0.24.0-nightly (5bb478a 2017-11-29)

First failing tor build

• https://travis-ci.org/tlyu/tor/jobs/312984357#L543
• rustc 1.24.0-nightly (5a2465e2b 2017-12-06)
• cargo 0.25.0-nightly (930f9d9 2017-12-05)

Cargo commits between 5bb478a and 930f9d9

• 5bb478a...930f9d9

[matklad]

@carols10cents were you able to reproduce it locally? I've https://github.com/tlyu/tor

[matklad]

...

g clone https://github.com/tlyu/tor.git 
cd tor/src/rust
cargo +nightly build --release --frozen --manifest-path tor_rust/Cargo.toml

And it does not complain.

[carols10cents]

@matklad neither I nor Alex have been able to reproduce this locally :( :( :(

[matklad]

One thing I've noticed is that only builds with TOR_RUST_DEPENDENCIES=true fail: https://travis-ci.org/tlyu/tor/builds/315398991

[carols10cents]

@isislovecruft what effect does the TOR_RUST_DEPENDENCIES env var have?

[carols10cents]

AHA https://github.com/tlyu/tor/blob/master/doc/HACKING/GettingStartedRust.md it turns on a use of cargo-vendor :) the plot thickens!!

[alexcrichton]

I've reproduced this locally and confirmed that this is a regression due to #4767

[alexcrichton]

I've posted a fix at #4816

@isislovecruft for a quick fix deleting this line should fix the regression and I think keep the build working (--frozen shouldn't strictly require that change of env var), or you could do something like set CARGO_HOME to /tmp (or something like that). In any case we can hopefully get a fix into nightlies pretty soon!

[isislovecruft]

TOR_RUST_DEPENDENCIES, when present, should point to a repo containing vendored dependencies (using cargo-vendor), as documented here. The repo is hosted (for now) at https://gitweb.torproject.org/user/Sebastian/tor-rust-dependencies.git and it only contains the libc dependency. If TOR_RUST_DEPENDENCIES isn't present, then the --enable-cargo-online-mode flag must be passed to ./configure, which tells cargo to run without --frozen.

[isislovecruft]

Oh nice, and @carols10cents and @alexcrichton already figured it out by the time I got here to respond! :D