A rustc compiled from the rust-*-src tarballs can't be used with the std binaries from s.r-l.o/dist

[japaric]

Originally reported here

In the reported case the rust package provided by Arch Linux was compiled from the rustc-1.8.0-src.tar.gz tarball. Thus their rustc binary doesn't have a git commit hash encoded in it:

$ rustc -V
rust-1.8.0

And if you try to use this rustc to cross compile using the std binaries from s.r-l.o you get the following error:

$ cargo build --target=i686-pc-windows-gnu`
error: the crate `std` has been compiled with rustc 1.8.0 (db2939409 2016-04-11), which is incompatible with this version of rustc [E0514]

At the center of this issue is this check, this rustc_version function and this CFG_VERSION env variable which is set by the build system.

In the case of the std crate since it was compiled from a git checkout the value returned by rustc_version is 1.8.0 ($SOME_HASH $SOME_DATE) whereas the same function just returns 1.8.0 in the case of a rustc that was compiled from a source tarball. The mismatch between these two strings is what causes the error reported above. But the error message is wrong because both binaries where compiled from the exact same source code and can be used together.

How to fix this? Possible solutions:

• Ask package maintaners to always build from a git checkout. This way the "rustc_version" string always contains a git commit hash.
• Tweak our build system so that building from a git checkout and from a source tarball yield the same "rustc_version" string. Two possible implementations:
  • Drop the commit hash/date part from the "rustc_version" string. This may be OK for the stable and beta channels because you can tell them apart from looking at the version number (e.g. 1.7.0 vs 1.8.0) but that's not the case for nightlies -- two consecutive nightlies would have the same version number (e.g. 1.10.0-nightly).
  • Include the commit hash part in the rustc-*-src.tar.gz tarballs, perhaps as a .commit-hash file and tweak the build system to use the contents of this file in the "rustc_version" string when one is not building from a git checkout.

Thoughts? @alexcrichton @brson

[brson]

I've never considered that it should be possible to mix binaries from compilers of different origin - our metadata situation used to be so screwed up that I just assumed it wouldn't be, and I suspect we intentionally made it impossible at some point in the past.

I don't know if even today it's possible. I would assume that e.g. changing cfg values will invalidate the metadata, though perhaps we embed and validate cfg compatibility by now.

[brson]

After considering it further, I don't see why this shouldn't be possible. This case wouldn't be mixing crates produced by different compilers. It's just reading a complete collection of crates produced by a different compiler.

[brson]

There may still be some conditional values embedded in the compiler that could influence crate behavior.

Use of unstable features could be one, but that is mostly determined by the version number.

Include the commit hash part in the rustc-*-src.tar.gz tarballs, perhaps as a .commit-hash file and tweak the build system to use the contents of this file in the "rustc_version" string when one is not building from a git checkout.

Something like this I think.

[arielb1]

@brson

The original problem is that the metadata format occasionally changes between nightlies, so that rustc 1.8.0 must not read the metadata created by rustc 1.8.0 (777777777) 2015-01-01.

[arielb1]

I think we should just compare the commit hash - that means storing it in the tarball.

[Mark-Simulacrum]

So this is a duplicate of #31393 (which I've closed in favor of this), but the conclusion reached there appears to be incorrect to me.

The currently checked string originates from CFG_VERSION, but we should be using CFG_VER_HASH.

@alexcrichton noted in #31393 that if we only check git commits it'll suffice for "these should link together." However, that won't work for this issue's case since there is no commit hash for the version, as far as I can tell. I'm not sure what the right steps are in that case, so not marking as E-easy for the time being. @alexcrichton Any thoughts on this? Could you propose a solution?

[alexcrichton]

I believe it boils down to these lines of code in the build system right now. They don't work (return None) when built from a tarball, and we need them to operate the same in and out of a tarball.

As to how to do that, I'd be fine with just generating some file during the creation of the source tarball.

[steveklabnik]

Triage: the code is now here: https://github.com/rust-lang/rust/blob/master/src/bootstrap/tool.rs#L244-L253

[pwnorbitals]

Do we have any progress on this issue ? Doesn't seem to be much activity

[Zapeth]

Just got this error when trying to use the std binaries for the armv7-unknown-linux-gnueabihf target with src-compiled rustc x86_64-pc-windows-gnu (mingw package)

Is this just an issue of mismatched metadata (commit hash/date) that causes the compilation error, or is the problem deeper rooted (i.e. compilation for target necessary)?

[jyn514]

I think this is just an issue of mismatched metadata; the compiler was built from the same sources, so it should be able to read libraries built by the same sources, even if they're a different physical binary.

It turns out that we already vendor the version hash when creating the rustc-src component: https://github.com/rust-lang/rust/blob/d50583eaecdf88b5ee59e1d77fe3b4a8d87cbe6e/src/bootstrap/dist.rs#L882

Mentoring instructions: In

rust/src/bootstrap/lib.rs

Line 1335 in 85c87f6

let mut version = self.rust_info.version(self, &self.version);

, check if self.rust_info.is_git().is_none(). If so, manually add the git commit from git-commit-hash. It's ok to panic if that file isn't present; the only supported way to build without git sources is through the vendored source tarball.

@rustbot label +E-mentor +E-easy

[dawnofmidnight]

@rustbot claim