Rust should not link sanitizer runtimes unconditionally #64629
Labels
A-linkage
Area: linking into static, shared libraries and binaries
A-sanitizers
Area: Sanitizers for correctness and code quality
C-bug
Category: This is a bug.
T-compiler
Relevant to the compiler team, which will review and decide on the PR/issue.
Comments
csmoe
added
the
A-linkage
jonas-schievink
added
A-sanitizers
choller
added a commit
to choller/rust
that referenced
this issue
Sep 25, 2019
bors
added a commit
that referenced
this issue
Oct 2, 2019
Only add sanitizer runtimes when linking an executable (#64629). This change modifies the code to only add sanitizer runtimes if we are linking an executable, as those runtimes should never be part of libraries. I successfully compiled `mozilla-central` with ASan using this patch.
bors
added a commit
that referenced
this issue
Oct 7, 2019
Only add sanitizer runtimes when linking an executable (#64629). This change modifies the code to only add sanitizer runtimes if we are linking an executable, as those runtimes should never be part of libraries. I successfully compiled `mozilla-central` with ASan using this patch.
|
I'm reopening this because the fix I made is not complete. There is a second place where we inject the sanitizer runtime and that is the |
choller
added a commit
to choller/rust
that referenced
this issue
Oct 17, 2019
Centril
added a commit
to Centril/rust
that referenced
this issue
Oct 19, 2019
build-std compatible sanitizer support ### Motivation When using `-Z sanitizer=*` feature it is essential that both user code and standard library is instrumented. Otherwise the utility of sanitizer will be limited, or its use will be impractical like in the case of memory sanitizer. The recently introduced cargo feature build-std makes it possible to rebuild standard library with arbitrary rustc flags. Unfortunately, those changes alone do not make it easy to rebuild standard library with sanitizers, since runtimes are dependencies of std that have to be build in specific environment, generally not available outside rustbuild process. Additionally rebuilding them requires presence of llvm-config and compiler-rt sources. The goal of changes proposed here is to make it possible to avoid rebuilding sanitizer runtimes when rebuilding the std, thus making it possible to instrument standard library for use with sanitizer with simple, although verbose command: ``` env CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS=-Zsanitizer=thread cargo test -Zbuild-std --target x86_64-unknown-linux-gnu ``` ### Implementation * Sanitizer runtimes are no long packed into crates. Instead, libraries build from compiler-rt are used as is, after renaming them into `librusc_rt.*`. * rustc obtains runtimes from target libdir for default sysroot, so that they are not required in custom build sysroots created with build-std. * The runtimes are only linked-in into executables to address issue rust-lang#64629. (in previous design it was hard to avoid linking runtimes into static libraries produced by rustc as demonstrated by sanitizer-staticlib-link test, which still passes despite changes made in rust-lang#64780). * When custom llvm-config is specified during build process, the sanitizer runtimes will be obtained from there instead of begin rebuilding from sources in src/llvm-project/compiler-rt. This should be preferable since runtimes used should match instrumentation passes. For example there have been nine version of address sanitizer ABI. Note this marked as a draft PR, because it is currently untested on OS X (I would appreciate any help there). cc @kennytm, @japaric, @Firstyear, @choller
Centril
added a commit
to Centril/rust
that referenced
this issue
Oct 19, 2019
build-std compatible sanitizer support ### Motivation When using `-Z sanitizer=*` feature it is essential that both user code and standard library is instrumented. Otherwise the utility of sanitizer will be limited, or its use will be impractical like in the case of memory sanitizer. The recently introduced cargo feature build-std makes it possible to rebuild standard library with arbitrary rustc flags. Unfortunately, those changes alone do not make it easy to rebuild standard library with sanitizers, since runtimes are dependencies of std that have to be build in specific environment, generally not available outside rustbuild process. Additionally rebuilding them requires presence of llvm-config and compiler-rt sources. The goal of changes proposed here is to make it possible to avoid rebuilding sanitizer runtimes when rebuilding the std, thus making it possible to instrument standard library for use with sanitizer with simple, although verbose command: ``` env CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS=-Zsanitizer=thread cargo test -Zbuild-std --target x86_64-unknown-linux-gnu ``` ### Implementation * Sanitizer runtimes are no long packed into crates. Instead, libraries build from compiler-rt are used as is, after renaming them into `librusc_rt.*`. * rustc obtains runtimes from target libdir for default sysroot, so that they are not required in custom build sysroots created with build-std. * The runtimes are only linked-in into executables to address issue rust-lang#64629. (in previous design it was hard to avoid linking runtimes into static libraries produced by rustc as demonstrated by sanitizer-staticlib-link test, which still passes despite changes made in rust-lang#64780). * When custom llvm-config is specified during build process, the sanitizer runtimes will be obtained from there instead of begin rebuilding from sources in src/llvm-project/compiler-rt. This should be preferable since runtimes used should match instrumentation passes. For example there have been nine version of address sanitizer ABI. Note this marked as a draft PR, because it is currently untested on OS X (I would appreciate any help there). cc @kennytm, @japaric, @Firstyear, @choller
bors
added a commit
that referenced
this issue
Oct 20, 2019
Avoid injecting sanitizer runtimes into staticlibs (#64629). This fixes the remaining issue in `creader.rs` and also fixes the expected test failure. I have explicitly turned the `$(CC)` call into a negative check with the `!` to ensure that this command is really failing (if it is not, then either the runtime is attached to the lib or the lib has not been instrumented and both would be an error). I've also borrowed `program.rs` and the additional `rustc` invocation from @tmiasko 's PR since he pointed out that using `-fsanitize=address` with `$(CC)` for linking could fail if the sanitizer runtimes on the system are incompatible. With this toolchain I was able to compile Firefox locally without any linker errors. I am still seeing races with Rust in TSan but I assume that is because I did not build with `-Z build-std`.
bors
added a commit
that referenced
this issue
Oct 22, 2019
build-std compatible sanitizer support ### Motivation When using `-Z sanitizer=*` feature it is essential that both user code and standard library is instrumented. Otherwise the utility of sanitizer will be limited, or its use will be impractical like in the case of memory sanitizer. The recently introduced cargo feature build-std makes it possible to rebuild standard library with arbitrary rustc flags. Unfortunately, those changes alone do not make it easy to rebuild standard library with sanitizers, since runtimes are dependencies of std that have to be build in specific environment, generally not available outside rustbuild process. Additionally rebuilding them requires presence of llvm-config and compiler-rt sources. The goal of changes proposed here is to make it possible to avoid rebuilding sanitizer runtimes when rebuilding the std, thus making it possible to instrument standard library for use with sanitizer with simple, although verbose command: ``` env CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS=-Zsanitizer=thread cargo test -Zbuild-std --target x86_64-unknown-linux-gnu ``` ### Implementation * Sanitizer runtimes are no long packed into crates. Instead, libraries build from compiler-rt are used as is, after renaming them into `librusc_rt.*`. * rustc obtains runtimes from target libdir for default sysroot, so that they are not required in custom build sysroots created with build-std. * The runtimes are only linked-in into executables to address issue #64629. (in previous design it was hard to avoid linking runtimes into static libraries produced by rustc as demonstrated by sanitizer-staticlib-link test, which still passes despite changes made in #64780). * When custom llvm-config is specified during build process, the sanitizer runtimes will be obtained from there instead of begin rebuilding from sources in src/llvm-project/compiler-rt. This should be preferable since runtimes used should match instrumentation passes. For example there have been nine version of address sanitizer ABI. Note this marked as a draft PR, because it is currently untested on OS X (I would appreciate any help there). cc @kennytm, @japaric, @Firstyear, @choller
tmandry
added a commit
to tmandry/rust
that referenced
this issue
Oct 30, 2019
build-std compatible sanitizer support ### Motivation When using `-Z sanitizer=*` feature it is essential that both user code and standard library is instrumented. Otherwise the utility of sanitizer will be limited, or its use will be impractical like in the case of memory sanitizer. The recently introduced cargo feature build-std makes it possible to rebuild standard library with arbitrary rustc flags. Unfortunately, those changes alone do not make it easy to rebuild standard library with sanitizers, since runtimes are dependencies of std that have to be build in specific environment, generally not available outside rustbuild process. Additionally rebuilding them requires presence of llvm-config and compiler-rt sources. The goal of changes proposed here is to make it possible to avoid rebuilding sanitizer runtimes when rebuilding the std, thus making it possible to instrument standard library for use with sanitizer with simple, although verbose command: ``` env CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS=-Zsanitizer=thread cargo test -Zbuild-std --target x86_64-unknown-linux-gnu ``` ### Implementation * Sanitizer runtimes are no long packed into crates. Instead, libraries build from compiler-rt are used as is, after renaming them into `librusc_rt.*`. * rustc obtains runtimes from target libdir for default sysroot, so that they are not required in custom build sysroots created with build-std. * The runtimes are only linked-in into executables to address issue rust-lang#64629. (in previous design it was hard to avoid linking runtimes into static libraries produced by rustc as demonstrated by sanitizer-staticlib-link test, which still passes despite changes made in rust-lang#64780). * When custom llvm-config is specified during build process, the sanitizer runtimes will be obtained from there instead of begin rebuilding from sources in src/llvm-project/compiler-rt. This should be preferable since runtimes used should match instrumentation passes. For example there have been nine version of address sanitizer ABI. Note this marked as a draft PR, because it is currently untested on OS X (I would appreciate any help there). cc @kennytm, @japaric, @Firstyear, @choller
bors
added a commit
that referenced
this issue
Dec 11, 2019
build-std compatible sanitizer support ### Motivation When using `-Z sanitizer=*` feature it is essential that both user code and standard library is instrumented. Otherwise the utility of sanitizer will be limited, or its use will be impractical like in the case of memory sanitizer. The recently introduced cargo feature build-std makes it possible to rebuild standard library with arbitrary rustc flags. Unfortunately, those changes alone do not make it easy to rebuild standard library with sanitizers, since runtimes are dependencies of std that have to be build in specific environment, generally not available outside rustbuild process. Additionally rebuilding them requires presence of llvm-config and compiler-rt sources. The goal of changes proposed here is to make it possible to avoid rebuilding sanitizer runtimes when rebuilding the std, thus making it possible to instrument standard library for use with sanitizer with simple, although verbose command: ``` env CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS=-Zsanitizer=thread cargo test -Zbuild-std --target x86_64-unknown-linux-gnu ``` ### Implementation * Sanitizer runtimes are no long packed into crates. Instead, libraries build from compiler-rt are used as is, after renaming them into `librusc_rt.*`. * rustc obtains runtimes from target libdir for default sysroot, so that they are not required in custom build sysroots created with build-std. * The runtimes are only linked-in into executables to address issue #64629. (in previous design it was hard to avoid linking runtimes into static libraries produced by rustc as demonstrated by sanitizer-staticlib-link test, which still passes despite changes made in #64780). cc @kennytm, @japaric, @Firstyear, @choller
bors
added a commit
that referenced
this issue
Jan 10, 2020
build-std compatible sanitizer support ### Motivation When using `-Z sanitizer=*` feature it is essential that both user code and standard library is instrumented. Otherwise the utility of sanitizer will be limited, or its use will be impractical like in the case of memory sanitizer. The recently introduced cargo feature build-std makes it possible to rebuild standard library with arbitrary rustc flags. Unfortunately, those changes alone do not make it easy to rebuild standard library with sanitizers, since runtimes are dependencies of std that have to be build in specific environment, generally not available outside rustbuild process. Additionally rebuilding them requires presence of llvm-config and compiler-rt sources. The goal of changes proposed here is to make it possible to avoid rebuilding sanitizer runtimes when rebuilding the std, thus making it possible to instrument standard library for use with sanitizer with simple, although verbose command: ``` env CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS=-Zsanitizer=thread cargo test -Zbuild-std --target x86_64-unknown-linux-gnu ``` ### Implementation * Sanitizer runtimes are no long packed into crates. Instead, libraries build from compiler-rt are used as is, after renaming them into `librusc_rt.*`. * rustc obtains runtimes from target libdir for default sysroot, so that they are not required in custom build sysroots created with build-std. * The runtimes are only linked-in into executables to address issue #64629. (in previous design it was hard to avoid linking runtimes into static libraries produced by rustc as demonstrated by sanitizer-staticlib-link test, which still passes despite changes made in #64780). cc @kennytm, @japaric, @Firstyear, @choller
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I've noticed today that apparently Rust adds sanitizer runtimes such as the ASan runtime during linking and I found this runtime inside a static archive.
As far as I know, these runtimes should not be part of static archives or DSOs, they should only be linked when creating an executable. Adding them to DSOs causes two runtimes to exist when loading the DSO into an ASan binary (we had this problem longer ago in mozilla-central and it was fixed in Clang, see [1]). Adding the runtime to static archives causes linker problems when linking the archive to a C++ binary built with ASan because Clang will add the ASan runtime a second time when linking the executable. For some reason this worked before (maybe if the runtimes used are exactly identical the linker will deduplicate them) but today during building with Rust nightly I got duplicate symbol errors indicating two conflicting ASan RTs.
From what I can tell, Rust should not link these runtimes except if it is producing an executable. The respective code is called here independent of
crate_type:rust/src/librustc_codegen_ssa/back/link.rs
Line 1376 in ea3ba36
[1] https://github.com/llvm-mirror/clang/blob/7dbdcfcb827c68b0c380de289a2d7526666b8771/lib/Driver/ToolChains/CommonArgs.cpp#L645
The text was updated successfully, but these errors were encountered: