no_mangle/used static is only present in output when in reachable module

[phil-opp]

Statics that are marked no_mangle and/or used only get to the linker if they are in a reachable module of a reachable crate. The static itself does not need to be used in code, only some function in the same module. This previously worked in our project, so this seems like a regression.

Example

The issue is best explained with a small example crate named bug_test, which can be found here:

src/lib.rs:

pub mod foo {
    #[no_mangle]
    #[used]
    pub static STATIC: [u32; 10] = [1; 10];

    pub fn hello() {}
}

pub fn bar() {
    foo::hello(); // STATIC not present if commented out
}

src/main.rs:

extern crate bug_test;

fn main() {
    bug_test::bar(); // STATIC not present if commented out 
}

Linker script linker.ld:

SECTIONS
{
    .static : ALIGN(4)
    {
        KEEP(*(.rodata.STATIC));
    }
}

Build using:

RUSTFLAGS='-Z pre-link-args=-Tlinker.ld' cargo build

Show contents of .static section:

> objdump -s -j".static" target/debug/bug_test

target/debug/bug_test:     file format elf64-x86-64

Contents of section .static:
 3354c 01000000 01000000 01000000 01000000  ................
 3355c 01000000 01000000 01000000 01000000  ................
 3356c 01000000 01000000                    ........  

Comment out one of the STATIC not present if commented out lines and recompile. Then STATIC no longer exist in the output and the .static section is empty:

> objdump -s -j".static" target/debug/bug_test

target/debug/bug_test:     file format elf64-x86-64

objdump: section '.static' mentioned in a -j option, but not found in any input file

Versions

> rustc --version
rustc 1.27.0-nightly (ac3c2288f 2018-04-18)
> cargo --version
cargo 1.26.0-nightly (008c36908 2018-04-13)
> ld --version
GNU ld (GNU Binutils for Ubuntu) 2.26.1

Edit: Added the used attribute.

[phil-opp]

cc @oli-obk

[nagisa]

Doesn’t seem like a bug to me.

The library bug_test is a rlib, so it has no guarantees as to what appears in the binary or what doesn’t. Eventually rlib probably is expected to not contain any executable code whatsoever and not use a linker, which would break your linker script approach either way.

In your scenario it is entirely up to Rust whether:

1. it uses a linker when generating code for the intermediate crates;
2. it emits any binary or linkable code into the rlib or just aggregates intermediate representations and generates all the code in one go during the final binary compilation;
3. it optimises out unused globals for the binary (in absence of preventative attributes such as #[used]) before they reach the linker.

[petrochenkov]

IIRC, @joshtriplett reported that lang team discussed this a few months ago and decided to make link-time visibility of no_mangle items based purely on their pub annotation and not reachability.
I can't find that comment though and supposedly there was no progress since then.

[joshtriplett]

@petrochenkov I recall that as well, and that sounds accurate.

[oli-obk]

IIRC we tried both the used attribute and making it public and it gave us the same behaviour.

[phil-opp]

IIRC we tried both the used attribute and making it public and it gave us the same behaviour.

Yes, used does not change anything. This behaves exactly the same:

// src/lib.rs

#![feature(used)]

pub mod foo {
    #[no_mangle]
    #[used]
    pub static STATIC: [u32; 10] = [1; 10];

    pub fn hello() {}
}

pub fn bar() {
    foo::hello(); // STATIC not present if commented out
}

Edit: I updated the title and the issue description to reflect this.

[phil-opp]

So @japaric explained that this is a problem with our linker script and has nothing to do with the no_mangle and used attributes: #40289 (comment). The fix is to add an EXTERN(STATIC); declaration to the linker script, which tells the linker to keep looking for a STATIC symbol, even if all other symbols are already resolved.

[dtolnay]

Reopening because people continue to have problems with #[used] statics failing to appear, even in use cases without a custom linker script.

• inventory::submit!() doesn't work unless code from downstream crate is actually used dtolnay/inventory#7
• Does not collect plugins defined outside of crate root dtolnay/inventory#9
• https://users.rust-lang.org/t/link-section-in-libraries-discarded/28349

[CJKay]

FYI I managed to work around this in my case by doing the following:

• In the linker script, specify EXTERN(xyz) where xyz is a symbol you are trying to export (you actually only need to do this for one symbol if all your symbols are in one module, presumably because it stops the linker from removing anything else in that library (object file?))
• Mark the symbol as #[no_mangle] and pub
• pub use the symbol from the binary

Realistically, this is not a long-term solution - the first step is reasonable, but steps 2 and 3 are just hacks. There's no reason the symbol should have to be given an unmangled name, and definitely no reason the binary should have to pollute its own code to work around unusual linker behaviour.

Additionally, some of the time rustc warns that the #[no_mangle] is not used, but I can't reproduce that reliably:

 --> src/lib.rs:3:1
  |
3 | #[no_mangle]
  | ^^^^^^^^^^^^
  |
  = note: #[warn(unused_attributes)] on by default```

[oli-obk]

Additionally, some of the time rustc warns that the #[no_mangle] is not used, but I can't reproduce that reliably:

That should be fixed on master, can you confirm that it is fixed for you now?

[zachreizner]

I ran into this issue today, and strangely only in debug builds. With --release, the #[used] static items from an otherwise unreachable module appeared in the resulting binary. My theory is that the divergent behavior is due the default incremental builds in debug mode. Adding this to my Cargo.toml seems to fix this issue, but it feels way to brittle to rely on:

[profile.dev]
incremental = false

[Shnatsel]

This issue is still present for me on rustc 1.45.1. It occurs in release mode both with and without LTO. Using #[link_section = "foo"] doesn't help. Actually using the static in some other code does preserve it.

Comment out the println! with a function call to reproduce: https://github.com/Shnatsel/rust-audit/blob/master/hello-auditable/src/main.rs

Declaration of the static: https://github.com/Shnatsel/rust-audit/blob/master/auditable/src/lib.rs

[carbotaniuman]

If anyone read my last message, I think that's just my computer getting hit by cosmic rays/some hardware bug, because trying to reproduce the fix led to cascading failures across my Linux install, sigh, so much time wasted. Sorry for any noise.

Running everything again on my laptop now and the case where you're including a static into the final main.rs appears to work with #[used(linker)]. Everything else, such as linkme, inventory won't get solved with #[used(linker)] due to the linker behavior mentioned above, and it looks like it'll require proper compiler support. It should be possible if #93791 doesn't get merged, to force all extern objects into their own CGU.

[bjorn3]

Has this been fixed by #95604?

Edit: according to the description for #95604, yes. It just didn't get picked up by github.