Re-enable lto support guarded behind feature flags

[NobodyXu]

• fat-lto for fat lto, will override thin-lto if specified
• thin-lto for thin lto, will fallback to fat-lto is not supported

Signed-off-by: Jiahao XU [email protected]

[NobodyXu]

The previous bug with lto should be fixed by rust-lang/cc-rs#757

@gyscos I recommend to upgrade to new cc once it lands.

But still, I want to be safe here by putting them behind feature flags so that users can decide whether to enable fat-lto or thin-lto.

[gyscos]

Thanks for the work!

[gyscos]

Hmmm at this moment running cargo test --features thin-lto from zstd-safe fails for me.
(Running it from zstd-sys does not matter because the tests don't actually link anything.)

[NobodyXu]

Hmmm at this moment running cargo test --features thin-lto from zstd-safe fails for me. (Running it from zstd-sys does not matter because the tests don't actually link anything.)

Hmmm both fat-lto and thin-lto works on my M1 macbook.

Well, at least we can override thin-lto with fat-lto, which should work more reliably on more targets.

[gyscos]

It also fails with --features fat-lto. (Running on a x86_64 linux machine).

Indeed, running it on a M1 macbook running macOS, the tests pass.

Might be because on the linux machine, cc is gcc, while it's clang on macOS.

[NobodyXu]

@gyscos It should get better with rust-lang/cc-rs#757 once cc releases a new version.

[gyscos]

Aaah right I had an explicit override for gcc before. Now it only relies on cc detection, which still hasn't released. Sorry for the confusion.

(Was just trying to understand why it's now working worse for me than before disabling LTO.)

[NobodyXu]

Now it only relies on cc detection, which still hasn't released.

My fix for cc is actually really trivial if you look at the PR: It simply checks for the exit status.

I don't know why it didn't check for that but this should improve things a bit, when compiler fails without any stderr output.

[gyscos]

(This post may sound trivial for anyone used to LTO, but I was not, so here's my journey.)

Soo I just realized this only works if the C lib is compiled by clang (or another LLVM-based compiler), otherwise the intermediate compiled file will only contain gcc's flavor of IR, which rust cannot use.

So at the moment, both fat-lto and thin-lto only successfully compile if CC is set to clang, and if the linker is given to proper option.

In addition, -Clinker-plugin-lto is probably recommended to actually run LTO, though it appears to at least compile fine if omitted.

TLDR, In my case, I had to run:

RUSTFLAGS="-Clinker-plugin-lto -Clink-arg=-fuse-ld=lld" CC=clang cargo test -vv --features thin-lto

Unsurprisingly, on platforms where clang is already the default compiler/linker, these overrides are not required.

It also means that, since this can currently only work with clang, there never was a real need for the better gcc lto-support detection from cc-rs. :^/