Apply dllimport in ThinLTO

[Zoxc]

This partially reverts #103353 by properly applying dllimport if -Z dylib-lto is passed. That PR should probably fully be reverted as it looks quite sketchy. We don't know locally if the entire crate graph would be statically linked.

This should hopefully be sufficient to make ThinLTO work for rustc on Windows.

r? @wesleywiser

---

Edit: This PR is changed to just generally revert #103353.

[lqd]

We can make try builds also run some tests now, so we could test this PR with it:

• re-apply https://github.com/rust-lang/rust/pull/103591/files
• set DIST_TRY_BUILD to 0
• switch the try builder to x64-msvc
• launch a try build and see what happens

[lqd]

@bors try

[bors]

⌛ Trying commit ea72d12 with merge af61c39...

[bors]

☀️ Try build successful - checks-actions
Build commit: af61c39 (af61c395e5842831b05bff9ab29b95ebd08a2b34)

[Zoxc]

I'm a bit confused what you meant about underlying issues in #128947?

[lqd]

In recent t-compiler meetings, Wesley said he'd be looking at this PR soon and testing with the original internal repro.

I think #81408, #109114, #103353, and this PR are all pointing at unknown issues, with dllimports on our side that I guess are more or less "tracked" via #27438, and/or lld bugs on msvc (or the converse with link.exe, #109114 (comment) seems to show a damned if we do damned if we don't kind of deal).

On top of this, the fact that we're using debug_asserts, that we don't enable anywhere, to do CLI validation is concerning. We also only CLI validate -Clinker-plugin-lto while #103353's workaround also looks at -Clto. This would likely have caught #109067/#109114.

It doesn't look like we have tests for #81408's behavior, only for what the workaround is expected to produce. So we don't know if the lld issue would reproduce or has been fixed, if we land this PR.

[lqd]

To be clear, I want us to land the PR. I'm just not sure we understand the issues, workarounds, and what other miscompilations we could uncover, don't have sufficient test coverage, and can't even crater any of this on windows, and last time this happened no one noticed before hitting stable.

tl;dr: we’re in a tough spot to make changes confidently imho

So with all of the above and previous comment, are you 100% confident this is fine as-is?

[Zoxc]

@lqd Just to clarify, you're just talking about this PR and not applying ThinLTO to the compiler on msvc right?

[Zoxc]

It looks like #103353 also causes #127979 so it might make sense to fully revert it instead. I suspect Bevy is not using -Z dylib-lto.

[tgross35]

On top of this, the fact that we're using debug_asserts, that we don't enable anywhere, to do CLI validation is concerning.

What is this referring to? I assume not #127979 (comment) since many of these were changed in #127995.

[lqd]

What is this referring to? I assume not #127979 (comment)

That was the one yes, the change is recent and didn't show in the github UI in this older PR until it was rebased yesterday. The assertion is still not checking -Clto on master. But that may be moot if we land this full revert.

@lqd Just to clarify, you're just talking about this PR and not applying ThinLTO to the compiler on msvc right?

The latter is scarier, but #109114 (comment) seems to be saying that the same situation can happen on both lld and msvc's link, and that fixing one breaks the other (and that may be what happened with #127979). So I wished that we had a more in-depth look at the entire situation rather than a back and forth of regressions.

I wish to know whether this comment is still accurate. And if we're indeed "disallowing dynamic linking" with -Clinker-plugin-lto but not -Clto=thin.

                // ThinLTO can't handle this workaround in all cases, so we don't
                // emit the attrs. Instead we make them unnecessary by disallowing
                // dynamic linking when linker plugin based LTO is enabled.

I also wish for some involvement from people more familiar with the situation like @wesleywiser and @michaelwoerister or other windows experts.

It looks like #103353 also causes #127979 so it might make sense to fully revert it instead. I suspect Bevy is not using -Z dylib-lto.

Right, they probably don't (but do recommend -Zthreads=0 which is audacious :).

What happens with this PR on the repros in #81408? A revert is just preferring to accept #81408 instead of #127979, and both were encountered in the wild. Bevy seems to use lld on windows anyways, so their users turning on ThinLTO can easily encounter either issue. Not great.

The good thing out of #127979 is that we have a minimal repro to make a test.

[Zoxc]

I tested a reproducer in #81408 with this PR applied. It crashed with lld, but not Microsoft's linker, so that's likely just a lld bug.

A revert is just preferring to accept #81408 instead of #127979

One is a lld bug, the other is a rustc bug introduced in #103353 and thus affects all linkers. It's pretty clear what should be fixed there.

[lqd]

Ok then until wesley/michael can take a look, it'll be worthwhile to have tests in this PR that reproduce the issue on CI for sure.

FWIW I tested bash's repro of #127979, and it reproduced. I then updated VS build tools to test with this PR (because of linker errors on rustc-driver with download-ci-llvm, and it didn't even help; but stage1 built when compiling llvm).

With these build tools v16.11.38, bash's repro doesn't trigger the bug anymore, with link.exe or rust-lld, in release or debug, with this PR or master.

I also tested the original bevy quickstart repo where the issue appeared, with this PR. It crashed when using rust-lld, but doesn't work with link.exe either (doesn't crash, just that "nothing happens", unlike in debug mode). And things were the same on nightly with link.exe.

🔥🔥🔥🔥🔥🔥🔥🔥
🔥 🐶 this is fine 🔥
🔥🔥🔥🔥🔥🔥🔥🔥

[Zoxc]

bash's repro doesn't trigger the bug anymore

Apparently after #128469 (which is in this branch now) it no longer reproduces, assume that's why you couldn't reproduce it? That doesn't seem to contain a fix though.

[lqd]

assume that's why you couldn't reproduce it?

Ah yes:

• it reproduces on the nightly that precedes the rollup (nightly-2024-07-31) and on its parent commit
• it reproduces on master with std: implement the once_wait feature #127567 from that rollup reverted
• it doesn't reproduce with this PR + std: implement the once_wait feature #127567 reverted

As for bevy_quickstart:

• "nothing happens" unexpectedly on yesterday's nightly (91376f4)
• "entered unreachable code" ICE on master with std: implement the once_wait feature #127567 reverted
• "entered unreachable code" ICE with this PR + std: implement the once_wait feature #127567 reverted. Are you able to reproduce this?

[lqd]

In case it helps, here's a non-regression test for #81408. It will pass on master and with #129079, but will fail with this PR of course.

[janhohenheim]

@lqd regarding your earlier comment: any recommendations for making Bevy's set of recommended compiler settings less audacious? I'd be ready to open a PR there to fix them up :)
Most of these were not chosen with careful deliberation but more of an "I chose this and that and them benchmarked" approach.

[lqd]

Just that -Zthreads is known to have issues, deadlocks and ICEs — but I guess test driving it and opening reproducible issues should they appear, could also be worthwhile. (And it also can work fine)

[bors]

☔ The latest upstream changes (presumably #130506) made this pull request unmergeable. Please resolve the merge conflicts.

[bors]

☔ The latest upstream changes (presumably #132581) made this pull request unmergeable. Please resolve the merge conflicts.