Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CompilerSupportLibraries_jll] Upgrade to libraries from GCC 12 #45582

Merged
merged 12 commits into from
Nov 3, 2022

Conversation

giordano
Copy link
Contributor

@giordano giordano commented Jun 4, 2022

Usual memo to self:

  • change the version number in stdlib/CompilerSupportLibraries_jll/Project.toml
  • update CSL_NEXT_GLIBCXX_VERSION in deps/csl.mk
  • update hashes with make -f contrib/refresh_checksums.mk -j csl
  • since the soversion of libgcc_s for x86_64-darwin changed, update all references to the version number wherever it's mentioned (I hope I got them all)

Note 1: this lets us effectively test in CI #44829 is fixed
Note 2: I marked this to be backported to julia v1.6, but this will first require backporting #45249, which seems to be stalling.

@giordano giordano added building Build system, or building Julia or its dependencies external dependencies Involves LLVM, OpenBLAS, or other linked libraries backport 1.6 Change should be backported to release-1.6 backport 1.8 Change should be backported to release-1.8 labels Jun 4, 2022
@giordano
Copy link
Contributor Author

giordano commented Jun 4, 2022

I have no idea what's wrong with Windows (lots of undefined references to llvm functions at linking time for libjulia-internal.dll).

@giordano giordano added this to the 1.8 milestone Jun 4, 2022
@giordano
Copy link
Contributor Author

giordano commented Jun 4, 2022

Alright, we have a problem. In https://github.com/JuliaPackaging/Yggdrasil/blob/7639eb0a7a1ec9d21cb8701925b4ec4f0257423e/C/CompilerSupportLibraries/build_tarballs.jl we always use the latest version of libgomp and libstdc++, independently from the version of libgfortran (this is for example important to solve the issue with undefined references to GLIBCXX symbols). However, GCC 12 changed the ABI of libgcc_s for x86_64-darwin, which is now called libgcc_s.1.1.dylib instead of libgcc_s.1.dylib. But this means in the libgfortran{3,4} versions of CSL we have libstdc++ which requires libgcc_s.1.1.dylib (because this comes from the latest version) but all the other libraries (e.g. libgfortran) which require libgcc_s.1.dylib, so they can't really coexist. I'm not sure how to address this issue. @staticfloat ideas? A datapoint: as far as I can tell, the macOS version of libtsdc++ doesn't have the infamous GLIBCXX symbols, so maybe we can not use the latest version of libstdc++ on macOS?

@staticfloat
Copy link
Member

but all the other libraries (e.g. libgfortran) which require libgcc_s.1.dylib, so they can't really coexist.

Do we know that they actually can't co-exist? 😈

@giordano
Copy link
Contributor Author

giordano commented Jun 6, 2022

Julia now builds on x86_64-darwin. Still linking errors on Windows for libjulia-internal.dll, no clue why. Why libgcc would even affect resolution of llvm symbols?

@KristofferC KristofferC mentioned this pull request Jun 7, 2022
36 tasks
@fxcoudert
Copy link
Contributor

Not sure what the windows error is due to, but the macOS part looks good to me.

@giordano
Copy link
Contributor Author

Linking on windows now works (without having changed anything here), but Julia fails to bootstrap. What's wrong always with Windows?

@giordano
Copy link
Contributor Author

Right, this requires #45649, otherwise the Float16 tests are failing.

Going back to Windows, the error now is:

ERROR: `ccall` requires the compiler
make[1]: *** [sysimage.mk:58: /cygdrive/c/buildbot/worker/package_win64/build/usr/lib/julia/corecompiler.ji] Error 1

which looks similar to issues mentioned in #45857 and #45641 (comment). If that's the case, it means Julia is picking the wrong libstdc++? I can't debug much myself anything related to Windows, so I'd need help from someone with access to a Windows box 🙂

@JeffBezanson
Copy link
Member

What's the status of this? Still seems to have build problems. Is this release-blocking?

@giordano
Copy link
Contributor Author

I can't do much about Windows, I don't have a Windows box (and zero experience with Windows anyway). I'm surprised by the macOS failure, also because it was working before the last rebase, but haven't had the time to look into that. It's probably not a release blocker strictly speaking, but it'd be nice to get this in: just see at the people having problems with the old libstdc++ shipped by Julia in JuliaGL/GLFW.jl#198.

@IanButterworth
Copy link
Member

Just doing a merge-up and rerun, given Mosé is on vacation for 3 weeks. Just in case it all goes green

@IanButterworth
Copy link
Member

MacOS now built and I believe the test fail is a known FileWatching issue, but the windows issues remain and it's not obvious to me why

@inkydragon
Copy link
Member

inkydragon commented Aug 3, 2022

image
image

missing _ZSt14__once_functor in libstdc++-6.dll


Update: What is the full version number of our gcc12?

Maybe we need this patch?

libstdc++: Check for TLS support on mingw cross-compilers
Native mingw builds enable TLS, but crosses don't because we don't use
GCC_CHECK_TLS in the cross-compiler config.

gcc-mirror/gcc@cc1e288

@giordano giordano merged commit 10fcec9 into JuliaLang:master Nov 3, 2022
@giordano giordano deleted the mg/csl-gcc12 branch November 3, 2022 09:36
@vtjnash
Copy link
Member

vtjnash commented Nov 4, 2022

This commit message is a bit messy, and seems to claim to add debug checks and revert commits that aren't part of Julia. Please be conscientious when merging so that they are easily readable to others.

@vtjnash
Copy link
Member

vtjnash commented Nov 9, 2022

I think this broke compilation with gcc <= 10 on Windows, due to something about the lib/*.a files being incompatible or something. Building in wine after this commit results in:

    JULIA julia-win32/usr/lib/julia/corecompiler.ji
ERROR: Unable to load dependent library Z:\data\vtjnash\julia\julia-win32\usr\bin\../bin/libjulia-internal.dll
Message:Bad EXE format for %1. 
make[1]: *** [/data/vtjnash/julia/sysimage.mk:61: /data/vtjnash/julia/julia-win32/usr/lib/julia/corecompiler.ji] Error 1
make: *** [/data/vtjnash/julia/Makefile:84: julia-sysimg-ji] Error 2

If we then do

$ /bin/tar -xmvf deps/srccache/CompilerSupportLibraries.v0.5.4+0.i686-w64-mingw32-libgfortran5.tar.gz -C julia-win32/usr lib

to get the lib files from the previous version back and then re-link julia (e.g. rm usr/bin/libjulia-internal.dll && make), it works again.

Perhaps we now need to avoid installing these lib/*.a files, and only get the updated/upgraded dll files?

@staticfloat
Copy link
Member

staticfloat commented Nov 9, 2022

We have been thinking about eliminating .a files from JLLs for a while now. They are often more trouble than they're worth.

@giordano
Copy link
Contributor Author

giordano commented Nov 9, 2022

@vchuravy needed them for #47184. Static libraries for Windows were introduced in #47332, not here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 1.8 Change should be backported to release-1.8 building Build system, or building Julia or its dependencies external dependencies Involves LLVM, OpenBLAS, or other linked libraries JLLs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Needs libstdc++.so.6.0.30
10 participants