Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Mingw-w64 for Win32 #8996

Closed
klutzy opened this issue Sep 5, 2013 · 21 comments
Closed

Use Mingw-w64 for Win32 #8996

klutzy opened this issue Sep 5, 2013 · 21 comments
Assignees
Labels
O-windows Operating system: Windows P-medium Medium priority
Milestone

Comments

@klutzy
Copy link
Contributor

klutzy commented Sep 5, 2013

We currently use mingw for win32 platform. However mingw does not support win64 (#1237), so we have to use mingw-w64 (or msvc: #1768).
Although mingw and mingw-w64 looks similar, implementations differ: we have issues due to mingw which are already solved on mingw-w64. (#8663, #8859)

Instead, we may switch to mingw-w64 for win32. (Yes, mingw-w64 supports win32 despite its name.) then we can reduce platform/runtime differences between win32/win64.

Previously @thadguidry posted about Qt's discussion on mingw/mingw-w64. (original discussion)

also cc @vadimcn :)

@klutzy
Copy link
Contributor Author

klutzy commented Sep 5, 2013

Some more thoughts:

mingw-w64 has no 'official' binary: I think mingw-builds is most widely used, but I guess rubenvb's personal build is also quite popular.

For mingw-builds, there are many options available: platform (win32/win64), thread model (posix/win32 for both platform) and stack unwinding model (sjlj/dwarf for 32-bit, sjlj/seh for 64-bit). I've only used win32 (threads) and dwarf/win32 & seh/win64 (unwinding) so far. We have to choose some model among them for 'official' rust binary because performance and compatibility matter.

On stack unwinding models:

Actually I've already built win32 rust on mingw-w64 (platform 32-bit, threads win32, exception dwarf), but make check-fast failed mysteriously and rustc works horribly slowly. I don't know any details so far. Anyone who tried this before?

@thadguidry
Copy link
Contributor

There are /testing versions in the mingw-builds sourceforge repository as well. Also, there are build scripts to build a mingw-build, version yourself, which is found here: http://sourceforge.net/p/mingwbuilds/code/ci/master/tree/

@klutzy I also have been able to build win32 rust, and noticed the same issues as you on my Win7 64 bit Intel PC

@thadguidry
Copy link
Contributor

@klutzy rubenvb's builds are coordinated with a few of the Mingw developers/contributors...namely jon_y. But the his builds might not incorporate the very latest releases available of gcc. The mingw-build guys seem to keep a bit more up to date. Of note is the inclusion of Clang with rubenvb's builds.... that alone might sway a vote of confidence. It is probably best to ask someone like jon_y or others on the official Mingw mailing list itself.

@thadguidry
Copy link
Contributor

@klutzy We also ideally need a way to pass down the -L option during ./configure , since I try different toolchains, all usually within the same basic PATH of c:\mingw-toolchains. What ends up happening is that the ld.exe during the LLVM configure steps gets the wrong library path through the LDFLAGS and ends up throwing an error: C compiler cannot create executables See 'config.log' for more details...


configure:2124: $? = 0
configure:2131: gcc -m32 -v >&5
Using built-in specs.
COLLECT_GCC=c:\mingw-toolchains\x64-4.8.1-release-win32-seh-rev5\mingw64\bin\gcc.exe
COLLECT_LTO_WRAPPER=c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../libexec/gcc/x86_64-w64-mingw32/4.8.1/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../../../src/gcc-4.8.1/configure --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --prefix=/mingw64 --with-sysroot=/tmp/x64-481-win32-seh-r5/mingw64 --enable-shared --enable-static --disable-multilib --enable-languages=ada,c,c++,fortran,objc,obj-c++,lto --enable-libstdcxx-time=yes --enable-threads=win32 --enable-libgomp --enable-lto --enable-graphite --enable-checking=release --enable-fully-dynamic-string --enable-version-specific-runtime-libs --disable-isl-version-check --disable-cloog-version-check --disable-libstdcxx-pch --disable-libstdcxx-debug --enable-bootstrap --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-gnu-as --with-gnu-ld --with-arch=nocona --with-tune=core2 --with-libiconv --with-system-zlib --with-gmp=/tmp/mingw-prereq/x86_64-w64-mingw32-static --with-mpfr=/tmp/mingw-prereq/x86_64-w64-mingw32-static --with-mpc=/tmp/mingw-prereq/x86_64-w64-mingw32-static --with-isl=/tmp/mingw-prereq/x86_64-w64-mingw32-static --with-cloog=/tmp/mingw-prereq/x86_64-w64-mingw32-static --enable-cloog-backend=isl --with-pkgversion='rev5, Built by MinGW-W64 project' --with-bugurl=http://sourceforge.net/projects/mingw-w64 CFLAGS='-O2 -pipe -I/tmp/x64-481-win32-seh-r5/libs/include -I/tmp/mingw-prereq/x64-zlib/include -I/tmp/mingw-prereq/x86_64-w64-mingw32-static/include' CXXFLAGS='-O2 -pipe -I/tmp/x64-481-win32-seh-r5/libs/include -I/tmp/mingw-prereq/x64-zlib/include -I/tmp/mingw-prereq/x86_64-w64-mingw32-static/include' CPPFLAGS= LDFLAGS='-pipe -L/tmp/x64-481-win32-seh-r5/libs/lib -L/tmp/mingw-prereq/x64-zlib/lib -L/tmp/mingw-prereq/x86_64-w64-mingw32-static/lib -L/tmp/x64-481-win32-seh-r5/mingw64/opt/lib '
Thread model: win32
gcc version 4.8.1 (rev5, Built by MinGW-W64 project)
configure:2134: $? = 0
configure:2141: gcc -m32 -V >&5
gcc.exe: error: unrecognized command line option '-V'
gcc.exe: fatal error: no input files
compilation terminated.
configure:2144: $? = 1
configure:2167: checking for C compiler default output file name
configure:2194: gcc -m32 -m32 -m32 conftest.c >&5
c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/../../../../x86_64-w64-mingw32/lib/libmingw32.a when searching for -lmingw32
c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/../../../../x86_64-w64-mingw32/lib\libmingw32.a when searching for -lmingw32
c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/../../../../x86_64-w64-mingw32/lib/libmingw32.a when searching for -lmingw32
c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lmingw32
c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/libgcc.a when searching for -lgcc
c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1\libgcc.a when searching for -lgcc
c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/libgcc.a when searching for -lgcc
c:/mingw-toolchains/x64-4.8.1-release-win32-seh-rev5/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.8.1/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lgcc

@vadimcn
Copy link
Contributor

vadimcn commented Sep 12, 2013

Yes, rustc built with mingw-w64 (from mingw-builds) is very slow. Profiling points towards libwinpthreads' implementation of mutexes. I've checked out the source, and apparently they are implemented using Windows semaphores, which would explain why they are so slow: win32 semaphores are kernel objects, so every lock/unlock entails a syscall.

On the other hand, pthreads-win32 library, used by the regular mingw, looks much more reasonable - it first tries to obtain a lock via interlocked exchange, and only then falls back onto syscalls.

Here's the bug I've opened in mingw-w64 issue tracker: https://sourceforge.net/p/mingw-w64/bugs/344/

@klutzy
Copy link
Contributor Author

klutzy commented Sep 14, 2013

For record, I built rust on mingw-w64/32-bit where gcc version is 4.8.1. The failure was at test/run-pass/extern-pass-TwoU64s-ref.rs:27.
I got the failure on latest mingw, so this seems unrelated to mingw-w64 but related to recent gcc.

@thadguidry
Copy link
Contributor

@klutzy In that case, we could REALLY use your notes on a new Wiki Page (or GIST) so that all of us can compare how your actually doing this and your step by steps. Can you do this please ?

@klutzy
Copy link
Contributor Author

klutzy commented Sep 14, 2013

@thadguidry On mingw-64's 32-bit toolchain, I did it really straightforwardly: 1) set msys to use mingw-w64's toolchain. 2) copy some mingw dlls (e.g. libgcc_s_dw2-1.dll) to <builddir>/<host>/stage0/bin/, 3) make :)
I think there was no issues around building rustc.

@thadguidry
Copy link
Contributor

I cannot get gcc to be found in the PATH at all, with a clean installation of MinGW with the new steps..https://github.com/mozilla/rust/wiki/Note-getting-started-developing-Rust I need help from you in how you were able to get gcc found correctly on Win7 using those steps. When I do "which gcc", it cannot be found. Can you enlighten me ? How are you launching Msys on Win7 ? Did you change your path and where ? Did you change your /etc/profile in Msys , and how so ? That's what I need help understanding. Start with a clean slate, and try the steps on getting started, and let me know where things are incorrect and how they differed for you...

@klutzy
Copy link
Contributor Author

klutzy commented Sep 15, 2013

Oh my. mingw installer does not set msys configuration at all. I didn't recognized it since I always did it manually.
Installing msys via mingw-get, then run msys.bat at C:/path/to/mingw/msys/1.0/. In msys, execute the following line:

sh /postinstall/pi.sh

This script asks mingw directory (C:/path/to/mingw), which contains /bin or /lib directories.
I'm not sure whether you should restart msys.bat or not, but anyway after restart which gcc should indicate /mingw/bin/gcc.exe.

It is same for mingw-w64 except that you should download msys manually.

@klutzy
Copy link
Contributor Author

klutzy commented Sep 15, 2013

make check-fast failure is due to #9205.

tests expected to fail:

  • extern-pass-TwoU64s-ref
  • extern-return-TwoU64s
  • struct-return

@thadguidry
Copy link
Contributor

Klutzy,

Your a hero ! That did the trick. And it looks like it was just a simple
omission on my part on MinGW's own Getting Started wiki page.
It appears that the real reason was a needed /fstab entry... which that
post install script performs. Nice.

I will update our Getting Started wiki.

Thanks !

On Sat, Sep 14, 2013 at 11:31 PM, klutzy [email protected] wrote:

Oh my. mingw installer set no msys configuration at all. I didn't
recognized it since I always did it manually.
To use msys, run msys.bat and type the following:

$ sh /postinstall/pi.sh

This script asks mingw directory, which contains /bin or /lib directories.
I'm not sure whether you should restart msys.bat or not, anyway after
restart which gcc should indicate /mingw/bin/gcc.exe.

It is same for mingw-w64 except that you should obtain msys manuallyhttp://sourceforge.net/apps/trac/mingw-w64/wiki/MSYS
.


Reply to this email directly or view it on GitHubhttps://github.com//issues/8996#issuecomment-24464238
.

-Thad
Thad on Freebase.com http://www.freebase.com/view/en/thad_guidry
Thad on LinkedIn http://www.linkedin.com/in/thadguidry/

@thestinger
Copy link
Contributor

Yet another issue this would fix: #10315

@pnkfelix
Copy link
Member

pnkfelix commented Nov 7, 2013

accepted for P-high, want this for 1.0.

@klutzy
Copy link
Contributor Author

klutzy commented Nov 20, 2013

(NOTE: here i686-w64-mingw32 means mingw-w64 32bit (i686), and i686-pc-mingw means mingw what we use. Yes it's confusing...)

If you run ../configure && make on i686-w64-mingw32, configure thinks it's on i686-pc-mingw32, and downloads corresponding snapshot then build stage2. I've checked it last week and it even passed make check. The only issue I've found is that rustc.exe -v shows host: i686-pc-mingw32.
Similarly, you may provide i686-w64-mingw32 "bootstrap" binary by copying mingw one. (I haven't tried this)

In the other hand, mingw-w64 provides a lot of options: exception model (dwarf / sjlj) and threading model (win32 / posix). I'm curious if this affects some compatibility.
Debian provides sjlj-win32. Our mingw uses dwarf-win32. I've always used dwarf-win32 but I got rustc.exe which is slower than @luqmana's one (from #10578), so I'm going to rebuild it on sjlj-win32 and see what made it bad.

@klutzy
Copy link
Contributor Author

klutzy commented Nov 20, 2013

I found rustllvm.dll depends on winpthreads.dll because llvm is built with pthreads, so I added --disable-pthreads to LLVM_OPTS and built llvm. Now it seems to run as fast as cross-one or mingw does.

@vadimcn
Copy link
Contributor

vadimcn commented Nov 20, 2013

@klutzy As I mentioned here, the reason for slowness seems to be sub-optimal implementation of mutexes in libwinpthreads that is bundled with mingw-w64.

I could be wrong, of course, so perhaps this should be looked over by a second pair of eyes. I've used the VerySleepy profiler.

@klutzy
Copy link
Contributor Author

klutzy commented Nov 22, 2013

Steps to get "bootstrapped" rustc: (outdated: see below)

  • Apply Allow cross-compiling with mingw64. #10578.
  • modify configure: LLVM_OPTS = "$LLVM_OPTS --disable-pthreads". This is purely for speed; may require more investigation as @vadimcn said.
  • ../configure --build=i686-w64-mingw32 --enable-rust-local --rust-local-root=/path/to/rust-stage0
    where /path/to/rust-stage0 contains latest snapshot of mingw.
  • make. It may fail due to missing mingw dlls: When rust-local option is enabled, it copies rustc.exe and some dlls to stage0 directory. However it does not copy mingw runtime dlls. (I copied them manually for now.)

Again, I used mingw-builds x32-4.8.1-win32-dwarf-rev5 (threading model = win32 / exception model = dwarf).
It seems to work well, but make check-stage2-std fails. Needs more investigation.

@klutzy
Copy link
Contributor Author

klutzy commented Nov 23, 2013

The failure was due to wrong linker: i686-w64-mingw32-gcc.exe was used instead of g++.exe. #10578 had an issue which is now fixed. Now it passes make check-fast. Yay!

@klutzy klutzy mentioned this issue Nov 24, 2013
bors added a commit that referenced this issue Nov 26, 2013
This patchset fixes some parts broken on Win64.

This also adds `--disable-pthreads` flags to llvm on mingw-w64 archs (both 32-bit and 64-bit, not mingw) due to bad performance. See #8996 for discussion.
@klutzy
Copy link
Contributor Author

klutzy commented Dec 20, 2013

Now ../configure --build=i686-w64-mingw32 && make builds properly working rustc. (there is no need to use --rust-local-root)

@brson brson self-assigned this Apr 8, 2014
@brson
Copy link
Contributor

brson commented Apr 11, 2014

The bots are switched over to mingw-w64 and I've attempted to update the docs on the wiki to reflect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
O-windows Operating system: Windows P-medium Medium priority
Projects
None yet
Development

No branches or pull requests

6 participants