Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build with Mingw-w64 #6170

Open
ghost opened this issue Jun 9, 2018 · 23 comments
Open

Build with Mingw-w64 #6170

ghost opened this issue Jun 9, 2018 · 23 comments
Labels
kind:feature platform:windows-gnu Windows support based on the MinGW-w64 toolchain + MSYS2

Comments

@ghost
Copy link

ghost commented Jun 9, 2018

Crystal is in the process of supporting Visual Studio. However Mingw-w64 should
be supported as well.

http://mingw-w64.org

@sam0x17
Copy link
Contributor

sam0x17 commented Jun 10, 2018

I'm all for this because I like the idea of being able to build windows binaries without windows, but not to the detriment / timeliness of the current windows development being done. That said, maybe this will make windows CI easier? or no?

@straight-shoota
Copy link
Member

Of course, Mingw64 should be supported. It doesn't feel like a pressing issue to me, though.

I don't no much about the practical differences between MSVC and Mingw64 in regards to Crystal supporting it. But, as I alread said in #5430 (comment), the current effort should focus on porting the stdlib to Win32 API. That needs to be done anyway and still requires a lot of work. Supporting an alternate tool chain doesn't gain anything until that is at least somewhat matured.

this will make windows CI easier?

Windows CI needs to run on windows anyway, so there is no point in cross compiling from a different platform.

@faustinoaq
Copy link
Contributor

faustinoaq commented Oct 17, 2018

Windows CI needs to run on windows anyway, so there is no point in cross compiling from a different platform.

Azure CI looks very interesting. It supports Windows, Linux and Mac and is free fpr Open Source Projects: https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/ @bcardiff @RX14 👀

@straight-shoota
Copy link
Member

travis-ci has also announced windows support: https://blog.travis-ci.com/2018-10-11-windows-early-release

That should be fairly easy to integrate into the CI setup

@mominshaikhdevs
Copy link

mominshaikhdevs commented Mar 15, 2023

Since Crystal is maturing support towards MSVC and Win32 APIs directly instead of mingw-w64 (#26 (comment)), Is this still relevant?

@straight-shoota
Copy link
Member

straight-shoota commented Mar 15, 2023

Technically yes. Mingw-w64 is a different target than msvc and supporting it is still a goal I think. Although it's not actively driven at the moment due to low importance.
So I guess we could also close this issue because it's not directly actionable, with the option to re-open once there's activity towards supporting this platform.
I expect it shouldn't be much work, but also probably not many people care about it due to native Win32 support and the overall decline of Mingw due to WSL.

@mominshaikhdevs
Copy link

Exactly. The Crystal team can always re-open the issue if there is enough demand for it.

@ghost
Copy link

ghost commented Mar 17, 2023

Mingw-w64 is a different target than msvc and supporting it is still a goal I think.

correct. which is the reason language like Rust offer a MSVC and GNU build for Windows, the latter being MinGW-w64.

So I guess we could also close this issue because it's not directly actionable

how is it not actionable? wouldn't adding support for it be an action?

re-open once there's activity towards supporting this platform

please dont do that. you have over 1000 issues open, no harm in leaving this open. closing by a maintainer sends a strong signal that this is essentially WONTFIX, regardless of any reason given.

not many people care about it due to native Win32 support and the overall decline of Mingw due to WSL.

both of these items are false. Native Win32 support is only marginally useful, as developing with that environment requires an install of Visual Studio, which is at least 100 times the install size of MSYS2 and similar options. secondly, WSL has no impact on MinGW/MSYS2, because WSL is a Linux emulator, so you take a huge performance hit using that option, whereas MSYS2 offers native Windows output.

@straight-shoota
Copy link
Member

how is it not actionable? wouldn't adding support for it be an action?

Actionable means to have a clear path what's necessary to achieve this, or at least the intention to discuss that and determine a path.

you have over 1000 issues open, no harm in leaving this open. closing by a maintainer sends a strong signal that this is essentially WONTFIX, regardless of any reason given.

Yes, we'd like to reduce that number. There's not much value in having lots of stale issues. We know that Mingw is not supported and that support for it would be welcome. But no need to keep an issue around that does nothing.
There are dozens if not hundreds of somewhat reasonably relevant platforms which Crystal does not currently support. It doesn't make sense to keep open issues just to remind of that.

both of these items are false. Native Win32 support is only marginally useful, as developing with that environment requires an install of Visual Studio, which is at least 100 times the install size of MSYS2 and similar options. secondly, WSL has no impact on MinGW/MSYS2, because WSL is a Linux emulator, so you take a huge performance hit using that option, whereas MSYS2 offers native Windows output.

Sorry, I have a hard time making any sense of "Native Win32 support is only marginally useful".
This has been one of the most requested features since almost the beginning of Crystal (#26). And even though we're not yet fully there, it's already used by many people. I fail to see any marginality of the most popular desktop operating system's native environment.

I also beg to differ that "WSL has no impact on MinGW/MSYS2". MSYS2 is a tool for using Unix programs on Windows. I used that myself back in the day. But I've long moved to WSL because it's a much more complete Unix environment with an actual Linux kernel interface that offers a lot more possibilities. Sure, WSL is not a replacement for all use cases of MSYS2. But for me, it is a much better one. And I'm pretty sure for many others as well.

@ghost
Copy link

ghost commented Mar 17, 2023

Yes, we'd like to reduce that number. There's not much value in having lots of stale issues. We know that Mingw is not supported and that support for it would be welcome. But no need to keep an issue around that does nothing. There are dozens if not hundreds of somewhat reasonably relevant platforms which Crystal does not currently support. It doesn't make sense to keep open issues just to remind of that.

I get the desire to want to close issues, but closing them doesn't fix anything, it just makes you feel better. they should be closed when they are actually fixed, or when they decision is made that they will never be fixed.

Sorry, I have a hard time making any sense of "Native Win32 support is only marginally useful". This has been one of the most requested features since almost the beginning of Crystal (#26).

sorry I was just using your language. a better comment would be "MSVC support is only marginally useful". Windows developers use MSVC when its the only option. Ideally MSYS2 support would also exist, as it has several benefits over MSVC.

I also beg to differ that "WSL has no impact on MinGW/MSYS2". MSYS2 is a tool for using Unix programs on Windows.

MSYS2 is able to produce native Windows programs:

https://packages.msys2.org/base/mingw-w64-gcc

Sure, WSL is not a replacement for all use cases of MSYS2.

its not a replacement at all. WSL "programs" are linux programs that work on Windows via an emulator. MSYS2 produces native Windows programs that rely on the Windows API. the two couldn't be more different.

@straight-shoota
Copy link
Member

straight-shoota commented Mar 17, 2023

I get the desire to want to close issues, but closing them doesn't fix anything, it just makes you feel better. they should be closed when they are actually fixed, or when they decision is made that they will never be fixed.

Keeping it open also does nothing towards getting it resolved. Closing issues with no forseeable activity helps to keep the amount of currently relevant issues manageable.
And I think it's more honest to close the issue to make clear that it's not an active goal. The Core Team won't pursue Mingw support, so perhaps it might be more accurate to describe this as "wontfix". Although it's not a hard one. We're accepting contributions from the community. But until someone picks that up and starts actively working on it, this issue is just a zombie.

@HertzDevil
Copy link
Contributor

HertzDevil commented Mar 4, 2024

To be clear, there are a few different toolchains possible on Windows:

Toolchain Compiler Triple Flags Additional runtimes
Microsoft Visual C++ cl.exe x86_64-pc-windows-msvc windows msvc win32 VCRedist, UCRT
MinGW-w64 (native) gcc.exe x86_64-w64-windows-gnu windows gnu win32 VCRedist, (UCRT or MSVCRT)
MinGW-w64 (MSYS) gcc.exe x86_64-w64-windows-gnu windows gnu unix msys-2.0.dll
Cygwin gcc.exe x86_64-unknown-cygwin
x86_64-pc-windows-cygnus
windows gnu unix cygwin1.dll

MSYS is the "MSYS" environment of MSYS2, as indicated by $MSYSTEM; native is any other MSYS2 environment (e.g. UCRT64), or a stand-alone installation like WinLibs. Clang and LLVM are unmaintained and extremely outdated on Cygwin's repository, so we will forget about that for the moment.

While the majority of our MSVC toolchain work is completed, some deserves extra attention regarding what happens if other Windows toolchains are going to be supported, such as the meaning of "default shell". There is also the issue that MinGW-w64's license will always be more permissive than MSVC's, and that could be a deal breaker for some people. So I think now is a good time to revisit those alternative toolchains.

@ysbaddaden
Copy link
Contributor

I see little value in supporting Cygwin or MSYS-2.0 since we support the official Windows API: there shouldn't be any portability issues of crystal code.

I can see value in MinGW-w64 as an alternative to MSVC, especially if we could link an EXE from linux or macos (though I'm not sure about the runtime in that case).

@beta-ziliani
Copy link
Member

I see little value in supporting Cygwin or MSYS-2.0 since we support the official Windows API: there shouldn't be any portability issues of crystal code.

+1

especially if we could link an EXE from linux or macos

What? 😮

@straight-shoota
Copy link
Member

especially if we could link an EXE from linux or macos

What? 😮

Sure. Linking against msys we wouldn't need any Windows system libraries, so it should be entirely feasible to cross-link executables.

@beta-ziliani
Copy link
Member

Ah, right, thanks for clarifying 🙇

@HertzDevil
Copy link
Contributor

You can cross-link Windows executables straight from non-Windows systems using x86_64-w64-mingw32-gcc, as long as all the third-party dependencies are also cross-linked. MinGW-w64 maintains its own import libraries for the Win32 API (which are always dynamically linked), and that's why you don't need anything from a Visual Studio installation for this to work.

x86_64-w64-windows-gnu is not merely about switching the compiler, by the way; the build environment should also answer questions like whether programs like sh.exe or make.exe or pkg-config.exe are expected to exist at build time, which makes a difference for things like library version detection. There is a reason why RubyInstaller comes with the MSYS2 UCRT64 environment or requires you to bring your own.


Anyway, here is a worked example of using MinGW-w64 stand-alone on Windows:

  • Install WinLibs, choosing Win64 + UCRT runtime + POSIX threads, to a location like %USERPROFILE%\mingw64;

  • Compile and install PCRE2 for MinGW-w64: (do not use the MSVC libs that come with Crystal)

    git clone https://github.com/PCRE2Project/pcre2.git
    cd pcre2
    mkdir build
    cd build
    set PATH=%USERPROFILE%\mingw64\bin;%PATH%
    cmake .. -G "MinGW Makefiles" -DBUILD_SHARED_LIBS=ON
    cmake --build .
    cmake -DCMAKE_INSTALL_PREFIX=%USERPROFILE%\crystal-libs-mingw64 -P cmake_install.cmake
    
  • Cross-compile the following code to get a test.obj file:

    class String
      def bytesize
        @bytesize
      end
    
      def to_unsafe
        pointerof(@c)
      end
    end
    
    @[Link("kernel32")]
    lib LibC
      alias HANDLE = Void*
      alias DWORD = UInt32
      alias BOOL = Int32
      alias SizeT = UInt64
    
      STD_OUTPUT_HANDLE = 0xFFFFFFF5_u32
    
      fun GetStdHandle(nStdHandle : DWORD) : HANDLE
      fun WriteFile(hFile : HANDLE, lpBuffer : Void*, nNumberOfBytesToWrite : DWORD, lpNumberOfBytesWritten : DWORD*, lpOverlapped : Void*) : BOOL
    
      fun malloc(size : SizeT) : Void*
    end
    
    @[Link("pcre2-8")]
    lib LibPCRE2
      alias Int = Int32
    
      CONFIG_VERSION = 11
    
      fun config = pcre2_config_8(what : UInt32, where : Void*) : Int
    end
    
    def print(msg)
      LibC.WriteFile(LibC.GetStdHandle(LibC::STD_OUTPUT_HANDLE), msg, msg.bytesize, out _, nil)
    end
    
    print("Hello world on ")
    print(Crystal::TARGET_TRIPLE)
    print("\nUsing PCRE2 version ")
    
    len = LibPCRE2.config(LibPCRE2::CONFIG_VERSION, nil)
    buf = LibC.malloc(len).as(UInt8*)
    LibPCRE2.config(LibPCRE2::CONFIG_VERSION, buf)
    LibC.WriteFile(LibC.GetStdHandle(LibC::STD_OUTPUT_HANDLE), buf, len, out _, nil)
    crystal build --prelude=empty --cross-compile --target=x86_64-w64-windows-gnu test.cr
    
  • Link the object file using the MinGW-w64 GCC:

    %USERPROFILE%\mingw64\bin\x86_64-w64-mingw32-gcc.exe test.obj -lpcre2-8 -lkernel32 -lmsvcrt -L%USERPROFILE%\crystal-libs-mingw64\lib
    
  • Copy %USERPROFILE%\crystal-libs-mingw64\bin\libpcre2-8.dll to the current directory; (note the lib prefix which is absent in our MSVC build, so if we are not cross-compiling we need to adjust the dll: parameter for @[Link])

  • Run a.exe:

    Hello world on x86_64-w64-windows-gnu
    Using PCRE2 version 10.43 2024-02-16
    

We can supposedly symlink src/lib_c/x86_64-windows-msvc/ to src/lib_c/x86_64-windows-gnu/, but Crystal needs to know whether it is inside an MSYS2 environment or not, because an MSYS / Cygwin port would have used that same directory.

@HertzDevil HertzDevil moved this to To investigate in Windows Support Mar 15, 2024
@crysbot
Copy link

crysbot commented Jul 25, 2024

This issue has been mentioned on Crystal Forum. There might be relevant details there:

https://forum.crystal-lang.org/t/the-easiest-way-to-prepare-a-ready-to-use-windows-executable-on-linux/7040/2

@HertzDevil HertzDevil moved this from To investigate to In Progress in Windows Support Oct 9, 2024
@HertzDevil
Copy link
Contributor

HertzDevil commented Oct 11, 2024

Apart from cross-compilation, it is also possible to build and distribute Crystal itself using x86_64-windows-gnu as the default triple. Now is the time to consider whether such a distribution would be portable, like the MSVC one, or it would rely on an entire POSIX environment. The main difference is in how the standard library detects the version information of third-party libraries like OpenSSL:

{% if flag?(:win32) %}
{% from_libressl = false %}
{% ssl_version = nil %}
{% for dir in Crystal::LIBRARY_PATH.split(Crystal::System::Process::HOST_PATH_DELIMITER) %}
{% unless ssl_version %}
{% config_path = "#{dir.id}\\openssl_VERSION" %}
{% if config_version = read_file?(config_path) %}
{% ssl_version = config_version.chomp %}
{% end %}
{% end %}
{% end %}
{% ssl_version ||= "0.0.0" %}
{% else %}
{% from_libressl = (`hash pkg-config 2> /dev/null || printf %s false` != "false") &&
(`test -f $(pkg-config --silence-errors --variable=includedir libcrypto)/openssl/opensslv.h || printf %s false` != "false") &&
(`printf "#include <openssl/opensslv.h>\nLIBRESSL_VERSION_NUMBER" | ${CC:-cc} $(pkg-config --cflags --silence-errors libcrypto || true) -E -`.chomp.split('\n').last != "LIBRESSL_VERSION_NUMBER") %}
{% ssl_version = `hash pkg-config 2> /dev/null && pkg-config --silence-errors --modversion libcrypto || printf %s 0.0.0`.split.last.gsub(/[^0-9.]/, "") %}
{% end %}

On MSYS2, assuming a non-portable Crystal installation, there is probably no need for an openssl_VERSION file, so we could change the flag?(:win32) check into flag?(:msvc). But then the backticks on the other branch do not work: (this is of course related to #9030)

In src\openssl\lib_crypto.cr:16:27

 16 | {% from_libressl = (`hash pkg-config 2> /dev/null || printf %s false` != "false") &&
                          ^
Error: error executing command: hash pkg-config 2> /dev/null || printf %s false: The system cannot find the file specified.

They could actually be fixed by wrapping them in sh -c '...', because then the whole command string parses like a single process invocation:

      {% from_libressl = (`sh -c 'hash pkg-config 2> /dev/null || printf %s false'` != "false") &&
                         (`sh -c 'test -f $(pkg-config --silence-errors --variable=includedir libcrypto)/openssl/opensslv.h || printf %s false'` != "false") &&
                         (`sh -c 'printf "#include <openssl/opensslv.h>\nLIBRESSL_VERSION_NUMBER" | ${CC:-cc} $(pkg-config --cflags --silence-errors libcrypto || true) -E -'`.chomp.split('\n').last != "LIBRESSL_VERSION_NUMBER") %}
      {% ssl_version = `sh -c 'hash pkg-config 2> /dev/null && pkg-config --silence-errors --modversion libcrypto || printf %s 0.0.0'`.split.last.gsub(/[^0-9.]/, "") %}

Yet now the initial cross-compilation to x86_64-windows-gnu fails, because it tries to execute these shell commands on an MSVC toolchain host, with or without sh -c. (See also #14376.)

straight-shoota pushed a commit that referenced this issue Oct 11, 2024
Resolves part of #6170. These series of patches allow `--cross-compile --target=x86_64-windows-gnu` to mostly work:

* The `@[ThreadLocal]` annotation and its corresponding LLVM attribute seem to break when targetting `x86_64-windows-gnu`, so Win32 TLS is used instead. This is only needed for `Thread.current`.
* Since MinGW uses `libgcc`, and Crystal relies on the underlying C++ ABI to raise exceptions, we use the Itanium ABI's `_Unwind_*` functions, along with a thin personality function wrapper. ([GCC itself does the same](https://github.com/gcc-mirror/gcc/blob/68afc7acf609be2b19ec05c8393c2ffc7f4adb4a/libgcc/unwind-c.c#L238-L246). See also [Language-specific handler](https://learn.microsoft.com/en-us/cpp/build/exception-handling-x64?view=msvc-170#language-specific-handler) from the Windows x64 ABI docs.)
* MinGW binaries now come with DWARF debug information, so they work under GDB, maybe under LLDB, and probably poorly under the Visual Studio Debugger.
* There is no need to mangle symbol names the same way MSVC binaries do.
* `--cross-compile` now prints ld-style linker flags, rather than MSVC ones. This is still incomplete and includes remnants of the MSVC toolchain like the `/ENTRY` flag; they will be fixed later once we get to native compiler builds.
* `src/lib_c/x86_64-windows-gnu` is now a symlink to `src/lib_c/x86_64-windows-msvc`, since both toolchains are targetting the same Win32 APIs. (This is not Cygwin nor MSYS2's MSYS environment.)
* Lib funs now use the Win32 C ABI, instead of the SysV ABI.
* On MinGW we use GMP proper, and there is no need for MPIR.

After building a local compiler, `bin\crystal build --cross-compile --target=x86_64-windows-gnu` will generate an object file suitable for linking under MinGW-w64. At a minimum, this object file depends on Boehm GC and libiconv, although they can be skipped using `-Dgc_none` and `-Dwithout_iconv` respectively. Then we could use MSYS2's UCRT64 environment to link the final executable:

```
$ pacman -Sy mingw-w64-ucrt-x86_64-gc mingw-w64-ucrt-x86_64-pcre2 mingw-w64-ucrt-x86_64-libiconv mingw-w64-ucrt-x86_64-gmp
$ cc test.obj `pkg-config bdw-gc iconv libpcre2-8 gmp --libs` -lDbgHelp -lole32
```

Stack traces do not work correctly yet. Also note that MSYS2's DLL names are different from the ones distributed with MSVC Crystal, and that cross-compilation never copies the DLL dependencies to the output directory. To make the executable run outside MSYS2, use `dumpbin /dependents` from the MSVC developer prompt to obtain the dependencies, then copy them manually from the MSYS2 `/ucrt64/bin` folder.
@HertzDevil
Copy link
Contributor

HertzDevil commented Oct 18, 2024

LLVM is similar, if we assume that a MinGW-w64-based compiler is only portable within an MSYS2 shell. First the flag?(:win32) guard for the llvm_VERSION-specific macro code can be turned into flag?(:msvc). Then, since the macro backtick still delegates to LibC.CreateProcessW, POSIX shell scripts aren't considered executable on Windows, and now we must attach a shell here:

    {% llvm_config = env("LLVM_CONFIG") || `sh #{__DIR__}/ext/find-llvm-config`.stringify %}

This is somewhat related to #9030 and shows that we probably shouldn't assume /bin/sh to be the default shell on x86_64-windows-gnu.


If $LLVM_CONFIG isn't set, the next line will normally fail:

    {% llvm_version = `#{llvm_config.id} --version`.stringify %}

This is not a parsing issue, but simply because LibC.CreateProcessW cannot locate /ucrt64/bin/llvm-config.exe. (This is a root-relative path, so it might find something else actually.) We have to use cygpath inside src/llvm/ext/find-llvm-config itself:

if [ "$LLVM_CONFIG" ]; then
  case "$(uname -s)" in
    MSYS_NT*|MINGW32_NT*|MINGW64_NT*)
      printf "%s" "$(cygpath -w "$LLVM_CONFIG")"
      ;;
    *)
      printf "%s" "$LLVM_CONFIG"
      ;;
  esac
else
  # ...
fi

or alternatively, as described above, wrap each macro backtick invocation within sh -c, since MSYS2 understands /ucrt64/bin. The first option also makes Makefile work under MSYS2.

straight-shoota pushed a commit that referenced this issue Oct 19, 2024
This is a continuation of #15070 that allows a compiler built with MinGW-w64 to itself build programs correctly. Resolves part of #6170.

* Because linker flags for GCC may now be executed on a Windows environment, we use the correct form of argument quoting. We also drop `-rdynamic` since that only makes sense for ELF executables.
* Targetting `x86_64-windows-gnu`, including normal compilations from such a Crystal compiler, will not copy dependent DLLs to the output directory. Crystal itself and programs built under MSYS2 will just work as long as the proper environment is used. You are on your own here, although `ldd` exists on MSYS2 so that you don't need the MSVC build tools for this.
* The correct GCC compiler flag to select `wmain` over `main` as the C entry point is `-municode`. (The system entry point is presumably `_start` now.)
* `legacy_stdio_definitions.obj` doesn't exist on MinGW-w64, so we disable it outside MSVC.
* For build command lines that are too long on Windows, we use GCC's response file support.

To build a MinGW-w64 compiler:

```cmd
@Rem on the MSVC developer prompt
make -fMakefile.win crystal
bin\crystal build --cross-compile --target=x86_64-windows-gnu src\compiler\crystal.cr -Dwithout_interpreter
```

```sh
# on MSYS2's UCRT64 environment
pacman -Sy \
  mingw-w64-ucrt-x86_64-gc mingw-w64-ucrt-x86_64-pcre2 mingw-w64-ucrt-x86_64-libiconv \
  mingw-w64-ucrt-x86_64-zlib mingw-w64-ucrt-x86_64-openssl mingw-w64-ucrt-x86_64-llvm
cc crystal.obj -o crystal \
  $(pkg-config bdw-gc libpcre2-8 iconv zlib openssl --libs) \
  $(llvm-config --libs --system-libs --ldflags) \
  -lDbgHelp -lole32 -lWS2_32
export CRYSTAL_PATH='lib;$ORIGIN\src'
export CRYSTAL_LIBRARY_PATH=''
```

Now you can run or build a considerable number of files from here, such as `./crystal.exe samples/2048.cr` and `./crystal.exe spec spec/std/regex_spec.cr`. Notable omissions are OpenSSL and LLVM, as fixing their version detection macros is a bit complicated.

The interpreter is not supported. Most likely, `Crystal::Loader` would have a GCC-style `.parse`, but the rest of the functionality would be identical to the MSVC `LoadLibraryExW`-based loader.

~~Also, some invocations like `./crystal.exe spec spec/std/json` will fail since the whole command line string is too long. Similar to MSVC, [GCC also handles response files starting with `@`](https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html), so this can be implemented later; a workaround is to use `--single-module`.~~

For reference, here are all the useful MSYS2 packages and their corresponding pkg-config names:

| MSYS2 package name             | pkg-config name |
|-|-|
| mingw-w64-ucrt-x86_64-gc       | bdw-gc     |
| mingw-w64-ucrt-x86_64-pcre2    | libpcre2-8 |
| mingw-w64-ucrt-x86_64-libiconv | iconv      |
| mingw-w64-ucrt-x86_64-gmp      | gmp        |
| mingw-w64-ucrt-x86_64-zlib     | zlib       |
| mingw-w64-ucrt-x86_64-libxml2  | libxml-2.0 |
| mingw-w64-ucrt-x86_64-libyaml  | yaml-0.1   |
| mingw-w64-ucrt-x86_64-openssl  | openssl    |
| mingw-w64-ucrt-x86_64-libffi   | libffi     |
| mingw-w64-ucrt-x86_64-llvm     | _(use llvm-config instead)_ |
@HertzDevil
Copy link
Contributor

HertzDevil commented Oct 22, 2024

Currently, MSYS2 only provides a Cygwin-based Git, which unlike Git for Windows doesn't handle Windows symlinks correctly. That means when attempting to check out this repository, one of the following things will happen:

  • core.symlinks=false: All symbolic links become plain text files, in particular the src/lib_c/x86_64-windows-gnu directory.
  • core.symlinks=true: ln will fail since lib/markd/lib and lib/reply/lib are recursive and symlinks become hard copies by default.
  • MSYS=winsymlinks:nativestrict core.symlinks=true: Some symlinks will still fail to create, although MSYS='winsymlinks:nativestrict' git restore --source=HEAD :/ at the repository root will successfully restore everything.

This could be a problem for Shards as well.

straight-shoota pushed a commit that referenced this issue Oct 23, 2024
Cross-compiles a MinGW-w64-based Crystal compiler from Ubuntu, then links it on MSYS2's UCRT64 environment. Resolves part of #6170.

The artifact includes the compiler, all dependent DLLs, and the source code only. It is not a complete installation since it is missing e.g. the documentation and the licenses, but it is sufficient for bootstrapping further native compiler builds within MSYS2.

The resulting binary is portable within MSYS2 and can be executed from anywhere inside an MSYS2 shell, although compilation requires `mingw-w64-ucrt-x86_64-cc`, probably `mingw-w64-ucrt-x86_64-pkgconf`, plus the respective development libraries listed in #15077. The DLLs bundled under `bin/` are needed to even start Crystal since they are dynamically linked at load time; they are not strictly needed if Crystal is always run only within MSYS2, but that is the job of an actual `PKGBUILD` file.
straight-shoota pushed a commit that referenced this issue Oct 24, 2024
Introduces new methods for extracting COFF debug information from programs in the PE format, integrating them into Crystal's existing DWARF parsing functionality. Resolves part of #6170.

It is questionable whether reusing `src/exception/call_stack/elf.cr` for MinGW-w64 is appropriate, since nothing here is in the ELF format, but this PR tries to avoid moving existing code around, save for the old `Exception::CallStack.setup_crash_handler` as it remains the only common portion between MSVC and MinGW-w64.
@HertzDevil HertzDevil added platform:windows-gnu Windows support based on the MinGW-w64 toolchain + MSYS2 and removed platform:windows Windows support based on the MSVC toolchain / Win32 API labels Oct 24, 2024
@straight-shoota
Copy link
Member

I suppose we could work with core.symlinks=false.
It requires the compiler to resolve symlink text files which shouldn't be too hard to implement.

@HertzDevil
Copy link
Contributor

HertzDevil commented Oct 31, 2024

While we have been using the UCRT64 environment so far, the same cross-compilation also works for CLANG64 and MINGW64, but an even faster alternative is to download the "portable" artifact and do CRYSTAL=.../x86_64-mingw-w64-crystal/bin/crystal.exe make in that new environment. Their differences are:

  • cc and ld in CLANG64 are actually Clang and LLD, so all compiled binaries (including the compiler itself) depend on libc++.dll and libunwind.dll, rather than libstdc++-6.dll and libgcc_s_seh-1.dll.
  • MINGW64 links programs against the older MSVCRT runtime msvcrt.dll (not to be confused with msvcrt.lib, the MSVC toolchain's library for the C runtime startup code), instead of the UCRT. This older runtime's implementation for LibC.strtof and LibC.strtod is not C99-compliant, and fails to handle infinities and NaNs, affecting String#to_f and #to_f32. Ideally this will be fixed as part of Functions that depend on the current C locale #11952 though. Additionally, the @[Link] annotation in src/lib_c.cr needs to be commented out, otherwise C symbols will conflict.

The CLANGARM64 environment is only meaningful on an ARM64 Windows host, but it should make porting to ARM64 relatively straightforward.

The MSYS environment is a bit different. LLVM never officially supports MSYS, so if you ever see an x86_64-windows-msys triple, it originates from the numerous unmaintained patches exclusive to the MSYS2 package repository. I don't think there is much interest in getting Crystal to work there, now that WSL exists.

CTC97 pushed a commit to CTC97/crystal that referenced this issue Nov 9, 2024
)

Cross-compiles a MinGW-w64-based Crystal compiler from Ubuntu, then links it on MSYS2's UCRT64 environment. Resolves part of crystal-lang#6170.

The artifact includes the compiler, all dependent DLLs, and the source code only. It is not a complete installation since it is missing e.g. the documentation and the licenses, but it is sufficient for bootstrapping further native compiler builds within MSYS2.

The resulting binary is portable within MSYS2 and can be executed from anywhere inside an MSYS2 shell, although compilation requires `mingw-w64-ucrt-x86_64-cc`, probably `mingw-w64-ucrt-x86_64-pkgconf`, plus the respective development libraries listed in crystal-lang#15077. The DLLs bundled under `bin/` are needed to even start Crystal since they are dynamically linked at load time; they are not strictly needed if Crystal is always run only within MSYS2, but that is the job of an actual `PKGBUILD` file.
CTC97 pushed a commit to CTC97/crystal that referenced this issue Nov 9, 2024
Introduces new methods for extracting COFF debug information from programs in the PE format, integrating them into Crystal's existing DWARF parsing functionality. Resolves part of crystal-lang#6170.

It is questionable whether reusing `src/exception/call_stack/elf.cr` for MinGW-w64 is appropriate, since nothing here is in the ELF format, but this PR tries to avoid moving existing code around, save for the old `Exception::CallStack.setup_crash_handler` as it remains the only common portion between MSVC and MinGW-w64.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:feature platform:windows-gnu Windows support based on the MinGW-w64 toolchain + MSYS2
Projects
None yet
Development

No branches or pull requests

9 participants