Segmentation fault with MinGW-w64 due to -lucrt

[GalaxySnail]

The minimal reproducer:

# hello.pyx
from libc.stdio cimport printf

def say_hello():
    printf("Hello %d world!\n", 42)

# setup.py
from setuptools import setup
from Cython.Build import cythonize

setup(
    ext_modules=cythonize("hello.pyx"),
)

Build and run it in msys2 environment:

$ python -c 'import sys; print(sys.version)'
3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]

$ CC=/mingw64/bin/gcc python setup.py build_ext -i
...

$ python -X dev -c 'import hello; hello.say_hello()'
Windows fatal exception: access violation

Current thread 0x0000140c (most recent call first):
  File "<string>", line 1 in <module>
zsh: segmentation fault  python -X dev -c 'import hello; hello.say_hello()'

This is because mingw64-gcc links msvcrt by default [1], but distutils links ucrt, which causes incompatibility.

It was introduced in #41 and was reported in #109 a year ago. Simply removing "ucrt" can fix this.

---

For the relationship between msvcrt and ucrt on MinGW-w64, this post talked about it clearly.

tl; dr: -lucrt never works for gcc. On msys2 mingw64 environment, it causes segfault; On msys2 ucrt64 environment, it's a noop. It's not msys2-specific, in fact all MinGW-w64 gcc builds are the same. I have tested on w64devkit and cygwin, the results are at the bottom.

In MinGW-w64, libmsvcrt.a is just an alias, which varies with --with-default-msvcrt in the configuration step. By default, it's msvcrt, which means libmsvcrt.a is an alias for libmsvcrt-os.a. Most MinGW-w64 builds are configured like this, so does msys2 mingw64 environment.[2] In this case, the compiled executable will dynamically link to msvcrt.dll. --with-default-msvcrt can also be set to ucrt, which is msys2 ucrt64 environment does, means libmsvcrt.a is an alias for libucrt.a, and the compiled executable will dynamically link to ucrtbase.dll.

$ sha256sum /mingw64/lib/lib{msvcrt*,ucrt.a}
b7c701c34572d672f19ddb6dbd4d28c5f8ea9d08391184a76edc65b7ca2e3811 */mingw64/lib/libmsvcrt.a
b7c701c34572d672f19ddb6dbd4d28c5f8ea9d08391184a76edc65b7ca2e3811 */mingw64/lib/libmsvcrt-os.a
3385e765d960fa88a833e5be96a07d8ea1fc4772de5992576e7f7f4eff952da1 */mingw64/lib/libucrt.a

$ sha256sum /ucrt64/lib/lib{msvcrt*,ucrt.a}
3bc0a7a52d0b45c39f2b399e4b5843320246bb6c136c78dc738d8143a7d7664c */ucrt64/lib/libmsvcrt.a
988e3f3f60af32dad4991d05ac4d46e55bfee4b257ea6bb38b38b91d1d57bd77 */ucrt64/lib/libmsvcrt-os.a
3bc0a7a52d0b45c39f2b399e4b5843320246bb6c136c78dc738d8143a7d7664c */ucrt64/lib/libucrt.a

In both cases, libgcc is implicitly linked with -lmsvcrt (remember, it's an alias).

$ gcc -dumpspecs | grep -B 1 msvcrt
*libgcc:
%{mthreads:-lmingwthrd} -lmingw32      %{static|static-libgcc:-lgcc -lgcc_eh}  %{!static:    %{!static-libgcc:      %{!shared:        %{!shared-libgcc:-lgcc -lgcc_eh}        %{shared-libgcc:-lgcc_s -lgcc}       }      %{shared:-lgcc_s -lgcc}     }   }     -lmoldname -lmingwex %{!mcrtdll=*:-lmsvcrt} %{mcrtdll=*:-l%*} -lkernel32

So, on msys2 mingw64 environment, libgcc is always linked to msvcrt.dll. If -lucrt is passed, the C program will link to ucrtbase.dll, which is incompatible with libgcc and will cause segmentation faults. On msys2 ucrt64 environment, obviously, ucrt is linked implicitly, so it's not necessary to pass -lucrt.

For gcc, if you want the program to link to ucrt properly on a MinGW-w64 --with-default-msvcrt=msvcrt environment, the only way is to dump gcc's specs and modify it (by gcc -dumpspecs, which has been metioned in that post). imba-tjd/mingw64ccompiler is doing the right thing.

By the way, even the simplest C hello world compiled with gcc -lucrt behaves the same here.

---

click to expand

First, generate hello.c from hello.pyx with cython:

$ cython -3 hello.pyx

msys2 mingw64 prefix

without -lucrt

$ /mingw64/bin/gcc -shared "-IC:/Program Files/Python310/include" hello.c "-LC:/Program Files/Python310" "-LC:/Program Files/Python310/Libs" -lpython310 -o hello.pyd

$ python -X dev -c 'import hello; hello.say_hello()'
Hello 42 world!

$ objdump -p hello.pyd | grep 'DLL Name'
        DLL Name: KERNEL32.dll
        DLL Name: msvcrt.dll
        DLL Name: python310.dll

with -lucrt

$ /mingw64/bin/gcc -shared "-IC:/Program Files/Python310/include" hello.c -lucrt "-LC:/Program Files/Python310" "-LC:/Program Files/Python310/Libs" -lpython310 -o hello.pyd

$ python -c 'import hello; hello.say_hello()'
zsh: segmentation fault  python -c 'import hello; hello.say_hello()'

$ objdump -p hello.pyd | grep 'DLL Name'
        DLL Name: KERNEL32.dll
        DLL Name: msvcrt.dll
        DLL Name: api-ms-win-crt-environment-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-time-l1-1-0.dll
        DLL Name: python310.dll

msys2 ucrt64 prefix

without -lucrt

$ /ucrt64/bin/gcc -shared "-IC:/Program Files/Python310/include" hello.c "-LC:/Program Files/Python310" "-LC:/Program Files/Python310/Libs" -lpython310 -o hello.pyd

$ python -c 'import hello; hello.say_hello()'
Hello 42 world!

$ objdump -p hello.pyd | grep 'DLL Name'
        DLL Name: KERNEL32.dll
        DLL Name: api-ms-win-crt-environment-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-time-l1-1-0.dll
        DLL Name: python310.dll

with -lucrt

$ /ucrt64/bin/gcc -shared "-IC:/Program Files/Python310/include" hello.c -lucrt "-LC:/Program Files/Python310" "-LC:/Program Files/Python310/Libs" -lpython310 -o hello.pyd

$ python -c 'import hello; hello.say_hello()'
Hello 42 world!

$ objdump -p hello.pyd | grep 'DLL Name'
        DLL Name: KERNEL32.dll
        DLL Name: api-ms-win-crt-environment-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-time-l1-1-0.dll
        DLL Name: python310.dll

w64devkit 1.17.0

• home page: https://github.com/skeeto/w64devkit
• download: https://github.com/skeeto/w64devkit/releases/download/v1.17.0/w64devkit-mini-1.17.0.zip

$ sha256sum x86_64-w64-mingw32/lib/libmsvcrt.a x86_64-w64-mingw32/lib/libmsvcrt-os.a x86_64-w64-mingw32/lib/libucrt.a
e136d6158cf573080aba84c6a910824cc97ba00a966372af06a3e799c0ed5500  x86_64-w64-mingw32/lib/libmsvcrt.a
e136d6158cf573080aba84c6a910824cc97ba00a966372af06a3e799c0ed5500  x86_64-w64-mingw32/lib/libmsvcrt-os.a
931553676e70104c98e95bcbfd063f1ab24ab9a1ab5cc7d0161c686940550e38  x86_64-w64-mingw32/lib/libucrt.a

without -lucrt

$ gcc -shared "-IC:/Program Files/Python310/include" hello.c "-LC:/Program Files/Python310" "-LC:/Program Files/Python310/Libs" -lpython310 -o hello.pyd

$ python -c 'import hello; hello.say_hello()'
Hello 42 world!

$ objdump -p hello.pyd | grep 'DLL Name'
        DLL Name: KERNEL32.dll
        DLL Name: msvcrt.dll
        DLL Name: python310.dll

with -lucrt

$ gcc -shared "-IC:/Program Files/Python310/include" hello.c -lucrt "-LC:/Program Files/Python310" "-LC:/Program Files/Python310/Libs" -lpython310 -o hello.pyd

$ python -c 'import hello; hello.say_hello()'
zsh: segmentation fault  python -c 'import hello; hello.say_hello()'

$ objdump -p hello.pyd | grep 'DLL Name'
        DLL Name: KERNEL32.dll
        DLL Name: msvcrt.dll
        DLL Name: api-ms-win-crt-environment-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-time-l1-1-0.dll
        DLL Name: python310.dll

cygwin mingw64-gcc

• home page: https://cygwin.com/
• package name: mingw64-x86_64-gcc-core
• package version: 11.3.0-1

$ sha256sum /usr/x86_64-w64-mingw32/sys-root/mingw/lib/lib{msvcrt*,ucrt.a}
da55e191ae4ef503db42d36e400ca3dcc8f66e608f762e6a725aca9441088c38 */usr/x86_64-w64-mingw32/sys-root/mingw/lib/libmsvcrt.a
da55e191ae4ef503db42d36e400ca3dcc8f66e608f762e6a725aca9441088c38 */usr/x86_64-w64-mingw32/sys-root/mingw/lib/libmsvcrt-os.a
672f28150c81fb070f5688334f6f2f011f7120ab79f857cc2ccd9ac21e8f49bb */usr/x86_64-w64-mingw32/sys-root/mingw/lib/libucrt.a

without -lucrt

$ x86_64-w64-mingw32-gcc -shared "-IC:/Program Files/Python310/include" hello.c "-LC:/Program Files/Python310" "-LC:/Program Files/Python310/Libs" -lpython310 -o hello.pyd

$ python -c 'import hello; hello.say_hello()'
Hello 42 world!

$ objdump -p hello.pyd | grep 'DLL Name'
        DLL Name: KERNEL32.dll
        DLL Name: msvcrt.dll
        DLL Name: python310.dll

with -lucrt

$ x86_64-w64-mingw32-gcc -shared "-IC:/Program Files/Python310/include" hello.c -lucrt "-LC:/Program Files/Python310" "-LC:/Program Files/Python310/Libs" -lpython310 -o hello.pyd

$ python -c 'import hello; hello.say_hello()'
zsh: segmentation fault  python -c 'import hello; hello.say_hello()'

$ objdump -p hello.pyd | grep 'DLL Name'
        DLL Name: KERNEL32.dll
        DLL Name: msvcrt.dll
        DLL Name: api-ms-win-crt-environment-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-time-l1-1-0.dll
        DLL Name: python310.dll

---

CC @imba-tjd

[imba-tjd]

AFAIK CPython by default is using ucrt. Is it ok to use msvcrt in C extensions in this case?

Indeed -lucrt in a default-msvcrt=msvcrt mingw causes issue. And indeed if -lmsvcrt is an alias, no need to add -lucrt. But if the first question is no, maybe we can do more.

[GalaxySnail]

AFAIK CPython by default is using ucrt. Is it ok to use msvcrt in C extensions in this case?

Yes, it's OK. Only if you pass a libc object to a dll that is linked to an incompatible libc, it will fail. Otherwise, it's fine. For example:

// mylib.h
int say_hello(FILE *fp);

// mylib.c
#include <stdio.h>
#include "mylib.h"

int say_hello(FILE *fp)
{
    return fprintf(fp, "Hello %d world!\n", 42);
}

// main.c
#include <stdio.h>
#include "mylib.h"

int main(void)
{
    say_hello(stdout);
    return 0;
}

$ /ucrt64/bin/gcc -shared -Wall mylib.c -o mylib.dll
$ /mingw64/bin/gcc -Wall main.c mylib.dll -o main.exe
$ ./main
zsh: segmentation fault  ./main

However, Python/C API doesn't need any libc objects, so it doesn't matter. Just like this:

// mylib.h
int say_hello(void);

// mylib.c
#include <stdio.h>
#include "mylib.h"

int say_hello(void)
{
    return fprintf(stdout, "Hello %d world!\n", 42);
}

// main.c
#include <stdio.h>
#include "mylib.h"

int main(void)
{
    say_hello();
    return 0;
}

$ /ucrt64/bin/gcc -shared -Wall mylib.c -o mylib.dll
$ /mingw64/bin/gcc -Wall main.c mylib.dll -o main.exe
$ ./main.exe
Hello 42 world!