Windows: exporting a tuple leads to "A dynamic link library (DLL) initialization routine failed."

[paugier]

I discovered a funny bug affecting Transonic users on Windows when using Visual Studio and clang-cl (but not MinGW, which explains why I didn't detect that with our CI on Github Actions). Pythran files written by Transonic export a tuple of strings, which usually work fine but not for this specific case.

It is quite simple to reproduce with a Windows machine (details because working on Windows is new for me):

• install python 3.11 from the Microsoft Store
• install Visual Studio 2022 with clang-cl
• open "x64 native tools command prompt for VS 2022" (to target x64)

We get:

clang-cl --version
clang version 17.0.3
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\Llvm\x64\bin

python --version
Python 3.11.8

• create a venv, activate it and install pythran

python -m venv .venv
.venv\Scripts\activate.bat
pip install pythran

• create a file simple.py

#pythran export my_tuple
my_tuple = ("a",)

• Set the environment variables with

set CC=clang-cl
set CXX=clang-cl

• create the extension with pythran simple.py

• import the extension python -c "import simple" gives:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: DLL load failed while importing simple: A dynamic link library (DLL) initialization routine failed.

Note: actually one can also reproduce simply with an environment based on conda-forge created with mamba create -n env_pythran pythran clang (in this case clang==18.1.2).

Even though I guess exporting a tuple from a pythran file is not common, this bug is quite annoying for me because it affects all Pythran extensions produced through Transonic on Windows for conda-forge 🙂 Therefore, I think I'm going to quickly release a Transonic which does not export this tuple to overcome this bug.

[paugier]

Additional information about this bug:

• I cannot reproduce on Linux with conda-forge.
• transonic 0.6.4 avoids this bug for most cases 🙂
• this bug can also be triggered by exporting a dictionary 🙁 , which is much more annoying because it is used for some extensions by Transonic and it cannot be simply avoided. I'm going to give a minimum example next week when I have access to a Windows machine.

[serge-sans-paille]

It's difficult for me to debug that one as I don't have access to a windows machine. I confirm I can't reproduce on windows. Could you try the following:

pythran -E a.py
sed -i -e 's/PyModule_AddObject/PyModule_AddObjectRef/g' a.cpp
pythran a.cpp
python -c 'import a; print(a.my_tuple)'

I don't see why this would change the behavior (it should actually introduce a leak), just trying.

[paugier]

Replacing PyModule_AddObject by PyModule_AddObjectRef changes nothing.

If I comment PyModule_AddObject(theModule, "my_tuple", my_tuple);, same thing.

If I comment static PyObject* my_tuple = to_python(__pythran_simple::my_tuple()());, the extension can be imported (but of course without my_tuple).

Then, I tried to replace this line with

Py_ssize_t size = 2;
PyObject* my_tuple = PyTuple_New(size);

I again get the same error.

Then, I tried to manually write a minimal cpp file exporting one tuple and I again get the same error, but I really don't know if it is correct.

#include <Python.h>

Py_ssize_t size = 2;
PyObject* my_tuple = PyTuple_New(size);

static PyMethodDef methods[] = {
    {NULL, NULL, 0, NULL}  /* Sentinel */
};

static struct PyModuleDef mod_def = {
    PyModuleDef_HEAD_INIT,
    "my_mod",
    "",
    -1,
    methods,
    NULL,
    NULL,
    NULL,
    NULL
};

PyObject *PyInit_my_mod(void)
{
    PyObject* theModule = PyModule_Create(&mod_def);
    PyModule_AddObject(theModule, "my_tuple", my_tuple);
    return theModule;
}

I tried to check on Linux but using pythran to compile such minimal cpp file does not work on Linux ("ImportError: dynamic module does not define module export function (PyInit_my_mod)").