c-shared (DLL) library created for Windows fails to initialize runtime when called from C# application

[rickfillion]

What version of Go are you using (go version)?

1.10 darwin/amd64
Also tried 1.10.2, 1.10.3, 1.11beta2

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/rfillion/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/rfillion/go"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/sw/bykzdlq92_nfwpxflx4m_8jm0000gp/T/go-build405501066=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I have a very very simple main.go file:

package main

import "C"
import "fmt"

func main() {
	fmt.Println(Bar())
	fmt.Println(Bar2())	
	return 
}

//export Bar
func Bar() C.long {
	fmt.Println("GOT HERE")
	return 54
}

//export Bar2
func Bar2() *C.char {
	return C.CString("Hello World From Go!")
}

I'm cross-compiling it as a DLL for Windows/x86 from my AMD64 MacOS 10.13 device via:

GOOS=windows GOARCH=386 CGO_ENABLED=1 CC=i686-w64-mingw32-gcc go build -buildmode=c-shared -o main.dll main.go

Then I'm attempting to make using of this DLL from within a C# application compiled with Visual Studios 2017 on Windows 10. Complete source for this application is:

using System;
using System.Runtime.InteropServices;

namespace x86Hullo
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Huuullllloooo!!");
            Console.WriteLine("Before Bar");
            GoFunctions.Bar();
            Console.WriteLine("After Bar");
            Console.WriteLine("Goodbye.");
        }
    }


    static class GoFunctions
    {
        [DllImport(@"C:\Users\rfillion\main.dll", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.StdCall)]
        public static extern long Bar();
    }
}

What did you expect to see?

I would expect to see console output that looks like:

Huuullllloooo!!
Before Bar
GOT HERE
After Bar
Goodbye.

What did you see instead?

Huuullllloooo!!
Before Bar

The process is then hung. If I pause the process in the debugger I see the following:

Looking at the dissassembled version of the DLL, it seems that it's stuck in _cgo_wait_runtime_init_done's while true loop checking whether the runtime is up.

When I compile this same go file as a unix .so file and link it with a C program on the Mac everything works exactly as expected.

Is this expected to work? It was my understanding that as of Go 1.10 this was supposed to work.

[ianlancetaylor]

CC @nadiasvertex

[alexbrainman]

GOOS=windows GOARCH=386 CGO_ENABLED=1 CC=i686-w64-mingw32-gcc go build -buildmode=c-shared -o main.dll main.go

Why GOARCH=386? Your C# executable is 32-bit or 64-bit?

You Go code says

func Bar() C.long

Your C# code must match parameters and return value

[DllImport(@"C:\Users\rfillion\main.dll", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.StdCall)]
public static extern long Bar();

Where in your C# code, it says that Bar takes no parameters and returns C.long (whatever C.long is)?

I also tried building your C# program, and build fails.

c:\Users\Alex\dev\src\issue\go\26714>type a.cs
namespace x86Hullo
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Huuullllloooo!!");
            Console.WriteLine("Before Bar");
            GoFunctions.Bar();
            Console.WriteLine("After Bar");
            Console.WriteLine("Goodbye.");
        }
    }


    static class GoFunctions
    {
        [DllImport(@"C:\Users\rfillion\main.dll", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.StdCall)]
        public static extern long Bar();
    }
}
c:\Users\Alex\dev\src\issue\go\26714>csc a.cs
Microsoft (R) Visual C# Compiler version 2.3.2.62019 (b7354608)
Copyright (C) Microsoft Corporation. All rights reserved.

a.cs(18,10): error CS0246: The type or namespace name 'DllImportAttribute' could not be found (are you missing a using directive or an assembly reference?)
a.cs(18,10): error CS0246: The type or namespace name 'DllImport' could not be found (are you missing a using directive or an assembly reference?)
a.cs(18,51): error CS0246: The type or namespace name 'CharSet' could not be found (are you missing a using directive or an assembly reference?)
a.cs(18,61): error CS0103: The name 'CharSet' does not exist in the current context
a.cs(18,78): error CS0246: The type or namespace name 'CallingConvention' could not be found (are you missing a using directive or an assembly reference?)
a.cs(18,98): error CS0103: The name 'CallingConvention' does not exist in the current context

c:\Users\Alex\dev\src\issue\go\26714>

What am I doing wrong?

Alex

[nadiasvertex]

I think his C# program snippet is incomplete. At a minimum you need to have

using System;
using System.Runtime.InteropServices;

at the top in order to make that work.

I'm not really sure that C#'s P/Invoke is going to work with a mingw-created DLL. At least, not without some fiddling. In theory the runtime should just be initialized, but the current DLL is built with the expectation that mingw's C runtime will be doing the initialization, not MSVC's C runtime. I don't know how different they are, or what is involved in making it work. As noted below, it might just be a case of giving gcc/mingw the right commands.

See also:

• https://lucidhelix.wordpress.com/2010/04/29/mingw-dll-pinvoke/
• http://www.mingw.org/wiki/Interoperability_of_Libraries_Created_by_Different_Compiler_Brands
• http://www.mingw.org/wiki/MSVC_and_MinGW_DLLs

[rickfillion]

Why GOARCH=386? Your C# executable is 32-bit or 64-bit?

My C# executable is 32bit, so 386 should be the right value there.

Where in your C# code, it says that Bar takes no parameters and returns C.long (whatever C.long is)?

It seems right to me, but maybe I'm interpretting this incorrectly?
https://golang.org/cmd/cgo/#hdr-Go_references_to_C

I think his C# program snippet is incomplete. At a minimum you need to have...

You're correct. My editor window was scrolled down slightly and I forgot the using statements. I've edited my post to correct that.

I'm not really sure that C#'s P/Invoke is going to work with a mingw-created DLL. At least, not without some fiddling. In theory the runtime should just be initialized, but the current DLL is built with the expectation that mingw's C runtime will be doing the initialization, not MSVC's C runtime. I don't know how different they are, or what is involved in making it work. As noted below, it might just be a case of giving gcc/mingw the right commands.

Thanks. I'll check out those links and see what I can learn from them.

I've seen that some developers have had luck prior to Go 1.10 by building a static library, then wrapping that in a DLL that was built without Go. I've not had luck with that approach yet, but I haven't spent all that much time there yet.

https://github.com/z505/goDLL
https://stackoverflow.com/questions/48208098/using-generated-golang-dll-to-return-string-or-c-char

I'd love to get an understanding of what's expected to work with Go's 1.10 support of DLLs.

[alexbrainman]

My editor window was scrolled down slightly and I forgot the using statements. I've edited my post to correct that.

Now it all works for me:

c:\Users\Alex\dev\src\issue\go\26714>go version
go version devel +5c11480631 Fri Aug 10 20:02:31 2018 +0000 windows/amd64

c:\Users\Alex\dev\src\issue\go\26714>gcc --version
gcc (GCC) 4.9.1
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


c:\Users\Alex\dev\src\issue\go\26714>type main.go
package main

import "C"
import "fmt"

func main() {
        fmt.Println(Bar())
        fmt.Println(Bar2())
        return
}

//export Bar
func Bar() C.long {
        fmt.Println("GOT HERE")
        return 54
}

//export Bar2
func Bar2() *C.char {
        return C.CString("Hello World From Go!")
}

c:\Users\Alex\dev\src\issue\go\26714>go build -buildmode=c-shared -o main.dll main.go

c:\Users\Alex\dev\src\issue\go\26714>type a.cs
using System;
using System.Runtime.InteropServices;

namespace x86Hullo
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Huuullllloooo!!");
            Console.WriteLine("Before Bar");
            GoFunctions.Bar();
            Console.WriteLine("After Bar");
            Console.WriteLine("Goodbye.");
        }
    }


    static class GoFunctions
    {
        [DllImport(@"c:\Users\Alex\dev\src\issue\go\26714\main.dll", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.StdCall)]
        public static extern long Bar();
    }
}
c:\Users\Alex\dev\src\issue\go\26714>csc a.cs && a.exe
Microsoft (R) Visual C# Compiler version 2.3.2.62019 (b7354608)
Copyright (C) Microsoft Corporation. All rights reserved.

Huuullllloooo!!
Before Bar
GOT HERE
After Bar
Goodbye.

c:\Users\Alex\dev\src\issue\go\26714>

Alex

[rickfillion]

That's interesting, @alexbrainman.

If I'm understanding this correctly, this means that the worries about MSVC vs GCC/mingw compilation for this case shouldn't be an issue.

The big differences I'm seeing between what you're doing and what I'm doing are:

• You're using gcc instead of mingw
• You're not doing any cross-compilation

For our purposes that might be fine. I would have expected cross-compilation to work though. Is there any chance that you could send me the compiled DLL for this case? I'd love to poke around and see where/how it deviates from the one I've built.

Any idea why the cross-compilation would have resulted in the issue I saw?

[rickfillion]

Oh, I suppose another difference between your setup and mine is your go version. I'm using the stable build and you're on devel. Is it expected to work as you're showing on stable?

[alexbrainman]

If I'm understanding this correctly, this means that the worries about MSVC vs GCC/mingw compilation for this case shouldn't be an issue.

I do not know what you are referring to.

You're using gcc instead of mingw

I use gcc from mingw package that I installed on my WIndows.

You're not doing any cross-compilation

I am not. My mingw is installed on my Windows, and I run built program on my Windows.
I have never used mingw on Linux or Darwin, so I would not know how it works.

Is there any chance that you could send me the compiled DLL for this case? I'd love to poke around and see where/how it deviates from the one I've built.

Sure. How do you want to do that?

Any idea why the cross-compilation would have resulted in the issue I saw?

The other thing that different is your dll is 32-bit. My both exe and dll are 64-bit. If your exe and dll GOARCH do not match, it would not work.

Mind you, I do not use C#, so I would not know how it works and what is required.

Alex

[rickfillion]

If I'm understanding this correctly, this means that the worries about MSVC vs GCC/mingw compilation for this case shouldn't be an issue.
I do not know what you are referring to.

I'm referring to
#26714 (comment)

I use gcc from mingw package that I installed on my WIndows.

I tried to mimic what you're doing yesterday and installed mingw on Windows, but I was hitting different problems. I'm now getting

System.BadImageFormatException: 'An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)'

C:\Users\rfillion>gcc --version
gcc (x86_64-win32-seh-rev0, Built by MinGW-W64 project) 8.1.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


C:\Users\rfillion>go version
go version go1.10.3 windows/amd64

What's interesting is that if I compile the C# file with csc as you did as opposed to MSVC's IDE, it works (WOO!). So that gives me a pile more to try out.

Is there any chance that you could send me the compiled DLL for this case? I'd love to poke around and see where/how it deviates from the one I've built.
Sure. How do you want to do that?

That shouldn't be necessary anymore now that I have a version of it that works.

[alexbrainman]

I'm referring to
#26714 (comment)

OK

System.BadImageFormatException: 'An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)'

Googling for 0x8007000B, first hit, I get

https://stackoverflow.com/questions/18007967/net-framework-error-hresult-0x8007000b

Perhaps it is what is happening to you. You DLL is 386, while your executable amd64. This is not allowed!

Alex

[rickfillion]

Perhaps it is what is happening to you. You DLL is 386, while your executable amd64. This is not allowed!

As it turns out, you're absolutely right (again). I had it set to "Any CPU" in Visual Studios, with the assumption that this would select x64 since i'm on an x64 system. I was wrong. When I manually selected x64, it began working as expected.

So now the only change is using gcc/mingw on Windows itself as opposed to having used it from a Mac and doing some cross-compilation.

At this point I've got something that I think is workable, but I'll keep poking at this to see if I can determine what's causing the differences between the native build and the cross compiled build.

[ianlancetaylor]

Sounds like things are basically working, so closing. Please comment if you disagree.