runtime: c-shared with LD_PRELOAD does not work for some programs

[AkihiroSuda]

I'm trying to make my own LD_PRELOAD-able library like this:

// compile: go build -buildmode=c-shared -o example.so example.go
// tested with Go 1.5 on Ubuntu 15.04 amd64
package main
import "C"
func main() {}
//export dummy
func dummy() C.int { return 42 }

This example.so works well with several simple programs such as ls, cat, cp and so on.

$ LD_PRELOAD=./example.so ls

However, some programs such as firefox, google-chrome, gdb get segfault immediately (or sometimes just get hung up).

I suspect this is a bug of the Go runtime.

Anyone can please look on this?

Note that C-implementation seems working well with any program:

// compile: gcc -Wall -fPIC -shared -o example-c.so example.c
int dummy() { return 42; }

[AkihiroSuda]

A backtrace extracted from a core of gedit:

(gdb) 
#0  runtime.sigaltstack () at /usr/local/go/src/runtime/sys_linux_amd64.s:355
#1  0x00007f5f8a98ce00 in runtime.signalstack (s=0xc820000300) at /usr/local/go/src/runtime/os1_linux.go:337
#2  0x00007f5f8a98c7d5 in runtime.minit () at /usr/local/go/src/runtime/os1_linux.go:216
#3  0x00007f5f8a9948f8 in runtime.needm (x=0 '\000') at /usr/local/go/src/runtime/proc1.go:968
#4  0x00007f5f8a9bae26 in runtime.cgocallback_gofunc () at /usr/local/go/src/runtime/asm_amd64.s:742
#5  0x00007f5f8a9bad5a in runtime.cgocallback () at /usr/local/go/src/runtime/asm_amd64.s:714
#6  0x00007f5f8a9a0e1b in runtime.badsignal (sig=17) at /usr/local/go/src/runtime/sigqueue.go:169
#7  0x00007f5f8a9a0262 in runtime.sigtrampgo (sig=17, info=0x22599f0, ctx=0x22598c0) at /usr/local/go/src/runtime/signal_linux.go:90
#8  0x00007f5f8a9bc85b in runtime.sigtramp () at /usr/local/go/src/runtime/sys_linux_amd64.s:234
#9  0x00007f5f8a9bc860 in runtime.sigtramp () at /usr/local/go/src/runtime/sys_linux_amd64.s:235
#10 0x0000000000000001 in ?? ()
#11 0x0000000000000000 in ?? ()

The bug occurs when getg() returns nil in sigtrampgo().

Possible related issues: #10268 #12277

[ianlancetaylor]

Thanks for the backtrace. This is the Go signal handler receiving a SIGCHLD signal. It's not a problem for getg to return nil in sigtrampgo. That just means that a signal occurred on a non-Go thread. The Go code is going to do some shuffling and eventually raise the signal on a C thread, with the C signal handler installed. That should cause the right thing to happen.

However, in this backtrace, the shuffling fails because the sigaltstack system call fails. I think that sigaltstack must be returning EPERM: an attempt was made to change the alternate signal stack while the process was already executing on the alternate signal stack. That could happen if the C program create a thread, called sigaltstack on that thread, and then received a signal. That might explain why you are not seeing a problem with relatively simple programs, but only with relatively complex ones.

If that is indeed the problem, the solution may be relatively simple: if sigaltstack fails due to EPERM, we should fetch the current alternative signal stack and use it to set the value of m.gsignal.stack.

[AkihiroSuda]

Hi, @ianlancetaylor

Thank you very much for your comment.

After applying this fix (for commit dab0da), gedit works well.

diff --git a/src/runtime/os1_linux.go b/src/runtime/os1_linux.go
index c23dc30..99a2560 100644
--- a/src/runtime/os1_linux.go
+++ b/src/runtime/os1_linux.go
@@ -326,7 +326,15 @@ func getsig(i int32) uintptr {
 }

 func signalstack(s *stack) {
-       var st sigaltstackt
+       var st, oldst sigaltstackt
+       sigaltstack(nil, &oldst)
+       if oldst.ss_flags & _SS_ONSTACK != 0 {
+               if s != nil {
+                       s.lo = uintptr(unsafe.Pointer(oldst.ss_sp))
+                       s.hi = s.lo + oldst.ss_size
+               }
+               return
+       }
        if s == nil {
                st.ss_flags = _SS_DISABLE
        } else {
diff --git a/src/runtime/os2_linux.go b/src/runtime/os2_linux.go
index 71f36eb..6ce5e40 100644
--- a/src/runtime/os2_linux.go
+++ b/src/runtime/os2_linux.go
@@ -5,6 +5,7 @@
 package runtime

 const (
+       _SS_ONSTACK  = 1
        _SS_DISABLE  = 2
        _NSIG        = 65
        _SI_USER     = 0

However, firefox, thunderbird and google-chrome still does not work.

When I run these ones with gdb, I can get various kinds of strange error messages.
I suspect this flaky behavior is caused by some C stack corruption.

Could you please look on this?

$ LD_PRELOAD=./example.so firefox --debug
(gdb) run
Starting program: /usr/lib/firefox/firefox 
/bin/bash: : command not found
During startup program exited with code 127.

$ LD_PRELOAD=./example.so firefox --debug
(gdb) run
Starting program: /usr/lib/firefox/firefox

malloc: unknown:0: assertion botched
malloc: block on free list clobbered
Aborting...SIGABRT: abort
PC=0x7ffff72d6267 m=0

goroutine 0 [idle]:

goroutine 1 [running]:
runtime.systemstack_switch()
        /home/suda/go/src/runtime/asm_amd64.s:216 fp=0xc82002e770 sp=0xc82002e768
runtime.main()
        /home/suda/go/src/runtime/proc.go:49 +0x66 fp=0xc82002e7c0 sp=0xc82002e770
runtime.goexit()
        /home/suda/go/src/runtime/asm_amd64.s:1744 +0x1 fp=0xc82002e7c8 sp=0xc82002e7c0

goroutine 17 [chan receive (nil chan), locked to thread]:
runtime.gopark(0x0, 0x0, 0x7ffff7b74370, 0x17, 0x10, 0x2)
        /home/suda/go/src/runtime/proc.go:185 +0x169
runtime.chanrecv(0x7ffff7b4bb20, 0x0, 0x0, 0x1, 0x0)
        /home/suda/go/src/runtime/chan.go:374 +0x8b
runtime.chanrecv1(0x7ffff7b4bb20, 0x0, 0x0)
        /home/suda/go/src/runtime/chan.go:349 +0x2b
runtime.cgocallbackg1()
        /home/suda/go/src/runtime/cgocall.go:196 +0x94
runtime.cgocallbackg()
        /home/suda/go/src/runtime/cgocall.go:177 +0xd9
runtime.cgocallback_gofunc(0x0, 0x0, 0x0)
        /home/suda/go/src/runtime/asm_amd64.s:799 +0x5d
runtime.goexit()
        /home/suda/go/src/runtime/asm_amd64.s:1744 +0x1

goroutine 18 [syscall, locked to thread]:
runtime.goexit()
        /home/suda/go/src/runtime/asm_amd64.s:1744 +0x1

rax    0x0
rbx    0x4d68f0
rcx    0xffffffffffffffff
rdx    0x6
rdi    0x4cc5
rsi    0x4cc6
rbp    0x3
rsp    0x7ffff7081a48
r8     0x7ffff7667960
r9     0x7ffff7082700
r10    0x8
r11    0x206
r12    0x0
r13    0x0
r14    0x70eb80
r15    0x18
rip    0x7ffff72d6267
rflags 0x206
cs     0x33
fs     0x0
gs     0x0

-----

SIGQUIT: quit
PC=0x7ffff7b424da m=0

goroutine 0 [idle]:
runtime.rtsigprocmask(0xc800000002, 0xc8200098f8, 0x0)
        /home/suda/go/src/runtime/sys_linux_amd64.s:206 +0x1a
runtime.updatesigmask(0x0, 0x100000000)
        /home/suda/go/src/runtime/os1_linux.go:351 +0xf3
runtime.crash()
        /home/suda/go/src/runtime/signal1_unix.go:193 +0x33
runtime.sighandler(0xc800000006, 0xc820009bb0, 0xc820009a80, 0x7ffff7db0f00)
        /home/suda/go/src/runtime/signal_amd64x.go:200 +0x6b2
runtime.sigtrampgo(0x6, 0xc820009bb0, 0xc820009a80)
        /home/suda/go/src/runtime/signal_linux.go:94 +0x97
runtime.sigtramp(0x1, 0x0, 0xc820002000, 0x0, 0x7fa0, 0x7ffff7667960, 0x7ffff7082700, 0x8, 0x206, 0x0, ...)
        /home/suda/go/src/runtime/sys_linux_amd64.s:234 +0x1b
runtime.sigreturn(0x0, 0xc820002000, 0x0, 0x7fa0, 0x7ffff7667960, 0x7ffff7082700, 0x8, 0x206, 0x0, 0x0, ...)
        /home/suda/go/src/runtime/sys_linux_amd64.s:238
goroutine 1 [running]:
During startup program terminated with signal SIGSEGV, Segmentation fault.

$ LD_PRELOAD=./example.so firefox --debug
(gdb) run
Starting program: /usr/lib/firefox/firefox 

malloc: .././variables.c:3994: assertion botched
malloc: block on free list clobbered
Aborting...During startup program terminated with signal SIGABRT, Aborted.

[gopherbot]

CL https://golang.org/cl/17903 mentions this issue.

[majid3612]

Guys, I have the same problem with LD_PRELOAD firefox, google-chrome and some other programs. Is there any solution for that? I appreciate your response in advance.