net.Conn.Close() race condition can crash the runtime.

[gopherbot]

by jesse.dailey:

Before filing a bug, please check whether it has been fixed since
the latest release: run "hg pull -u" and retry what you did to
reproduce the problem.  Thanks.

What steps will reproduce the problem?
1. Have some go routine reading a socket, and launching a goroutine handler

for {
  packet, err :=  readPacket(sock);
  if valid(packet) {
    go process(packet,sock);
  }
}
  
2. Have the handling goroutine Close() the socket.
func process(packet, sock *net.Conn) {
 time.Sleep(1000); // do some stuff
 sock.Close();
}
3. Now if you repeat this under stress, it won't take long before the 
Close() will happen while the Read() is blocked.  At which point the 
runtime dies.

The error is "epoll_ctl: no such file or directory", and the stack points 
to AddFD() in fd.go


What is the expected output? What do you see instead?

I've attached a fully fleshed out example, it's an echo server, and a 
simple client for submitting enough requests to crash it.

You run the echo server, then run the client.

What you should see is 100 lines of '#: defghijklm', where # ranges from 1-
10.  Instead, partway through it will begin 'connection refused' messages.

From the echo server you will see:
Epoll delete fd=7: bad file descriptor
pollServer AddFD  9 :  epoll_ctl: no such file or directory
                                  
panic PC=0xb7e10038               
net·*pollServer·AddFD+0x153 /opt/go/src/pkg/net/fd.go:133
        net·*pollServer·AddFD(0xb7e0b5c0, 0xb7e96e00, 0x72, 0x64)
net·*pollServer·Run+0x269 /opt/go/src/pkg/net/fd.go:255
        net·*pollServer·Run(0xb7e0b5c0, 0x809d6fc)
goexit /opt/go/src/pkg/runtime/proc.c:135
        goexit()                  
0xb7e0b5c0 unknown pc

and the runtime is dead.

What should happen is that the blocked Read() should be woken up by the 
Close() and return an error (similar to EOF).

What is your $GOOS?  $GOARCH?
GOOS=linux (Debian lenny)
GOARCH=386

Which revision are you sync'ed to?  (hg log -l 1)

changeset:   4024:1d9d926b1aa7
tag:         tip
user:        Russ Cox <[email protected]>
date:        Thu Nov 12 23:38:48 2009 -0800

Attachments:

1. echo.go (754 bytes)
2. client.go (892 bytes)
3. Makefile (70 bytes)

[gopherbot]

Comment 1 by jesse.dailey:

I should probably comment that yes... no one would ever write an ECHO server this way.
But, for protocols like FastCGI, you need to be able to multiplex requests over a 
single socket, and any request might have a flag that says 'Close this when you are 
done', so the request handler needs to be able to Close() the socket outside of the 
goroutine that is Read()ing it.

[agl]

Comment 2:

Status changed to Accepted.

[agl]

Comment 3:

http://golang.org/cl/152130

Status changed to Started.

[agl]

Comment 4:

This issue was closed by revision ef8f483.

Status changed to Fixed.

Merged into issue #-.

[gopherbot]

Comment 5 by emilliken:

I am still seeing this issue:
$ hg log -l 1
changeset:   4186:2f32e74ab96e
tag:         tip
user:        Adam Langley <[email protected]>
date:        Sat Nov 21 15:53:03 2009 -0800
The following is a nonsensical demo of the crash:
package main
import(
    "io";
    "time";
    "fmt";
    "net";
)
func proxy(dst, src net.Conn) {
    io.Copy(dst, src);
}
func main() {
    c1, err := net.Dial("tcp4", "", "www.google.com:80");
    if err != nil {
        fmt.Printf("error1\n");
    }
    c2, err := net.Dial("tcp4", "", "www.google.com:80");
    if err != nil {
        fmt.Printf("error2\n");
    }
    go proxy(c1, c2);
    go proxy(c2, c1);
    time.Sleep(2*1000*1000*1000);
    c1.Close();
    c2.Close();
    fmt.Printf("part 2\n");
    c3, err := net.Dial("tcp4", "", "www.google.com:80");
    if err != nil {
        fmt.Printf("error1\n");
    }
    c4, err := net.Dial("tcp4", "", "www.google.com:80");
    if err != nil {
        fmt.Printf("error2\n");
    }
    go proxy(c3, c4);
    go proxy(c4, c3);
    time.Sleep(2*1000*1000*1000);
    c3.Close();
    c4.Close();
}
results in:
part 2
pollServer AddFD  3 :  epoll_ctl: no such file or directory 
panic PC=0x7f1677cdbfd8
net·*pollServer·AddFD+0x123 /home/eric/go/src/pkg/net/fd.go:138
    net·*pollServer·AddFD(0x77ca8200, 0x7f16, 0x77d0f280, 0x7f16, 0x72, ...)
net·*pollServer·Run+0x263 /home/eric/go/src/pkg/net/fd.go:260
    net·*pollServer·Run(0x77ca8200, 0x7f16)
goexit /home/eric/go/src/pkg/runtime/proc.c:135
    goexit()
0x7f1677ca8200 unknown pc