Added runtime.KeepAlive for the new stricter 1.8 GC #48

darstahl · 2017-05-01T21:37:47Z

fixes #41

As far as I can tell, this should contain all the runtime.KeepAlive calls needed to conform to the new argument liveness guarantees in Go1.8.x

It turns out that the main issue was due to passing a Go pointer via native calls to the ioCompletionProcessor, causing the new garbage collection rules to cleanup the struct containing the channel even though it did not have a finalizer.

/cc @jhowardmsft @jstarks PTAL

Thanks @mappu for the initial investigation and repro test.

This also updates zsyscall_windows.go with the new 1.8 go generated code, though it is not related to the KeepAlive, it should be done with the update to 1.8.

Signed-off-by: Darren Stahl [email protected]

jstarks · 2017-05-01T22:29:03Z

file.go

+		// code to ioCompletionProcessor, c and bytes must both remain alive
+		// until the channel read is complete.
+		runtime.KeepAlive(c)
+		runtime.KeepAlive(bytes)


bytes [](start = 20, length = 5)

I don't think we need to do anything with bytes here (it's not even a pointer).

jstarks · 2017-05-01T22:29:48Z

file.go

@@ -221,5 +228,7 @@ func (f *win32File) SetWriteDeadline(t time.Time) error {
 }

 func (f *win32File) Flush() error {
-	return syscall.FlushFileBuffers(f.handle)
+	err := syscall.FlushFileBuffers(f.handle)
+	runtime.KeepAlive(f)


runtime.KeepAlive(f) [](start = 1, length = 20)

should we just remove the finalizer on win32File?

That works for me.

Also removing BackupFileReader finalizer

jstarks · 2017-05-01T22:36:15Z

fileinfo.go

@@ -36,6 +38,8 @@ func SetFileBasicInfo(f *os.File, bi *FileBasicInfo) error {
 	if err := setFileInformationByHandle(syscall.Handle(f.Fd()), fileBasicInfo, (*byte)(unsafe.Pointer(bi)), uint32(unsafe.Sizeof(*bi))); err != nil {
 		return &os.PathError{Op: "SetFileInformationByHandle", Path: f.Name(), Err: err}
 	}
+	runtime.KeepAlive(f)
+	runtime.KeepAlive(bi)


runtime.KeepAlive(bi) [](start = 1, length = 21)

I don't think this is necessary; the cgo rules should keep bi alive across the call to setFileInformationByHandle.

This one I'm unsure about. After (*byte)(unsafe.Pointer(bi)) returns the pointer, is bi still considered live? How do the new runtime argument liveness rules apply to cgo rules? bi is no longer accessible as soon as unsafe.Pointer returns, so based on my understanding, a finalizer could run on bi before the call to c.

It doesn't matter though, as bi does not have a finalizer on it, so I can remove this regardless, but I don't see any documents regarding the new liveness rules when passing pointers to cgo.

unsafe.Pointer(x) and (*byte)(x) are casts, not function calls, so they're presumably handled differently. So I don't think that's true.

Ah, ok. Removed.

jstarks · 2017-05-01T22:36:38Z

pipe.go

 	if err != nil {
 		return 0, &os.PathError{Op: "open", Path: path, Err: err}
 	}
+	runtime.KeepAlive(sa)


runtime.KeepAlive(sa) [](start = 1, length = 21)

ditto here

jstarks · 2017-05-01T22:37:26Z

pipe.go

-	var sa securityAttributes
-	sa.Length = uint32(unsafe.Sizeof(sa))
+	sa := &securityAttributes{}
+	sa.Length = uint32(unsafe.Sizeof(*sa))
 	if securityDescriptor != nil {
 		sa.SecurityDescriptor = &securityDescriptor[0]


&securityDescriptor[0] [](start = 26, length = 22)

This actually violates cgo rules: https://golang.org/cmd/cgo/

Probably we should be duplicating this into a C-allocated buffer.

jstarks · 2017-05-01T22:46:07Z

err = syscall.WriteFile(f.handle, b, &bytes, &c.o)

I'm worried that b could be collected before asyncIo returns... I think we need to keep it alive explicitly. I'm not sure if KeepAlive directly on a slice does anything or if you need KeepAlive on a pointer to the first element of the slice.

(Technically this violates cgo rules, but we don't have an alternative other than copying the data to a C buffer before doing the WriteFile.)

Refers to: file.go:216 in bb2dfe9. [](commit_id = bb2dfe9, deletion_comment = False)

jstarks · 2017-05-01T22:46:26Z

err = syscall.WriteFile(f.handle, b, &bytes, &c.o)

And of course we need to do this everywhere else we call asyncIo as well.

In reply to: 298452715 [](ancestors = 298452715)

Refers to: file.go:216 in bb2dfe9. [](commit_id = bb2dfe9, deletion_comment = False)

jstarks · 2017-05-02T00:09:01Z

🕐

darstahl · 2017-05-02T02:18:59Z

Tested via manual GC invocations, b is fair game for the GC as soon as WriteFile returns. runtime.KeepAlive(b) prevents the cleanup of the whole slice and backing array.

Adding the KeepAlives

darstahl · 2017-05-02T21:44:02Z

Updated.

jstarks · 2017-05-03T19:32:43Z

file.go

@@ -221,5 +228,6 @@ func (f *win32File) SetWriteDeadline(t time.Time) error {
 }

 func (f *win32File) Flush() error {
-	return syscall.FlushFileBuffers(f.handle)
+	err := syscall.FlushFileBuffers(f.handle)
+	return err


unnecessary

jstarks · 2017-05-03T19:35:23Z

Looks good after a minor tweak and separating the CBytes thing out as a separate commit.

Signed-off-by: Darren Stahl <[email protected]>

darstahl · 2017-05-03T19:48:09Z

Split into two commits and nit fixed.

Signed-off-by: Darren Stahl <[email protected]>

Automatic merge from submit-queue Support windows in dockershim **What this PR does / why we need it**: This is the 2nd part for #45927 . The non-cri implementation dockertools was removed from kubelet v1.7 . Part of previous work for supporting windows container lies in v1.6 dockertools, this PR is to port them to dockershim. Main reference file in v1.6 dockertools windows support: https://github.com/kubernetes/kubernetes/blob/v1.6.4/pkg/kubelet/dockertools/docker_manager_windows.go **Which issue this PR fixes** 45927, for now catching up the implementation of v1.6 **Special notes for your reviewer**: The code change includes 4 parts, put them together as we discussed in #46089 1. Update go-winio package to a newer version 'go-winio' package is used by docker client. This change is to bring the support for Go v1.8, specifically included in the PR: microsoft/go-winio#48 Otherwise it will produce a lot of error like in: fsouza/go-dockerclient#648 2. Add os dependent getSecurityOpts helper method. seccomp not supported on windows Corresponding code in v1.6: https://github.com/kubernetes/kubernetes/blob/v1.6.4/pkg/kubelet/dockertools/docker_manager_windows.go#L78 3. Add updateCreateConfig. Allow user specified network mode setting. This is to be compatible with what kube-proxy package does on Windows. Also, there is a Linux section in both sandbox config and container config： LinuxPodSandboxConfig， LinuxContainerConfig. And that section later goes to Config and HostConfig section under docker container createConfig. Ideally hostconfig section should be dependent on host os, while config should depend on container image os. To simplify the case, here it assumes that windows host supports windows type container image only. It needs to be updated when kubernetes is to support windows host running linux container image or the like. Corresponding code in v1.6: https://github.com/kubernetes/kubernetes/blob/v1.6.4/pkg/kubelet/dockertools/docker_manager_windows.go#L57 4. Add podIpCache in dockershim. For v1.6 windows implementation, it still does not use sandbox, thus only allow single container to be exposed. Here added a cache for saving container IP, to get adapted to the new CRI api. Corresponding code in v1.6: No sandbox: https://github.com/kubernetes/kubernetes/blob/v1.6.4/pkg/kubelet/dockertools/docker_manager_windows.go#L66 Use container id as pod ip: https://github.com/kubernetes/kubernetes/blob/v1.6.4/pkg/kubelet/dockertools/docker_manager.go#L2727 **Release note**:

* Removed debug file * Formatted with gofmt after files were modified via GitHub web interface * Updated go-winio dependency to latest version. This update includes fixes from this PR: microsoft/go-winio#48 That fixes: microsoft/go-winio#41 And should fix #111.

jstarks reviewed May 1, 2017

View reviewed changes

darstahl force-pushed the Go1.8 branch from bb2dfe9 to bb53aa8 Compare May 2, 2017 21:43

fsouza mentioned this pull request May 2, 2017

Panic on Windows Server 2016 fsouza/go-dockerclient#648

Closed

darstahl mentioned this pull request May 2, 2017

Update go to 1.8 moby/moby#29030

Merged

jstarks reviewed May 3, 2017

View reviewed changes

darstahl added 2 commits May 3, 2017 12:46

Allocate securityDescriptor in C memory

23ad5a7

Signed-off-by: Darren Stahl <[email protected]>

Added runtime.KeepAlive for the new stricter 1.8 GC

e9792bc

Signed-off-by: Darren Stahl <[email protected]>

darstahl force-pushed the Go1.8 branch from bb53aa8 to e9792bc Compare May 3, 2017 19:47

jstarks merged commit 13736c3 into microsoft:master May 3, 2017

darstahl deleted the Go1.8 branch May 3, 2017 19:50

schmichael added a commit to hashicorp/nomad that referenced this pull request May 4, 2017

Update go-winio to get microsoft/go-winio#48

c969e56

schmichael mentioned this pull request May 4, 2017

Panic on Windows Server 2016 hashicorp/nomad#2343

Closed

samuelkarp mentioned this pull request May 4, 2017

Update to Go 1.9 aws/amazon-ecs-agent#681

Closed

darstahl added a commit to darstahl/go-winio that referenced this pull request May 5, 2017

Remove cgo as a dependency added by microsoft#48

de045f7

Signed-off-by: Darren Stahl <[email protected]>

darstahl mentioned this pull request May 5, 2017

Remove cgo as a dependency added by #49 #51

Merged

rfay mentioned this pull request May 8, 2017

Cross-compile Linux->Windows fails on latest commit: "undefined: securityAttributes" #52

Closed

karataliu mentioned this pull request Jun 1, 2017

Support windows in dockershim kubernetes/kubernetes#46744

Merged

mappu mentioned this pull request Jun 13, 2017

panic: "unexpected signal during runtime execution" DataDog/zstd#13

Closed

darstahl mentioned this pull request Aug 2, 2017

Windows Kubelet crashes with the below stacktrace #57

Closed

PaulMaddox mentioned this pull request Sep 8, 2017

Update microsoft/go-winio dependency aws/aws-sam-cli#112

Merged

richardpen mentioned this pull request Nov 17, 2017

Update go-winio to get rid of the runtime error aws/amazon-ecs-agent#1091

Closed

yhlee-aws mentioned this pull request Jan 23, 2018

Update to Go 1.9 aws/amazon-ecs-agent#1210

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added runtime.KeepAlive for the new stricter 1.8 GC #48

Added runtime.KeepAlive for the new stricter 1.8 GC #48

darstahl commented May 1, 2017

jstarks May 1, 2017

jstarks May 1, 2017

darstahl May 2, 2017

darstahl May 2, 2017

jstarks May 1, 2017

darstahl May 2, 2017

jstarks May 2, 2017

darstahl May 2, 2017

jstarks May 1, 2017

jstarks May 1, 2017

jstarks commented May 1, 2017

jstarks commented May 1, 2017

jstarks commented May 2, 2017

darstahl commented May 2, 2017

darstahl commented May 2, 2017

jstarks May 3, 2017

jstarks commented May 3, 2017

darstahl commented May 3, 2017

Added runtime.KeepAlive for the new stricter 1.8 GC #48

Added runtime.KeepAlive for the new stricter 1.8 GC #48

Conversation

darstahl commented May 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jstarks commented May 1, 2017

jstarks commented May 1, 2017

jstarks commented May 2, 2017

darstahl commented May 2, 2017

darstahl commented May 2, 2017

Choose a reason for hiding this comment

jstarks commented May 3, 2017

darstahl commented May 3, 2017