Bump Go to 1.9.2

[thaJeztah]

let's see what fails (or not) 👍

fixes #35152

[AkihiroSuda]

interesting

08:48:07 --- FAIL: TestHashFile (0.07s)
08:48:07 	tarsum_test.go:66: invalid checksum. expected 1149ab94af7be6cc1da1335e398f24ee1cf4926b720044d229969dfc248ae7ec, got 55dfeb344351ab27f59aa60ebb0ed12025a2f2f4677bf77d26ea7a671274a9ca
08:48:07 --- FAIL: TestHashSubdir (0.05s)
08:48:07 	tarsum_test.go:103: invalid checksum. expected d7f8d6353dee4816f9134f4156bf6a9d470fdadfb5d89213721f7e86744a4e69, got 74a3326b8e766ce63a8e5232f22e9dd895be647fb3ca7d337e5e0a9b3da8ef28
08:48:07 FAIL
08:48:07 coverage: 15.8% of statements
08:48:07 FAIL	github.com/docker/docker/builder/remotecontext	0.333s

[thaJeztah]

ping @unclejack @tonistiigi ^^

[AkihiroSuda]

git bisect says golang/go@66b5a2f is the first commit causing this failure:

commit 66b5a2f3f0b1d94f69763114a85a977f5bc0277a
Author: Lars Jeppesen <[email protected]>                
Date:   Sat Apr 29 23:25:34 2017 +0200  
                                                                                            
    archive/tar: remove file type bits from mode field                         
                           
    When writing tar files by using the FileInfoHeader                                   
    the type bits was set in the mode field of the header
    This is not correct according to the standard (GNU/Posix) and 
    other implementations.       
        
    Fixed #20150
                                                
    Change-Id: I3be7d946a1923ad5827cf45c696546a5e287ebba
    Reviewed-on: https://go-review.googlesource.com/42093
    Reviewed-by: Joe Tsai <[email protected]>
    Run-TryBot: Joe Tsai <[email protected]>
    TryBot-Result: Gobot Gobot <[email protected]>

[tonistiigi]

@AkihiroSuda Thanks for the debug.

I'm not sure what's the best approach here. Do we want to break build cache for everyone or fork the archive.FileInfoHeader and keep using the wrong headers? Also related: moby/buildkit#38

[AkihiroSuda]

Although build cache are not persistent data, people will get surprised if caches were broken by just updating Go compiler.
So my opinion is to keep using the wrong header until we integrate buildkit to the engine, but I don't have any strong opinion.
WDYT?

[AkihiroSuda]

I'll try to open a separate PR for filling file bits

[AkihiroSuda]

opened #33935

[tiborvass]

Can we confirm that the impact is only on caching? What about tar layers?

[tonistiigi]

@tiborvass Layers use tar-split copy of the original headers and don't regenerate them.

[thaJeztah]

Rebased, because #33935 was merged; let's see if it goes green this time

[tophj-ibm]

z failure looks serious, can you take a look @michael-holzheu ?

from the log

SIGILL: illegal instruction
PC=0x283186 m=0 sigcode=2

[utzb]

The z golang issue is understood and addressed. Thanks for attempting in the first place!

[thaJeztah]

Thanks @utzb - is that fix upstream (golang), or an issue fixed in this repository? (IOW, should a rebase fix it, or do we need to wait for the next Go beta/release)

[utzb]

The fix will be in upstream golang. With a little luck, it will be in 1.9beta3, but for sure in the first rc.
I verified test-unit and test-integration-cli run smooth using a golang with an early version of the fix.

[thaJeztah]

Thanks, good to hear!

[utzb]

Just FYI, this is now golang/go#21048 (https://go-review.googlesource.com/c/49250/)

[thaJeztah]

bumped to 1.9rc1

[tiborvass]

Since we are bumping Go versions, we should probably update archive/tar as well.
Here is the changelog for archive/tar: https://gist.github.com/tiborvass/f9303e496644c77bd11ba08ec00d9fa0

cc @tonistiigi @dmcgowan

[thaJeztah]

the generate script was updated in b877fc3, but looks like the script wasn't run to update the dockerfiles

[darstahl]

I'm investigating a blocker on Windows that prevents pulling some images. Not sure how CI passed tbh.

Edit: Oh, CI just side loads the base images. I'm not sure how that bypasses the bug, but it's new information to help me track it down.

[thaJeztah]

Removing the [do not merge] I think we can review this now; the changes made for the behaviour-change in Windows are a bit quirky, and looking at those, perhaps it should be a docker/cli test, so I added a TODO for that https://github.com/moby/moby/pull/33892/files#diff-f77ac0f89cdcfa4259ac67bced3d418cR39

@darrenstahlmsft was there still something to look into from your side?

ping @AkihiroSuda @vdemeester @cpuguy83 @yongtang PTAL

[vdemeester]

LGTM 🐮

[cpuguy83]

LGTM

[cpuguy83]

👎 Checking error strings, but the current error handling for this endpoint is not good enough as a true "not found" error would be seen as the container not being found, so we need more data in the error message to differentiate a "file not found" from a "container not found".

[thaJeztah]

@cpuguy83 agree it's pretty dirty; I didn't see a better option directly, other than splitting the test into a unit test (docker/cli), and an API test here (and may require changes to the endpoint itself as you mentioned).

Do you want me to look into this in this PR, or should we open an issue for tracking that effort?

[cpuguy83]

We don't need it for this PR.

[dnephin]

LGTM

[unclejack]

LGTM

[darstahl]

@thaJeztah yes, Windows needs microsoft/hcsshim#144

[darstahl]

Please hold the merge until that is in

[thaJeztah]

Thanks for checking @darrenstahlmsft - temporarily changed back to WIP

[thaJeztah]

#35554 was merged; rebased this PR, and removed "WIP" again; this should be ready to go now

[cpuguy83]

LGTM

[vieux]

LGTM