cmd/link: TestFallocate fallocate_test error

[rehack]

What version of Go are you using (go version)?

$ go version
go version go1.12.9 darwin/amd64

Does this issue reproduce with the latest release?

no

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/rehack/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/rehack/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/webserver/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/webserver/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/1x/wz0y29vj6wg9frw1vcwb8sl40000gn/T/go-build191766201=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Compile Go from source go1.15.src

➜  src ./all.bash

What did you expect to see?

No FAIL

What did you see instead?

ok      cmd/go/internal/renameio        0.505s
ok      cmd/go/internal/search  0.017s
ok      cmd/go/internal/test    0.029s
ok      cmd/go/internal/txtar   0.017s
ok      cmd/go/internal/web     0.022s
ok      cmd/go/internal/work    0.027s
ok      cmd/gofmt       0.046s
ok      cmd/internal/buildid    0.258s
ok      cmd/internal/dwarf      0.019s
ok      cmd/internal/edit       0.015s
ok      cmd/internal/goobj      0.863s
ok      cmd/internal/goobj2     0.015s
ok      cmd/internal/moddeps    1.456s
ok      cmd/internal/obj        0.016s
ok      cmd/internal/obj/arm64  0.131s
ok      cmd/internal/obj/ppc64  0.100s
ok      cmd/internal/obj/riscv  0.096s
ok      cmd/internal/obj/x86    10.811s
ok      cmd/internal/objabi     0.015s
ok      cmd/internal/src        0.016s
ok      cmd/internal/test2json  0.091s
ok      cmd/link        24.156s
ok      cmd/link/internal/benchmark     0.016s
--- FAIL: TestFallocate (0.01s)
    fallocate_test.go:61: unexpected disk usage: got 2040 blocks, want 2048
    fallocate_test.go:61: unexpected disk usage: got 6136 blocks, want 6144
FAIL
FAIL    cmd/link/internal/ld    13.383s
ok      cmd/link/internal/loader        0.034s
ok      cmd/link/internal/sym   0.016s
ok      cmd/nm  7.003s
ok      cmd/objdump     6.613s
ok      cmd/oldlink/internal/sym        0.016s
ok      cmd/pack        3.216s
ok      cmd/trace       0.080s
ok      cmd/vet 15.012s
FAIL
go tool dist: Failed: exit status 1

[AlexRouSg]

@Codesboy can you confirm that you are using the released 1.15 sources and not the beta/alpha.

cc @cherrymui
As this was supposed to be fixed in #39905

[rehack]

@AlexRouSg YES . I use go 1.15 sources: https://golang.org/dl/go1.15.src.tar.gz
It fixed in a4ba411
but this problem has bobbed up again.I need help.

[AlexRouSg]

If you need it to compile urgently, you can try editing src/cmd/link/internal/ld/fallocate_test.go and adding a t.Skip("fallocate is not supported") at the start of the function.

[odeke-em]

Thank you for this report @Codesboy and welcome to the Go project! Thank you for the response @AlexRouSg and suggestion for a fix.

@Codesboy your report says that you are using Go1.12.9 yet you just replied to @AlexRouSg in #41127 (comment) that you are using Go1.15. Could you please re-verify the bug report? Could you perhaps run git log | head -n 10 in your Go tree and report that? Thank you.

[networkimprov]

Hi Emmanuel, I think he's building the 1.15 source tree with 1.12.9.

[odeke-em]

Thanks @networkimprov for the clarification, my apologies, I missed that in the report (and was reading on my iPad) @Codesboy stated the problem.

[rehack]

@networkimprov @odeke-em Yes! i'm installing Go1.15 from source(https://golang.org/dl/go1.15.src.tar.gz) with go compiler version 1.12.9.
Thanks in advance for your help.

[cherrymui]

Thanks for the report. What version of macOS are you running? What file system are you running on (if it is not the default)? It is interesting that it got 8 fewer blocks. Is the failure reproducible? Thanks.

[rehack]

macOS version: High Sierra 10.13.6

➜  ~ uname -a
Darwin rehackdeiMac.local 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 18 21:21:34 PDT 2020; root:xnu-4570.71.82.5~1/RELEASE_X86_64 x86_64

Files System: APFS

Steps to reproduce

command steps

➜  ~ wget https://golang.org/dl/go1.15.src.tar.gz
--2020-09-01 10:07:14--  https://golang.org/dl/go1.15.src.tar.gz
正在连接 127.0.0.1:1087... 已连接。
已发出 Proxy 请求，正在等待回应... 302 Found
位置：https://dl.google.com/go/go1.15.src.tar.gz [跟随至新的 URL]
--2020-09-01 10:07:15--  https://dl.google.com/go/go1.15.src.tar.gz
正在连接 127.0.0.1:1087... 已连接。
已发出 Proxy 请求，正在等待回应... 200 OK
长度：23002901 (22M) [application/octet-stream]
正在保存至: “go1.15.src.tar.gz”

go1.15.src.tar.gz                                       100%[============================================================================================================================>]  21.94M  1.21MB/s  用时 22s     

2020-09-01 10:07:38 (1019 KB/s) - 已保存 “go1.15.src.tar.gz” [23002901/23002901])

➜  ~ tar zxvf go1.15.src.tar.gz 
➜  ~ cd go/src 
➜  src ./all.bash 
warning: GOPATH set to GOROOT (/Users/rehack/go) has no effect
Building Go cmd/dist using /usr/local/devenv/go. (go1.12.9 darwin/amd64)
Building Go toolchain1 using /usr/local/devenv/go.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for darwin/amd64.
warning: GOPATH set to GOROOT (/Users/rehack/go) has no effect

##### Testing packages.
ok      archive/tar     0.080s
ok      archive/zip     0.082s
ok      bufio   0.103s
ok      bytes   0.415s
ok      compress/bzip2  0.089s
ok      compress/flate  1.462s
ok      compress/gzip   0.039s
ok      compress/lzw    0.030s
ok      compress/zlib   0.046s
ok      container/heap  0.027s
ok      container/list  0.020s
ok      container/ring  0.025s
ok      context 0.035s
ok      crypto  0.020s
ok      crypto/aes      0.043s
ok      crypto/cipher   0.045s
ok      crypto/des      0.026s
ok      crypto/dsa      0.021s
ok      crypto/ecdsa    0.177s
ok      crypto/ed25519  0.107s
ok      crypto/elliptic 0.062s
ok      crypto/hmac     0.020s
ok      crypto/internal/subtle  0.021s
ok      crypto/md5      0.023s
ok      crypto/rand     0.044s
ok      crypto/rc4      0.054s
ok      crypto/rsa      0.120s
ok      crypto/sha1     0.050s
ok      crypto/sha256   0.024s
ok      crypto/sha512   0.019s
ok      crypto/subtle   0.031s
ok      crypto/tls      1.325s
ok      crypto/x509     2.155s
ok      database/sql    0.637s
ok      database/sql/driver     0.022s
ok      debug/dwarf     0.039s
ok      debug/elf       0.047s
ok      debug/gosym     0.021s
ok      debug/macho     0.031s
ok      debug/pe        0.054s
ok      debug/plan9obj  0.022s
ok      encoding/ascii85        0.023s
ok      encoding/asn1   0.042s
ok      encoding/base32 0.035s
ok      encoding/base64 0.023s
ok      encoding/binary 0.151s
ok      encoding/csv    0.029s
ok      encoding/gob    0.041s
ok      encoding/hex    0.022s
ok      encoding/json   0.353s
ok      encoding/pem    0.044s
ok      encoding/xml    0.036s
ok      errors  0.019s
ok      expvar  0.021s
ok      flag    0.131s
ok      fmt     0.151s
ok      go/ast  0.022s
ok      go/build        4.481s
ok      go/constant     0.022s
ok      go/doc  0.103s
ok      go/format       0.030s
ok      go/importer     0.170s
ok      go/internal/gccgoimporter       0.037s
ok      go/internal/gcimporter  0.681s
ok      go/internal/srcimporter 11.453s
ok      go/parser       0.056s
ok      go/printer      0.222s
ok      go/scanner      0.028s
ok      go/token        0.033s
ok      go/types        1.207s
ok      hash    0.021s
ok      hash/adler32    0.025s
ok      hash/crc32      0.021s
ok      hash/crc64      0.021s
ok      hash/fnv        0.020s
ok      hash/maphash    0.111s
ok      html    0.020s
ok      html/template   0.100s
ok      image   0.180s
ok      image/color     0.040s
ok      image/draw      0.096s
ok      image/gif       0.352s
ok      image/jpeg      0.338s
ok      image/png       0.054s
ok      index/suffixarray       0.194s
ok      internal/cpu    0.079s
ok      internal/fmtsort        0.020s
ok      internal/poll   0.043s
ok      internal/profile        0.026s
ok      internal/reflectlite    0.070s
ok      internal/singleflight   0.030s
ok      internal/trace  0.061s
ok      internal/unsafeheader   0.019s
ok      internal/xcoff  0.034s
ok      io      0.053s
ok      io/ioutil       0.044s
ok      log     0.023s
ok      log/syslog      1.249s
ok      math    0.026s
ok      math/big        0.919s
ok      math/bits       0.022s
ok      math/cmplx      0.022s
ok      math/rand       0.165s
ok      mime    0.198s
ok      mime/multipart  0.300s
ok      mime/quotedprintable    0.020s
ok      net     2.240s
ok      net/http        14.766s
ok      net/http/cgi    0.923s
ok      net/http/cookiejar      0.028s
ok      net/http/fcgi   0.024s
ok      net/http/httptest       0.051s
ok      net/http/httptrace      0.022s
ok      net/http/httputil       0.065s
ok      net/http/internal       0.020s
ok      net/http/pprof  5.116s
ok      net/internal/socktest   0.014s
ok      net/mail        0.016s
ok      net/rpc 0.058s
ok      net/rpc/jsonrpc 0.034s
ok      net/smtp        0.030s
ok      net/textproto   0.016s
ok      net/url 0.028s
ok      os      1.210s
ok      os/exec 1.063s
ok      os/signal       2.180s
ok      os/user 0.107s
ok      path    0.030s
ok      path/filepath   0.098s
ok      plugin  0.042s
ok      reflect 0.270s
ok      regexp  0.087s
ok      regexp/syntax   0.685s
ok      runtime 77.568s
ok      runtime/debug   0.077s
ok      runtime/internal/atomic 0.032s
ok      runtime/internal/math   0.022s
ok      runtime/internal/sys    0.024s
ok      runtime/pprof   7.809s
ok      runtime/race    0.430s
ok      runtime/trace   0.885s
ok      sort    0.093s
ok      strconv 1.427s
ok      strings 0.220s
ok      sync    0.431s
ok      sync/atomic     0.053s
ok      syscall 0.189s
ok      testing 0.330s
ok      testing/iotest  0.018s
ok      testing/quick   0.084s
ok      text/scanner    0.022s
ok      text/tabwriter  0.021s
ok      text/template   0.061s
ok      text/template/parse     0.024s
ok      time    2.094s
ok      unicode 0.025s
ok      unicode/utf16   0.020s
ok      unicode/utf8    0.023s
ok      cmd/addr2line   3.549s
ok      cmd/api 11.828s
ok      cmd/asm/internal/asm    1.917s
ok      cmd/asm/internal/lex    0.055s
ok      cmd/compile     0.023s
ok      cmd/compile/internal/gc 24.612s
ok      cmd/compile/internal/logopt     0.756s
ok      cmd/compile/internal/ssa        0.839s
ok      cmd/compile/internal/syntax     0.033s
ok      cmd/compile/internal/test       0.019s [no tests to run]
ok      cmd/compile/internal/types      0.019s
ok      cmd/cover       5.147s
ok      cmd/doc 0.302s
ok      cmd/fix 3.910s
ok      cmd/go  110.720s
ok      cmd/go/internal/auth    0.028s
ok      cmd/go/internal/cache   0.454s
ok      cmd/go/internal/generate        0.034s
ok      cmd/go/internal/get     1.279s
ok      cmd/go/internal/imports 0.029s
ok      cmd/go/internal/load    0.027s
ok      cmd/go/internal/lockedfile      0.121s
ok      cmd/go/internal/lockedfile/internal/filelock    0.060s
ok      cmd/go/internal/modconv 0.042s
ok      cmd/go/internal/modfetch        1.301s
ok      cmd/go/internal/modfetch/codehost       0.028s
ok      cmd/go/internal/modfetch/zip_sum_test   0.036s
ok      cmd/go/internal/modload 0.022s
ok      cmd/go/internal/mvs     0.024s
ok      cmd/go/internal/par     0.039s
ok      cmd/go/internal/renameio        0.759s
ok      cmd/go/internal/search  0.065s
ok      cmd/go/internal/test    0.180s
ok      cmd/go/internal/txtar   0.021s
ok      cmd/go/internal/web     0.043s
ok      cmd/go/internal/work    0.034s
ok      cmd/gofmt       0.103s
ok      cmd/internal/buildid    0.718s
ok      cmd/internal/dwarf      0.019s
ok      cmd/internal/edit       0.044s
ok      cmd/internal/goobj      2.813s
ok      cmd/internal/goobj2     0.024s
ok      cmd/internal/moddeps    4.863s
ok      cmd/internal/obj        0.020s
ok      cmd/internal/obj/arm64  0.326s
ok      cmd/internal/obj/ppc64  0.194s
ok      cmd/internal/obj/riscv  0.114s
ok      cmd/internal/obj/x86    11.026s
ok      cmd/internal/objabi     0.021s
ok      cmd/internal/src        0.022s
ok      cmd/internal/test2json  0.246s
ok      cmd/link        30.221s
ok      cmd/link/internal/benchmark     0.050s
--- FAIL: TestFallocate (0.04s)
    fallocate_test.go:61: unexpected disk usage: got 2040 blocks, want 2048
    fallocate_test.go:61: unexpected disk usage: got 6136 blocks, want 6144
FAIL
FAIL    cmd/link/internal/ld    49.869s
ok      cmd/link/internal/loader        0.083s
ok      cmd/link/internal/sym   0.022s
ok      cmd/nm  10.828s
ok      cmd/objdump     14.224s
ok      cmd/oldlink/internal/sym        0.033s
ok      cmd/pack        6.511s
ok      cmd/trace       0.239s
ok      cmd/vet 21.919s
FAIL
go tool dist: Failed: exit status 1
➜  src 

@cherrymui

[cherrymui]

Thanks. I don't see anything weird. I cannot reproduce it on my machine (macOS 10.15) or the builders. I'm not sure why it got 8 fewer blocks...
@jeremyfaller @aclements any idea?

[gopherbot]

Change https://golang.org/cl/262837 mentions this issue: cmd/link: TestFallocate fallocate_test error #41127

[ianlancetaylor]

I am seeing TestFallocate errors moderately often on tip on my GNU/Linux laptop.

--- FAIL: TestFallocate (0.00s)
    fallocate_test.go:61: unexpected disk usage: got 6152 blocks, want 6144

I don't see any failures on the builders. I don't know what is different about my laptop.

[ianlancetaylor]

Just tried again, and got

--- FAIL: TestFallocate (0.00s)
    fallocate_test.go:61: unexpected disk usage: got 4104 blocks, want 4096
    fallocate_test.go:61: unexpected disk usage: got 6152 blocks, want 6144

[aclements]

Allocating too few blocks is somewhat concerning, but allocating a few too many is much less. I could see unusual file system configurations doing this. E.g., maybe there's some reason the file system reports one block size but actually operates with a bigger block size in some cases, or it speculatively allocates ahead. Maybe we should make the test just accept that?

[ianlancetaylor]

SGTM, sent a CL.

[gopherbot]

Change https://golang.org/cl/282555 mentions this issue: cmd/link: accept extra blocks in TestFallocate

[cherrymui]

There was some bug in our linker that we erroneously allocated too many blocks on Darwin. Maybe we could check that it doesn't allocate way too many blocks?

[ianlancetaylor]

CL 282555 has been merged, but of course feel free to adjust to whatever level you think is appropriate. As you can see above what I saw on my system was that it sometimes allocated an extra 8 blocks.

[bcmills]

Is this the same issue as #42005?

[cherrymui]

I don't think this is the same as #42005 . This is on darwin. #42005 is specifically for ZFS on Linux.

That said, with the CL above I think we can call this issue fixed.