all: ensure that tests do not write to the current directory #28387

bcmills · 2018-10-25T15:55:15Z

In #27957, @hyangah noticed that the tests for compress/bzip2 fail when GOROOT is not writable, and those tests are run whenever we run go test all in module mode (which is intended to be a useful default).

As noted in #28386, tests should not assume that they can write to the current directory. We should ensure that none of the tests in the standard library make that mistake.

The text was updated successfully, but these errors were encountered:

tkivisik · 2018-10-27T20:27:17Z

Does that mean the following pattern would be fine (create and use a temporary folder)
e.g.

wd, _ := os.Getwd()
dir, err := ioutil.TempDir("", "testing")
if err != nil {
	log.Fatal(err)
}
defer os.RemoveAll(dir) // clean up
// change and restore finally
if err := os.Chdir(dir); err != nil {
	return
}
defer os.Chdir(wd)
// Create necessary files for a test
// Run tests

dpinela · 2018-10-28T16:18:16Z

@bcmills It isn't the tests for compress/bzip2 that fail in #27957, it's the ones for compress/flate, and in that particular case it only happens when they would fail anyway - they write the actual output to testdata/*.got when actual != expected. Considering these files need to outlive the test, I'm not sure where else we should put them. On the other hand, this case might be fine - if your GOROOT is read-only, then you're probably using a stable release of Go, which presumably means the tests are passing; if they're not, you're using a dev tree, which is almost certainly writable.

gopherbot · 2018-10-28T17:46:23Z

Change https://golang.org/cl/145283 mentions this issue: os: ensure tests pass even if GOROOT is read-only

We achieve this by always running all tests that create files in a fresh temporary directory, rather than just on darwin/{arm,arm64}. As a bonus, this lets us simplify the cleanup code for these tests and assume their working directory starts out empty. Updates #28387 Change-Id: I952007ae390a2451c9a368da26c7f9f5af64b2ba Reviewed-on: https://go-review.googlesource.com/c/145283 Reviewed-by: Brad Fitzpatrick <[email protected]> Run-TryBot: Brad Fitzpatrick <[email protected]> TryBot-Result: Gobot Gobot <[email protected]>

bcmills · 2018-10-29T17:57:33Z

@tkivisik, yes, but in most cases you shouldn't even need to bother with Getwd and Chdir.

I've found this pattern pretty helpful:

	dir, err := ioutil.TempDir("", path.Base(t.Name()))
	if err != nil {
		t.Fatal(err)
	}
	defer os.RemoveAll(dir)

I wonder whether we should just make that a method on *testing.T: the os.RemoveAll doesn't strictly need to be deferred right there at the call site (we can run it at any point after the test function returns), and it might be useful to have a standard flag (along the lines of -work) to leave the temporary directory in place.

bcmills · 2018-10-29T18:05:26Z

@dpinela

Considering these files need to outlive the test, I'm not sure where else we should put them.

Do they belong in side-files at all, given that the -update flag is not set? It seems like most regressions will be fairly small diffs, in which case we could log the first few lines of the diff — and that would also make the failure logs more useful if the reader doesn't have filesystem access (such as in the case of a TryBot failure).

If someone wants to dig in and fix a massive regression, they can always use -update and look at the diff using git diff instead of a separate side file.

bcmills · 2018-10-29T18:06:38Z

if your GOROOT is read-only, then you're probably using a stable release of Go, which presumably means the tests are passing

That's not a valid assumption in general. Platform- or architecture-dependent bugs and non-hermetic tests can all lead to failures even with a stable release.

gopherbot · 2018-10-30T23:43:13Z

Change https://golang.org/cl/146119 mentions this issue: go/internal/gcimporter: ensure tests pass even if GOROOT is read-only

This mainly entails writing compiler output files to a temporary directory, as well as the corrupted files in TestVersionHandling. Updates #28387 Change-Id: I6b3619a91fff27011c7d73daa4febd14a6c5c348 Reviewed-on: https://go-review.googlesource.com/c/146119 Run-TryBot: Robert Griesemer <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Robert Griesemer <[email protected]>

darkfeline · 2019-02-04T09:07:04Z

The tests for os appear to be affected by this (the bug currently only mentions compress/bzip2 so I thought I should point it out). (#29676)

--- FAIL: TestStatError (0.00s)
    os_test.go:200: symlink no-such-file symlink: permission denied
--- FAIL: TestHardLink (0.00s)
    os_test.go:694: open "hardlinktestto" failed: open hardlinktestto: permission denied
--- FAIL: TestSymlink (0.00s)
    os_test.go:780: Create("symlinktestto") failed: open symlinktestto: permission denied
--- FAIL: TestLongSymlink (0.00s)
    os_test.go:847: symlink "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", "longsymlinktestfrom" failed: symlink 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef longsymlinktestfrom: permission denied
--- FAIL: TestRename (0.00s)
    os_test.go:868: open "renamefrom" failed: open renamefrom: permission denied
--- FAIL: TestRenameOverwriteDest (0.00s)
    os_test.go:896: write file "renameto" failed: open renameto: permission denied
--- FAIL: TestAppend (0.00s)
    os_test.go:1684: Open: open append.txt: permission denied
--- FAIL: TestSameFile (0.00s)
    os_test.go:1766: Create(a): open a: permission denied
FAIL
FAIL	os	18.381s

dpinela · 2019-02-09T16:39:30Z

@darkfeline Is that on tip? I think these failures are already fixed there. Just ran them again to check.

darkfeline · 2019-02-18T03:34:14Z

@dpinela yes it is fixed on master, I had only checked the latest release.

gopherbot · 2019-02-19T19:59:24Z

Change https://golang.org/cl/163037 mentions this issue: cmd/vet: do not write test vet binary to GOROOT

gopherbot · 2019-02-19T20:24:58Z

Change https://golang.org/cl/162987 mentions this issue: unix: do not invoke Mkfifo with a relative path in a read-only directory

Updates golang/go#28387 Change-Id: Ibcdc3f9cb3dc43b86b7e7d3539ed592219e54aba Reviewed-on: https://go-review.googlesource.com/c/162987 Run-TryBot: Bryan C. Mills <[email protected]> Reviewed-by: Brad Fitzpatrick <[email protected]> TryBot-Result: Gobot Gobot <[email protected]>

Updates #28387 Change-Id: Ie5a5f1f798eb5900f9c7bdef165abcca02dd0dde Reviewed-on: https://go-review.googlesource.com/c/163037 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>

Updates golang#28387 Change-Id: Ie5a5f1f798eb5900f9c7bdef165abcca02dd0dde Reviewed-on: https://go-review.googlesource.com/c/163037 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>

Updates #28387 Updates #30316 Change-Id: I48c6dd8619ea9602e9617ce11dfa05f1c70a485d Reviewed-on: https://go-review.googlesource.com/c/go/+/207958 Reviewed-by: Jay Conrod <[email protected]>

This test runs 'go test -race -i runtime/race' and checks that it did not overwrite cmd/cgo. If GOROOT/pkg is read-only and GOROOT/pkg/$GOOS_$GOARCH_race is not already populated, as are the conditions if the Go toolchain was installed from source as root using 'make.bash', then 'go test -race -i' itself will fail because it cannot install packages to GOROOT/pkg. However, such a failure is not relevant to the test: even if 'go test -race -i' fails, we can still verify that it did not incidentally overwrite cmd/cgo. Updates #28387 Updates #30316 Change-Id: Iff2f75a0aeb4c926290ac3062c83695604522078 Reviewed-on: https://go-review.googlesource.com/c/go/+/207959 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Jay Conrod <[email protected]>

…t test Updates #28387 Updates #30316 Change-Id: I31e04c89f2cc226f9b5110f14c8b80a18e937efb Reviewed-on: https://go-review.googlesource.com/c/go/+/207960 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Jay Conrod <[email protected]>

gopherbot · 2019-11-19T19:23:15Z

Change https://golang.org/cl/207965 mentions this issue: cmd/go: consolidate TestInstalls into gopath_install script test

TestInstalls was already mostly redundant with TestInstallInto{GOPATH,GOBIN}, except for one additional check for the install location of cmd/fix. We can't assume that GOROOT is writable in general, so we also can't assume that the test will be able to reinstall cmd/fix at run time. Moreover, other processes running in parallel may expect to invoke cmd/fix themselves, so this test temporarily removing it could induce systemwide flakes. We could carefully construct a parallel GOROOT and install cmd/fix into it, but we can get *almost* as much coverage — at a much lower cost — by checking the output of 'go list' instead of actually rebuilding and reinstalling the binary. Updates #28387 Updates #30316 Change-Id: Id49f44a68b0c52dfabb84c665f63c4e7db58dd49 Reviewed-on: https://go-review.googlesource.com/c/go/+/207965 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Jay Conrod <[email protected]>

gopherbot · 2019-11-20T15:32:57Z

Change https://golang.org/cl/208117 mentions this issue: misc/cgo/testcarchive: avoid writing to GOROOT in tests

gopherbot · 2019-11-20T19:42:10Z

Change https://golang.org/cl/208119 mentions this issue: misc/cgo/testcshared: avoid writing to GOROOT in tests

gopherbot · 2019-11-20T21:57:30Z

Change https://golang.org/cl/208123 mentions this issue: message/pipeline: avoid writing to the testdata directory

gopherbot · 2019-11-20T22:15:09Z

Change https://golang.org/cl/208124 mentions this issue: misc/cgo/fortran: avoid writing to $PWD

The bash script that drives this test needs to know whether the fortran compiler works, but it doesn't actually care about the generated binary. Write that binary to /dev/null. Updates #28387 Updates #30316 Change-Id: I4f86da1aeb939fc205f467511fc69235a6a9af26 Reviewed-on: https://go-review.googlesource.com/c/go/+/208124 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Brad Fitzpatrick <[email protected]>

Also add a -testwork flag to facilitate debugging the test itself. Three of the tests of this package invoked 'go install -i -buildmode=c-archive' in order to generate an archive as well as multiple C header files. Unfortunately, the behavior of the '-i' flag is inappropriately broad for this use-case: it not only generates the library and header files (as desired), but also attempts to install a number of (unnecessary) archive files for transitive dependencies to GOROOT/pkg/$GOOS_$GOARCH_shared, which may not be writable — for example, if GOROOT is owned by the root user but the test is being run by a non-root user. Instead, for now we generate the header files for transitive dependencies separately by running 'go tool cgo -exportheader'. In the future, we should consider how to improve the ergonomics for generating transitive header files without coupling that to unnecessary library installation. Updates #28387 Updates #30316 Updates #35715 Change-Id: I3d483f84e22058561efe740aa4885fc3f26137b5 Reviewed-on: https://go-review.googlesource.com/c/go/+/208117 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>

The tests in this package invoked 'go install -i -buildmode=c-shared' in order to generate an archive as well as multiple C header files. Unfortunately, the behavior of the '-i' flag is inappropriately broad for this use-case: it not only generates the library and header files (as desired), but also attempts to install a number of (unnecessary) archive files for transitive dependencies to GOROOT/pkg/$GOOS_$GOARCH_testcshared_shared, which may not be writable — for example, if GOROOT is owned by the root user but the test is being run by a non-root user. Instead, for now we generate the header files for transitive dependencies separately by running 'go tool cgo -exportheader'. In the future, we should consider how to improve the ergonomics for generating transitive header files without coupling that to unnecessary library installation. Updates #28387 Updates #30316 Updates #35715 Change-Id: I622426a860828020d98f7040636f374e5c766d28 Reviewed-on: https://go-review.googlesource.com/c/go/+/208119 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>

If this test is run as a dependency of some other module, the testdata directory will be read-only. Moreover, if temporary files are written to the working directory they are likely to interfere with concurrent version-control operations, such as 'git commit -a'. Updates golang/go#28387 Change-Id: I15cd9408c63f9b6aed50facbfefa26299392052f Reviewed-on: https://go-review.googlesource.com/c/text/+/208123 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Marcel van Lohuizen <[email protected]>

gopherbot · 2019-11-22T21:58:33Z

Change https://golang.org/cl/208482 mentions this issue: misc/cgo/testshared: do not write to GOROOT

gopherbot · 2019-11-22T21:58:36Z

Change https://golang.org/cl/208481 mentions this issue: misc: remove use of relative directories in overlayDir functions

It turns out that the relative-path support never worked in the first place. It had been masked by the fact that we ~never invoke overlayDir with an absolute path, which caused filepath.Rel to always return an error, and overlayDir to always fall back to absolute paths. Since the absolute paths seem to be working fine (and are simpler), let's stick with those. As far as I can recall, the relative paths were only a space optimization anyway. Updates #28387 Updates #30316 Change-Id: Ie8cd28f3c41ca6497ace2799f4193d7f5dde7a37 Reviewed-on: https://go-review.googlesource.com/c/go/+/208481 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Jay Conrod <[email protected]>

Instead of installing shared libraries to GOROOT/pkg, clone the necessary files into a new GOROOT and run there. Given that we now have a build cache, ideally we should not need to install into GOROOT/pkg at all, but we can't fix that during the 1.14 code freeze. Updates #28387 Updates #28553 Updates #30316 Change-Id: I83084a8ca29a5dffcd586c7fccc3f172cac57cc6 Reviewed-on: https://go-review.googlesource.com/c/go/+/208482 Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Gobot Gobot <[email protected]> Reviewed-by: Jay Conrod <[email protected]>

bcmills · 2019-11-25T17:14:21Z

This is now fixed for the standard library. Leaving open for the x/ modules.

gopherbot · 2021-01-15T21:46:18Z

Change https://golang.org/cl/284293 mentions this issue: cmd/link: remove GOROOT write in TestBuildForTvOS

Tests should avoid writing to GOROOT when possible. Such writes would fail if GOROOT is non-writeable, and it can interfere with other tests that don't expect GOROOT to change during test execution. Updates #28387. Change-Id: I7d72614f218df3375540f5c2f9c9f8c11034f602 Reviewed-on: https://go-review.googlesource.com/c/go/+/284293 Trust: Dmitri Shuralyov <[email protected]> Reviewed-by: Cherry Zhang <[email protected]> Reviewed-by: Bryan C. Mills <[email protected]>

gopherbot · 2022-03-04T17:28:57Z

Change https://go.dev/cl/390075 mentions this issue: cmd/getgo: exec main from TestMain instead of running 'go build' in tests

…ests This was noticed from https://build.golang.org/log/da703ece9e1626eaeabf485e1a3a8180a6bde512, but I suspect not relevant to the getgo test failure observed there. Updates golang/go#28387 Change-Id: I1a156e780beabb13b4df6fd5313d4785aeb26e97 Reviewed-on: https://go-review.googlesource.com/c/tools/+/390075 Trust: Bryan Mills <[email protected]> Run-TryBot: Bryan Mills <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> gopls-CI: kokoro <[email protected]> TryBot-Result: Gopher Robot <[email protected]>

bcmills · 2024-03-13T16:19:40Z

@golang/release, do we know whether the LUCI builder infrastructure can better support making the working directory read-only when running x/ repo tests?

bcmills added the Testing An issue that has been verified to require only test changes, not just a test failure. label Oct 25, 2018

bcmills added this to the Go1.12 milestone Oct 25, 2018

bcmills added NeedsFix The path to resolution is known, but the work has not been done. help wanted labels Oct 25, 2018

bcmills mentioned this issue Oct 25, 2018

cmd/go: document that directory patterns only match dirs in current module #27957

Open

ianlancetaylor modified the milestones: Go1.12, Unplanned Nov 29, 2018

bcmills mentioned this issue Dec 14, 2018

cmd/go: TestNewReleaseRebuildsStalePackagesInGOPATH modifies code in GOROOT #29263

Closed

dpinela mentioned this issue Jan 12, 2019

cmd/go: go test all should not run standard library tests #29676

Closed

darkfeline mentioned this issue Feb 4, 2019

path/filepath: TestBug3486 fails if GOROOT/test is removed #29713

Open

ianlancetaylor mentioned this issue Feb 11, 2019

os: Testing re. hardlinks failing on macs. #30179

Closed

bcmills mentioned this issue Feb 19, 2019

x/build: make GOROOT read-only before running tests #30316

Open

ianlancetaylor modified the milestones: Go1.14, Unreleased Dec 5, 2019

bcmills mentioned this issue May 4, 2020

cmd/go: "package … is not in GOROOT" is confusing in module mode when the package would exist in GOPATH mode #38812

Open

bcmills mentioned this issue Jan 12, 2021

go/types: TestCheck and TestStdlib fail when installed from binary distribution #43232

Closed

bcmills mentioned this issue Jan 19, 2021

os: spurious TestDirFS failures due to directory mtime skew on Windows #42637

Closed

bcmills mentioned this issue Sep 30, 2021

cmd/go: tests fail when GOROOT is stale #48698

Closed

bcmills removed their assignment Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

all: ensure that tests do not write to the current directory #28387

all: ensure that tests do not write to the current directory #28387

bcmills commented Oct 25, 2018

tkivisik commented Oct 27, 2018

dpinela commented Oct 28, 2018

gopherbot commented Oct 28, 2018

bcmills commented Oct 29, 2018

bcmills commented Oct 29, 2018

bcmills commented Oct 29, 2018

gopherbot commented Oct 30, 2018

darkfeline commented Feb 4, 2019

dpinela commented Feb 9, 2019

darkfeline commented Feb 18, 2019

gopherbot commented Feb 19, 2019

gopherbot commented Feb 19, 2019

gopherbot commented Nov 19, 2019

gopherbot commented Nov 20, 2019

gopherbot commented Nov 20, 2019

gopherbot commented Nov 20, 2019

gopherbot commented Nov 20, 2019

gopherbot commented Nov 22, 2019

gopherbot commented Nov 22, 2019

bcmills commented Nov 25, 2019

gopherbot commented Jan 15, 2021

gopherbot commented Mar 4, 2022

bcmills commented Mar 13, 2024

all: ensure that tests do not write to the current directory #28387

all: ensure that tests do not write to the current directory #28387

Comments

bcmills commented Oct 25, 2018

tkivisik commented Oct 27, 2018

dpinela commented Oct 28, 2018

gopherbot commented Oct 28, 2018

bcmills commented Oct 29, 2018

bcmills commented Oct 29, 2018

bcmills commented Oct 29, 2018

gopherbot commented Oct 30, 2018

darkfeline commented Feb 4, 2019

dpinela commented Feb 9, 2019

darkfeline commented Feb 18, 2019

gopherbot commented Feb 19, 2019

gopherbot commented Feb 19, 2019

gopherbot commented Nov 19, 2019

gopherbot commented Nov 20, 2019

gopherbot commented Nov 20, 2019

gopherbot commented Nov 20, 2019

gopherbot commented Nov 20, 2019

gopherbot commented Nov 22, 2019

gopherbot commented Nov 22, 2019

bcmills commented Nov 25, 2019

gopherbot commented Jan 15, 2021

gopherbot commented Mar 4, 2022

bcmills commented Mar 13, 2024