cmd/go: 'go test -h' prints much more than other -h flags

[mvdan]

$ go build -h
usage: go build [-o output] [-i] [build flags] [packages]
Run 'go help build' for details.
$ go get -h
usage: go get [-d] [-f] [-t] [-u] [-v] [-fix] [-insecure] [build flags] [packages]
Run 'go help get' for details.
$ go install -h
usage: go install [-i] [build flags] [packages]
Run 'go help install' for details.
$ go fmt -h
usage: go fmt [-n] [-x] [packages]
Run 'go help fmt' for details.
$ go env -h
usage: go env [-json] [var ...]
Run 'go help env' for details.
$ go test -h
usage: go test [build/test flags] [packages] [build/test flags & test binary flags]

In addition to the build flags, the flags handled by 'go test' itself are:

        -args
            Pass the remainder of the command line (everything after -args)
            to the test binary, uninterpreted and unchanged.
            Because this flag consumes the remainder of the command line,
            the package list (if present) must appear before this flag.

        -c
            Compile the test binary to pkg.test but do not run it
            (where pkg is the last element of the package's import path).
            The file name can be changed with the -o flag.

        -exec xprog
            Run the test binary using xprog. The behavior is the same as
            in 'go run'. See 'go help run' for details.

        -i
            Install packages that are dependencies of the test.
            Do not run the test.

        -json
            Convert test output to JSON suitable for automated processing.
            See 'go doc test2json' for the encoding details.

        -o file
            Compile the test binary to the named file.
            The test still runs (unless -c or -i is specified).

The test binary also accepts flags that control execution of the test; these
flags are also accessible by 'go test'.

        -bench regexp
            Run only those benchmarks matching a regular expression.
            By default, no benchmarks are run.
            To run all benchmarks, use '-bench .' or '-bench=.'.
            The regular expression is split by unbracketed slash (/)
            characters into a sequence of regular expressions, and each
            part of a benchmark's identifier must match the corresponding
            element in the sequence, if any. Possible parents of matches
            are run with b.N=1 to identify sub-benchmarks. For example,
            given -bench=X/Y, top-level benchmarks matching X are run
            with b.N=1 to find any sub-benchmarks matching Y, which are
            then run in full.

        -benchtime t
            Run enough iterations of each benchmark to take t, specified
            as a time.Duration (for example, -benchtime 1h30s).
            The default is 1 second (1s).

        -count n
            Run each test and benchmark n times (default 1).
            If -cpu is set, run n times for each GOMAXPROCS value.
            Examples are always run once.

        -cover
            Enable coverage analysis.
            Note that because coverage works by annotating the source
            code before compilation, compilation and test failures with
            coverage enabled may report line numbers that don't correspond
            to the original sources.

        -covermode set,count,atomic
            Set the mode for coverage analysis for the package[s]
            being tested. The default is "set" unless -race is enabled,
            in which case it is "atomic".
            The values:
                set: bool: does this statement run?
                count: int: how many times does this statement run?
                atomic: int: count, but correct in multithreaded tests;
                        significantly more expensive.
            Sets -cover.

        -coverpkg pattern1,pattern2,pattern3
            Apply coverage analysis in each test to packages matching the patterns.
            The default is for each test to analyze only the package being tested.
            See 'go help packages' for a description of package patterns.
            Sets -cover.

        -cpu 1,2,4
            Specify a list of GOMAXPROCS values for which the tests or
            benchmarks should be executed. The default is the current value
            of GOMAXPROCS.

        -failfast
            Do not start new tests after the first test failure.

        -list regexp
            List tests, benchmarks, or examples matching the regular expression.
            No tests, benchmarks or examples will be run. This will only
            list top-level tests. No subtest or subbenchmarks will be shown.

        -parallel n
            Allow parallel execution of test functions that call t.Parallel.
            The value of this flag is the maximum number of tests to run
            simultaneously; by default, it is set to the value of GOMAXPROCS.
            Note that -parallel only applies within a single test binary.
            The 'go test' command may run tests for different packages
            in parallel as well, according to the setting of the -p flag
            (see 'go help build').

        -run regexp
            Run only those tests and examples matching the regular expression.
            For tests, the regular expression is split by unbracketed slash (/)
            characters into a sequence of regular expressions, and each part
            of a test's identifier must match the corresponding element in
            the sequence, if any. Note that possible parents of matches are
            run too, so that -run=X/Y matches and runs and reports the result
            of all tests matching X, even those without sub-tests matching Y,
            because it must run them to look for those sub-tests.

        -short
            Tell long-running tests to shorten their run time.
            It is off by default but set during all.bash so that installing
            the Go tree can run a sanity check but not spend time running
            exhaustive tests.

        -timeout d
            If a test binary runs longer than duration d, panic.
            If d is 0, the timeout is disabled.
            The default is 10 minutes (10m).

        -v
            Verbose output: log all tests as they are run. Also print all
            text from Log and Logf calls even if the test succeeds.

        -vet list
            Configure the invocation of "go vet" during "go test"
            to use the comma-separated list of vet checks.
            If list is empty, "go test" runs "go vet" with a curated list of
            checks believed to be always worth addressing.
            If list is "off", "go test" does not run "go vet" at all.

The following flags are also recognized by 'go test' and can be used to
profile the tests during execution:

        -benchmem
            Print memory allocation statistics for benchmarks.

        -blockprofile block.out
            Write a goroutine blocking profile to the specified file
            when all tests are complete.
            Writes test binary as -c would.

        -blockprofilerate n
            Control the detail provided in goroutine blocking profiles by
            calling runtime.SetBlockProfileRate with n.
            See 'go doc runtime.SetBlockProfileRate'.
            The profiler aims to sample, on average, one blocking event every
            n nanoseconds the program spends blocked. By default,
            if -test.blockprofile is set without this flag, all blocking events
            are recorded, equivalent to -test.blockprofilerate=1.

        -coverprofile cover.out
            Write a coverage profile to the file after all tests have passed.
            Sets -cover.

        -cpuprofile cpu.out
            Write a CPU profile to the specified file before exiting.
            Writes test binary as -c would.

        -memprofile mem.out
            Write an allocation profile to the file after all tests have passed.
            Writes test binary as -c would.

        -memprofilerate n
            Enable more precise (and expensive) memory allocation profiles by
            setting runtime.MemProfileRate. See 'go doc runtime.MemProfileRate'.
            To profile all memory allocations, use -test.memprofilerate=1.

        -mutexprofile mutex.out
            Write a mutex contention profile to the specified file
            when all tests are complete.
            Writes test binary as -c would.

        -mutexprofilefraction n
            Sample 1 in n stack traces of goroutines holding a
            contended mutex.

        -outputdir directory
            Place output files from profiling in the specified directory,
            by default the directory in which "go test" is running.

        -trace trace.out
            Write an execution trace to the specified file before exiting.

Each of these flags is also recognized with an optional 'test.' prefix,
as in -test.v. When invoking the generated test binary (the result of
'go test -c') directly, however, the prefix is mandatory.

The 'go test' command rewrites or removes recognized flags,
as appropriate, both before and after the optional package list,
before invoking the test binary.

For instance, the command

        go test -v -myflag testdata -cpuprofile=prof.out -x

will compile the test binary and then run it as

        pkg.test -test.v -myflag testdata -test.cpuprofile=prof.out

(The -x flag is removed because it applies only to the go command's
execution, not to the test itself.)

The test flags that generate profiles (other than for coverage) also
leave the test binary in pkg.test for use when analyzing the profiles.

When 'go test' runs a test binary, it does so from within the
corresponding package's source code directory. Depending on the test,
it may be necessary to do the same when invoking a generated test
binary directly.

The command-line package list, if present, must appear before any
flag not known to the go test command. Continuing the example above,
the package list would have to appear before -myflag, but could appear
on either side of -v.

When 'go test' runs in package list mode, 'go test' caches successful
package test results to avoid unnecessary repeated running of tests. To
disable test caching, use any test flag or argument other than the
cacheable flags. The idiomatic way to disable test caching explicitly
is to use -count=1.

To keep an argument for a test binary from being interpreted as a
known flag or a package name, use -args (see 'go help test') which
passes the remainder of the command line through to the test binary
uninterpreted and unaltered.

For instance, the command

        go test -v -args -x -v

will compile the test binary and then run it as

        pkg.test -test.v -x -v

Similarly,

        go test -args math

will compile the test binary and then run it as

        pkg.test math

In the first example, the -x and the second -v are passed through to the
test binary unchanged and with no effect on the go command itself.
In the second example, the argument math is passed through to the test
binary, instead of being interpreted as the package list.

I know that this issue body is massive - it helps me prove my point. All go <cmd> -h outputs are tiny and quick, yet test's is the exact opposite for no apparent reason.

If this much information is needed, I suggest putting it in go help test, or if it doesn't fit there, perhaps another help document like go help testing.

[mvdan]

I found this while investigating #26998, since both test and vet handle their flags in a special way via the cmdflag package.

[gopherbot]

Change https://golang.org/cl/129318 mentions this issue: cmd/go: fix 'go vet -h' to print the right text

[rsc]

/cc @robpike

[mvdan]

Bump @robpike?

[robpike]

Looks like the same sort of fix as in CL 129318 should work. Not sure why I'm being pinged here.

[mvdan]

Perhaps I should have been clearer - the two issues are unrelated. The problem here is that go test has two help texts.

go test -h is 251 lines, and can be found at src/cmd/go/internal/test/test.go:172.

go help test is 117 lines, and can be found at src/cmd/go/internal/test/test.go:48.

It does look like the two texts overlap a bit, but they're definitely not the same.

I raised this issue because all other subcommands do have short and long usage texts, but the short one is usually two lines. In this case, the short one is actually the longest, and spans hundreds of lines.

[mvdan]

To further clarify - I simply need help deciding what to do here. To me, go test -h should output two lines, like the rest of subcommands. The question then is what to do with the current text. Do we replace go help test with that text? Do we simply get rid of it?

[robpike]

It seems clear that go test -h should output two lines and the two long things should be merged into one that is shown with go help test.

[rsc]

go test -h should be fixed to output 2 lines.
The 'go help test' is OK as is. It does NOT need to be lengthened.
(Old 'go test -h' was like 'go help test' + 'go help testflag'. They can remain separate.)

[gopherbot]

Change https://golang.org/cl/144558 mentions this issue: cmd/go: make 'go test -h' print two lines