gencmdref script ending in runtime error

[RichardLitt]

I am trying to move bin/gencmdref to the ipfs/website directly, because the Makefile there uses it, it has a hardcoded to path to where it is stored on @jbenet's machine, and there's now been two PRs to the script in this repository to fix an issue in ipfs/website (#1733 and #1723).

Running ipfs commands | gencmdref breaks, however, and I'm not sure why. Here is my exit code:

panic: runtime error: index out of range

goroutine 1 [running]:
github.com/ipfs/go-ipfs/commands/cli.parseArgs(0x8210fb4d0, 0x0, 0x3, 0x82111c008, 0xf41a80, 0x2, 0x2, 0x0, 0xf42220, 0x0, ...)
    /go/src/github.com/ipfs/go-ipfs/commands/cli/parse.go:266 +0x10ea
github.com/ipfs/go-ipfs/commands/cli.Parse(0x8210f23d0, 0x3, 0x3, 0x82111c008, 0xf42220, 0x0, 0x0, 0x821249d00, 0x0, 0x0, ...)
    /go/src/github.com/ipfs/go-ipfs/commands/cli/parse.go:45 +0x52b
main.(*cmdInvocation).Parse(0x8210f2840, 0x88212a2110, 0x8210fb440, 0x8210f23d0, 0x3, 0x3, 0x0, 0x0)
    /go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:236 +0x89
main.main()
    /go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:107 +0x3db

goroutine 18 [chan receive]:
github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics.init.1.func2()
    /go/src/github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics/metrics.go:321 +0x73
created by github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics.init.1
    /go/src/github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics/metrics.go:328 +0x7b

goroutine 19 [syscall]:
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37
Traceback (most recent call last):
  File "gencmdref", line 38, in <module>
    main()
  File "gencmdref", line 28, in main
    print run((line + ' --help').split(' ')).strip()
  File "gencmdref", line 11, in run
    return check_output(cmd)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['ipfs', 'dht', 'put', '--help']' returned non-zero exit status 2

[whyrusleeping]

does piping ipfs commands to anything else also break?

[RichardLitt]

Nope. ipfs commands | grep 'ipfs' runs perfect.

[RichardLitt]

Clarification: ipfs commands | py gencmdref breaks. ipfs commands | gencmdref doesn't do anything.

[jbenet]

run it like:

ipfs commands | ./gencmdref

but yeah i see the same error as @RichardLitt -- after a bunch of good output, i get the stuff below. this is weird because running ipfs dht put --help runs fine on its own. even weirder: it breaks on the cli only (i.e. running without a daemon!)

## ipfs dht put

panic: runtime error: index out of range

goroutine 1 [running]:
github.com/ipfs/go-ipfs/commands/cli.parseArgs(0x821364d50, 0x0, 0x3, 0x82111a008, 0xf45e40, 0x2, 0x2, 0x0, 0xf465e0, 0x0, ...)
    /go/src/github.com/ipfs/go-ipfs/commands/cli/parse.go:266 +0x10ea
github.com/ipfs/go-ipfs/commands/cli.Parse(0x8210f62d0, 0x3, 0x3, 0x82111a008, 0xf465e0, 0x0, 0x0, 0x821251d00, 0x0, 0x0, ...)
    /go/src/github.com/ipfs/go-ipfs/commands/cli/parse.go:45 +0x52b
main.(*cmdInvocation).Parse(0x821360340, 0x88213220f8, 0x821364cc0, 0x8210f62d0, 0x3, 0x3, 0x0, 0x0)
    /go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:236 +0x89
main.main()
    /go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:107 +0x3db

goroutine 17 [chan receive]:
github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics.init.1.func2()
    /go/src/github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics/metrics.go:321 +0x73
created by github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics.init.1
    /go/src/github.com/ipfs/go-ipfs/Godeps/_workspace/src/github.com/codahale/metrics/metrics.go:328 +0x7b

goroutine 18 [syscall]:
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37
Traceback (most recent call last):
  File "bin/gencmdref", line 38, in <module>
    main()
  File "bin/gencmdref", line 28, in main
    print run((line + ' --help').split(' ')).strip()
  File "bin/gencmdref", line 11, in run
    return check_output(cmd)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['ipfs', 'dht', 'put', '--help']' returned non-zero exit status 2

[jbenet]

@RichardLitt where's the hardcoded path? is that in the website repo?

[jbenet]

ah yep: https://github.com/ipfs/website/blob/master/src/ipfs.io/docs/commands/Makefile#L3 -- though that's pretty standard and likely to match most users' installation of go. maybe slightly better would be to do, there:

gencmdref:
    wget -o gencmdref https://raw.githubusercontent.com/ipfs/go-ipfs/master/bin/gencmdref
    # or replace this o/ with an https://ipfs.io/ipfs/<hash> url, to get security
    chmod +x gencmdref

cmdref.md: gencmdref
    ipfs commands | ./gencmdref >$@

[jbenet]

(or yeah, move it there altogether)

[RichardLitt]

So, this is still breaking. And we can't update the website with that being broken.

@whyrusleeping can you help? or @noffle or @lgierth, maybe?

[whyrusleeping]

looks like this is an issue with ipfs dht put, repro with:

echo "ipfs dht put" | ./gencmdref

[whyrusleeping]

more concretely:

echo "" | ipfs dht put

[whyrusleeping]

it seems to be because we give it a stdin argument, and it counts that as an 'available argument' even though we don't have anything. Then, since the first argument isnt one that can be satisfied by stdin, attempts to grab the first input (that doesnt exist), and panics because of array boundary breaking

[RichardLitt]

Thanks @whyrusleeping.