[0.10.1] panic: runtime error: slice bounds out of range

[jackzampolin]

Instances are clustered and restart of any of the nodes causes this panic:

panic: runtime error: slice bounds out of range

goroutine 71 [running]:
github.com/influxdb/influxdb/tsdb/engine/tsm1.(*indirectIndex).UnmarshalBinary(0xc222850f00, 0x7f715
abea3c9, 0x7efa2a, 0x7efa32, 0x0, 0x0)
        /tmp/tmp.n7AHL1nb9U/src/github.com/influxdb/influxdb/tsdb/engine/tsm1/reader.go:639 +0x45f

Nodes cannot be restarted currently. The logs from the nodes are attached

Gist with logs

Tagging @joelegasse per his request

[joelegasse]

Would it be possible to get a copy of the database that is causing the issue?

Is this limited to 0.10.1? Or is this also seen in 0.10.3?

[joelegasse]

This appears to be a corrupt/truncated index from where the code is panicking. We can (and should) add some checks to make sure everything doesn't crash, but that wouldn't fix the ability to load the data, or even prevent the corruption if we happen to be creating these corrupt indexes ourselves.

[jwilder]

Should be fixed via #6082

[beckettsean]

related to #5889