Failed async BookKeeper writes should cause peer to to restart

[lbradstreet]

See https://github.com/onyx-platform/onyx/blob/0.8.x/src/onyx/state/log/bookkeeper.clj#L67

If a given write has failed, then the task's local state is no longer going to be in sync with the played back log, and now new log entries will be written. The peer should either rollback to the old state and create a new ledger, or should suicide, causing a new peer replay the state and start writing to a new ledger. The unacked messages will then be replayed.

I suggest we do the second first, then create a new issue to implement the first at some point. I believe implementing the first is worthwhile because in case of a partition we may not want all the grouping peers to restart at the same time, and would rather them attempt to recover. It may be tricky to do so however.

[lbradstreet]

Given #410, maybe we should only restart if the write failed and we're still writing to the same ledger as the original write.

[lbradstreet]

Confirmed to be an issue by jepsen.

[lbradstreet]

Fixed in 4d3684e.