Is the Raft Read Optimization actually (easily?) possible?

[otoolep]

API call: https://pkg.go.dev/github.com/hashicorp/raft#Raft.CommitIndex

Discussion on Google Groups: https://groups.google.com/g/raft-dev/c/4QlyV0aptEQ/m/1bCWuAU7AgAJ

Background CommitIndex is nominally available to clients of this library so those clients can execute the Read Optimization described in section 6.4 of the Raft dissertation Processing read-only queries more efficiently. To summarize the process:

1. Read the current commit index
2. Verify leadership status
3. Wait until the FSM has applied the log entry indicated the commit index read in step 1
4. Do the read and return the results to the client.

However step 1 is not valid unless the Leader has committed at least one log entry after gaining Leadership (see the Google Group discussion above, which includes input from Diego). This implementation does do that here: https://github.com/hashicorp/raft/blob/v1.7.1/raft.go#L565

Any suggestions on how clients of this library can determine when that log entry has been committed? Without this knowledge it's not safe to read CommitIndex for the purposes of the Read Optimizatoin. I can think of a few ways, but haven't looked into the details:

1. Register for leadership changes and when a node is elected leader start a polling operation looking for some sign a Noop has been committed
2. Changes to this library so that it signals that this log has been committed
3. A new attribute on the library -- call it NoopCommitIndex which would be set whenever a Noop is committed. This might help clients figure it out.
4. Something else that I've missed?

[otoolep]

A change such as the following might work: #622

The library would need other code such that this value was store, exposed, etc. Then the Read Optimization could be implemented by first ensuring (blocking if needed) CommitIndex >= NoopCommitindex and then using the value of CommitIndex. At least, I think so.

I can work on a PR, unless you have other ideas.

[otoolep]

See #625 for a proposed fix for this issue.