diff --git a/spec/abci/abci++_app_requirements.md b/spec/abci/abci++_app_requirements.md
index 006cf65110c..b721e4e2b9d 100644
--- a/spec/abci/abci++_app_requirements.md
+++ b/spec/abci/abci++_app_requirements.md
@@ -6,52 +6,52 @@ title: Requirements for the Application
# Requirements for the Application
- [Requirements for the Application](#requirements-for-the-application)
- - [Formal Requirements](#formal-requirements)
- - [Consensus Connection Requirements](#consensus-connection-requirements)
- - [Mempool Connection Requirements](#mempool-connection-requirements)
- - [Managing the Application state and related topics](#managing-the-application-state-and-related-topics)
- - [Connection State](#connection-state)
- - [Concurrency](#concurrency)
- - [FinalizeBlock](#finalizeblock)
- - [Commit](#commit)
- - [Candidate States](#candidate-states)
- - [States and ABCI++ Connections](#states-and-abci-connections)
- - [Consensus Connection](#consensus-connection)
- - [Mempool Connection](#mempool-connection)
- - [Replay Protection](#replay-protection)
- - [Info/Query Connection](#infoquery-connection)
- - [Snapshot Connection](#snapshot-connection)
- - [Transaction Results](#transaction-results)
- - [Gas](#gas)
- - [Specifics of `ResponseCheckTx`](#specifics-of-responsechecktx)
- - [Specifics of `ExecTxResult`](#specifics-of-exectxresult)
- - [Updating the Validator Set](#updating-the-validator-set)
- - [Consensus Parameters](#consensus-parameters)
- - [List of Parameters](#list-of-parameters)
- - [BlockParams.MaxBytes](#blockparamsmaxbytes)
- - [BlockParams.MaxGas](#blockparamsmaxgas)
- - [EvidenceParams.MaxAgeDuration](#evidenceparamsmaxageduration)
- - [EvidenceParams.MaxAgeNumBlocks](#evidenceparamsmaxagenumblocks)
- - [EvidenceParams.MaxBytes](#evidenceparamsmaxbytes)
- - [ValidatorParams.PubKeyTypes](#validatorparamspubkeytypes)
- - [VersionParams.App](#versionparamsapp)
- - [ABCIParams.VoteExtensionsEnableHeight](#abciparamsvoteextensionsenableheight)
- - [Updating Consensus Parameters](#updating-consensus-parameters)
- - [`InitChain`](#initchain)
- - [`FinalizeBlock`, `PrepareProposal`/`ProcessProposal`](#finalizeblock-prepareproposalprocessproposal)
- - [`Query`](#query)
- - [Query Proofs](#query-proofs)
- - [Peer Filtering](#peer-filtering)
- - [Paths](#paths)
- - [Crash Recovery](#crash-recovery)
- - [State Sync](#state-sync)
- - [Taking Snapshots](#taking-snapshots)
- - [Bootstrapping a Node](#bootstrapping-a-node)
- - [Snapshot Discovery](#snapshot-discovery)
- - [Snapshot Restoration](#snapshot-restoration)
- - [Snapshot Verification](#snapshot-verification)
- - [Transition to Consensus](#transition-to-consensus)
- - [Application configuration required to switch to ABCI 2.0](#application-configuration-required-to-switch-to-abci-20)
+ - [Formal Requirements](#formal-requirements)
+ - [Consensus Connection Requirements](#consensus-connection-requirements)
+ - [Mempool Connection Requirements](#mempool-connection-requirements)
+ - [Managing the Application state and related topics](#managing-the-application-state-and-related-topics)
+ - [Connection State](#connection-state)
+ - [Concurrency](#concurrency)
+ - [FinalizeBlock](#finalizeblock)
+ - [Commit](#commit)
+ - [Candidate States](#candidate-states)
+ - [States and ABCI++ Connections](#states-and-abci-connections)
+ - [Consensus Connection](#consensus-connection)
+ - [Mempool Connection](#mempool-connection)
+ - [Replay Protection](#replay-protection)
+ - [Info/Query Connection](#infoquery-connection)
+ - [Snapshot Connection](#snapshot-connection)
+ - [Transaction Results](#transaction-results)
+ - [Gas](#gas)
+ - [Specifics of `ResponseCheckTx`](#specifics-of-responsechecktx)
+ - [Specifics of `ExecTxResult`](#specifics-of-exectxresult)
+ - [Updating the Validator Set](#updating-the-validator-set)
+ - [Consensus Parameters](#consensus-parameters)
+ - [List of Parameters](#list-of-parameters)
+ - [BlockParams.MaxBytes](#blockparamsmaxbytes)
+ - [BlockParams.MaxGas](#blockparamsmaxgas)
+ - [EvidenceParams.MaxAgeDuration](#evidenceparamsmaxageduration)
+ - [EvidenceParams.MaxAgeNumBlocks](#evidenceparamsmaxagenumblocks)
+ - [EvidenceParams.MaxBytes](#evidenceparamsmaxbytes)
+ - [ValidatorParams.PubKeyTypes](#validatorparamspubkeytypes)
+ - [VersionParams.App](#versionparamsapp)
+ - [ABCIParams.VoteExtensionsEnableHeight](#abciparamsvoteextensionsenableheight)
+ - [Updating Consensus Parameters](#updating-consensus-parameters)
+ - [`InitChain`](#initchain)
+ - [`FinalizeBlock`, `PrepareProposal`/`ProcessProposal`](#finalizeblock-prepareproposalprocessproposal)
+ - [`Query`](#query)
+ - [Query Proofs](#query-proofs)
+ - [Peer Filtering](#peer-filtering)
+ - [Paths](#paths)
+ - [Crash Recovery](#crash-recovery)
+ - [State Sync](#state-sync)
+ - [Taking Snapshots](#taking-snapshots)
+ - [Bootstrapping a Node](#bootstrapping-a-node)
+ - [Snapshot Discovery](#snapshot-discovery)
+ - [Snapshot Restoration](#snapshot-restoration)
+ - [Snapshot Verification](#snapshot-verification)
+ - [Transition to Consensus](#transition-to-consensus)
+ - [Application configuration required to switch to ABCI 2.0](#application-configuration-required-to-switch-to-abci-20)
## Formal Requirements
@@ -74,7 +74,7 @@ returns via `ResponsePrepareProposal` to CometBFT, also known as the prepared pr
Process *p*'s prepared proposal can differ in two different rounds where *p* is the proposer.
-* Requirement 1 [`PrepareProposal`, timeliness]: If *p*'s Application fully executes prepared blocks in
+- Requirement 1 [`PrepareProposal`, timeliness]: If *p*'s Application fully executes prepared blocks in
`PrepareProposal` and the network is in a synchronous period while processes *p* and *q* are in *rp*,
then the value of *TimeoutPropose* at *q* must be such that *q*'s propose timer does not time out
(which would result in *q* prevoting `nil` in *rp*).
@@ -87,7 +87,7 @@ compromise liveness because even though `TimeoutPropose` is used as the initial
value for proposal timeouts, CometBFT will be dynamically adjust these timeouts
such that they will eventually be enough for completing `PrepareProposal`.
-* Requirement 2 [`PrepareProposal`, tx-size]: When *p*'s Application calls `ResponsePrepareProposal`, the
+- Requirement 2 [`PrepareProposal`, tx-size]: When *p*'s Application calls `ResponsePrepareProposal`, the
total size in bytes of the transactions returned does not exceed `RequestPrepareProposal.max_tx_bytes`.
Busy blockchains might seek to gain full visibility into transactions in CometBFT's mempool,
@@ -99,7 +99,7 @@ Under these settings, the aggregated size of all transactions may exceed `Reques
Hence, Requirement 2 ensures that the size in bytes of the transaction list returned by the application will never
cause the resulting block to go beyond its byte size limit.
-* Requirement 3 [`PrepareProposal`, `ProcessProposal`, coherence]: For any two correct processes *p* and *q*,
+- Requirement 3 [`PrepareProposal`, `ProcessProposal`, coherence]: For any two correct processes *p* and *q*,
if *q*'s CometBFT calls `RequestProcessProposal` on *up*,
*q*'s Application returns Accept in `ResponseProcessProposal`.
@@ -112,12 +112,12 @@ likely hit the bug at the same time. This would result in most (or all) processe
serious consequences on CometBFT's liveness that this entails. Due to its criticality, Requirement 3 is a
target for extensive testing and automated verification.
-* Requirement 4 [`ProcessProposal`, determinism-1]: `ProcessProposal` is a (deterministic) function of the current
+- Requirement 4 [`ProcessProposal`, determinism-1]: `ProcessProposal` is a (deterministic) function of the current
state and the block that is about to be applied. In other words, for any correct process *p*, and any arbitrary block *u*,
if *p*'s CometBFT calls `RequestProcessProposal` on *u* at height *h*,
then *p*'s Application's acceptance or rejection **exclusively** depends on *u* and *sp,h-1*.
-* Requirement 5 [`ProcessProposal`, determinism-2]: For any two correct processes *p* and *q*, and any arbitrary
+- Requirement 5 [`ProcessProposal`, determinism-2]: For any two correct processes *p* and *q*, and any arbitrary
block *u*,
if *p*'s (resp. *q*'s) CometBFT calls `RequestProcessProposal` on *u* at height *h*,
then *p*'s Application accepts *u* if and only if *q*'s Application accepts *u*.
@@ -145,7 +145,7 @@ Let *erp* be the vote extension that the Application of a
Let *wrp* be the proposed block that *p*'s CometBFT passes to the Application via `RequestExtendVote`
in round *r*, height *h*.
-* Requirement 6 [`ExtendVote`, `VerifyVoteExtension`, coherence]: For any two different correct
+- Requirement 6 [`ExtendVote`, `VerifyVoteExtension`, coherence]: For any two different correct
processes *p* and *q*, if *q* receives *erp* from *p* in height *h*, *q*'s
Application returns Accept in `ResponseVerifyVoteExtension`.
@@ -157,13 +157,13 @@ However, if there is a (deterministic) bug in `ExtendVote` or `VerifyVoteExtensi
we will face the same liveness issues as described for Requirement 5, as Precommit messages with invalid vote
extensions will be discarded.
-* Requirement 7 [`VerifyVoteExtension`, determinism-1]: `VerifyVoteExtension` is a (deterministic) function of
+- Requirement 7 [`VerifyVoteExtension`, determinism-1]: `VerifyVoteExtension` is a (deterministic) function of
the current state, the vote extension received, and the prepared proposal that the extension refers to.
In other words, for any correct process *p*, and any arbitrary vote extension *e*, and any arbitrary
block *w*, if *p*'s (resp. *q*'s) CometBFT calls `RequestVerifyVoteExtension` on *e* and *w* at height *h*,
then *p*'s Application's acceptance or rejection **exclusively** depends on *e*, *w* and *sp,h-1*.
-* Requirement 8 [`VerifyVoteExtension`, determinism-2]: For any two correct processes *p* and *q*,
+- Requirement 8 [`VerifyVoteExtension`, determinism-2]: For any two correct processes *p* and *q*,
and any arbitrary vote extension *e*, and any arbitrary block *w*,
if *p*'s (resp. *q*'s) CometBFT calls `RequestVerifyVoteExtension` on *e* and *w* at height *h*,
then *p*'s Application accepts *e* if and only if *q*'s Application accepts *e*.
@@ -178,12 +178,12 @@ Requirements 7 and 8 can be violated by a bug inducing non-determinism in
Extra care should be put in the implementation of `ExtendVote` and `VerifyVoteExtension`.
As a general rule, `VerifyVoteExtension` SHOULD always accept the vote extension.
-* Requirement 9 [*all*, no-side-effects]: *p*'s calls to `RequestPrepareProposal`,
+- Requirement 9 [*all*, no-side-effects]: *p*'s calls to `RequestPrepareProposal`,
`RequestProcessProposal`, `RequestExtendVote`, and `RequestVerifyVoteExtension` at height *h* do
not modify *sp,h-1*.
-* Requirement 10 [`ExtendVote`, `FinalizeBlock`, non-dependency]: for any correct process *p*,
+- Requirement 10 [`ExtendVote`, `FinalizeBlock`, non-dependency]: for any correct process *p*,
and any vote extension *e* that *p* received at height *h*, the computation of
*sp,h* does not depend on *e*.
@@ -191,10 +191,10 @@ The call to correct process *p*'s `RequestFinalizeBlock` at height *h*, with blo
passed as parameter, creates state *sp,h*.
Additionally, *p*'s `FinalizeBlock` creates a set of transaction results *Tp,h*.
-* Requirement 11 [`FinalizeBlock`, determinism-1]: For any correct process *p*,
+- Requirement 11 [`FinalizeBlock`, determinism-1]: For any correct process *p*,
*sp,h* exclusively depends on *sp,h-1* and *vp,h*.
-* Requirement 12 [`FinalizeBlock`, determinism-2]: For any correct process *p*,
+- Requirement 12 [`FinalizeBlock`, determinism-2]: For any correct process *p*,
the contents of *Tp,h* exclusively depend on *sp,h-1* and *vp,h*.
Note that Requirements 11 and 12, combined with the Agreement property of consensus ensure
@@ -204,14 +204,14 @@ Also, notice that neither `PrepareProposal` nor `ExtendVote` have determinism-re
requirements associated.
Indeed, `PrepareProposal` is not required to be deterministic:
-* *up* may depend on *vp* and *sp,h-1*, but may also depend on other values or operations.
-* *vp = vq ⇏ up = uq*.
+- *up* may depend on *vp* and *sp,h-1*, but may also depend on other values or operations.
+- *vp = vq ⇏ up = uq*.
Likewise, `ExtendVote` can also be non-deterministic:
-* *erp* may depend on *wrp* and *sp,h-1*,
+- *erp* may depend on *wrp* and *sp,h-1*,
but may also depend on other values or operations.
-* *wrp = wrq ⇏
+- *wrp = wrq ⇏
erp = erq*
### Mempool Connection Requirements
@@ -228,7 +228,7 @@ we define *CheckTxCodetx,p,h* as the singleton value of *CheckTxCodes
If *CheckTxCodestx,p,h* is not a singleton set, *CheckTxCodetx,p,h* is undefined.
Let predicate *OK(CheckTxCodetx,p,h)* denote whether *CheckTxCodetx,p,h* is `SUCCESS`.
-* Requirement 13 [`CheckTx`, eventual non-oscillation]: For any transaction *tx*,
+- Requirement 13 [`CheckTx`, eventual non-oscillation]: For any transaction *tx*,
there exists a boolean value *b*,
and a height *hstable* such that,
for any correct process *p*,
@@ -327,22 +327,22 @@ Likewise, CometBFT calls `ProcessProposal` upon reception of a proposed block fr
network. The proposed block's data
that is disclosed to the Application by these two methods is the following:
-* the transaction list
-* the `LastCommit` referring to the previous block
-* the block header's hash (except in `PrepareProposal`, where it is not known yet)
-* list of validators that misbehaved
-* the block's timestamp
-* `NextValidatorsHash`
-* Proposer address
+- the transaction list
+- the `LastCommit` referring to the previous block
+- the block header's hash (except in `PrepareProposal`, where it is not known yet)
+- list of validators that misbehaved
+- the block's timestamp
+- `NextValidatorsHash`
+- Proposer address
The Application may decide to *immediately* execute the given block (i.e., upon `PrepareProposal`
or `ProcessProposal`). There are two main reasons why the Application may want to do this:
-* *Avoiding invalid transactions in blocks*.
+- *Avoiding invalid transactions in blocks*.
In order to be sure that the block does not contain *any* invalid transaction, there may be
no way other than fully executing the transactions in the block as though it was the *decided*
block.
-* *Quick `FinalizeBlock` execution*.
+- *Quick `FinalizeBlock` execution*.
Upon reception of the decided block via `FinalizeBlock`, if that same block was executed
upon `PrepareProposal` or `ProcessProposal` and the resulting state was kept in memory, the
Application can simply apply that state (faster) to the main state, rather than reexecuting
@@ -481,8 +481,8 @@ or validation should fail before it can use more resources than it requested.
When `MaxGas > -1`, CometBFT enforces the following rules:
-* `GasWanted <= MaxGas` for every transaction in the mempool
-* `(sum of GasWanted in a block) <= MaxGas` when proposing a block
+- `GasWanted <= MaxGas` for every transaction in the mempool
+- `(sum of GasWanted in a block) <= MaxGas` when proposing a block
If `MaxGas == -1`, no rules about gas are enforced.
@@ -500,7 +500,7 @@ it can use `PrepareProposal` and `ProcessProposal` to enforce that `(sum of GasW
in all proposed or prevoted blocks,
we have:
-* `(sum of GasUsed in a block) <= MaxGas` for every block
+- `(sum of GasUsed in a block) <= MaxGas` for every block
The `GasUsed` field is ignored by CometBFT.
@@ -561,21 +561,21 @@ duplicates, the block execution will fail irrecoverably.
Structure `ValidatorUpdate` contains a public key, which is used to identify the validator:
The public key currently supports three types:
-* `ed25519`
-* `secp256k1`
-* `sr25519`
+- `ed25519`
+- `secp256k1`
+- `sr25519`
Structure `ValidatorUpdate` also contains an `ìnt64` field denoting the validator's new power.
Applications must ensure that
`ValidatorUpdate` structures abide by the following rules:
-* power must be non-negative
-* if power is set to 0, the validator must be in the validator set; it will be removed from the set
-* if power is greater than 0:
- * if the validator is not in the validator set, it will be added to the
+- power must be non-negative
+- if power is set to 0, the validator must be in the validator set; it will be removed from the set
+- if power is greater than 0:
+ - if the validator is not in the validator set, it will be added to the
set with the given power
- * if the validator is in the validator set, its power will be adjusted to the given power
-* the total power of the new validator set must not exceed `MaxTotalVotingPower`, where
+ - if the validator is in the validator set, its power will be adjusted to the given power
+- the total power of the new validator set must not exceed `MaxTotalVotingPower`, where
`MaxTotalVotingPower = MaxInt64 / 8`
Note the updates returned after processing the block at height `H` will only take effect
@@ -622,7 +622,7 @@ This is enforced by the consensus algorithm.
This implies a maximum transaction size that is `MaxBytes`, less the expected size of
the header, the validator set, and any included evidence in the block.
-The Application should be aware that honest validators _may_ produce and
+The Application should be aware that honest validators *may* produce and
broadcast blocks with up to the configured `MaxBytes` size.
As a result, the consensus
[timeout parameters](../../docs/core/configuration.md#consensus-timeouts-explained)
@@ -790,8 +790,8 @@ include vote extensions yet, but `ExtendVote` and `VerifyVoteExtension` will
be called. Then, when reaching height `H+1`, `PrepareProposal` will
include the vote extensions from height `H`. For all heights after `H`
-* vote extensions cannot be disabled,
-* they are mandatory: all precommit messages sent MUST have an extension
+- vote extensions cannot be disabled,
+- they are mandatory: all precommit messages sent MUST have an extension
attached. Nevertheless, the application MAY provide 0-length
extensions.
@@ -863,9 +863,9 @@ For such applications, the `AppHash` provides a much more efficient way to verif
ABCI applications can take advantage of more efficient light-client proofs for
their state as follows:
-* return the Merkle root of the deterministic application state in
+- return the Merkle root of the deterministic application state in
`ResponseFinalizeBlock.Data`. This Merkle root will be included as the `AppHash` in the next block.
-* return efficient Merkle proofs about that application state in `ResponseQuery.Proof`
+- return efficient Merkle proofs about that application state in `ResponseQuery.Proof`
that can be verified using the `AppHash` of the corresponding block.
For instance, this allows an application's light-client to verify proofs of
@@ -900,9 +900,9 @@ the list should match the `AppHash` being verified against.
When CometBFT connects to a peer, it sends two queries to the ABCI application
using the following paths, with no additional data:
-* `/p2p/filter/addr/`, where `` denote the IP address and
+- `/p2p/filter/addr/`, where `` denote the IP address and
the port of the connection
-* `p2p/filter/id/`, where `` is the peer node ID (ie. the
+- `p2p/filter/id/`, where `` is the peer node ID (ie. the
pubkey.Address() for the peer's PubKey)
If either of these queries return a non-zero ABCI code, CometBFT will refuse
@@ -932,6 +932,7 @@ committed state of the app. The app MUST return information consistent with the
last block for which it successfully completed `Commit`.
The three steps performed before the state of a height is considered persisted are:
+
- The block is stored by CometBFT in the blockstore
- CometBFT has stored the state returned by the application through `FinalizeBlockResponse`
- The application has committed its state within `Commit`.
@@ -965,6 +966,7 @@ and the operations for this height are repeated entirely):
Based on the sequence of these events, CometBFT will panic if any of the steps in the sequence happen out of order,
that is if:
+
- The application has persisted a block at a height higher than the blocked saved during `state_stored`.
- The `block_stored` step persisted a block at a height smaller than the `state_stored`
- And the difference between the heights of the blocks persisted by `state_stored` and `block_stored` is more
@@ -999,20 +1001,20 @@ Applications that want to support state syncing must take state snapshots at reg
this is accomplished is entirely up to the application. A snapshot consists of some metadata and
a set of binary chunks in an arbitrary format:
-* `Height (uint64)`: The height at which the snapshot is taken. It must be taken after the given
+- `Height (uint64)`: The height at which the snapshot is taken. It must be taken after the given
height has been committed, and must not contain data from any later heights.
-* `Format (uint32)`: An arbitrary snapshot format identifier. This can be used to version snapshot
+- `Format (uint32)`: An arbitrary snapshot format identifier. This can be used to version snapshot
formats, e.g. to switch from Protobuf to MessagePack for serialization. The application can use
this when restoring to choose whether to accept or reject a snapshot.
-* `Chunks (uint32)`: The number of chunks in the snapshot. Each chunk contains arbitrary binary
+- `Chunks (uint32)`: The number of chunks in the snapshot. Each chunk contains arbitrary binary
data, and should be less than 16 MB; 10 MB is a good starting point.
-* `Hash ([]byte)`: An arbitrary hash of the snapshot. This is used to check whether a snapshot is
+- `Hash ([]byte)`: An arbitrary hash of the snapshot. This is used to check whether a snapshot is
the same across nodes when downloading chunks.
-* `Metadata ([]byte)`: Arbitrary snapshot metadata, e.g. chunk hashes for verification or any other
+- `Metadata ([]byte)`: Arbitrary snapshot metadata, e.g. chunk hashes for verification or any other
necessary info.
For a snapshot to be considered the same across nodes, all of these fields must be identical. When
@@ -1023,14 +1025,14 @@ application via the ABCI `ListSnapshots` method to discover available snapshots,
snapshot chunks via `LoadSnapshotChunk`. The application is free to choose how to implement this
and which formats to use, but must provide the following guarantees:
-* **Consistent:** A snapshot must be taken at a single isolated height, unaffected by
+- **Consistent:** A snapshot must be taken at a single isolated height, unaffected by
concurrent writes. This can be accomplished by using a data store that supports ACID
transactions with snapshot isolation.
-* **Asynchronous:** Taking a snapshot can be time-consuming, so it must not halt chain progress,
+- **Asynchronous:** Taking a snapshot can be time-consuming, so it must not halt chain progress,
for example by running in a separate thread.
-* **Deterministic:** A snapshot taken at the same height in the same format must be identical
+- **Deterministic:** A snapshot taken at the same height in the same format must be identical
(at the byte level) across nodes, including all metadata. This ensures good availability of
chunks, and that they fit together across nodes.
@@ -1115,9 +1117,10 @@ Once the snapshots have all been restored, CometBFT gathers additional informati
bootstrapping the node (e.g. chain ID, consensus parameters, validator sets, and block headers)
from the genesis file and light client RPC servers. It also calls `Info` to verify the following:
-* that the app hash from the snapshot it has delivered to the Application matches the apphash
+- that the app hash from the snapshot it has delivered to the Application matches the apphash
stored in the next height's block
-* that the version that the Application returns in `ResponseInfo` matches the version in the
+
+- that the version that the Application returns in `ResponseInfo` matches the version in the
current height's block header
Once the state machine has been restored and CometBFT has gathered this additional
@@ -1141,8 +1144,9 @@ There is a newly introduced [**consensus parameter**](./abci%2B%2B_app_requireme
This parameter represents the height at which vote extensions are
required for consensus to proceed, with 0 being the default value (no vote extensions).
A chain can enable vote extensions either:
-* at genesis by setting `VoteExtensionsEnableHeight` to be equal, e.g., to the `InitialHeight`
-* or via the application logic by changing the `ConsensusParam` to configure the
+
+- at genesis by setting `VoteExtensionsEnableHeight` to be equal, e.g., to the `InitialHeight`
+- or via the application logic by changing the `ConsensusParam` to configure the
`VoteExtensionsEnableHeight`.
Once the (coordinated) upgrade to ABCI 2.0 has taken place, at height *hu*,
diff --git a/spec/abci/abci++_basic_concepts.md b/spec/abci/abci++_basic_concepts.md
index bb71d1b3690..a5db36d9b20 100644
--- a/spec/abci/abci++_basic_concepts.md
+++ b/spec/abci/abci++_basic_concepts.md
@@ -6,21 +6,21 @@ title: Overview and basic concepts
## Outline
- [Overview and basic concepts](#overview-and-basic-concepts)
- - [ABCI++ vs. ABCI](#abci-vs-abci)
- - [Methods overview](#methods-overview)
- - [Consensus/block execution methods](#consensusblock-execution-methods)
- - [Mempool methods](#mempool-methods)
- - [Info methods](#info-methods)
- - [State-sync methods](#state-sync-methods)
- - [Other methods](#other-methods)
- - [Proposal timeout](#proposal-timeout)
- - [Deterministic State-Machine Replication](#deterministic-state-machine-replication)
- - [Events](#events)
- - [Evidence](#evidence)
- - [Errors](#errors)
- - [`CheckTx`](#checktx)
- - [`ExecTxResult` (as part of `FinalizeBlock`)](#exectxresult-as-part-of-finalizeblock)
- - [`Query`](#query)
+ - [ABCI++ vs. ABCI](#abci-vs-abci)
+ - [Methods overview](#methods-overview)
+ - [Consensus/block execution methods](#consensusblock-execution-methods)
+ - [Mempool methods](#mempool-methods)
+ - [Info methods](#info-methods)
+ - [State-sync methods](#state-sync-methods)
+ - [Other methods](#other-methods)
+ - [Proposal timeout](#proposal-timeout)
+ - [Deterministic State-Machine Replication](#deterministic-state-machine-replication)
+ - [Events](#events)
+ - [Evidence](#evidence)
+ - [Errors](#errors)
+ - [`CheckTx`](#checktx)
+ - [`ExecTxResult` (as part of `FinalizeBlock`)](#exectxresult-as-part-of-finalizeblock)
+ - [`Query`](#query)
# Overview and basic concepts
@@ -74,21 +74,19 @@ call sequences of these methods.
proposer to perform application-dependent work in a block before proposing it.
This enables, for instance, batch optimizations to a block, which has been empirically
demonstrated to be a key component for improved performance. Method `PrepareProposal` is called
- every time CometBFT is about to broadcast a Proposal message and _validValue_ is `nil`.
+ every time CometBFT is about to broadcast a Proposal message and *validValue* is `nil`.
CometBFT gathers outstanding transactions from the
mempool, generates a block header, and uses them to create a block to propose. Then, it calls
`RequestPrepareProposal` with the newly created proposal, called *raw proposal*. The Application
- can make changes to the raw proposal, such as modifying the set of transactions or the order
- in which they appear, and returns the
- (potentially) modified proposal, called *prepared proposal* in the `ResponsePrepareProposal`
- call.
+ can make changes to the raw proposal, such as reordering, adding and removing transactions, before returning the
+ (potentially) modified proposal, called *prepared proposal* in the `ResponsePrepareProposal`.
The logic modifying the raw proposal MAY be non-deterministic.
- [**ProcessProposal:**](./abci++_methods.md#processproposal) It allows a validator to
perform application-dependent work in a proposed block. This enables features such as immediate
block execution, and allows the Application to reject invalid blocks.
- CometBFT calls it when it receives a proposal and _validValue_ is `nil`.
+ CometBFT calls it when it receives a proposal and *validValue* is `nil`.
The Application cannot modify the proposal at this point but can reject it if
invalid. If that is the case, the consensus algorithm will prevote `nil` on the proposal, which has
strong liveness implications for CometBFT. As a general rule, the Application
@@ -114,10 +112,10 @@ call sequences of these methods.
This has a negative impact on liveness, i.e., if vote extensions repeatedly cannot be
verified by correct validators, the consensus algorithm may not be able to finalize a block even if sufficiently
many (+2/3) validators send precommit votes for that block. Thus, `VerifyVoteExtension`
- should be used with special care.
+ should be implemented with special care.
As a general rule, an Application that detects an invalid vote extension SHOULD
accept it in `ResponseVerifyVoteExtension` and ignore it in its own logic. CometBFT calls it when
- a process receives a precommit message with a (possibly empty) vote extension.
+ a process receives a precommit message with a (possibly empty) vote extension, for the current height. It is not called for precommit votes received after the height is concluded but while waiting to accumulate more precommit votes.
The logic in `VerifyVoteExtension` MUST be deterministic.
- [**FinalizeBlock:**](./abci++_methods.md#finalizeblock) It delivers a decided block to the
@@ -251,7 +249,8 @@ time during `FinalizeBlock`; they must only apply state changes in `Commit`.
Additionally, vote extensions or the validation thereof (via `ExtendVote` or
`VerifyVoteExtension`) must *never* have side effects on the current state.
-They can only be used when their data is provided in a `RequestPrepareProposal` call.
+They can only be used when their data is provided in a `RequestPrepareProposal` call but, again,
+without side effects to the app state.
If there is some non-determinism in the state machine, consensus will eventually
fail as nodes disagree over the correct values for the block header. The
diff --git a/spec/abci/abci++_comet_expected_behavior.md b/spec/abci/abci++_comet_expected_behavior.md
index 55ce4d397c5..d9269ca5380 100644
--- a/spec/abci/abci++_comet_expected_behavior.md
+++ b/spec/abci/abci++_comet_expected_behavior.md
@@ -260,15 +260,17 @@ historical commits and potential optimizations, are discussed in detail in [RFC-
## Handling upgrades to ABCI 2.0
If applications upgrade to ABCI 2.0, CometBFT internally ensures that the [application setup](./abci%2B%2B_app_requirements.md#application-configuration-required-to-switch-to-abci-20) is reflected in its operation.
-CometBFT retrieves from the application configuration the value of `VoteExtensionsEnableHeight`( *he*,),
+CometBFT retrieves from the application configuration the value of `VoteExtensionsEnableHeight`( _he_,),
the height at which vote extensions are required for consensus to proceed, and uses it to determine the data it stores and data it sends to a peer that is catching up.
-Namely, upon saving the block for a given height *h* in the block store at decision time
-* if *h ≥ he*, the corresponding extended commit that was used to decide locally is saved as well
-* if *h < he*, there are no changes to the data saved
+Namely, upon saving the block for a given height _h_ in the block store at decision time
-In the catch-up mechanism, when a node *f* realizes that another peer is at height *hp*, which is more than 2 heights behind,
-* if *hp ≥ he*, *f* uses the extended commit to
+* if _h ≥ he_, the corresponding extended commit that was used to decide locally is saved as well
+* if _h < he_, there are no changes to the data saved
+
+In the catch-up mechanism, when a node _f_ realizes that another peer is at height _hp_, which is more than 2 heights behind height _hf_,
+
+* if _hp ≥ he_, _f_ uses the extended commit to
reconstruct the precommit votes with their corresponding extensions
-* if *hp < he*, *f* uses the canonical commit to reconstruct the precommit votes,
+* if _hp < he_, _f_ uses the canonical commit to reconstruct the precommit votes,
as done for ABCI 1.0 and earlier.
diff --git a/spec/abci/abci++_example_scenarios.md b/spec/abci/abci++_example_scenarios.md
index 1a7de60b91d..2af4665ceb1 100644
--- a/spec/abci/abci++_example_scenarios.md
+++ b/spec/abci/abci++_example_scenarios.md
@@ -44,11 +44,12 @@ application needs to account for any number of rounds, where each round can exhi
behaviours. Recall that the application is unaware of the internals of consensus and thus of the rounds.
# Possible scenarios
-The unknown number of rounds we can have when following the consensus algorithm yields a vast number of
-scenarios we can expect. Listing them all is unfeasible. However, here we give several of them and draw the
+
+The unknown number of rounds we can have when following the consensus algorithm yields a vast number of
+scenarios we can expect. Listing them all is unfeasible. However, here we give several of them and draw the
main conclusions. Specifically, we will show that before block $X$ is decided:
-
-1. On a correct node, `PrepareProposal` may be called multiple times and for different blocks ([**Scenario 1**](#scenario-1)).
+
+1. On a correct node, `PrepareProposal` may be called multiple times and for different blocks ([**Scenario 1**](#scenario-1)).
1. On a correct node, `ProcessProposal` may be called multiple times and for different blocks ([**Scenario 2**](#scenario-2)).
1. On a correct node, `PrepareProposal` and `ProcessProposal` for block $X$ may not be called ([**Scenario 3**](#scenario-3)).
1. On a correct node, `PrepareProposal` and `ProcessProposal` may not be called at all ([**Scenario 4**](#scenario-4)).
@@ -56,12 +57,12 @@ main conclusions. Specifically, we will show that before block $X$ is decided:
## Basic information
-Each scenario is presented from the perspective of a process $p$. More precisely, we show what happens in
-each round's $step$ of the [Tendermint consensus algorithm](https://arxiv.org/pdf/1807.04938.pdf). While in
-practice the consensus algorithm works with respect to voting power of the validators, in this document
-we refer to number of processes (e.g., $n$, $f+1$, $2f+1$) for simplicity. The legend is below:
+Each scenario is presented from the perspective of a process $p$. More precisely, we show what happens in
+each round's $step$ of the [Tendermint consensus algorithm](https://arxiv.org/pdf/1807.04938.pdf). While in
+practice the consensus algorithm works with respect to voting power of the validators, in this document
+we refer to number of processes (e.g., $n$, $f+1$, $2f+1$) for simplicity. The legend is below:
-### Round X:
+### Round X
1. **Propose:** Describes what happens while $step_p = propose$.
1. **Prevote:** Describes what happens while $step_p = prevote$.
@@ -71,60 +72,60 @@ we refer to number of processes (e.g., $n$, $f+1$, $2f+1$) for simplicity. The l
$p$ calls `ProcessProposal` many times with different values.
-### Round 0:
-
-1. **Propose:** The proposer of this round is a Byzantine process, and it chooses not to send the proposal
-message. Therefore, $p$'s $timeoutPropose$ expires, it sends $Prevote$ for $nil$, and it does not call
-`ProcessProposal`. All correct processes do the same.
-1. **Prevote:** $p$ eventually receives $2f+1$ $Prevote$ messages for $nil$ and starts $timeoutPrevote$.
-When $timeoutPrevote$ expires it sends $Precommit$ for $nil$.
-1. **Precommit:** $p$ eventually receives $2f+1$ $Precommit$ messages for $nil$ and starts $timeoutPrecommit$.
-When it expires, it moves to the next round.
-
-### Round 1:
-
-1. **Propose:** A correct process is the proposer in this round. Its $validValue$ is $nil$, and it is free
-to generate and propose a new block $Y$. Process $p$ receives this proposal in time, calls `ProcessProposal`
-for block $Y$, and broadcasts a $Prevote$ message for it.
-1. **Prevote:** Due to network asynchrony less than $2f+1$ processes send $Prevote$ for this block.
-Therefore, $p$ does not update $validValue$ in this round.
-1. **Precommit:** Since less than $2f+1$ processes send $Prevote$, no correct process will lock on this
-block and send $Precommit$ message. As a consequence, $p$ does not decide on $Y$.
-
-### Round 2:
-
-1. **Propose:** Same as in [**Round 1**](#round-1), just another correct process is the proposer, and it
-proposes another value $Z$. Process $p$ receives the proposal on time, calls `ProcessProposal` for new block
-$Z$, and broadcasts a $Prevote$ message for it.
+### Round 0
+
+1. **Propose:** The proposer of this round is a Byzantine process, and it chooses not to send the proposal
+message. Therefore, $p$'s $timeoutPropose$ expires, it sends $Prevote$ for $nil$, and it does not call
+`ProcessProposal`. All correct processes do the same.
+1. **Prevote:** $p$ eventually receives $2f+1$ $Prevote$ messages for $nil$ and starts $timeoutPrevote$.
+When $timeoutPrevote$ expires it sends $Precommit$ for $nil$.
+1. **Precommit:** $p$ eventually receives $2f+1$ $Precommit$ messages for $nil$ and starts $timeoutPrecommit$.
+When it expires, it moves to the next round.
+
+### Round 1
+
+1. **Propose:** A correct process is the proposer in this round. Its $validValue$ is $nil$, and it is free
+to generate and propose a new block $Y$. Process $p$ receives this proposal in time, calls `ProcessProposal`
+for block $Y$, and broadcasts a $Prevote$ message for it.
+1. **Prevote:** Due to network asynchrony less than $2f+1$ processes send $Prevote$ for this block.
+Therefore, $p$ does not update $validValue$ in this round.
+1. **Precommit:** Since less than $2f+1$ processes send $Prevote$, no correct process will lock on this
+block and send $Precommit$ message. As a consequence, $p$ does not decide on $Y$.
+
+### Round 2
+
+1. **Propose:** Same as in [**Round 1**](#round-1), just another correct process is the proposer, and it
+proposes another value $Z$. Process $p$ receives the proposal on time, calls `ProcessProposal` for new block
+$Z$, and broadcasts a $Prevote$ message for it.
1. **Prevote:** Same as in [**Round 1**](#round-1).
1. **Precommit:** Same as in [**Round 1**](#round-1).
-Rounds like these can continue until we have a round in which process $p$ updates its $validValue$ or until
-we reach round $r$ where process $p$ decides on a block. After that, it will not call `ProcessProposal`
-anymore for this height.
+Rounds like these can continue until we have a round in which process $p$ updates its $validValue$ or until
+we reach round $r$ where process $p$ decides on a block. After that, it will not call `ProcessProposal`
+anymore for this height.
-## Scenario 2
+## Scenario 2
$p$ calls `PrepareProposal` many times with different values.
-### Round 0:
+### Round 0
-1. **Propose:** Process $p$ is the proposer in this round. Its $validValue$ is $nil$, and it is free to
-generate and propose new block $Y$. Before proposing, it calls `PrepareProposal` for $Y$. After that, it
-broadcasts the proposal, delivers it to itself, calls `ProcessProposal` and broadcasts $Prevote$ for it.
-1. **Prevote:** Due to network asynchrony less than $2f+1$ processes receive the proposal on time and send
-$Prevote$ for it. Therefore, $p$ does not update $validValue$ in this round.
-1. **Precommit:** Since less than $2f+1$ processes send $Prevote$, no correct process will lock on this
-block and send non-$nil$ $Precommit$ message. As a consequence, $p$ does not decide on $Y$.
+1. **Propose:** Process $p$ is the proposer in this round. Its $validValue$ is $nil$, and it is free to
+generate and propose new block $Y$. Before proposing, it calls `PrepareProposal` for $Y$. After that, it
+broadcasts the proposal, delivers it to itself, calls `ProcessProposal` and broadcasts $Prevote$ for it.
+1. **Prevote:** Due to network asynchrony less than $2f+1$ processes receive the proposal on time and send
+$Prevote$ for it. Therefore, $p$ does not update $validValue$ in this round.
+1. **Precommit:** Since less than $2f+1$ processes send $Prevote$, no correct process will lock on this
+block and send non-$nil$ $Precommit$ message. As a consequence, $p$ does not decide on $Y$.
-After this round, we can have multiple rounds like those in [Scenario 1](#scenario-1). The important thing
-is that process $p$ should not update its $validValue$. Consequently, when process $p$ reaches the round
-when it is again the proposer, it will ask the mempool for the new block again, and the mempool may return a
-different block $Z$, and we can have the same round as [Round 0](#round-0-1) just for a different block. As
-a result, process $p$ calls `PrepareProposal` again but for a different value. When it reaches round $r$
-some process will propose block $X$ and if $p$ receives $2f+1$ $Precommit$ messages, it will decide on this
-value.
+After this round, we can have multiple rounds like those in [Scenario 1](#scenario-1). The important thing
+is that process $p$ should not update its $validValue$. Consequently, when process $p$ reaches the round
+when it is again the proposer, it will ask the mempool for the new block again, and the mempool may return a
+different block $Z$, and we can have the same round as [Round 0](#round-0-1) just for a different block. As
+a result, process $p$ calls `PrepareProposal` again but for a different value. When it reaches round $r$
+some process will propose block $X$ and if $p$ receives $2f+1$ $Precommit$ messages, it will decide on this
+value.
## Scenario 3
@@ -140,24 +141,24 @@ so it did not call `ProcessProposal`, and
* if $p$ was the proposer it proposed some other value $\neq X$.
-### Round $r$:
+### Round $r$
-1. **Propose:** A correct process is the proposer in this round, and it proposes block $X$.
+1. **Propose:** A correct process is the proposer in this round, and it proposes block $X$.
Due to asynchrony, the proposal message arrives to process $p$ after its $timeoutPropose$
expires and it sends $Prevote$ for $nil$. Consequently, process $p$ does not call
`ProcessProposal` for block $X$. However, the same proposal arrives at other processes
before their $timeoutPropose$ expires, and they send $Prevote$ for this proposal.
-1. **Prevote:** Process $p$ receives $2f+1$ $Prevote$ messages for proposal $X$, updates correspondingly its
-$validValue$ and $lockedValue$ and sends $Precommit$ message. All correct processes do the same.
-1. **Precommit:** Finally, process $p$ receives $2f+1$ $Precommit$ messages, and decides on block $X$.
+1. **Prevote:** Process $p$ receives $2f+1$ $Prevote$ messages for proposal $X$, updates correspondingly its
+$validValue$ and $lockedValue$ and sends $Precommit$ message. All correct processes do the same.
+1. **Precommit:** Finally, process $p$ receives $2f+1$ $Precommit$ messages, and decides on block $X$.
## Scenario 4
-[Scenario 3](#scenario-3) can be translated into a scenario where $p$ does not call `PrepareProposal` and
-`ProcessProposal` at all. For this, it is necessary that process $p$ is not the proposer in any of the
-rounds $0 <= r' <= r$ and that due to network asynchrony or Byzantine proposer, it does not receive the
-proposal before $timeoutPropose$ expires. As a result, it will enter round $r$ without calling
-`PrepareProposal` and `ProcessProposal` before it, and as shown in Round $r$ of [Scenario 3](#scenario-3) it
+[Scenario 3](#scenario-3) can be translated into a scenario where $p$ does not call `PrepareProposal` and
+`ProcessProposal` at all. For this, it is necessary that process $p$ is not the proposer in any of the
+rounds $0 <= r' <= r$ and that due to network asynchrony or Byzantine proposer, it does not receive the
+proposal before $timeoutPropose$ expires. As a result, it will enter round $r$ without calling
+`PrepareProposal` and `ProcessProposal` before it, and as shown in Round $r$ of [Scenario 3](#scenario-3) it
will decide in this round. Again without calling any of these two calls.
diff --git a/spec/abci/abci++_methods.md b/spec/abci/abci++_methods.md
index 5783970ec82..56554af2e05 100644
--- a/spec/abci/abci++_methods.md
+++ b/spec/abci/abci++_methods.md
@@ -330,8 +330,9 @@ title: Methods
* `RequestPrepareProposal`'s parameters `txs`, `misbehavior`, `height`, `time`,
`next_validators_hash`, and `proposer_address` are the same as in `RequestProcessProposal`
and `RequestFinalizeBlock`.
- * `RequestPrepareProposal.local_last_commit` is a set of the precommit votes that allowed the
- decision of the previous block, together with their corresponding vote extensions.
+ * `RequestPrepareProposal.local_last_commit` is a set of the precommit votes for the previous
+ height, including the ones that led to the decision of the previous block,
+ together with their corresponding vote extensions.
* The `height`, `time`, and `proposer_address` values match the values from the header of the
proposed block.
* `RequestPrepareProposal` contains a preliminary set of transactions `txs` that CometBFT
@@ -378,7 +379,7 @@ title: Methods
-->
* If CometBFT fails to validate the `ResponsePrepareProposal`, CometBFT will assume the
Application is faulty and crash.
- * The implementation of `PrepareProposal` can be non-deterministic.
+ * The implementation of `PrepareProposal` MAY be non-deterministic.
#### When does CometBFT call "PrepareProposal" ?
@@ -404,6 +405,9 @@ and _p_'s _validValue_ is `nil`:
* modify transactions (e.g. aggregate them). As explained above, this compromises client traceability, unless
it is implemented at the Application level.
* reorder transactions - the Application reorders transactions in the list
+ * the Application MAY use the vote extensions in the commit info to modify the proposal, in which case it is suggested
+ that extensions be validated in the same maner as done in `VerifyVoteExtension`, since extensions of votes included
+ in the commit info after the minimum of +2/3 had been reached are not verified.
4. The Application includes the transaction list (whether modified or not) in the return parameters
(see the rules in section _Usage_), and returns from the call.
5. _p_ uses the (possibly) modified block as _p_'s proposal in round _r_, height _h_.
@@ -449,7 +453,7 @@ the consensus algorithm will use it as proposal and will not call `RequestPrepar
* The height and time values match the values from the header of the proposed block.
* If `ResponseProcessProposal.status` is `REJECT`, consensus assumes the proposal received
is not valid.
- * The Application MAY fully execute the block — immediate execution
+ * The Application MAY fully execute the block (immediate execution)
* The implementation of `ProcessProposal` MUST be deterministic. Moreover, the value of
`ResponseProcessProposal.status` MUST **exclusively** depend on the parameters passed in
the call to `RequestProcessProposal`, and the last committed Application state
@@ -596,6 +600,12 @@ message for round _r_, height _h_ from validator _q_ (_q_ ≠ _p_):
structure in calls to `RequestPrepareProposal`, in rounds of height _h + 1_ where _p_ is the proposer.
* `REJECT`, _p_ will deem the Precommit message invalid and discard it.
+When a node _p_ is in consensus round _0_, height _h_, and _p_ receives a Precommit
+message for CommitRound _r_, height _h-1_ from validator _q_ (_q_ ≠ _p_), _p_
+MAY add the Precommit message and associated extension to [ExtendedCommitInfo](#extendedcommitinfo)
+without calling `RequestVerifyVoteExtension` to verify it.
+
+
### FinalizeBlock
#### Parameters and Types