Add Ostracon specification skeleton (Finschia#567)

* Define spec * Update spec/core/data_structures.md Co-authored-by: Toshimasa Nasu <[email protected]> * Update spec/p2p/v0.34/configuration.md Co-authored-by: Toshimasa Nasu <[email protected]> * Update README.md --------- Co-authored-by: Toshimasa Nasu <[email protected]>
tnasu · Mar 10, 2023 · cc6bd13 · cc6bd13
1 parent b77283b
commit cc6bd13
Show file tree

Hide file tree

Showing 11 changed files with 559 additions and 0 deletions.
diff --git a/spec/README.md b/spec/README.md
@@ -0,0 +1,78 @@
+# Ostracon Spec
+
+This document is a specification of the Ostracon. Ostracon is designed to have many compatibilities with CometBFT. As such, the Ostracon spec is based on the [CometBFT v0.34.x Spec](https://github.com/cometbft/cometbft/tree/v0.34.x/spec). The Ostracon also has many unique improvements to improve security and performance. This specification clarifies the differences between CometBFT and defines the base data structures, how they are validated, and how they are communicated over the network.
+
+## Contents
+
+> **Note**
+> ![oc](./static/oc.svg) represents items with Ostracon-specific improvements added. It also describes the major changes.
+
+> **Note**
+> ![tm](./static/tm.svg) represents items with specifications equivalent to CometBFT. As such, the article links directly to CometBFT's repository.
+
+### Introduction
+
+- [![oc](./static/oc.svg)Overview](./introduction/overview.md)
+
+### Core
+
+- [![oc](./static/oc.svg)Data Structure](./core/data_structures.md): Ostacon uses a more secure proposer election algorithm. Add definitions for the data structures required for this election algorithm.
+- [![tm](./static/tm.svg)Encoding](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/core/encoding.md)
+- [![tm](./static/tm.svg)Genesis](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/core/genesis.md)
+- [![oc](./static/oc.svg)State](./core/state.md): Ostacon uses a more secure proposer election algorithm. Add the data required for this election algorithm to the state.
+
+### Consensus Protocol
+
+- [![tm](./static/tm.svg)Consensus Algorithm](https://github.com/cometbft/cometbft/blob/v0.34.x/spec//consensus/consensus.md)
+- [![tm](./static/tm.svg)BFT Time](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/consensus/bft-time.md)
+- [![oc](./static/oc.svg)Proposer Selection](./consensus/proposer-selection.md): Ostacon adopts a proposer selection algorithm using VRF. Using VRF makes the election unpredictable and makes the Proposer election more secure.
+- [![tm](./static/tm.svg)Creating a proposal](https://github.com/cometbft/cometbft/blob/v0.34.x/spec//consensus/creating-proposal.md)
+- [![tm](./static/tm.svg)Siging](https://github.com/cometbft/cometbft/blob/v0.34.x/spec//consensus/signing.md)
+- [![tm](./static/tm.svg)Write-Ahead Log](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/consensus/wal.md)
+
+#### Proposer-Based Timestamps
+
+- [![tm](./static/tm.svg)Overview](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/consensus/proposer-based-timestamp/pbts_001_draft.md)
+- [![tm](./static/tm.svg)System Model and Properties](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md)
+- [![tm](./static/tm.svg)Protocol Specification](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/consensus/proposer-based-timestamp/pbts-algorithm_001_draft.md)
+
+### Light-Client
+
+- [![tm](./static/tm.svg)Spec](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/light-client/README.md)
+
+### P2P and Network Protocols
+
+- [![tm](./static/tm.svg)Node](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/node.md)
+- [![tm](./static/tm.svg)Peer](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/peer.md)
+- [![tm](./static/tm.svg)Connection](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/connection.md)
+- [![tm](./static/tm.svg)Config](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/config.md)
+#### Messages Type
+
+- [![tm](./static/tm.svg)Block Sync](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/messages/block-sync.md)
+- [![tm](./static/tm.svg)Mempool](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/messages/mempool.md)
+- [![tm](./static/tm.svg)Evidence](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/messages/evidence.md)
+- [![tm](./static/tm.svg)State Sync](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/messages/state-sync.md)
+- [![tm](./static/tm.svg)Peer Exchange (PEX)](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/messages/pex.md)
+- [![tm](./static/tm.svg)Consensus](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/messages/consensus.md)
+
+#### v0.34
+
+- [![tm](./static/tm.svg)Transport](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/v0.34/transport.md)
+- [![tm](./static/tm.svg)Switch](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/v0.34/switch.md)
+- [![tm](./static/tm.svg)PEX Reactor](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/v0.34/pex.md)
+- [![tm](./static/tm.svg)Peer Exchange protocol](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/v0.34/pex-protocol.md)
+- [![tm](./static/tm.svg)Address Book](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/v0.34/addressbook.md)
+- [![tm](./static/tm.svg)Peer Manager](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/v0.34/peer_manager.md)
+- [![tm](./static/tm.svg)Type](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/v0.34/types.md)
+- [![oc](./static/oc.svg)Configuration](./p2p/v0.34/configuration.md): Ostracon allows each reactor to process messages asynchronously in separate threads. Ostracon adds parameters related to that.
+
+### RPC
+
+- [![oc](./static/oc.svg)Spec](./rpc/README.md): Add a entropy to the Block and BlockByHash API response.
+
+### ABCI
+
+- [![tm](./static/tm.svg)Overview](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/abci/README.md)
+- [![oc](./static/oc.svg)Methods and Types](./abci/abci.md): Ostracon adds new ABCI Methods.
+- [![oc](./static/oc.svg)Applications](./abci/apps.md): Ostracon improves the mempool connection cycle to reduce the time the mempool is locked.
+- [![tm](./static/tm.svg)Client and Server](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/abci/client-server.md)
diff --git a/spec/abci/abci.md b/spec/abci/abci.md
@@ -0,0 +1,74 @@
+# Methods and Types
+
+Please ensure you've first read the spec for [CometBFT Methods and Types](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/abci/abci.md). Here only defines the difference between CometBFT.
+
+## Connections
+
+#### **Mempool** connection
+
+Ostracon handles the `BeginRecheckTx` and `EndRecheckTx` calls in addition to `CheckTx`.
+
+## Messages
+
+### BeginBlock
+
+Ostracon adds a entropy to the [BeginBlock](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/abci/abci.md#BeginBlock) response.
+
+* **Request**:
+
+    | Name                 | Type                                          | Description                                                                                                       | Field Number |
+    |----------------------|-----------------------------------------------|-------------------------------------------------------------------------------------------------------------------|--------------|
+    | hash                 | bytes                                         | The block's hash. This can be derived from the block header.                                                      | 1            |
+    | header               | [Header](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/core/data_structures.md#header)   | The block header.                                                                                                 | 2            |
+    | last_commit_info     | [LastCommitInfo](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/abci/abci.md#lastcommitinfo)             | Info about the last commit, including the round, and the list of validators and which ones signed the last block. | 3            |
+    | byzantine_validators | repeated [Evidence](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/abci/abci.md#evidence)                | List of evidence of validators that acted maliciously.                                                            | 4            |
+    | entropy              | [Entropy](../core/data_structures.md#entropy) | The block's entropy.                                                                                              | 1000         |
+
+* **Response**:
+
+    | Name   | Type                      | Description                         | Field Number |
+    |--------|---------------------------|-------------------------------------|--------------|
+    | events | repeated [Event](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/abci/abci.md#events) | type & Key-Value events for indexing | 1           |
+
+* **Usage**:
+    * Signals the beginning of a new block.
+    * Called prior to any `DeliverTx` method calls.
+    * The header contains the height, timestamp, and more - it exactly matches the
+    CometBFT block header. We may seek to generalize this in the future.
+    * The `LastCommitInfo` and `ByzantineValidators` can be used to determine
+    rewards and punishments for the validators.
+    * The `entropy` can be used to determine the next validators set.
+
+### BeginRecheckTx
+
+* **Request**:
+
+    | Name   | Type                                                                                            | Description                    | Field Number |
+    |--------|-------------------------------------------------------------------------------------------------|--------------------------------|--------------|
+    | header | [Header](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/core/data_structures.md#header) | The block header.              | 1            |
+
+* **Response**:
+
+    | Name       | Type   | Description      | Field Number |
+    |------------|--------|------------------|--------------|
+    | code       | uint32 | Response code.   | 1            |
+
+* **Usage**:
+    * Signals the beginning of re-checking transactions.
+
+### EndRecheckTx
+
+* **Request**:
+
+    | Name   | Type  | Description                    | Field Number |
+    |--------|-------|--------------------------------|--------------|
+    | height | int64 | Height of the executing block. | 1            |
+
+* **Response**:
+
+    | Name       | Type   | Description    | Field Number |
+    |------------|--------|----------------|--------------|
+    | code       | uint32 | Response code. | 1            |
+
+* **Usage**:
+    * Signals the end of re-checking transactions.
diff --git a/spec/abci/apps.md b/spec/abci/apps.md
@@ -0,0 +1,30 @@
+# Applications
+
+Please ensure you've first read the spec for [CometBFT Applications](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/abci/apps.md). Here only defines the difference between CometBFT.
+
+## Connection State
+
+### Commit
+
+Ostracon fixes [CometBFT Commit](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/abci/apps.md#commit).
+
+In CometBFT、`Commit` assumes mempool is locked. It needs this assumption only for connection states sync. However `Commit` usually takes long time (about 500ms~1s), mempool is locked too long. Additionally, connection state sync only needs to be performed between `Commit` and rechecks. `BeginRecheckTx` and `EndRecheckTx` are added to notify the application of the start and end of the recheck so that connection states sync can be performed at the appropriate time.
+
+The PR [#160](https://github.com/line/ostracon/pull/160) contains this change.
+
+### BeginRecheckTx
+
+Before `BeginRecheckTx` is called, Ostracon locks and flushes the mempool so that no new messages will be received on the mempool connection. This provides an opportunity to safely update all four connection states to the latest committed state at once.
+
+After `EndRecheckTx` is called, it unlocks the mempool.
+
+### Mempool Connection
+
+Ostracon fixes [CometBFT Mempool Connection](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/abci/apps.md#mempool-connection).
+
+Ostracon calls `Commit` without locking the mempool connection. After `Commit`, Ostracon locks the mempool.
+
+After `BeginRecheckTx`, CheckTx is run again on all transactions that remain in the node's local mempool after filtering those included in the block.
+
+Finally, after `EndRecheckTx`, Ostracon will unlock
+the mempool connection. New transactions are once again able to be processed through CheckTx.
diff --git a/spec/consensus/proposer-selection.md b/spec/consensus/proposer-selection.md
@@ -0,0 +1,3 @@
+# Proposer Selection Procedure
+
+Ostacon adopts a proposer selection algorithm using VRF. Using VRF makes the election unpredictable and makes the Proposer election more secure. Learn more here: [VRF-based proposer election](https://github.com/torao/ostracon/blob/fix/merge_docs_from_lbm/docs/en/02-consensus.md#vrf-based-proposer-election)
diff --git a/spec/core/data_structures.md b/spec/core/data_structures.md
@@ -0,0 +1,56 @@
+# Data Structure
+
+Please ensure you've first read the spec for [CometBFT Data Structure](https://github.com/tendermint/tendermint/blob/v0.34.x/spec/core/data_structures.md). Here only defines the difference between CometBFT.
+
+## Block
+
+Ostracon extends [CometBFT Block](https://github.com/tendermint/tendermint/blob/v0.34.x/spec/core/data_structures.md#block). In Ostracon, Block consists of Entropy in addition to Header, Data, Evidence, and LastCommit.
+
+| Name | Type | Description | Validation |
+|------|------|-------------|------------|
+| ...  |      | Header, Data, Evidence, and LastCommit are the same as CometBFT. |            |
+| Entropy | [Entropy](#entropy) | Entropy represents height-specific complexity. This field contains infomation used proposer-election. | Must adhere to the validation rules of [entropy](#entropy) |
+
+## Execution
+
+`LastProofHash` is added to the state. Ostracon extends [CometBFT Execute](https://github.com/tendermint/tendermint/blob/v0.34.x/spec/core/data_structures.md#execution) as:
+
+```go
+func Execute(s State, app ABCIApp, block Block) State {
+ AppHash, ValidatorChanges, ConsensusParamChanges := app.ApplyBlock(block)
+ nextConsensusParams := UpdateConsensusParams(state.ConsensusParams, ConsensusParamChanges)
+ return State{
+  ChainID:         state.ChainID,
+  InitialHeight:   state.InitialHeight,
+  LastResults:     abciResponses.DeliverTxResults,
+  AppHash:         AppHash,
+  InitialHeight:   state.InitialHeight,
+  LastValidators:  state.Validators,
+  Validators:      state.NextValidators,
+  NextValidators:  UpdateValidators(state.NextValidators, ValidatorChanges),
+  ConsensusParams: nextConsensusParams,
+  Version: {
+   Consensus: {
+    AppVersion: nextConsensusParams.Version.AppVersion,
+   },
+  },
+  LastProofHash: ProofToHash(block.Entropy.Proof),
+ }
+}
+```
+
+Ostracon adds the following steps to the validation of a new block:
+
+- Validate the entropy in the block.
+    - Make sure the `Round` is <= current round.
+    - Make sure the `ProposerAddress` corresponds to the `Round`.
+    - Make sure the `Proof` corresponds to the `Round`.
+
+## Entropy
+
+Ostracon introduces Entropy as a new data structure. This represents height-specific complexity and is used by proposer-election. This consists of a vrf proof and round in which the proposer generates it.
+
+| Name | Type | Description | Validation |
+|------|------|-------------|------------|
+| Round | int32                     | Round in which proposer generate a vrf proof             | Must be >= 0 |
+| Proof | slice of bytes (`[]byte`) | Proof is a vrf proof | Length of proof must be == 0, == 81 (r2ishiguro), or == 80 (libsodium) |
diff --git a/spec/core/state.md b/spec/core/state.md
@@ -0,0 +1,28 @@
+# State
+
+Please ensure you've first read the spec for [CometBFT State](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/core/state.md). Here only defines the difference between CometBFT.
+
+Ostracon adds `LastProofHash` to state. `LastProofHash` is used by proposer selection in the current height.
+
+```go
+type State struct {
+    ChainID        string
+    InitialHeight  int64
+
+    LastBlockHeight int64
+    LastBlockID     types.BlockID
+    LastBlockTime   time.Time
+
+    Version     Version
+    LastResults []Result
+    AppHash     []byte
+
+    LastValidators ValidatorSet
+    Validators     ValidatorSet
+    NextValidators ValidatorSet
+
+    ConsensusParams ConsensusParams
+
+    LastProofHash []byte
+}
+```
diff --git a/spec/introduction/overview.md b/spec/introduction/overview.md
@@ -0,0 +1,18 @@
+# Overview
+
+See below for an overview of Ostracon.
+
+- [What is Ostracon](https://github.com/line/ostracon/blob/main/docs/en/01-overview.md)
+- [Consensus](https://github.com/line/ostracon/blob/main/docs/en/02-consensus.md)
+- [Transaction Sharing](https://github.com/line/ostracon/blob/main/docs/en/03-tx-sharing.md)
+
+## Optimization
+
+Ostracon has the following optimizations to improve performance:
+
+- Fixed each reactor to process messages asynchronously in separate threads.
+    - https://github.com/line/ostracon/issues/128
+    - https://github.com/line/ostracon/pull/135
+- Fixed some ABCI methdos to be executed concurrently.
+    - https://github.com/line/ostracon/pull/160
+    - https://github.com/line/ostracon/pull/163
diff --git a/spec/p2p/v0.34/configuration.md b/spec/p2p/v0.34/configuration.md
@@ -0,0 +1,17 @@
+# Ostracon p2p configuration
+
+Please ensure you've first read the spec for [CometBFT p2p configuration](https://github.com/cometbft/cometbft/blob/v0.34.x/spec/p2p/v0.34/configuration.md). Here only defines the difference between CometBFT.
+
+
+Ostracon allows each reactor to process messages asynchronously in separate threads. Ostracon adds parameters related to async receiving to configurable parameters in addition to parameters defined by CometBFT.
+
+| Parameter| Default| Description |
+| --- | --- | --- |
+| ... |     | The parameters defined by CometBFT |
+|	RecvAsync                   |  true | Set true to enable the async receiving in a reactor                  |
+|	PexRecvBufSize              |  1000 | Size of receive buffer used in async receiving of pex reactor        |
+|	EvidenceRecvBufSize         |  1000 | Size of receive buffer used in async receiving of evidence reactor   |
+|	MempoolRecvBufSize          |  1000 | Size of receive buffer used in async receiving of mempool reactor    |
+|	ConsensusRecvBufSize        |  1000 | Size of receive buffer used in async receiving of consensus reactor  |
+|	BlockchainRecvBufSize       |  1000 | Size of receive buffer used in async receiving of blockchain reactor |
+|	StatesyncRecvBufSize        |  1000 | Size of receive buffer used in async receiving of statesync reactor  |
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,3 @@
		# Proposer Selection Procedure

		Ostacon adopts a proposer selection algorithm using VRF. Using VRF makes the election unpredictable and makes the Proposer election more secure. Learn more here: [VRF-based proposer election](https://github.com/torao/ostracon/blob/fix/merge_docs_from_lbm/docs/en/02-consensus.md#vrf-based-proposer-election)