diff --git a/docs/build/architecture/PROCESS.md b/docs/build/architecture/PROCESS.md index e30a74062..ff8314731 100644 --- a/docs/build/architecture/PROCESS.md +++ b/docs/build/architecture/PROCESS.md @@ -3,7 +3,7 @@ 1. Copy the `adr-template.md` file. Use the following filename pattern: `adr-next_number-title.md` 2. Create a draft Pull Request if you want to get an early feedback. 3. Make sure the context and a solution is clear and well documented. -4. Add an entry to a list in the [README](README.md) file. +4. Add an entry to a list in the [README](./README.md) file. 5. Create a Pull Request to propose a new ADR. ## What is an ADR? diff --git a/docs/build/architecture/README.md b/docs/build/architecture/README.md index ce1ee432d..e5d350ff8 100644 --- a/docs/build/architecture/README.md +++ b/docs/build/architecture/README.md @@ -32,7 +32,7 @@ If recorded decisions turned out to be lacking, convene a discussion, record the ## Creating new ADR -Read about the [PROCESS](PROCESS.md). +Read about the [PROCESS](./PROCESS.md). ### Use RFC 2119 Keywords @@ -42,53 +42,55 @@ When writing ADRs, follow the same best practices for writing RFCs. When writing ### Accepted -* [ADR 002: SDK Documentation Structure](adr-002-docs-structure.md) -* [ADR 004: Split Denomination Keys](adr-004-split-denomination-keys.md) -* [ADR 006: Secret Store Replacement](adr-006-secret-store-replacement.md) -* [ADR 009: Evidence Module](adr-009-evidence-module.md) -* [ADR 010: Modular AnteHandler](adr-010-modular-antehandler.md) -* [ADR 019: Protocol Buffer State Encoding](adr-019-protobuf-state-encoding.md) -* [ADR 020: Protocol Buffer Transaction Encoding](adr-020-protobuf-transaction-encoding.md) -* [ADR 021: Protocol Buffer Query Encoding](adr-021-protobuf-query-encoding.md) -* [ADR 023: Protocol Buffer Naming and Versioning](adr-023-protobuf-naming.md) -* [ADR 029: Fee Grant Module](adr-029-fee-grant-module.md) -* [ADR 030: Message Authorization Module](adr-030-authz-module.md) -* [ADR 031: Protobuf Msg Services](adr-031-msg-service.md) -* [ADR 055: ORM](adr-055-orm.md) -* [ADR 058: Auto-Generated CLI](adr-058-auto-generated-cli.md) +* [ADR 002: SDK Documentation Structure](./adr-002-docs-structure.md) +* [ADR 004: Split Denomination Keys](./adr-004-split-denomination-keys.md) +* [ADR 006: Secret Store Replacement](./adr-006-secret-store-replacement.md) +* [ADR 009: Evidence Module](./adr-009-evidence-module.md) +* [ADR 010: Modular AnteHandler](./adr-010-modular-antehandler.md) +* [ADR 019: Protocol Buffer State Encoding](./adr-019-protobuf-state-encoding.md) +* [ADR 020: Protocol Buffer Transaction Encoding](./adr-020-protobuf-transaction-encoding.md) +* [ADR 021: Protocol Buffer Query Encoding](./adr-021-protobuf-query-encoding.md) +* [ADR 023: Protocol Buffer Naming and Versioning](./adr-023-protobuf-naming.md) +* [ADR 024: Coin Metadata](./adr-024-coin-metadata.md) +* [ADR 029: Fee Grant Module](./adr-029-fee-grant-module.md) +* [ADR 030: Message Authorization Module](./adr-030-authz-module.md) +* [ADR 031: Protobuf Msg Services](./adr-031-msg-service.md) +* [ADR 046: Module Params](./adr-046-module-params.md) +* [ADR 055: ORM](./adr-055-orm.md) +* [ADR 058: Auto-Generated CLI](./adr-058-auto-generated-cli.md) * [ADR 060: ABCI 1.0 (Phase I)](adr-060-abci-1.0.md) -* [ADR 061: Liquid Staking](adr-061-liquid-staking.md) +* [ADR 061: Liquid Staking](./adr-061-liquid-staking.md) ### Proposed -* [ADR 003: Dynamic Capability Store](adr-003-dynamic-capability-store.md) -* [ADR 011: Generalize Genesis Accounts](adr-011-generalize-genesis-accounts.md) -* [ADR 012: State Accessors](adr-012-state-accessors.md) -* [ADR 013: Metrics](adr-013-metrics.md) -* [ADR 016: Validator Consensus Key Rotation](adr-016-validator-consensus-key-rotation.md) -* [ADR 017: Historical Header Module](adr-017-historical-header-module.md) -* [ADR 018: Extendable Voting Periods](adr-018-extendable-voting-period.md) -* [ADR 022: Custom baseapp panic handling](adr-022-custom-panic-handling.md) -* [ADR 024: Coin Metadata](adr-024-coin-metadata.md) -* [ADR 027: Deterministic Protobuf Serialization](adr-027-deterministic-protobuf-serialization.md) -* [ADR 028: Public Key Addresses](adr-028-public-key-addresses.md) -* [ADR 032: Typed Events](adr-032-typed-events.md) -* [ADR 033: Inter-module RPC](adr-033-protobuf-inter-module-comm.md) -* [ADR 035: Rosetta API Support](adr-035-rosetta-api-support.md) -* [ADR 037: Governance Split Votes](adr-037-gov-split-vote.md) -* [ADR 038: State Listening](adr-038-state-listening.md) -* [ADR 039: Epoched Staking](adr-039-epoched-staking.md) -* [ADR 040: Storage and SMT State Commitments](adr-040-storage-and-smt-state-commitments.md) -* [ADR 046: Module Params](adr-046-module-params.md) -* [ADR 054: Semver Compatible SDK Modules](adr-054-semver-compatible-modules.md) -* [ADR 057: App Wiring](adr-057-app-wiring.md) -* [ADR 059: Test Scopes](adr-059-test-scopes.md) -* [ADR 062: Collections State Layer](adr-062-collections-state-layer.md) -* [ADR 063: Core Module API](adr-063-core-module-api.md) -* [ADR 065: Store V2](adr-065-store-v2.md) +* [ADR 003: Dynamic Capability Store](./adr-003-dynamic-capability-store.md) +* [ADR 011: Generalize Genesis Accounts](./adr-011-generalize-genesis-accounts.md) +* [ADR 012: State Accessors](./adr-012-state-accessors.md) +* [ADR 013: Metrics](./adr-013-metrics.md) +* [ADR 016: Validator Consensus Key Rotation](./adr-016-validator-consensus-key-rotation.md) +* [ADR 017: Historical Header Module](./adr-017-historical-header-module.md) +* [ADR 018: Extendable Voting Periods](./adr-018-extendable-voting-period.md) +* [ADR 022: Custom baseapp panic handling](./adr-022-custom-panic-handling.md) +* [ADR 027: Deterministic Protobuf Serialization](./adr-027-deterministic-protobuf-serialization.md) +* [ADR 028: Public Key Addresses](./adr-028-public-key-addresses.md) +* [ADR 032: Typed Events](./adr-032-typed-events.md) +* [ADR 033: Inter-module RPC](./adr-033-protobuf-inter-module-comm.md) +* [ADR 035: Rosetta API Support](./adr-035-rosetta-api-support.md) +* [ADR 037: Governance Split Votes](./adr-037-gov-split-vote.md) +* [ADR 038: State Listening](./adr-038-state-listening.md) +* [ADR 039: Epoched Staking](./adr-039-epoched-staking.md) +* [ADR 040: Storage and SMT State Commitments](./adr-040-storage-and-smt-state-commitments.md) +* [ADR 054: Semver Compatible SDK Modules](./adr-054-semver-compatible-modules.md) +* [ADR 057: App Wiring](./adr-057-app-wiring.md) +* [ADR 059: Test Scopes](./adr-059-test-scopes.md) +* [ADR 062: Collections State Layer](./adr-062-collections-state-layer.md) +* [ADR 063: Core Module API](./adr-063-core-module-api.md) +* [ADR 065: Store v2](./adr-065-store-v2.md) +* [ADR 067: Simulator v2](./adr-067-simulator-v2.md) ### Draft -* [ADR 044: Guidelines for Updating Protobuf Definitions](adr-044-protobuf-updates-guidelines.md) -* [ADR 047: Extend Upgrade Plan](adr-047-extend-upgrade-plan.md) -* [ADR 053: Go Module Refactoring](adr-053-go-module-refactoring.md) +* [ADR 044: Guidelines for Updating Protobuf Definitions](./adr-044-protobuf-updates-guidelines.md) +* [ADR 047: Extend Upgrade Plan](./adr-047-extend-upgrade-plan.md) +* [ADR 053: Go Module Refactoring](./adr-053-go-module-refactoring.md) +* [ADR 068: Preblock](./adr-068-preblock.md) diff --git a/docs/build/architecture/_category_.json b/docs/build/architecture/_category_.json index 87ceb9374..e0b1907a9 100644 --- a/docs/build/architecture/_category_.json +++ b/docs/build/architecture/_category_.json @@ -1,5 +1,5 @@ { "label": "ADRs", - "position": 11, + "position": 6, "link": null } \ No newline at end of file diff --git a/docs/build/architecture/adr-007-specialization-groups.md b/docs/build/architecture/adr-007-specialization-groups.md index 9a351dd1a..58f78abf5 100644 --- a/docs/build/architecture/adr-007-specialization-groups.md +++ b/docs/build/architecture/adr-007-specialization-groups.md @@ -174,4 +174,4 @@ type SpecializationGroup interface { ## References -* [dCERT ADR](adr-008-dCERT-group.md) +* [dCERT ADR](./adr-008-dCERT-group.md) diff --git a/docs/build/architecture/adr-008-dCERT-group.md b/docs/build/architecture/adr-008-dCERT-group.md index 2097bf1b9..2b2d2b826 100644 --- a/docs/build/architecture/adr-008-dCERT-group.md +++ b/docs/build/architecture/adr-008-dCERT-group.md @@ -32,7 +32,7 @@ vulnerability being patched on the live network. ## Decision The dCERT group is proposed to include an implementation of a `SpecializationGroup` -as defined in [ADR 007](adr-007-specialization-groups.md). This will include the +as defined in [ADR 007](./adr-007-specialization-groups.md). This will include the implementation of: * continuous voting @@ -168,4 +168,4 @@ they should all be severely slashed. ## References - [Specialization Groups ADR](adr-007-specialization-groups.md) + [Specialization Groups ADR](./adr-007-specialization-groups.md) diff --git a/docs/build/architecture/adr-009-evidence-module.md b/docs/build/architecture/adr-009-evidence-module.md index ded04a142..46ee5ebcb 100644 --- a/docs/build/architecture/adr-009-evidence-module.md +++ b/docs/build/architecture/adr-009-evidence-module.md @@ -17,7 +17,7 @@ evidence can be submitted, evaluated and verified resulting in some agreed upon penalty for any misbehavior committed by a validator, such as equivocation (double-voting), signing when unbonded, signing an incorrect state transition (in the future), etc. Furthermore, such a mechanism is paramount for any -[IBC](https://github.com/cosmos/ics/blob/master/ibc/2_IBC_ARCHITECTURE.md) or +[IBC](https://github.com/cosmos/ibc) or cross-chain validation protocol implementation in order to support the ability for any misbehavior to be relayed back from a collateralized chain to a primary chain so that the equivocating validator(s) can be slashed. @@ -178,5 +178,5 @@ type GenesisState struct { ## References * [ICS](https://github.com/cosmos/ics) -* [IBC Architecture](https://github.com/cosmos/ics/blob/master/ibc/1_IBC_ARCHITECTURE.md) +* [IBC Architecture](https://github.com/cosmos/ibc/blob/main/spec/ics-001-ics-standard/README.md) * [Tendermint Fork Accountability](https://github.com/tendermint/spec/blob/7b3138e69490f410768d9b1ffc7a17abc23ea397/spec/consensus/fork-accountability.md) diff --git a/docs/build/architecture/adr-013-metrics.md b/docs/build/architecture/adr-013-metrics.md index 33849b56c..b0808d462 100644 --- a/docs/build/architecture/adr-013-metrics.md +++ b/docs/build/architecture/adr-013-metrics.md @@ -21,7 +21,7 @@ and is rich with ecosystem tooling. We must also aim to integrate metrics into the Cosmos SDK in the most seamless way possible such that metrics may be added or removed at will and without much friction. To do this, we will use the -[go-metrics](https://github.com/armon/go-metrics) library. +[go-metrics](https://github.com/hashicorp/go-metrics) library. Finally, operators may enable telemetry along with specific configuration options. If enabled, metrics will be exposed via `/metrics?format={text|prometheus}` via the API server. diff --git a/docs/build/architecture/adr-017-historical-header-module.md b/docs/build/architecture/adr-017-historical-header-module.md index 573c632c2..54cd99845 100644 --- a/docs/build/architecture/adr-017-historical-header-module.md +++ b/docs/build/architecture/adr-017-historical-header-module.md @@ -4,6 +4,7 @@ * 26 November 2019: Start of first version * 2 December 2019: Final draft of first version +* 7 September 2023: Reduce HistoricalInfo type ## Context @@ -16,10 +17,11 @@ The application MUST store the most recent `n` headers in a persistent store. At The application MUST store this information by storing new headers immediately when handling `abci.RequestBeginBlock`: ```go -func BeginBlock(ctx sdk.Context, keeper HistoricalHeaderKeeper, req abci.RequestBeginBlock) abci.ResponseBeginBlock { +func BeginBlock(ctx sdk.Context, keeper HistoricalHeaderKeeper) error { info := HistoricalInfo{ - Header: ctx.BlockHeader(), - ValSet: keeper.StakingKeeper.GetAllValidators(ctx), // note that this must be stored in a canonical order + apphash: ctx.HeaderInfo().AppHash, + Time: ctx.HeaderInfo().Time, + NextValidatorsHash: ctx.CometInfo().NextValidatorsHash, } keeper.SetHistoricalInfo(ctx, ctx.BlockHeight(), info) n := keeper.GetParamRecentHeadersToStore() diff --git a/docs/build/architecture/adr-020-protobuf-transaction-encoding.md b/docs/build/architecture/adr-020-protobuf-transaction-encoding.md index 344a7fef5..4c30bb9eb 100644 --- a/docs/build/architecture/adr-020-protobuf-transaction-encoding.md +++ b/docs/build/architecture/adr-020-protobuf-transaction-encoding.md @@ -23,7 +23,7 @@ Accepted ## Context This ADR is a continuation of the motivation, design, and context established in -[ADR 019](adr-019-protobuf-state-encoding.md), namely, we aim to design the +[ADR 019](./adr-019-protobuf-state-encoding.md), namely, we aim to design the Protocol Buffer migration path for the client-side of the Cosmos SDK. Specifically, the client-side migration path primarily includes tx generation and @@ -205,7 +205,7 @@ message SignDoc { In order to sign in the default mode, clients take the following steps: 1. Serialize `TxBody` and `AuthInfo` using any valid protobuf implementation. -2. Create a `SignDoc` and serialize it using [ADR 027](adr-027-deterministic-protobuf-serialization.md). +2. Create a `SignDoc` and serialize it using [ADR 027](./adr-027-deterministic-protobuf-serialization.md). 3. Sign the encoded `SignDoc` bytes. 4. Build a `TxRaw` and serialize it for broadcasting. @@ -221,7 +221,7 @@ Signature verifiers do: 3. For each required signer: * Pull account number and sequence from the state. * Obtain the public key either from state or `AuthInfo`'s `signer_infos`. - * Create a `SignDoc` and serialize it using [ADR 027](adr-027-deterministic-protobuf-serialization.md). + * Create a `SignDoc` and serialize it using [ADR 027](./adr-027-deterministic-protobuf-serialization.md). * Verify the signature at the same list position against the serialized `SignDoc`. #### `SIGN_MODE_LEGACY_AMINO` @@ -312,7 +312,7 @@ enforce this. Currently, the REST and CLI handlers encode and decode types and txs via Amino JSON encoding using a concrete Amino codec. Being that some of the types dealt with -in the client can be interfaces, similar to how we described in [ADR 019](adr-019-protobuf-state-encoding.md), +in the client can be interfaces, similar to how we described in [ADR 019](./adr-019-protobuf-state-encoding.md), the client logic will now need to take a codec interface that knows not only how to handle all the types, but also knows how to generate transactions, signatures, and messages. diff --git a/docs/build/architecture/adr-021-protobuf-query-encoding.md b/docs/build/architecture/adr-021-protobuf-query-encoding.md index 76fd40feb..a90e807d5 100644 --- a/docs/build/architecture/adr-021-protobuf-query-encoding.md +++ b/docs/build/architecture/adr-021-protobuf-query-encoding.md @@ -11,11 +11,11 @@ Accepted ## Context This ADR is a continuation of the motivation, design, and context established in -[ADR 019](adr-019-protobuf-state-encoding.md) and -[ADR 020](adr-020-protobuf-transaction-encoding.md), namely, we aim to design the +[ADR 019](./adr-019-protobuf-state-encoding.md) and +[ADR 020](./adr-020-protobuf-transaction-encoding.md), namely, we aim to design the Protocol Buffer migration path for the client-side of the Cosmos SDK. -This ADR continues from [ADD 020](adr-020-protobuf-transaction-encoding.md) +This ADR continues from [ADD 020](./adr-020-protobuf-transaction-encoding.md) to specify the encoding of queries. ## Decision diff --git a/docs/build/architecture/adr-022-custom-panic-handling.md b/docs/build/architecture/adr-022-custom-panic-handling.md index 2cdce59f4..8cb5d9687 100644 --- a/docs/build/architecture/adr-022-custom-panic-handling.md +++ b/docs/build/architecture/adr-022-custom-panic-handling.md @@ -23,7 +23,7 @@ We propose middleware-solution, which could help developers implement the follow * call panic for specific error cases; It will also make `OutOfGas` case and `default` case one of the middlewares. -`Default` case wraps recovery object to an error and logs it ([example middleware implementation](#recovery-middleware)). +`Default` case wraps recovery object to an error and logs it ([example middleware implementation](#Recovery-middleware)). Our project has a sidecar service running alongside the blockchain node (smart contracts virtual machine). It is essential that node <-> sidecar connectivity stays stable for TXs processing. So when the communication breaks we need diff --git a/docs/build/architecture/adr-024-coin-metadata.md b/docs/build/architecture/adr-024-coin-metadata.md index 71bedac5b..91c7c4d9f 100644 --- a/docs/build/architecture/adr-024-coin-metadata.md +++ b/docs/build/architecture/adr-024-coin-metadata.md @@ -6,7 +6,7 @@ ## Status -Proposed +ACCEPTED ## Context diff --git a/docs/build/architecture/adr-027-deterministic-protobuf-serialization.md b/docs/build/architecture/adr-027-deterministic-protobuf-serialization.md index e19a45a73..66ce6e2b7 100644 --- a/docs/build/architecture/adr-027-deterministic-protobuf-serialization.md +++ b/docs/build/architecture/adr-027-deterministic-protobuf-serialization.md @@ -27,7 +27,7 @@ other cases as well. For signature verification in Cosmos SDK, the signer and verifier need to agree on the same serialization of a `SignDoc` as defined in -[ADR-020](adr-020-protobuf-transaction-encoding.md) without transmitting the +[ADR-020](./adr-020-protobuf-transaction-encoding.md) without transmitting the serialization. Currently, for block signatures we are using a workaround: we create a new [TxRaw](https://github.com/cosmos/cosmos-sdk/blob/9e85e81e0e8140067dd893421290c191529c148c/proto/cosmos/tx/v1beta1/tx.proto#L30) diff --git a/docs/build/architecture/adr-030-authz-module.md b/docs/build/architecture/adr-030-authz-module.md index 0454138d1..5aab72c5c 100644 --- a/docs/build/architecture/adr-030-authz-module.md +++ b/docs/build/architecture/adr-030-authz-module.md @@ -25,7 +25,7 @@ The concrete use cases which motivated this module include: delegated stake * "sub-keys" functionality, as originally proposed in [\#4480](https://github.com/cosmos/cosmos-sdk/issues/4480) which is a term used to describe the functionality provided by this module together with -the `fee_grant` module from [ADR 029](adr-029-fee-grant-module.md) and the [group module](https://github.com/cosmos/cosmos-sdk/tree/main/x/group). +the `fee_grant` module from [ADR 029](./adr-029-fee-grant-module.md) and the [group module](https://github.com/cosmos/cosmos-sdk/tree/main/x/group). The "sub-keys" functionality roughly refers to the ability for one account to grant some subset of its capabilities to other accounts with possibly less robust, but easier to use security measures. For instance, a master account representing diff --git a/docs/build/architecture/adr-031-msg-service.md b/docs/build/architecture/adr-031-msg-service.md index b8e4005d2..861f4b3f3 100644 --- a/docs/build/architecture/adr-031-msg-service.md +++ b/docs/build/architecture/adr-031-msg-service.md @@ -105,12 +105,12 @@ One consequence of this convention is that each `Msg` type can be the request pa ### Encoding -Encoding of transactions generated with `Msg` services do not differ from current Protobuf transaction encoding as defined in [ADR-020](adr-020-protobuf-transaction-encoding.md). We are encoding `Msg` types (which are exactly `Msg` service methods' request parameters) as `Any` in `Tx`s which involves packing the +Encoding of transactions generated with `Msg` services do not differ from current Protobuf transaction encoding as defined in [ADR-020](./adr-020-protobuf-transaction-encoding.md). We are encoding `Msg` types (which are exactly `Msg` service methods' request parameters) as `Any` in `Tx`s which involves packing the binary-encoded `Msg` with its type URL. ### Decoding -Since `Msg` types are packed into `Any`, decoding transactions messages are done by unpacking `Any`s into `Msg` types. For more information, please refer to [ADR-020](adr-020-protobuf-transaction-encoding.md#transactions). +Since `Msg` types are packed into `Any`, decoding transactions messages are done by unpacking `Any`s into `Msg` types. For more information, please refer to [ADR-020](./adr-020-protobuf-transaction-encoding.md#transactions). ### Routing @@ -122,7 +122,7 @@ For backward compatability, the old handlers are not removed yet. If BaseApp rec ### Module Configuration -In [ADR 021](adr-021-protobuf-query-encoding.md), we introduced a method `RegisterQueryService` +In [ADR 021](./adr-021-protobuf-query-encoding.md), we introduced a method `RegisterQueryService` to `AppModule` which allows for modules to register gRPC queriers. To register `Msg` services, we attempt a more extensible approach by converting `RegisterQueryService` @@ -198,5 +198,5 @@ Finally, closing a module to client API opens desirable OCAP patterns discussed * [Initial Github Issue \#7122](https://github.com/cosmos/cosmos-sdk/issues/7122) * [proto 3 Language Guide: Defining Services](https://developers.google.com/protocol-buffers/docs/proto3#services) * [Initial pre-`Any` `Msg` designs](https://docs.google.com/document/d/1eEgYgvgZqLE45vETjhwIw4VOqK-5hwQtZtjVbiXnIGc) -* [ADR 020](adr-020-protobuf-transaction-encoding.md) -* [ADR 021](adr-021-protobuf-query-encoding.md) +* [ADR 020](./adr-020-protobuf-transaction-encoding.md) +* [ADR 021](./adr-021-protobuf-query-encoding.md) diff --git a/docs/build/architecture/adr-033-protobuf-inter-module-comm.md b/docs/build/architecture/adr-033-protobuf-inter-module-comm.md index 79b71f718..28c69a910 100644 --- a/docs/build/architecture/adr-033-protobuf-inter-module-comm.md +++ b/docs/build/architecture/adr-033-protobuf-inter-module-comm.md @@ -11,8 +11,8 @@ Proposed ## Abstract This ADR introduces a system for permissioned inter-module communication leveraging the protobuf `Query` and `Msg` -service definitions defined in [ADR 021](adr-021-protobuf-query-encoding.md) and -[ADR 031](adr-031-msg-service.md) which provides: +service definitions defined in [ADR 021](./adr-021-protobuf-query-encoding.md) and +[ADR 031](./adr-031-msg-service.md) which provides: * stable protobuf based module interfaces to potentially later replace the keeper paradigm * stronger inter-module object capabilities (OCAPs) guarantees @@ -20,7 +20,7 @@ service definitions defined in [ADR 021](adr-021-protobuf-query-encoding.md) and ## Context -In the current Cosmos SDK documentation on the [Object-Capability Model](../../develop/advanced/10-ocap.md), it is stated that: +In the current Cosmos SDK documentation on the [Object-Capability Model](../../learn/advanced/10-ocap.md), it is stated that: > We assume that a thriving ecosystem of Cosmos SDK modules that are easy to compose into a blockchain application will contain faulty or malicious modules. @@ -48,7 +48,7 @@ just a simple string. So the `x/upgrade` module could mint tokens for the `x/sta ## Decision -Based on [ADR-021](adr-021-protobuf-query-encoding.md) and [ADR-031](adr-031-msg-service.md), we introduce the +Based on [ADR-021](./adr-021-protobuf-query-encoding.md) and [ADR-031](./adr-031-msg-service.md), we introduce the Inter-Module Communication framework for secure module authorization and OCAPs. When implemented, this could also serve as an alternative to the existing paradigm of passing keepers between modules. The approach outlined here-in is intended to form the basis of a Cosmos SDK v1.0 that provides the necessary @@ -60,8 +60,8 @@ addressed as amendments to this ADR. ### New "Keeper" Paradigm -In [ADR 021](adr-021-protobuf-query-encoding.md), a mechanism for using protobuf service definitions to define queriers -was introduced and in [ADR 31](adr-031-msg-service.md), a mechanism for using protobuf service to define `Msg`s was added. +In [ADR 021](./adr-021-protobuf-query-encoding.md), a mechanism for using protobuf service definitions to define queriers +was introduced and in [ADR 31](./adr-031-msg-service.md), a mechanism for using protobuf service to define `Msg`s was added. Protobuf service definitions generate two golang interfaces representing the client and server sides of a service plus some helper code. Here is a minimal example for the bank `cosmos.bank.Msg/Send` message type: @@ -77,7 +77,7 @@ type MsgServer interface { } ``` -[ADR 021](adr-021-protobuf-query-encoding.md) and [ADR 31](adr-031-msg-service.md) specifies how modules can implement the generated `QueryServer` +[ADR 021](./adr-021-protobuf-query-encoding.md) and [ADR 31](./adr-031-msg-service.md) specifies how modules can implement the generated `QueryServer` and `MsgServer` interfaces as replacements for the legacy queriers and `Msg` handlers respectively. In this ADR we explain how modules can make queries and send `Msg`s to other modules using the generated `QueryClient` @@ -161,7 +161,7 @@ denom prefix being restricted to certain modules (as discussed in ### `ModuleKey`s and `ModuleID`s A `ModuleKey` can be thought of as a "private key" for a module account and a `ModuleID` can be thought of as the -corresponding "public key". From the [ADR 028](adr-028-public-key-addresses.md), modules can have both a root module account and any number of sub-accounts +corresponding "public key". From the [ADR 028](./adr-028-public-key-addresses.md), modules can have both a root module account and any number of sub-accounts or derived accounts that can be used for different pools (ex. staking pools) or managed accounts (ex. group accounts). We can also think of module sub-accounts as similar to derived keys - there is a root key and then some derivation path. `ModuleID` is a simple struct which contains the module name and optional "derivation" path, @@ -248,7 +248,7 @@ func (key RootModuleKey) Invoke(ctx context.Context, method string, args, reply ### `AppModule` Wiring and Requirements -In [ADR 031](adr-031-msg-service.md), the `AppModule.RegisterService(Configurator)` method was introduced. To support +In [ADR 031](./adr-031-msg-service.md), the `AppModule.RegisterService(Configurator)` method was introduced. To support inter-module communication, we extend the `Configurator` interface to pass in the `ModuleKey` and to allow modules to specify their dependencies on other modules using `RequireServer()`: @@ -393,8 +393,8 @@ replacing `Keeper` interfaces altogether. ## References -* [ADR 021](adr-021-protobuf-query-encoding.md) -* [ADR 031](adr-031-msg-service.md) -* [ADR 028](adr-028-public-key-addresses.md) +* [ADR 021](./adr-021-protobuf-query-encoding.md) +* [ADR 031](./adr-031-msg-service.md) +* [ADR 028](./adr-028-public-key-addresses.md) * [ADR 030 draft](https://github.com/cosmos/cosmos-sdk/pull/7105) * [Object-Capability Model](https://docs.network.com/main/core/ocap) diff --git a/docs/build/architecture/adr-038-state-listening.md b/docs/build/architecture/adr-038-state-listening.md index 212d275d7..319d872be 100644 --- a/docs/build/architecture/adr-038-state-listening.md +++ b/docs/build/architecture/adr-038-state-listening.md @@ -8,7 +8,7 @@ * Add `ListenCommit`, flatten the state writes in a block to a single batch. * Remove listeners from cache stores, should only listen to `rootmulti.Store`. * Remove `HaltAppOnDeliveryError()`, the errors are propagated by default, the implementations should return nil if don't want to propogate errors. - +* 26/05/2023: Update with ABCI 2.0 ## Status @@ -20,7 +20,7 @@ This ADR defines a set of changes to enable listening to state changes of indivi ## Context -Currently, KVStore data can be remotely accessed through [Queries](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules/02-messages-and-queries.md#queries) +Currently, KVStore data can be remotely accessed through [Queries](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules/messages-and-queries.md#queries) which proceed either through Tendermint and the ABCI, or through the gRPC server. In addition to these request/response queries, it would be beneficial to have a means of listening to state changes as they occur in real time. @@ -82,7 +82,7 @@ We will configure the `Store` with a `MemoryListener` which will collect state c ```go // Store implements the KVStore interface with listening enabled. -// Operations are traced on each advanced KVStore call and written to any of the +// Operations are traced on each core KVStore call and written to any of the // underlying listeners with the proper key and operation permissions type Store struct { parent types.KVStore @@ -223,14 +223,10 @@ so that the service can group the state changes with the ABCI requests. // ABCIListener is the interface that we're exposing as a streaming service. type ABCIListener interface { - // ListenBeginBlock updates the streaming service with the latest BeginBlock messages - ListenBeginBlock(ctx context.Context, req abci.RequestBeginBlock, res abci.ResponseBeginBlock) error - // ListenEndBlock updates the steaming service with the latest EndBlock messages - ListenEndBlock(ctx types.Context, req abci.RequestEndBlock, res abci.ResponseEndBlock) error - // ListenDeliverTx updates the steaming service with the latest DeliverTx messages - ListenDeliverTx(ctx context.Context, req abci.RequestDeliverTx, res abci.ResponseDeliverTx) error - // ListenCommit updates the steaming service with the latest Commit messages and state changes - ListenCommit(ctx context.Context, res abci.ResponseCommit, changeSet []*store.StoreKVPair) error + // ListenFinalizeBlock updates the streaming service with the latest FinalizeBlock messages + ListenFinalizeBlock(ctx context.Context, req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) error + // ListenCommit updates the steaming service with the latest Commit messages and state changes + ListenCommit(ctx context.Context, res abci.ResponseCommit, changeSet []*StoreKVPair) error } ``` @@ -267,85 +263,27 @@ type BaseApp struct { #### ABCI Event Hooks -We will modify the `BeginBlock`, `EndBlock`, `DeliverTx` and `Commit` methods to pass ABCI requests and responses +We will modify the `FinalizeBlock` and `Commit` methods to pass ABCI requests and responses to any streaming service hooks registered with the `BaseApp`. ```go -func (app *BaseApp) BeginBlock(req abci.RequestBeginBlock) (res abci.ResponseBeginBlock) { - - ... - - // call the streaming service hook with the BeginBlock messages - for _, abciListener := range app.abciListeners { - ctx := app.deliverState.ctx - blockHeight := ctx.BlockHeight() - if app.abciListenersAsync { - go func(req abci.RequestBeginBlock, res abci.ResponseBeginBlock) { - if err := app.abciListener.ListenBeginBlock(ctx, req, res); err != nil { - app.logger.Error("BeginBlock listening hook failed", "height", blockHeight, "err", err) - } - }(req, res) - } else { - if err := app.abciListener.ListenBeginBlock(ctx, req, res); err != nil { - app.logger.Error("BeginBlock listening hook failed", "height", blockHeight, "err", err) - if app.stopNodeOnABCIListenerErr { - os.Exit(1) - } - } - } - } +func (app *BaseApp) FinalizeBlock(req abci.RequestFinalizeBlock) abci.ResponseFinalizeBlock { - return res -} -``` - -```go -func (app *BaseApp) EndBlock(req abci.RequestEndBlock) (res abci.ResponseEndBlock) { - - ... - - // call the streaming service hook with the EndBlock messages - for _, abciListener := range app.abciListeners { - ctx := app.deliverState.ctx - blockHeight := ctx.BlockHeight() - if app.abciListenersAsync { - go func(req abci.RequestEndBlock, res abci.ResponseEndBlock) { - if err := app.abciListener.ListenEndBlock(blockHeight, req, res); err != nil { - app.logger.Error("EndBlock listening hook failed", "height", blockHeight, "err", err) - } - }(req, res) - } else { - if err := app.abciListener.ListenEndBlock(blockHeight, req, res); err != nil { - app.logger.Error("EndBlock listening hook failed", "height", blockHeight, "err", err) - if app.stopNodeOnABCIListenerErr { - os.Exit(1) - } - } - } - } - - return res -} -``` - -```go -func (app *BaseApp) DeliverTx(req abci.RequestDeliverTx) abci.ResponseDeliverTx { - - var abciRes abci.ResponseDeliverTx + var abciRes abci.ResponseFinalizeBlock defer func() { - // call the streaming service hook with the EndBlock messages + // call the streaming service hook with the FinalizeBlock messages for _, abciListener := range app.abciListeners { - ctx := app.deliverState.ctx + ctx := app.finalizeState.ctx blockHeight := ctx.BlockHeight() if app.abciListenersAsync { - go func(req abci.RequestDeliverTx, res abci.ResponseDeliverTx) { - if err := app.abciListener.ListenDeliverTx(blockHeight, req, res); err != nil { - app.logger.Error("DeliverTx listening hook failed", "height", blockHeight, "err", err) + go func(req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) { + if err := app.abciListener.FinalizeBlock(blockHeight, req, res); err != nil { + app.logger.Error("FinalizeBlock listening hook failed", "height", blockHeight, "err", err) } }(req, abciRes) } else { - if err := app.abciListener.ListenDeliverTx(blockHeight, req, res); err != nil { - app.logger.Error("DeliverTx listening hook failed", "height", blockHeight, "err", err) + if err := app.abciListener.ListenFinalizeBlock(blockHeight, req, res); err != nil { + app.logger.Error("FinalizeBlock listening hook failed", "height", blockHeight, "err", err) if app.stopNodeOnABCIListenerErr { os.Exit(1) } @@ -455,18 +393,9 @@ syntax = "proto3"; message Empty {} -message ListenBeginBlockRequest { - RequestBeginBlock req = 1; - ResponseBeginBlock res = 2; -} -message ListenEndBlockRequest { - RequestEndBlock req = 1; - ResponseEndBlock res = 2; -} -message ListenDeliverTxRequest { - int64 block_height = 1; - RequestDeliverTx req = 2; - ResponseDeliverTx res = 3; +message ListenFinalizeBlockRequest { + RequestFinalizeBlock req = 1; + ResponseFinalizeBlock res = 2; } message ListenCommitRequest { int64 block_height = 1; @@ -476,9 +405,7 @@ message ListenCommitRequest { // plugin that listens to state changes service ABCIListenerService { - rpc ListenBeginBlock(ListenBeginBlockRequest) returns (Empty); - rpc ListenEndBlock(ListenEndBlockRequest) returns (Empty); - rpc ListenDeliverTx(ListenDeliverTxRequest) returns (Empty); + rpc ListenFinalizeBlock(ListenFinalizeBlockRequest) returns (Empty); rpc ListenCommit(ListenCommitRequest) returns (Empty); } ``` @@ -487,9 +414,7 @@ service ABCIListenerService { ... // plugin that doesn't listen to state changes service ABCIListenerService { - rpc ListenBeginBlock(ListenBeginBlockRequest) returns (Empty); - rpc ListenEndBlock(ListenEndBlockRequest) returns (Empty); - rpc ListenDeliverTx(ListenDeliverTxRequest) returns (Empty); + rpc ListenFinalizeBlock(ListenFinalizeBlockRequest) returns (Empty); rpc ListenCommit(ListenCommitRequest) returns (Empty); } ``` @@ -508,17 +433,7 @@ type GRPCClient struct { client ABCIListenerServiceClient } -func (m *GRPCClient) ListenBeginBlock(ctx context.Context, req abci.RequestBeginBlock, res abci.ResponseBeginBlock) error { - _, err := m.client.ListenBeginBlock(ctx, &ListenBeginBlockRequest{Req: req, Res: res}) - return err -} - -func (m *GRPCClient) ListenEndBlock(goCtx context.Context, req abci.RequestEndBlock, res abci.ResponseEndBlock) error { - _, err := m.client.ListenEndBlock(ctx, &ListenEndBlockRequest{Req: req, Res: res}) - return err -} - -func (m *GRPCClient) ListenDeliverTx(goCtx context.Context, req abci.RequestDeliverTx, res abci.ResponseDeliverTx) error { +func (m *GRPCClient) ListenFinalizeBlock(goCtx context.Context, req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) error { ctx := sdk.UnwrapSDKContext(goCtx) _, err := m.client.ListenDeliverTx(ctx, &ListenDeliverTxRequest{BlockHeight: ctx.BlockHeight(), Req: req, Res: res}) return err @@ -536,16 +451,8 @@ type GRPCServer struct { Impl baseapp.ABCIListener } -func (m *GRPCServer) ListenBeginBlock(ctx context.Context, req *ListenBeginBlockRequest) (*Empty, error) { - return &Empty{}, m.Impl.ListenBeginBlock(ctx, req.Req, req.Res) -} - -func (m *GRPCServer) ListenEndBlock(ctx context.Context, req *ListenEndBlockRequest) (*Empty, error) { - return &Empty{}, m.Impl.ListenEndBlock(ctx, req.Req, req.Res) -} - -func (m *GRPCServer) ListenDeliverTx(ctx context.Context, req *ListenDeliverTxRequest) (*Empty, error) { - return &Empty{}, m.Impl.ListenDeliverTx(ctx, req.Req, req.Res) +func (m *GRPCServer) ListenFinalizeBlock(ctx context.Context, req *ListenFinalizeBlockRequest) (*Empty, error) { + return &Empty{}, m.Impl.ListenFinalizeBlock(ctx, req.Req, req.Res) } func (m *GRPCServer) ListenCommit(ctx context.Context, req *ListenCommitRequest) (*Empty, error) { @@ -564,15 +471,7 @@ And the pre-compiled Go plugin `Impl`(*this is only used for plugins that are wr // ABCIListener is the implementation of the baseapp.ABCIListener interface type ABCIListener struct{} -func (m *ABCIListenerPlugin) ListenBeginBlock(ctx context.Context, req abci.RequestBeginBlock, res abci.ResponseBeginBlock) error { - // send data to external system -} - -func (m *ABCIListenerPlugin) ListenEndBlock(ctx context.Context, req abci.RequestBeginBlock, res abci.ResponseBeginBlock) error { - // send data to external system -} - -func (m *ABCIListenerPlugin) ListenDeliverTxBlock(ctx context.Context, req abci.RequestBeginBlock, res abci.ResponseBeginBlock) error { +func (m *ABCIListenerPlugin) ListenFinalizeBlock(ctx context.Context, req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) error { // send data to external system } @@ -669,9 +568,12 @@ func registerABCIListenerPlugin( exposeKeysStr := cast.ToStringSlice(appOpts.Get(keysKey)) exposedKeys := exposeStoreKeysSorted(exposeKeysStr, keys) bApp.cms.AddListeners(exposedKeys) - bApp.SetStreamingService(abciListener) - bApp.stopNodeOnABCIListenerErr = stopNodeOnErr - bApp.abciListenersAsync = async + app.SetStreamingManager( + storetypes.StreamingManager{ + ABCIListeners: []storetypes.ABCIListener{abciListener}, + StopNodeOnErr: stopNodeOnErr, + }, + ) } ``` diff --git a/docs/build/architecture/adr-040-storage-and-smt-state-commitments.md b/docs/build/architecture/adr-040-storage-and-smt-state-commitments.md index b089bfb1a..f60e3adcf 100644 --- a/docs/build/architecture/adr-040-storage-and-smt-state-commitments.md +++ b/docs/build/architecture/adr-040-storage-and-smt-state-commitments.md @@ -23,7 +23,7 @@ In the current design, IAVL is used for both data storage and as a Merkle Tree f * Each edge traversal requires a DB query. * Creating snapshots is [expensive](https://github.com/cosmos/cosmos-sdk/issues/7215#issuecomment-684804950). It takes about 30 seconds to export less than 100 MB of state (as of March 2020). * Updates in IAVL may trigger tree reorganization and possible O(log(n)) hashes re-computation, which can become a CPU bottleneck. -* The node structure is pretty expensive - it contains a standard tree node elements (key, value, left and right element) and additional metadata such as height, version (which is not required by the Cosmos SDK). The entire node is hashed, and that hash is used as the key in the underlying database, [ref](https://github.com/cosmos/iavl/blob/master/docs/node/03-node.md +* The node structure is pretty expensive - it contains a standard tree node elements (key, value, left and right element) and additional metadata such as height, version (which is not required by the Cosmos SDK). The entire node is hashed, and that hash is used as the key in the underlying database, [ref](https://github.com/cosmos/iavl/blob/master/docs/node/node.md ). Moreover, the IAVL project lacks support and a maintainer and we already see better and well-established alternatives. Instead of optimizing the IAVL, we are looking into other solutions for both storage and state commitments. diff --git a/docs/build/architecture/adr-042-group-module.md b/docs/build/architecture/adr-042-group-module.md index 834ec455e..52e94327d 100644 --- a/docs/build/architecture/adr-042-group-module.md +++ b/docs/build/architecture/adr-042-group-module.md @@ -22,7 +22,7 @@ The legacy amino multi-signature mechanism of the Cosmos SDK has certain limitat * It requires `legacy_amino` sign mode ([#8141](https://github.com/cosmos/cosmos-sdk/issues/8141)). While the group module is not meant to be a total replacement for the current multi-signature accounts, it provides a solution to the limitations described above, with a more flexible key management system where keys can be added, updated or removed, as well as configurable thresholds. -It's meant to be used with other access control modules such as [`x/feegrant`](adr-029-fee-grant-module.md) ans [`x/authz`](adr-030-authz-module.md) to simplify key management for individuals and organizations. +It's meant to be used with other access control modules such as [`x/feegrant`](./adr-029-fee-grant-module.md) ans [`x/authz`](adr-030-authz-module.md) to simplify key management for individuals and organizations. The proof of concept of the group module can be found in https://github.com/regen-network/regen-ledger/tree/master/proto/regen/group/v1alpha1 and https://github.com/regen-network/regen-ledger/tree/master/x/group. diff --git a/docs/build/architecture/adr-044-protobuf-updates-guidelines.md b/docs/build/architecture/adr-044-protobuf-updates-guidelines.md index 245adcffe..a5ea31316 100644 --- a/docs/build/architecture/adr-044-protobuf-updates-guidelines.md +++ b/docs/build/architecture/adr-044-protobuf-updates-guidelines.md @@ -78,7 +78,7 @@ Protobuf supports the [`deprecated` field option](https://developers.google.com/ As an example, the Cosmos SDK v0.42 to v0.43 update contained two Protobuf-breaking changes, listed below. Instead of bumping the package versions from `v1beta1` to `v1`, the SDK team decided to follow this guideline, by reverting the breaking changes, marking those changes as deprecated, and modifying the node implementation when processing messages with deprecated fields. More specifically: * The Cosmos SDK recently removed support for [time-based software upgrades](https://github.com/cosmos/cosmos-sdk/pull/8849). As such, the `time` field has been marked as deprecated in `cosmos.upgrade.v1beta1.Plan`. Moreover, the node will reject any proposal containing an upgrade Plan whose `time` field is non-empty. -* The Cosmos SDK now supports [governance split votes](adr-037-gov-split-vote.md). When querying for votes, the returned `cosmos.gov.v1beta1.Vote` message has its `option` field (used for 1 vote option) deprecated in favor of its `options` field (allowing multiple vote options). Whenever possible, the SDK still populates the deprecated `option` field, that is, if and only if the `len(options) == 1` and `options[0].Weight == 1.0`. +* The Cosmos SDK now supports [governance split votes](./adr-037-gov-split-vote.md). When querying for votes, the returned `cosmos.gov.v1beta1.Vote` message has its `option` field (used for 1 vote option) deprecated in favor of its `options` field (allowing multiple vote options). Whenever possible, the SDK still populates the deprecated `option` field, that is, if and only if the `len(options) == 1` and `options[0].Weight == 1.0`. #### 3. Fields MUST NOT be renamed diff --git a/docs/build/architecture/adr-045-check-delivertx-middlewares.md b/docs/build/architecture/adr-045-check-delivertx-middlewares.md index 756fa5a2d..60172977c 100644 --- a/docs/build/architecture/adr-045-check-delivertx-middlewares.md +++ b/docs/build/architecture/adr-045-check-delivertx-middlewares.md @@ -224,12 +224,12 @@ While the app developer can define and compose the middlewares of their choice, | TxDecoderMiddleware | This middleware takes in transaction raw bytes, and decodes them into a `sdk.Tx`. It replaces the `baseapp.txDecoder` field, so that BaseApp stays as thin as possible. Since most middlewares read the contents of the `sdk.Tx`, the TxDecoderMiddleware should be run first in the middleware stack. | | {Antehandlers} | Each antehandler is converted to its own middleware. These middlewares perform signature verification, fee deductions and other validations on the incoming transaction. | | IndexEventsTxMiddleware | This is a simple middleware that chooses which events to index in Tendermint. Replaces `baseapp.indexEvents` (which unfortunately still exists in baseapp too, because it's used to index Begin/EndBlock events) | -| RecoveryTxMiddleware | This index recovers from panics. It replaces baseapp.runTx's panic recovery described in [ADR-022](adr-022-custom-panic-handling.md). | +| RecoveryTxMiddleware | This index recovers from panics. It replaces baseapp.runTx's panic recovery described in [ADR-022](./adr-022-custom-panic-handling.md). | | GasTxMiddleware | This replaces the [`Setup`](https://github.com/cosmos/cosmos-sdk/blob/v0.43.0/x/auth/ante/setup.go) Antehandler. It sets a GasMeter on sdk.Context. Note that before, GasMeter was set on sdk.Context inside the antehandlers, and there was some mess around the fact that antehandlers had their own panic recovery system so that the GasMeter could be read by baseapp's recovery system. Now, this mess is all removed: one middleware sets GasMeter, another one handles recovery. | ### Similarities and Differences between Antehandlers and Middlewares -The middleware-based design builds upon the existing antehandlers design described in [ADR-010](adr-010-modular-antehandler.md). Even though the final decision of ADR-010 was to go with the "Simple Decorators" approach, the middleware design is actually very similar to the other [Decorator Pattern](adr-010-modular-antehandler.md#decorator-pattern) proposal, also used in [weave](https://github.com/iov-one/weave). +The middleware-based design builds upon the existing antehandlers design described in [ADR-010](./adr-010-modular-antehandler.md). Even though the final decision of ADR-010 was to go with the "Simple Decorators" approach, the middleware design is actually very similar to the other [Decorator Pattern](./adr-010-modular-antehandler.md#decorator-pattern) proposal, also used in [weave](https://github.com/iov-one/weave). #### Similarities with Antehandlers diff --git a/docs/build/architecture/adr-046-module-params.md b/docs/build/architecture/adr-046-module-params.md index 369cd043d..09d786f15 100644 --- a/docs/build/architecture/adr-046-module-params.md +++ b/docs/build/architecture/adr-046-module-params.md @@ -6,7 +6,7 @@ ## Status -Proposed +ACCEPTED ## Abstract diff --git a/docs/build/architecture/adr-047-extend-upgrade-plan.md b/docs/build/architecture/adr-047-extend-upgrade-plan.md index 3a4f3aac4..3500bb334 100644 --- a/docs/build/architecture/adr-047-extend-upgrade-plan.md +++ b/docs/build/architecture/adr-047-extend-upgrade-plan.md @@ -3,10 +3,11 @@ ## Changelog * Nov, 23, 2021: Initial Draft +* May, 16, 2023: Proposal ABANDONED. `pre_run` and `post_run` are not necessary anymore and adding the `artifacts` brings minor benefits. ## Status -PROPOSED Not Implemented +ABANDONED ## Abstract @@ -19,6 +20,7 @@ The `upgrade` module in conjunction with Cosmovisor are designed to facilitate a Users submit a software upgrade governance proposal containing an upgrade `Plan`. The [Plan](https://github.com/cosmos/cosmos-sdk/blob/v0.44.5/proto/cosmos/upgrade/v1beta1/upgrade.proto#L12) currently contains the following fields: + * `name`: A short string identifying the new version. * `height`: The chain height at which the upgrade is to be performed. * `info`: A string containing information about the upgrade. @@ -79,6 +81,7 @@ message UpgradeInstructions { ``` All fields in the `UpgradeInstructions` are optional. + * `pre_run` is a command to run prior to the upgraded chain restarting. If defined, it will be executed after halting and downloading the new artifact but before restarting the upgraded chain. The working directory this command runs from MUST be `{DAEMON_HOME}/cosmovisor/{upgrade name}`. diff --git a/docs/build/architecture/adr-050-sign-mode-textual-annex1.md b/docs/build/architecture/adr-050-sign-mode-textual-annex1.md index 13deec921..ff3acc8c5 100644 --- a/docs/build/architecture/adr-050-sign-mode-textual-annex1.md +++ b/docs/build/architecture/adr-050-sign-mode-textual-annex1.md @@ -63,7 +63,7 @@ Value Renderers describe how values of different Protobuf types should be encode ### `repeated` -* Applies to all `repeated` fields, except `cosmos.tx.v1beta1.TxBody#Messages`, which has a particular encoding (see [ADR-050](adr-050-sign-mode-textual.md)). +* Applies to all `repeated` fields, except `cosmos.tx.v1beta1.TxBody#Messages`, which has a particular encoding (see [ADR-050](./adr-050-sign-mode-textual.md)). * A repeated type has the following template: ``` @@ -275,7 +275,7 @@ The number 35 was chosen because it is the longest length where the hashed-and-p - byte arrays starting from length 36 will be be hashed to 32 bytes, which is 64 hex characters plus 15 spaces, and with the `SHA-256=` prefix, it takes 87 characters. Also, secp256k1 public keys have length 33, so their Textual representation is not their hashed value, which we would like to avoid. -Note: Data longer than 35 bytes are not rendered in a way that can be inverted. See ADR-050's [section about invertability](adr-050-sign-mode-textual.md#invertible-rendering) for a discussion. +Note: Data longer than 35 bytes are not rendered in a way that can be inverted. See ADR-050's [section about invertability](./adr-050-sign-mode-textual.md#invertible-rendering) for a discussion. #### Examples diff --git a/docs/build/architecture/adr-050-sign-mode-textual.md b/docs/build/architecture/adr-050-sign-mode-textual.md index efa4ace48..c5b51b223 100644 --- a/docs/build/architecture/adr-050-sign-mode-textual.md +++ b/docs/build/architecture/adr-050-sign-mode-textual.md @@ -28,7 +28,7 @@ This ADR specifies SIGN_MODE_TEXTUAL, a new string-based sign mode that is targe ## Context -Protobuf-based SIGN_MODE_DIRECT was introduced in [ADR-020](adr-020-protobuf-transaction-encoding.md) and is intended to replace SIGN_MODE_LEGACY_AMINO_JSON in most situations, such as mobile wallets and CLI keyrings. However, the [Ledger](https://www.ledger.com/) hardware wallet is still using SIGN_MODE_LEGACY_AMINO_JSON for displaying the sign bytes to the user. Hardware wallets cannot transition to SIGN_MODE_DIRECT as: +Protobuf-based SIGN_MODE_DIRECT was introduced in [ADR-020](./adr-020-protobuf-transaction-encoding.md) and is intended to replace SIGN_MODE_LEGACY_AMINO_JSON in most situations, such as mobile wallets and CLI keyrings. However, the [Ledger](https://www.ledger.com/) hardware wallet is still using SIGN_MODE_LEGACY_AMINO_JSON for displaying the sign bytes to the user. Hardware wallets cannot transition to SIGN_MODE_DIRECT as: * SIGN_MODE_DIRECT is binary-based and thus not suitable for display to end-users. Technically, hardware wallets could simply display the sign bytes to the user. But this would be considered as blind signing, and is a security concern. * hardware cannot decode the protobuf sign bytes due to memory constraints, as the Protobuf definitions would need to be embedded on the hardware device. @@ -54,7 +54,7 @@ or to introduce or conclude a larger grouping. The text can contain the full range of Unicode code points, including control characters and nul. The device is responsible for deciding how to display characters it cannot render natively. -See [annex 2](adr-050-sign-mode-textual-annex2.md) for guidance. +See [annex 2](./adr-050-sign-mode-textual-annex2.md) for guidance. Screens have a non-negative indentation level to signal composite or nested structures. Indentation level zero is the top level. @@ -299,7 +299,7 @@ where: This is to prevent transaction hash malleability. The point #1 about invertiblity assures that transaction `body` and `auth_info` values are not malleable, but the transaction hash still might be malleable with point #1 only, because the SIGN_MODE_TEXTUAL strings don't follow the byte ordering defined in `body_bytes` and `auth_info_bytes`. Without this hash, a malicious validator or exchange could intercept a transaction, modify its transaction hash _after_ the user signed it using SIGN_MODE_TEXTUAL (by tweaking the byte ordering inside `body_bytes` or `auth_info_bytes`), and then submit it to Tendermint. -By including this hash in the SIGN_MODE_TEXTUAL signing payload, we keep the same level of guarantees as [SIGN_MODE_DIRECT](adr-020-protobuf-transaction-encoding.md). +By including this hash in the SIGN_MODE_TEXTUAL signing payload, we keep the same level of guarantees as [SIGN_MODE_DIRECT](./adr-020-protobuf-transaction-encoding.md). These bytes are only shown in expert mode, hence the leading `*`. @@ -314,13 +314,13 @@ Updates in the 1st category include changes of the `Screen` struct or its corres Updates in the 2nd category include changes to any of the value renderers or to the transaction envelope. For example, the ordering of fields in the envelope can be swapped, or the timestamp formatting can be modified. Since SIGN_MODE_TEXTUAL sends `Screen`s to the hardware device, this type of change do not need a hardware wallet application update. They are however state-machine-breaking, and must be documented as such. They require the coordination of SDK developers with client-side developers (e.g. CosmJS), so that the updates are released on both sides close to each other in time. -We define a spec version, which is an integer that must be incremented on each update of either category. This spec version will be exposed by the SDK's implementation, and can be communicated to clients. For example, SDK v0.48 might use the spec version 1, and SDK v0.49 might use 2; thanks to this versioning, clients can know how to craft SIGN_MODE_TEXTUAL transactions based on the target SDK version. +We define a spec version, which is an integer that must be incremented on each update of either category. This spec version will be exposed by the SDK's implementation, and can be communicated to clients. For example, SDK v0.50 might use the spec version 1, and SDK v0.51 might use 2; thanks to this versioning, clients can know how to craft SIGN_MODE_TEXTUAL transactions based on the target SDK version. The current spec version is defined in the "Status" section, on the top of this document. It is initialized to `0` to allow flexibility in choosing how to define future versions, as it would allow adding a field either in the SignDoc Go struct or in Protobuf in a backwards-compatible way. ## Additional Formatting by the Hardware Device -See [annex 2](adr-050-sign-mode-textual-annex2.md). +See [annex 2](./adr-050-sign-mode-textual-annex2.md). ## Examples @@ -328,9 +328,10 @@ See [annex 2](adr-050-sign-mode-textual-annex2.md). 2. A transaction with a bit of everything: [see transaction](https://github.com/cosmos/cosmos-sdk/blob/094abcd393379acbbd043996024d66cd65246fb1/tx/textual/internal/testdata/e2e.json#L71-L270). The examples below are stored in a JSON file with the following fields: -- `proto`: the representation of the transaction in ProtoJSON, -- `screens`: the transaction rendered into SIGN_MODE_TEXTUAL screens, -- `cbor`: the sign bytes of the transaction, which is the CBOR encoding of the screens. + +* `proto`: the representation of the transaction in ProtoJSON, +* `screens`: the transaction rendered into SIGN_MODE_TEXTUAL screens, +* `cbor`: the sign bytes of the transaction, which is the CBOR encoding of the screens. ## Consequences @@ -354,14 +355,14 @@ SIGN_MODE_TEXTUAL is purely additive, and doesn't break any backwards compatibil ## Further Discussions -* Some details on value renderers need to be polished, see [Annex 1](adr-050-sign-mode-textual-annex1.md). +* Some details on value renderers need to be polished, see [Annex 1](./adr-050-sign-mode-textual-annex1.md). * Are ledger apps able to support both SIGN_MODE_LEGACY_AMINO_JSON and SIGN_MODE_TEXTUAL at the same time? * Open question: should we add a Protobuf field option to allow app developers to overwrite the textual representation of certain Protobuf fields and message? This would be similar to Ethereum's [EIP4430](https://github.com/ethereum/EIPs/pull/4430), where the contract developer decides on the textual representation. * Internationalization. ## References -* [Annex 1](adr-050-sign-mode-textual-annex1.md) +* [Annex 1](./adr-050-sign-mode-textual-annex1.md) * Initial discussion: https://github.com/cosmos/cosmos-sdk/issues/6513 * Living document used in the working group: https://hackmd.io/fsZAO-TfT0CKmLDtfMcKeA?both diff --git a/docs/build/architecture/adr-054-semver-compatible-modules.md b/docs/build/architecture/adr-054-semver-compatible-modules.md index ed2a5f3f6..65a9a7268 100644 --- a/docs/build/architecture/adr-054-semver-compatible-modules.md +++ b/docs/build/architecture/adr-054-semver-compatible-modules.md @@ -37,7 +37,7 @@ In order to achieve this, we need to solve the following problems: many modules in the SDK independently 3. pernicious minor version incompatibilities introduced through correctly [evolving protobuf schemas](https://developers.google.com/protocol-buffers/docs/proto3#updating) - without correct [unknown field filtering](adr-020-protobuf-transaction-encoding.md#unknown-field-filtering) + without correct [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering) Note that all the following discussion assumes that the proto file versioning and state machine versioning of a module are distinct in that: @@ -145,7 +145,7 @@ with this update and use that for `foo/v2`. But this change is state machine breaking for `v1`. It requires changing the `ValidateBasic` method to reject the case where `amount` is zero, and it adds the `condition` field which should be rejected based -on [ADR 020 unknown field filtering](adr-020-protobuf-transaction-encoding.md#unknown-field-filtering). +on [ADR 020 unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering). So adding these changes as a patch on `v1` is actually incorrect based on semantic versioning. Chains that want to stay on `v1` of `foo` should not be importing these changes because they are incorrect for `v1.` @@ -174,9 +174,9 @@ on `v1` or `v2` and dynamically, it could choose to only use `condition` when `foo/v2` is available. Even if `bar/v2` were able to perform this check, however, how do we know that it is always performing the check properly. Without some sort of -framework-level [unknown field filtering](adr-020-protobuf-transaction-encoding.md#unknown-field-filtering), +framework-level [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering), it is hard to know whether these pernicious hard to detect bugs are getting into -our app and a client-server layer such as [ADR 033: Inter-Module Communication](adr-033-protobuf-inter-module-comm.md) +our app and a client-server layer such as [ADR 033: Inter-Module Communication](./adr-033-protobuf-inter-module-comm.md) may be needed to do this. ## Solutions @@ -265,9 +265,9 @@ of care to avoid these sorts of issues. This approach in and of itself does little to address any potential minor version incompatibilities and the -requisite [unknown field filtering](adr-020-protobuf-transaction-encoding.md#unknown-field-filtering). +requisite [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering). Likely some sort of client-server routing layer which does this check such as -[ADR 033: Inter-Module communication](adr-033-protobuf-inter-module-comm.md) +[ADR 033: Inter-Module communication](./adr-033-protobuf-inter-module-comm.md) is required to make sure that this is done properly. We could then allow modules to perform a runtime check given a `MsgClient`, ex: @@ -290,7 +290,7 @@ result in an undesirable performance hit depending on how complex this logic is. An alternate approach to solving the versioning problem is to change how protobuf code is generated and move modules mostly or completely in the direction of inter-module communication as described -in [ADR 033](adr-033-protobuf-inter-module-comm.md). +in [ADR 033](./adr-033-protobuf-inter-module-comm.md). In this paradigm, a module could generate all the types it needs internally - including the API types of other modules - and talk to other modules via a client-server boundary. For instance, if `bar` needs to talk to `foo`, it could generate its own version of `MsgDoSomething` as `bar/internal/foo/v1.MsgDoSomething` and just pass this to the @@ -309,7 +309,7 @@ to `foo/internal.MsgDoSomething` would be marshaling and unmarshaling in the ADR we needed to expose protobuf types in `Keeper` interfaces because the whole point is to try to keep these types `internal/` so that we don't end up with all the import version incompatibilities we've described above. However, because of the issue with minor version incompatibilities and the need -for [unknown field filtering](adr-020-protobuf-transaction-encoding.md#unknown-field-filtering), +for [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering), sticking with the `Keeper` paradigm instead of ADR 033 may be unviable to begin with. A more performant solution (that could maybe be adapted to work with `Keeper` interfaces) would be to only expose @@ -383,7 +383,7 @@ and would also need to use special tags and replace directives to make sure that versions. Note, however, that all of these ad-hoc approaches, would be vulnerable to the minor version compatibility issues -described above unless [unknown field filtering](adr-020-protobuf-transaction-encoding.md#unknown-field-filtering) +described above unless [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering) is properly addressed. ### Approach D) Avoid protobuf generated code in public APIs @@ -424,6 +424,7 @@ way that requires duplication and differing sets of design principles (protobuf while golang APIs would forbid it). Other downsides to this approach are: + * no clear roadmap to supporting modules in other languages like Rust * doesn't get us any closer to proper object capability security (one of the goals of ADR 033) * ADR 033 needs to be done properly anyway for the set of use cases which do need it @@ -432,7 +433,7 @@ Other downsides to this approach are: The latest **DRAFT** proposal is: -1. we are alignment on adopting [ADR 033](adr-033-protobuf-inter-module-comm.md) not just as an addition to the +1. we are alignment on adopting [ADR 033](./adr-033-protobuf-inter-module-comm.md) not just as an addition to the framework, but as a core replacement to the keeper paradigm entirely. 2. the ADR 033 inter-module router will accommodate any variation of approach (A) or (B) given the following rules: a. if the client type is the same as the server type then pass it directly through, @@ -446,7 +447,8 @@ languages, possibly executed within a WASM VM. ### Minor API Revisions To declare minor API revisions of proto files, we propose the following guidelines (which were already documented -in [cosmos.app.v1alpha module options](../proto/cosmos/app/v1alpha1/module.proto)): +in [cosmos.app.v1alpha module options](../../proto/cosmos/app/v1alpha1/module.proto)): + * proto packages which are revised from their initial version (considered revision `0`) should include a `package` * comment in some .proto file containing the test `Revision N` at the start of a comment line where `N` is the current revision number. @@ -489,7 +491,7 @@ type MsgClient interface { ### Unknown Field Filtering -To correctly perform [unknown field filtering](adr-020-protobuf-transaction-encoding.md#unknown-field-filtering), +To correctly perform [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering), the inter-module router can do one of the following: * use the `protoreflect` API for messages which support that @@ -522,6 +524,7 @@ func ProtoImage(protoImage []byte) Option {} ``` This approach allows us to support several ways protobuf files might be generated: + * proto files generated internally to a module (use `ProtoFiles`) * the API module approach with pinned file descriptors (use `ProtoImage`) * gogo proto (use `GzippedProtoFiles`) @@ -541,7 +544,7 @@ We propose defining these dependencies in the proto options of the module config We will also need to define how interface methods are defined on types that are serialized as `google.protobuf.Any`'s. In light of the desire to support modules in other languages, we may want to think of solutions that will accommodate -other languages such as plugins described briefly in [ADR 033](adr-033-protobuf-inter-module-comm.md#internal-methods). +other languages such as plugins described briefly in [ADR 033](./adr-033-protobuf-inter-module-comm.md#internal-methods). ### Testing @@ -724,5 +727,5 @@ Key outstanding discussions if we do adopt that direction are: * https://github.com/cosmos/cosmos-sdk/discussions/10368 * https://github.com/cosmos/cosmos-sdk/pull/11340 * https://github.com/cosmos/cosmos-sdk/issues/11899 -* [ADR 020](adr-020-protobuf-transaction-encoding.md) -* [ADR 033](adr-033-protobuf-inter-module-comm.md) +* [ADR 020](./adr-020-protobuf-transaction-encoding.md) +* [ADR 033](./adr-033-protobuf-inter-module-comm.md) diff --git a/docs/build/architecture/adr-057-app-wiring.md b/docs/build/architecture/adr-057-app-wiring.md index 0a23f3933..2e37ced66 100644 --- a/docs/build/architecture/adr-057-app-wiring.md +++ b/docs/build/architecture/adr-057-app-wiring.md @@ -22,13 +22,13 @@ which contains almost 100 lines of imports and is otherwise over 600 lines of mo generally copied to each new project. (Not to mention the additional boilerplate which gets copied in `simapp/simd`.) The large amount of boilerplate needed to bootstrap an app has made it hard to release independently versioned go -modules for Cosmos SDK modules as described in [ADR 053: Go Module Refactoring](adr-053-go-module-refactoring.md). +modules for Cosmos SDK modules as described in [ADR 053: Go Module Refactoring](./adr-053-go-module-refactoring.md). In addition to being very verbose and repetitive, `app.go` also exposes a large surface area for breaking changes as most modules instantiate themselves with positional parameters which forces breaking changes anytime a new parameter (even an optional one) is needed. -Several attempts were made to improve the current situation including [ADR 033: Internal-Module Communication](adr-033-protobuf-inter-module-comm.md) +Several attempts were made to improve the current situation including [ADR 033: Internal-Module Communication](./adr-033-protobuf-inter-module-comm.md) and [a proof-of-concept of a new SDK](https://github.com/allinbits/cosmos-sdk-poc). The discussions around these designs led to the current solution described here. @@ -149,7 +149,7 @@ to decode the app config in separate phases: 3. decode the app config as proto JSON using the protobuf type registry Because in [ADR 054: Protobuf Semver Compatible Codegen](https://github.com/cosmos/cosmos-sdk/pull/11802), each module -should use `internal` generated code which is not registered with the global protobuf registry, this code should provide +might use `internal` generated code which is not registered with the global protobuf registry, this code should provide an alternate way to register protobuf types with a type registry. In the same way that `.pb.go` files currently have a `var File_foo_proto protoreflect.FileDescriptor` for the file `foo.proto`, generated code should have a new member `var Types_foo_proto TypeInfo` where `TypeInfo` is an interface or struct with all the necessary info to register both @@ -246,7 +246,7 @@ func main() { So far we have described a system which is largely agnostic to the specifics of the SDK such as store keys, `AppModule`, `BaseApp`, etc. Improvements to these parts of the framework that integrate with the general app wiring framework -defined here are described in [ADR 061: Core Module API](./adr-063-core-module-api.md). +defined here are described in [ADR 063: Core Module API](./adr-063-core-module-api.md). ### Registration of Inter-Module Hooks @@ -299,6 +299,35 @@ generated. This will allow: Code generation requires that all providers and invokers and their parameters are exported and in non-internal packages. +### Module Semantic Versioning + +When we start creating semantically versioned SDK modules that are in standalone go modules, a state machine breaking +change to a module should be handled as follows: +- the semantic major version should be incremented, and +- a new semantically versioned module config protobuf type should be created. + +For instance, if we have the SDK module for bank in the go module `cosmossdk.io/x/bank` with the module config type +`cosmos.bank.module.v1.Module`, and we want to make a state machine breaking change to the module, we would: +- create a new go module `cosmossdk.io/x/bank/v2`, +- with the module config protobuf type `cosmos.bank.module.v2.Module`. + +This _does not_ mean that we need to increment the protobuf API version for bank. Both modules can support +`cosmos.bank.v1`, but `cosmossdk.io/x/bank/v2` will be a separate go module with a separate module config type. + +This practice will eventually allow us to use appconfig to load new versions of a module via a configuration change. + +Effectively, there should be a 1:1 correspondence between a semantically versioned go module and a +versioned module config protobuf type, and major versioning bumps should occur whenever state machine breaking changes +are made to a module. + +NOTE: SDK modules that are standalone go modules _should not_ adopt semantic versioning until the concerns described in +[ADR 054: Module Semantic Versioning](./adr-054-semver-compatible-modules.md) are +addressed. The short-term solution for this issue was left somewhat unresolved. However, the easiest tactic is +likely to use a standalone API go module and follow the guidelines described in this comment: https://github.com/cosmos/cosmos-sdk/pull/11802#issuecomment-1406815181. For the time-being, it is recommended that +Cosmos SDK modules continue to follow tried and true [0-based versioning](https://0ver.org) until an officially +recommended solution is provided. This section of the ADR will be updated when that happens and for now, this section +should be considered as a design recommendation for future adoption of semantic versioning. + ## Consequences ### Backwards Compatibility @@ -337,4 +366,4 @@ light of code generation. It may be better to do this type registration with a D * https://github.com/google/wire * https://pkg.go.dev/github.com/cosmos/cosmos-sdk/container * https://github.com/cosmos/cosmos-sdk/pull/11802 -* [ADR 063](./adr-063-core-module-api.md) +* [ADR 063: Core Module API](./adr-063-core-module-api.md) diff --git a/docs/build/architecture/adr-059-test-scopes.md b/docs/build/architecture/adr-059-test-scopes.md index 060344593..d4ec5519c 100644 --- a/docs/build/architecture/adr-059-test-scopes.md +++ b/docs/build/architecture/adr-059-test-scopes.md @@ -56,7 +56,7 @@ Tests which exercise a whole module's function with dependencies mocked, are *jo These are almost like integration tests in that they exercise many things together but still use mocks. -Example 1 journey vs illustrative tests - [depinject's BDD style tests](https://github.com/cosmos/cosmos-sdk/blob/main/depinject/features/bindings.feature), show how we can +Example 1 journey vs illustrative tests - depinject's BDD style tests, show how we can rapidly build up many illustrative cases demonstrating behavioral rules without [very much code](https://github.com/cosmos/cosmos-sdk/blob/main/depinject/binding_test.go) while maintaining high level readability. Example 2 [depinject table driven tests](https://github.com/cosmos/cosmos-sdk/blob/main/depinject/provider_desc_test.go) diff --git a/docs/build/architecture/adr-063-core-module-api.md b/docs/build/architecture/adr-063-core-module-api.md index 034b01fab..743dd5fee 100644 --- a/docs/build/architecture/adr-063-core-module-api.md +++ b/docs/build/architecture/adr-063-core-module-api.md @@ -20,7 +20,7 @@ A new core API is proposed as a way to develop cosmos-sdk applications that will * more stable than the current framework * enable deterministic events and queries, * support event listeners -* [ADR 033: Protobuf-based Inter-Module Communication](adr-033-protobuf-inter-module-comm.md) clients. +* [ADR 033: Protobuf-based Inter-Module Communication](./adr-033-protobuf-inter-module-comm.md) clients. ## Context @@ -90,7 +90,7 @@ slower than more fast moving projects. ### Core Services The following "core services" are defined by the core API. All valid runtime module implementations should provide -implementations of these services to modules via both [dependency injection](adr-057-app-wiring.md) and +implementations of these services to modules via both [dependency injection](./adr-057-app-wiring.md) and manual wiring. The individual services described below are all bundled in a convenient `appmodule.Service` "bundle service" so that for simplicity modules can declare a dependency on a single service. @@ -256,7 +256,7 @@ type GenesisTarget = func(field string) (io.WriteCloser, error) All genesis objects for a given module are expected to conform to the semantics of a JSON object. Each field in the JSON object should be read and written separately to support streaming genesis. -The [ORM](adr-055-orm.md) and [collections](adr-062-collections-state-layer.md) both support +The [ORM](./adr-055-orm.md) and [collections](./adr-062-collections-state-layer.md) both support streaming genesis and modules using these frameworks generally do not need to write any manual genesis code. @@ -280,6 +280,17 @@ type HasGenesis interface { } ``` +#### Pre Blockers + +Modules that have functionality that runs before BeginBlock and should implement the has `HasPreBlocker` interfaces: + +```go +type HasPreBlocker interface { + AppModule + PreBlock(context.Context) error +} +``` + #### Begin and End Blockers Modules that have functionality that runs before transactions (begin blockers) or after transactions @@ -377,7 +388,7 @@ Additional `AppModule` extension interfaces either inside or outside of core wil these concerns. In the case of gogo proto and amino interfaces, the registration of these generally should happen as early -as possible during initialization and in [ADR 057: App Wiring](./adr-057-app-wiring.md), protobuf type registration +as possible during initialization and in [ADR 057: App Wiring](./adr-057-app-wiring-1.md), protobuf type registration happens before dependency injection (although this could alternatively be done dedicated DI providers). gRPC gateway registration should probably be handled by the runtime module, but the core API shouldn't depend on gRPC @@ -407,11 +418,11 @@ Crisis module invariants and simulations are subject to potential redesign and s defined in the crisis and simulation modules respectively. Extension interface for CLI commands will be provided via the `cosmossdk.io/client/v2` module and its -[autocli](adr-058-auto-generated-cli.md) framework. +[autocli](./adr-058-auto-generated-cli.md) framework. #### Example Usage -Here is an example of setting up a hypothetical `foo` v2 module which uses the [ORM](adr-055-orm.md) for its state +Here is an example of setting up a hypothetical `foo` v2 module which uses the [ORM](./adr-055-orm.md) for its state management and genesis. ```go @@ -444,6 +455,51 @@ should check this compatibility version and return an error if the current `Runt than the version of the core API that this runtime version can support. When new features are adding to the `core` module API that runtime modules are required to support, this version should be incremented. +### Runtime Modules + +The initial `runtime` module will simply be created within the existing `github.com/cosmos/cosmos-sdk` go module +under the `runtime` package. This module will be a small wrapper around the existing `BaseApp`, `sdk.Context` and +module manager and follow the Cosmos SDK's existing [0-based versioning](https://0ver.org). To move to semantic +versioning as well as runtime modularity, new officially supported runtime modules will be created under the +`cosmossdk.io/runtime` prefix. For each supported consensus engine a semantically-versioned go module should be created +with a runtime implementation for that consensus engine. For example: +- `cosmossdk.io/runtime/comet` +- `cosmossdk.io/runtime/comet/v2` +- `cosmossdk.io/runtime/rollkit` +- etc. + +These runtime modules should attempt to be semantically versioned even if the underlying consensus engine is not. Also, +because a runtime module is also a first class Cosmos SDK module, it should have a protobuf module config type. +A new semantically versioned module config type should be created for each of these runtime module such that there is a +1:1 correspondence between the go module and module config type. This is the same practice should be followed for every +semantically versioned Cosmos SDK module as described in [ADR 057: App Wiring](./adr-057-app-wiring.md). + +Currently, `github.com/cosmos/cosmos-sdk/runtime` uses the protobuf config type `cosmos.app.runtime.v1alpha1.Module`. +When we have a standalone v1 comet runtime, we should use a dedicated protobuf module config type such as +`cosmos.runtime.comet.v1.Module1`. When we release v2 of the comet runtime (`cosmossdk.io/runtime/comet/v2`) we should +have a corresponding `cosmos.runtime.comet.v2.Module` protobuf type. + +In order to make it easier to support different consensus engines that support the same core module functionality as +described in this ADR, a common go module should be created with shared runtime components. The easiest runtime components +to share initially are probably the message/query router, inter-module client, service register, and event router. +This common runtime module should be created initially as the `cosmossdk.io/runtime/common` go module. + +When this new architecture has been implemented, the main dependency for a Cosmos SDK module would be +`cosmossdk.io/core` and that module should be able to be used with any supported consensus engine (to the extent +that it does not explicitly depend on consensus engine specific functionality such as Comet's block headers). An +app developer would then be able to choose which consensus engine they want to use by importing the corresponding +runtime module. The current `BaseApp` would be refactored into the `cosmossdk.io/runtime/comet` module, the router +infrastructure in `baseapp/` would be refactored into `cosmossdk.io/runtime/common` and support ADR 033, and eventually +a dependency on `github.com/cosmos/cosmos-sdk` would no longer be required. + +In short, modules would depend primarily on `cosmossdk.io/core`, and each `cosmossdk.io/runtime/{consensus-engine}` +would implement the `cosmossdk.io/core` functionality for that consensus engine. + +On additional piece that would need to be resolved as part of this architecture is how runtimes relate to the server. +Likely it would make sense to modularize the current server architecture so that it can be used with any runtime even +if that is based on a consensus engine besides Comet. This means that eventually the Comet runtime would need to +encapsulate the logic for starting Comet and the ABCI app. + ### Testing A mock implementation of all services should be provided in core to allow for unit testing of modules @@ -500,8 +556,8 @@ as by providing service implementations by wrapping `sdk.Context`. ## References -* [ADR 033: Protobuf-based Inter-Module Communication](adr-033-protobuf-inter-module-comm.md) -* [ADR 057: App Wiring](./adr-057-app-wiring.md) -* [ADR 055: ORM](adr-055-orm.md) -* [ADR 028: Public Key Addresses](adr-028-public-key-addresses.md) +* [ADR 033: Protobuf-based Inter-Module Communication](./adr-033-protobuf-inter-module-comm.md) +* [ADR 057: App Wiring](./adr-057-app-wiring-1.md) +* [ADR 055: ORM](./adr-055-orm.md) +* [ADR 028: Public Key Addresses](./adr-028-public-key-addresses.md) * [Keeping Your Modules Compatible](https://go.dev/blog/module-compatibility) diff --git a/docs/build/architecture/adr-064-abci-2.0.md b/docs/build/architecture/adr-064-abci-2.0.md index 9a4718050..c0dc7f746 100644 --- a/docs/build/architecture/adr-064-abci-2.0.md +++ b/docs/build/architecture/adr-064-abci-2.0.md @@ -5,6 +5,8 @@ * 2023-01-17: Initial Draft (@alexanderbez) * 2023-04-06: Add upgrading section (@alexanderbez) * 2023-04-10: Simplify vote extension state persistence (@alexanderbez) +* 2023-07-07: Revise vote extension state persistence (@alexanderbez) +* 2023-08-24: Revise vote extension power calculations and staking interface (@davidterpay) ## Status @@ -105,10 +107,11 @@ type ExtendVoteHandler func(sdk.Context, abci.RequestExtendVote) abci.ResponseEx type VerifyVoteExtensionHandler func(sdk.Context, abci.RequestVerifyVoteExtension) abci.ResponseVerifyVoteExtension ``` -A new execution state, `voteExtensionState`, will be introduced and provided as -the `Context` that is supplied to both handlers. It will contain relevant metadata -such as the block height and block hash. Note, `voteExtensionState` is never -committed and will exist as ephemeral state only in the context of a single block. +An ephemeral context and state will be supplied to both handlers. The +context will contain relevant metadata such as the block height and block hash. +The state will be a cached version of the committed state of the application and +will be discarded after the execution of the handler, this means that both handlers +get a fresh state view and no changes made to it will be written. If an application decides to implement `ExtendVoteHandler`, it must return a non-nil `ResponseExtendVote.VoteExtension`. @@ -217,33 +220,37 @@ a default signature verification method which applications can use: ```go type ValidatorStore interface { - GetValidatorByConsAddr(sdk.Context, cryptotypes.Address) (cryptotypes.PubKey, error) + GetPubKeyByConsAddr(context.Context, sdk.ConsAddress) (cmtprotocrypto.PublicKey, error) } // ValidateVoteExtensions is a function that an application can execute in // ProcessProposal to verify vote extension signatures. func (app *BaseApp) ValidateVoteExtensions(ctx sdk.Context, currentHeight int64, extCommit abci.ExtendedCommitInfo) error { + votingPower := 0 + totalVotingPower := 0 + for _, vote := range extCommit.Votes { + totalVotingPower += vote.Validator.Power + if !vote.SignedLastBlock || len(vote.VoteExtension) == 0 { continue } - valConsAddr := cmtcrypto.Address(vote.Validator.Address) - - validator, err := app.validatorStore.GetValidatorByConsAddr(ctx, valConsAddr) - if err != nil { - return fmt.Errorf("failed to get validator %s for vote extension", valConsAddr) - } - - cmtPubKey, err := validator.CmtConsPublicKey() + valConsAddr := sdk.ConsAddress(vote.Validator.Address) + pubKeyProto, err := valStore.GetPubKeyByConsAddr(ctx, valConsAddr) if err != nil { - return fmt.Errorf("failed to convert public key: %w", err) + return fmt.Errorf("failed to get public key for validator %s: %w", valConsAddr, err) } if len(vote.ExtensionSignature) == 0 { return fmt.Errorf("received a non-empty vote extension with empty signature for validator %s", valConsAddr) } + cmtPubKey, err := cryptoenc.PubKeyFromProto(pubKeyProto) + if err != nil { + return fmt.Errorf("failed to convert validator %X public key: %w", valConsAddr, err) + } + cve := cmtproto.CanonicalVoteExtension{ Extension: vote.VoteExtension, Height: currentHeight - 1, // the vote extension was signed in the previous height @@ -260,8 +267,14 @@ func (app *BaseApp) ValidateVoteExtensions(ctx sdk.Context, currentHeight int64, return errors.New("received vote with invalid signature") } - return nil + votingPower += vote.Validator.Power } + + if (votingPower / totalVotingPower) < threshold { + return errors.New("not enough voting power for the vote extensions") + } + + return nil } ``` @@ -279,32 +292,22 @@ decision based on the vote extensions. #### Vote Extension Persistence In certain contexts, it may be useful or necessary for applications to persist -data derived from vote extensions. In order to facilitate this use case, we -propose to allow application developers to manually retrieve the `finalizeState` -context (see [`FinalizeBlock`](#finalizeblock-1) below). Using this context, -state can be directly written to `finalizeState`, which will be used during -`FinalizeBlock` and eventually committed to the application state. Note, since -`ProcessProposal` can timeout and thus require another round of consensus, we -will reset `finalizeState` in the beginning of `ProcessProposal`. +data derived from vote extensions. In order to facilitate this use case, we propose +to allow app developers to define a pre-Blocker hook which will be called +at the very beginning of `FinalizeBlock`, i.e. before `BeginBlock` (see below). -A `ProcessProposal` handler could look like the following: +Note, we cannot allow applications to directly write to the application state +during `ProcessProposal` because during replay, CometBFT will NOT call `ProcessProposal`, +which would result in an incomplete state view. ```go -func (h MyHandler) ProcessProposalHandler() sdk.ProcessProposalHandler { - return func(ctx sdk.Context, req abci.RequestProcessProposal) abci.ResponseProcessProposal { - for _, txBytes := range req.Txs { - _, err := h.app.ProcessProposalVerifyTx(txBytes) - if err != nil { - return abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT} - } - } - - fCtx := h.app.GetFinalizeState() - - // Any state changes that occur on the provided fCtx WILL be written to state! - h.myKeeper.SetVoteExtResult(fCtx, ...) +func (a MyApp) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeBlock) error { + voteExts := GetVoteExtensions(ctx, req.Txs) - return abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_ACCEPT} + // Process and perform some compute on vote extensions, storing any resulting + // state. + if err a.processVoteExtensions(ctx, voteExts); if err != nil { + return err } } ``` @@ -347,11 +350,20 @@ legacy ABCI types, e.g. `LegacyBeginBlockRequest` and `LegacyEndBlockRequest`. O we can come up with new types and names altogether. ```go -func (app *BaseApp) FinalizeBlock(req abci.RequestFinalizeBlock) abci.ResponseFinalizeBlock { - // merge any state changes from ProcessProposal into the FinalizeBlock state - app.MergeProcessProposalState() +func (app *BaseApp) FinalizeBlock(req abci.RequestFinalizeBlock) (*abci.ResponseFinalizeBlock, error) { + ctx := ... - beginBlockResp := app.beginBlock(ctx, req) + if app.preBlocker != nil { + ctx := app.finalizeBlockState.ctx + rsp, err := app.preBlocker(ctx, req) + if err != nil { + return nil, err + } + if rsp.ConsensusParamsChanged { + app.finalizeBlockState.ctx = ctx.WithConsensusParams(app.GetConsensusParams(ctx)) + } + } + beginBlockResp, err := app.beginBlock(req) appendBlockEventAttr(beginBlockResp.Events, "begin_block") txExecResults := make([]abci.ExecTxResult, 0, len(req.Txs)) @@ -360,7 +372,7 @@ func (app *BaseApp) FinalizeBlock(req abci.RequestFinalizeBlock) abci.ResponseFi txExecResults = append(txExecResults, result) } - endBlockResp := app.endBlock(ctx, req) + endBlockResp, err := app.endBlock(app.finalizeBlockState.ctx) appendBlockEventAttr(beginBlockResp.Events, "end_block") return abci.ResponseFinalizeBlock{ diff --git a/docs/build/architecture/adr-065-store-v2.md b/docs/build/architecture/adr-065-store-v2.md index 67c1830fd..8faed0463 100644 --- a/docs/build/architecture/adr-065-store-v2.md +++ b/docs/build/architecture/adr-065-store-v2.md @@ -64,9 +64,9 @@ See the [Storage Discussion](https://github.com/cosmos/cosmos-sdk/discussions/13 ## Alternatives -There was a previous attempt to refactor the storage layer described in [ADR-040](adr-040-storage-and-smt-state-commitments.md). +There was a previous attempt to refactor the storage layer described in [ADR-040](./adr-040-storage-and-smt-state-commitments.md). However, this approach mainly stems on the short comings of IAVL and various performance -issues around it. While there was a (partial) implementation of [ADR-040](adr-040-storage-and-smt-state-commitments.md), +issues around it. While there was a (partial) implementation of [ADR-040](./adr-040-storage-and-smt-state-commitments.md), it was never adopted for a variety of reasons, such as the reliance on using an SMT, which was more in a research phase, and some design choices that couldn't be fully agreed upon, such as the snap-shotting mechanism that would result in @@ -74,7 +74,7 @@ massive state bloat. ## Decision -We propose to build upon some of the great ideas introduced in [ADR-040](adr-040-storage-and-smt-state-commitments.md), +We propose to build upon some of the great ideas introduced in [ADR-040](./adr-040-storage-and-smt-state-commitments.md), while being a bit more flexible with the underlying implementations and overall less intrusive. Specifically, we propose to: diff --git a/docs/build/architecture/adr-067-simulator-v2.md b/docs/build/architecture/adr-067-simulator-v2.md new file mode 100644 index 000000000..9398aaef6 --- /dev/null +++ b/docs/build/architecture/adr-067-simulator-v2.md @@ -0,0 +1,194 @@ +# ADR 067: Simulator v2 + +## Changelog + +* June 01, 2023: Initial Draft (@alexanderbez) + +## Status + +DRAFT + +## Abstract + +The Cosmos SDK simulator is a tool that allows developers to test the entirety +of their application's state machine through the use of pseudo-randomized "operations", +which represent transactions. The simulator also provides primitives that ensures +there are no non-determinism issues and that the application's state machine can +be successfully exported and imported using randomized state. + +The simulator has played an absolutely critical role in the development and testing +of the Cosmos Hub and all the releases of the Cosmos SDK after the launch of the +Cosmos Hub. Since the Hub, the simulator has relatively not changed much, so it's +overdue for a revamp. + +## Context + +The current simulator, `x/simulation`, acts as a semi-fuzz testing suite that takes +in an integer that represents a seed into a PRNG. The PRNG is used to generate a +sequence of "operations" that are meant to reflect transactions that an application's +state machine can process. Through the use of the PRNG, all aspects of block production +and consumption are randomized. This includes a block's proposer, the validators +who both sign and miss the block, along with the transaction operations themselves. + +Each Cosmos SDK module defines a set of simulation operations that _attempt_ to +produce valid transactions, e.g. `x/distribution/simulation/operations.go`. These +operations can sometimes fail depending on the accumulated state of the application +within that simulation run. The simulator will continue to generate operations +until it has reached a certain number of operations or until it has reached a +fatal state, reporting results. This gives the ability for application developers +to reliably execute full range application simulation and fuzz testing against +their application. + +However, there are a few major drawbacks. Namely, with the advent of ABCI++, specifically +`FinalizeBlock`, the internal workings of the simulator no longer comply with how +an application would actually perform. Specifically, operations are executed +_after_ `FinalizeBlock`, whereas they should be executed _within_ `FinalizeBlock`. + +Additionally, the simulator is not very extensible. Developers should be able to +easily define and extend the following: + +* Consistency or validity predicates (what are known as invariants today) +* Property tests of state before and after a block is simulated + +In addition, we also want to achieve the following: + +* Consolidated weight management, i.e. define weights within the simulator itself + via a config and not defined in each module +* Observability of the simulator's execution, i.e. have easy to understand output/logs + with the ability to pipe those logs into some external sink +* Smart replay, i.e. the ability to not only rerun a simulation from a seed, but + also the ability to replay from an arbitrary breakpoint +* Run a simulation based off of real network state + +## Decision + +Instead of refactoring the existing simulator, `x/simulation`, we propose to create +a new package in the root of the Cosmos SDK, `simulator`, that will be the new +simulation framework. The simulator will more accurately reflect the complete +lifecycle of an ABCI application. + +Specifically, we propose a similar implementation and use of a `simulator.Manager`, +that exists today, that is responsible for managing the execution of a simulation. +The manager will wrap an ABCI application and will be responsible for the following: + +* Populating the application's mempool with a set of pseudo-random transactions + before each block, some of which may contain invalid messages. +* Selecting transactions and a random proposer to execute `PrepareProposal`. +* Executing `ProcessProposal`, `FinalizeBlock` and `Commit`. +* Executing a set of validity predicates before and after each block. +* Maintaining a CPU and RAM profile of the simulation execution. +* Allowing a simulation to stop and resume from a given block height. +* Simulation liveness of each validator per-block. + +From an application developer's perspective, they will only need to provide the +modules to be used in the simulator and the manager will take care of the rest. +In addition, they will not need to write their own simulation test(s), e.g. +non-determinism, multi-seed, etc..., as the manager will provide these as well. + +```go +type Manager struct { + app sdk.Application + mempool sdk.Mempool + rng rand.Rand + // ... +} +``` + +### Configuration + +The simulator's testing input will be driven by a configuration file, as opposed +to CLI arguments. This will allow for more extensibility and ease of use along with +the ability to have shared configuration files across multiple simulations. + +### Execution + +As alluded to previously, after the execution of each block, the manager will +generate a series of pseudo-random transactions and attempt to insert them into +the mempool via `BaseApp#CheckTx`. During the ABCI lifecycle of a block, this +mempool will be used to seed the transactions into a block proposal as it would +in a real network. This allows us to not only test the state machine, but also +test the ABCI lifecycle of a block. + +Statistics, such as total blocks and total failed proposals, will be collected, +logged and written to output after the full or partial execution of a simulation. +The output destination of these statistics will be configurable. + +```go +func (s *Simulator) SimulateBlock() { + rProposer := s.SelectRandomProposer() + rTxs := s.SelectTxs() + + prepareResp, err := s.app.PrepareProposal(&abci.RequestPrepareProposal{Txs: rTxs}) + // handle error + + processResp, err := s.app.ProcessProposal(&abci.RequestProcessProposal{ + Txs: prepareResp.Txs, + // ... + }) + // handle error + + // execute liveness matrix... + + _, err = s.app.FinalizeBlock(...) + // handle error + + _, err = s.app.Commit(...) + // handle error +} +``` + +Note, some application do not define or need their own app-side mempool, so we +propose that `SelectTxs` mimic CometBFT and just return FIFO-ordered transactions +from an ad-hoc simulator mempool. In the case where an application does define +its own mempool, it will simply ignore what is provided in `RequestPrepareProposal`. + +### Profiling + +The manager will be responsible for collecting CPU and RAM profiles of the simulation +execution. We propose to use [Pyroscope](https://pyroscope.io/docs/golang/) to +capture profiles and export them to a local file and via an HTTP endpoint. This +can be disabled or enabled by configuration. + +### Breakpoints + +Via configuration, a caller can express a height-based breakpoint that will allow +the simulation to stop and resume from a given height. This will allow for debugging +of CPU, RAM, and state. + +### Validity Predicates + +We propose to provide the ability for an application to provide the simulator a +set of validity predicates, i.e. invariant checkers, that will be executed before +and after each block. This will allow for the application to assert that certain +state invariants are held before and after each block. Note, as a consequence of +this, we propose to remove the existing notion of invariants from module production +execution paths and deprecate their usage all together. + +```go +type Manager struct { + // ... + preBlockValidator func(sdk.Context) error + postBlockValidator func(sdk.Context) error +} +``` + +## Consequences + +### Backwards Compatibility + +The new simulator package will not naturally not be backwards compatible with the +existing `x/simulation` module. However, modules will still be responsible for +providing pseudo-random transactions to the simulator. + +### Positive + +* Providing more intuitive and cleaner APIs for application developers +* More closely resembling the true lifecycle of an ABCI application + +### Negative + +* Breaking current Cosmos SDK module APIs for transaction generation + +## References + +* [Osmosis Simulation ADR](https://github.com/osmosis-labs/osmosis/blob/main/simulation/ADR.md) diff --git a/docs/build/architecture/adr-068-preblock.md b/docs/build/architecture/adr-068-preblock.md new file mode 100644 index 000000000..86692c412 --- /dev/null +++ b/docs/build/architecture/adr-068-preblock.md @@ -0,0 +1,61 @@ +# ADR 068: Preblock + +## Changelog + +* Sept 13, 2023: Initial Draft + +## Status + +DRAFT + +## Abstract + +Introduce `PreBlock`, which runs before begin blocker other modules, and allows to modify consensus parameters, and the changes are visible to the following state machine logics. + +## Context + +When upgrading to sdk 0.47, the storage format for consensus parameters changed, but in the migration block, `ctx.ConsensusParams()` is always `nil`, because it fails to load the old format using new code, it's supposed to be migrated by the `x/upgrade` module first, but unfortunately, the migration happens in `BeginBlocker` handler, which runs after the `ctx` is initialized. +When we try to solve this, we find the `x/upgrade` module can't modify the context to make the consensus parameters visible for the other modules, the context is passed by value, and sdk team want to keep it that way, that's good for isolations between modules. + +## Alternatives + +The first alternative solution introduced a `MigrateModuleManager`, which only includes the `x/upgrade` module right now, and baseapp will run their `BeginBlocker`s before the other modules, and reload context's consensus parameters in between. + +## Decision + +Suggested this new lifecycle method. + +### `PreBlocker` + +There are two semantics around the new lifecycle method: + +- It runs before the `BeginBlocker` of all modules +- It can modify consensus parameters in storage, and signal the caller through the return value. + +When it returns `ConsensusParamsChanged=true`, the caller must refresh the consensus parameter in the finalize context: +``` +app.finalizeBlockState.ctx = app.finalizeBlockState.ctx.WithConsensusParams(app.GetConsensusParams()) +``` + +The new ctx must be passed to all the other lifecycle methods. + + +## Consequences + +### Backwards Compatibility + +### Positive + +### Negative + +### Neutral + +## Further Discussions + +## Test Cases + +## References +* [1] https://github.com/cosmos/cosmos-sdk/issues/16494 +* [2] https://github.com/cosmos/cosmos-sdk/pull/16583 +* [3] https://github.com/cosmos/cosmos-sdk/pull/17421 +* [4] https://github.com/cosmos/cosmos-sdk/pull/17713 diff --git a/docs/build/architecture/adr-template.md b/docs/build/architecture/adr-template.md index dcf07343c..04b0450c6 100644 --- a/docs/build/architecture/adr-template.md +++ b/docs/build/architecture/adr-template.md @@ -8,7 +8,7 @@ {DRAFT | PROPOSED} Not Implemented -> Please have a look at the [PROCESS](PROCESS.md#adr-status) page. +> Please have a look at the [PROCESS](./PROCESS.md#adr-status) page. > Use DRAFT if the ADR is in a draft stage (draft PR) or PROPOSED if it's in review. ## Abstract diff --git a/docs/build/building-apps/02-app-mempool.md b/docs/build/building-apps/02-app-mempool.md index 1f591fc6d..630b94ba7 100644 --- a/docs/build/building-apps/02-app-mempool.md +++ b/docs/build/building-apps/02-app-mempool.md @@ -15,7 +15,7 @@ Notably it introduces the `PrepareProposal` and `ProcessProposal` steps of ABCI+ :::note Pre-requisite Readings -* [BaseApp](../../develop/advanced/00-baseapp.md) +* [BaseApp](../../learn/advanced/00-baseapp.md) ::: diff --git a/docs/build/building-apps/04-vote-extensions.md b/docs/build/building-apps/04-vote-extensions.md new file mode 100644 index 000000000..f78f4c677 --- /dev/null +++ b/docs/build/building-apps/04-vote-extensions.md @@ -0,0 +1,121 @@ +--- +sidebar_position: 1 +--- + +# Vote Extensions + +:::note Synopsis +This sections describes how the application can define and use vote extensions +defined in ABCI++. +::: + +## Extend Vote + +ABCI++ allows an application to extend a pre-commit vote with arbitrary data. This +process does NOT have be deterministic and the data returned can be unique to the +validator process. The Cosmos SDK defines `baseapp.ExtendVoteHandler`: + +```go +type ExtendVoteHandler func(Context, *abci.RequestExtendVote) (*abci.ResponseExtendVote, error) +``` + +An application can set this handler in `app.go` via the `baseapp.SetExtendVoteHandler` +`BaseApp` option function. The `sdk.ExtendVoteHandler`, if defined, is called during +the `ExtendVote` ABCI method. Note, if an application decides to implement +`baseapp.ExtendVoteHandler`, it MUST return a non-nil `VoteExtension`. However, the vote +extension can be empty. See [here](https://github.com/cometbft/cometbft/blob/v0.38.0-rc1/spec/abci/abci++_methods.md#extendvote) +for more details. + +There are many decentralized censorship-resistant use cases for vote extensions. +For example, a validator may want to submit prices for a price oracle or encryption +shares for an encrypted transaction mempool. Note, an application should be careful +to consider the size of the vote extensions as they could increase latency in block +production. See [here](https://github.com/cometbft/cometbft/blob/v0.38.0-rc1/docs/qa/CometBFT-QA-38.md#vote-extensions-testbed) +for more details. + +## Verify Vote Extension + +Similar to extending a vote, an application can also verify vote extensions from +other validators when validating their pre-commits. For a given vote extension, +this process MUST be deterministic. The Cosmos SDK defines `sdk.VerifyVoteExtensionHandler`: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/abci.go#L26-L27 +``` + +An application can set this handler in `app.go` via the `baseapp.SetVerifyVoteExtensionHandler` +`BaseApp` option function. The `sdk.VerifyVoteExtensionHandler`, if defined, is called +during the `VerifyVoteExtension` ABCI method. If an application defines a vote +extension handler, it should also define a verification handler. Note, not all +validators will share the same view of what vote extensions they verify depending +on how votes are propagated. See [here](https://github.com/cometbft/cometbft/blob/v0.38.0-rc1/spec/abci/abci++_methods.md#verifyvoteextension) +for more details. + +## Vote Extension Propagation + +The agreed upon vote extensions at height `H` are provided to the proposing validator +at height `H+1` during `PrepareProposal`. As a result, the vote extensions are +not natively provided or exposed to the remaining validators during `ProcessProposal`. +As a result, if an application requires that the agreed upon vote extensions from +height `H` are available to all validators at `H+1`, the application must propagate +these vote extensions manually in the block proposal itself. This can be done by +"injecting" them into the block proposal, since the `Txs` field in `PrepareProposal` +is just a slice of byte slices. + +`FinalizeBlock` will ignore any byte slice that doesn't implement an `sdk.Tx` so +any injected vote extensions will safely be ignored in `FinalizeBlock`. For more +details on propagation, see the [ABCI++ 2.0 ADR](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-064-abci-2.0.md#vote-extension-propagation--verification). + +### Recovery of injected Vote Extensions + +As stated before, vote extensions can be injected in a block proposal (along with +other transactions in the `Txs` field). The Cosmos SDK provides a pre-FinalizeBlock +hook to allow applications to recover vote extensions, perform any necessary +computation on them, and then store the results in the cached store. These results +will be available to the application during the subsequent `FinalizeBlock` call. + +An example of how a pre-FinalizeBlock hook could look like is shown below: + +```go +app.SetPreBlocker(func(ctx sdk.Context, req *abci.RequestFinalizeBlock) error { + allVEs := []VE{} // store all parsed vote extensions here + for _, tx := range req.Txs { + // define a custom function that tries to parse the tx as a vote extension + ve, ok := parseVoteExtension(tx) + if !ok { + continue + } + + allVEs = append(allVEs, ve) + } + + // perform any necessary computation on the vote extensions and store the result + // in the cached store + result := compute(allVEs) + err := storeVEResult(ctx, result) + if err != nil { + return err + } + + return nil +}) + +``` + +Then in an app's module the application can retrieve the result of the computation +of vote extensions from the cached store: + +```go +func (k Keeper) BeginBlocker(ctx context.Context) error { + // retrieve the result of the computation of vote extensions from the cached store + result, err := k.GetVEResult(ctx) + if err != nil { + return err + } + + // use the result of the computation of vote extensions + k.setSomething(result) + + return nil +} +``` \ No newline at end of file diff --git a/docs/build/building-modules/00-intro.md b/docs/build/building-modules/00-intro.md index 7bfaf8e98..31c2a4889 100644 --- a/docs/build/building-modules/00-intro.md +++ b/docs/build/building-modules/00-intro.md @@ -10,18 +10,18 @@ Modules define most of the logic of Cosmos SDK applications. Developers compose :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK application](../../develop/beginner/00-app-anatomy.md) -* [Lifecycle of a Cosmos SDK transaction](../../develop/beginner/01-tx-lifecycle.md) +* [Anatomy of a Cosmos SDK application](../../learn/beginner/00-app-anatomy.md) +* [Lifecycle of a Cosmos SDK transaction](../../learn/beginner/01-tx-lifecycle.md) ::: ## Role of Modules in a Cosmos SDK Application -The Cosmos SDK can be thought of as the Ruby-on-Rails of blockchain development. It comes with a core that provides the basic functionalities every blockchain application needs, like a [boilerplate implementation of the ABCI](../../develop/advanced/00-baseapp.md) to communicate with the underlying consensus engine, a [`multistore`](../../develop/advanced/04-store.md#multistore) to persist state, a [server](../../develop/advanced/03-node.md) to form a full-node and [interfaces](./09-module-interfaces.md) to handle queries. +The Cosmos SDK can be thought of as the Ruby-on-Rails of blockchain development. It comes with a core that provides the basic functionalities every blockchain application needs, like a [boilerplate implementation of the ABCI](../../learn/advanced/00-baseapp.md) to communicate with the underlying consensus engine, a [`multistore`](../../learn/advanced/04-store.md#multistore) to persist state, a [server](../../learn/advanced/03-node.md) to form a full-node and [interfaces](./09-module-interfaces.md) to handle queries. On top of this core, the Cosmos SDK enables developers to build modules that implement the business logic of their application. In other words, SDK modules implement the bulk of the logic of applications, while the core does the wiring and enables modules to be composed together. The end goal is to build a robust ecosystem of open-source Cosmos SDK modules, making it increasingly easier to build complex blockchain applications. -Cosmos SDK modules can be seen as little state-machines within the state-machine. They generally define a subset of the state using one or more `KVStore`s in the [main multistore](../../develop/advanced/04-store.md), as well as a subset of [message types](./02-messages-and-queries.md#messages). These messages are routed by one of the main components of Cosmos SDK core, [`BaseApp`](../../develop/advanced/00-baseapp.md), to a module Protobuf [`Msg` service](./03-msg-services.md) that defines them. +Cosmos SDK modules can be seen as little state-machines within the state-machine. They generally define a subset of the state using one or more `KVStore`s in the [main multistore](../../learn/advanced/04-store.md), as well as a subset of [message types](./02-messages-and-queries.md#messages). These messages are routed by one of the main components of Cosmos SDK core, [`BaseApp`](../../learn/advanced/00-baseapp.md), to a module Protobuf [`Msg` service](./03-msg-services.md) that defines them. ```text + @@ -74,8 +74,8 @@ As a result of this architecture, building a Cosmos SDK application usually revo While there are no definitive guidelines for writing modules, here are some important design principles developers should keep in mind when building them: -* **Composability**: Cosmos SDK applications are almost always composed of multiple modules. This means developers need to carefully consider the integration of their module not only with the core of the Cosmos SDK, but also with other modules. The former is achieved by following standard design patterns outlined [here](#main-components-of-sdk-modules), while the latter is achieved by properly exposing the store(s) of the module via the [`keeper`](./06-keeper.md). -* **Specialization**: A direct consequence of the **composability** feature is that modules should be **specialized**. Developers should carefully establish the scope of their module and not batch multiple functionalities into the same module. This separation of concerns enables modules to be re-used in other projects and improves the upgradability of the application. **Specialization** also plays an important role in the [object-capabilities model](../../develop/advanced/10-ocap.md) of the Cosmos SDK. +* **Composability**: Cosmos SDK applications are almost always composed of multiple modules. This means developers need to carefully consider the integration of their module not only with the core of the Cosmos SDK, but also with other modules. The former is achieved by following standard design patterns outlined [here](#main-components-of-cosmos-sdk-modules), while the latter is achieved by properly exposing the store(s) of the module via the [`keeper`](./06-keeper.md). +* **Specialization**: A direct consequence of the **composability** feature is that modules should be **specialized**. Developers should carefully establish the scope of their module and not batch multiple functionalities into the same module. This separation of concerns enables modules to be re-used in other projects and improves the upgradability of the application. **Specialization** also plays an important role in the [object-capabilities model](../../learn/advanced/10-ocap.md) of the Cosmos SDK. * **Capabilities**: Most modules need to read and/or write to the store(s) of other modules. However, in an open-source environment, it is possible for some modules to be malicious. That is why module developers need to carefully think not only about how their module interacts with other modules, but also about how to give access to the module's store(s). The Cosmos SDK takes a capabilities-oriented approach to inter-module security. This means that each store defined by a module is accessed by a `key`, which is held by the module's [`keeper`](./06-keeper.md). This `keeper` defines how to access the store(s) and under what conditions. Access to the module's store(s) is done by passing a reference to the module's `keeper`. ## Main Components of Cosmos SDK Modules @@ -83,8 +83,8 @@ While there are no definitive guidelines for writing modules, here are some impo Modules are by convention defined in the `./x/` subfolder (e.g. the `bank` module will be defined in the `./x/bank` folder). They generally share the same core components: * A [`keeper`](./06-keeper.md), used to access the module's store(s) and update the state. -* A [`Msg` service](./02-messages-and-queries.md#messages), used to process messages when they are routed to the module by [`BaseApp`](../../develop/advanced/00-baseapp.md#message-routing) and trigger state-transitions. -* A [query service](./04-query-services.md), used to process user queries when they are routed to the module by [`BaseApp`](../../develop/advanced/00-baseapp.md#query-routing). +* A [`Msg` service](./02-messages-and-queries.md#messages), used to process messages when they are routed to the module by [`BaseApp`](../../learn/advanced/00-baseapp.md#message-routing) and trigger state-transitions. +* A [query service](./04-query-services.md), used to process user queries when they are routed to the module by [`BaseApp`](../../learn/advanced/00-baseapp.md#query-routing). * Interfaces, for end users to query the subset of the state defined by the module and create `message`s of the custom types defined in the module. In addition to these components, modules implement the `AppModule` interface in order to be managed by the [`module manager`](./01-module-manager.md). diff --git a/docs/build/building-modules/01-module-manager.md b/docs/build/building-modules/01-module-manager.md index 7666cfec5..dfd17d5f5 100644 --- a/docs/build/building-modules/01-module-manager.md +++ b/docs/build/building-modules/01-module-manager.md @@ -5,7 +5,7 @@ sidebar_position: 1 # Module Manager :::note Synopsis -Cosmos SDK modules need to implement the [`AppModule` interfaces](#application-module-interfaces), in order to be managed by the application's [module manager](#module-manager). The module manager plays an important role in [`message` and `query` routing](../../develop/advanced/00-baseapp.md#routing), and allows application developers to set the order of execution of a variety of functions like [`PreBlocker`](../../develop/beginner/00-app-anatomy#preblocker) and [`BeginBlocker` and `EndBlocker`](../../develop/beginner/00-app-anatomy.md#begingblocker-and-endblocker). +Cosmos SDK modules need to implement the [`AppModule` interfaces](#application-module-interfaces), in order to be managed by the application's [module manager](#module-manager). The module manager plays an important role in [`message` and `query` routing](../../learn/advanced/00-baseapp.md#routing), and allows application developers to set the order of execution of a variety of functions like [`PreBlocker`](../../learn/beginner/00-app-anatomy#preblocker) and [`BeginBlocker` and `EndBlocker`](../../learn/beginner/00-app-anatomy.md#begingblocker-and-endblocker). ::: :::note Pre-requisite Readings @@ -46,7 +46,7 @@ The above interfaces are mostly embedding smaller interfaces (extension interfac * (legacy) [`module.HasInvariants`](#hasinvariants): The extension interface for registering invariants. * (legacy) [`module.HasConsensusVersion`](#hasconsensusversion): The extension interface for declaring a module consensus version. -The `AppModuleBasic` interface exists to define independent methods of the module, i.e. those that do not depend on other modules in the application. This allows for the construction of the basic application structure early in the application definition, generally in the `init()` function of the [main application file](../../develop/beginner/00-app-anatomy.md#core-application-file). +The `AppModuleBasic` interface exists to define independent methods of the module, i.e. those that do not depend on other modules in the application. This allows for the construction of the basic application structure early in the application definition, generally in the `init()` function of the [main application file](../../learn/beginner/00-app-anatomy.md#core-application-file). The `AppModule` interface exists to define inter-dependent module methods. Many modules need to interact with other modules, typically through [`keeper`s](./06-keeper.md), which means there is a need for an interface where modules list their `keeper`s and other methods that require a reference to another module's object. `AppModule` interface extension, such as `HasBeginBlocker` and `HasEndBlocker`, also enables the module manager to set the order of execution between module's methods like `BeginBlock` and `EndBlock`, which is important in cases where the order of execution between modules matters in the context of the application. @@ -224,7 +224,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/module/module.go https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/core/appmodule/module.go#L49-L52 ``` -* `Precommit(context.Context)`: This method gives module developers the option to implement logic that is automatically triggered during [`Commit'](../../develop/advanced/00-baseapp.md#commit) of each block using the [`finalizeblockstate`](../../develop/advanced/00-baseapp.md#state-updates) of the block to be committed. Implement empty if no logic needs to be triggered during `Commit` of each block for this module. +* `Precommit(context.Context)`: This method gives module developers the option to implement logic that is automatically triggered during [`Commit'](../../learn/advanced/00-baseapp.md#commit) of each block using the [`finalizeblockstate`](../../learn/advanced/00-baseapp.md#state-updates) of the block to be committed. Implement empty if no logic needs to be triggered during `Commit` of each block for this module. ### `HasPrepareCheckState` @@ -234,7 +234,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/core/appmodule/module. https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/core/appmodule/module.go#L49-L52 ``` -* `PrepareCheckState(context.Context)`: This method gives module developers the option to implement logic that is automatically triggered during [`Commit'](../../develop/advanced/00-baseapp.md#commit) of each block using the [`checkState`](../../develop/advanced/00-baseapp.md#state-updates) of the next block. Implement empty if no logic needs to be triggered during `Commit` of each block for this module. +* `PrepareCheckState(context.Context)`: This method gives module developers the option to implement logic that is automatically triggered during [`Commit'](../../learn/advanced/00-baseapp.md#commit) of each block using the [`checkState`](../../learn/advanced/00-baseapp.md#state-updates) of the next block. Implement empty if no logic needs to be triggered during `Commit` of each block for this module. ### Implementing the Application Module Interfaces @@ -272,15 +272,15 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/module/module.go It implements the following methods: -* `NewBasicManager(modules ...AppModuleBasic)`: Constructor function. It takes a list of the application's `AppModuleBasic` and builds a new `BasicManager`. This function is generally called in the `init()` function of [`app.go`](../../develop/beginner/00-app-anatomy.md#core-application-file) to quickly initialize the independent elements of the application's modules (click [here](https://github.com/cosmos/gaia/blob/main/app/app.go#L59-L74) to see an example). +* `NewBasicManager(modules ...AppModuleBasic)`: Constructor function. It takes a list of the application's `AppModuleBasic` and builds a new `BasicManager`. This function is generally called in the `init()` function of [`app.go`](../../learn/beginner/00-app-anatomy.md#core-application-file) to quickly initialize the independent elements of the application's modules (click [here](https://github.com/cosmos/gaia/blob/main/app/app.go#L59-L74) to see an example). * `NewBasicManagerFromManager(manager *Manager, customModuleBasics map[string]AppModuleBasic)`: Contructor function. It creates a new `BasicManager` from a `Manager`. The `BasicManager` will contain all `AppModuleBasic` from the `AppModule` manager using `CoreAppModuleBasicAdaptor` whenever possible. Module's `AppModuleBasic` can be overridden by passing a custom AppModuleBasic map -* `RegisterLegacyAminoCodec(cdc *codec.LegacyAmino)`: Registers the [`codec.LegacyAmino`s](../../develop/advanced/05-encoding.md#amino) of each of the application's `AppModuleBasic`. This function is usually called early on in the [application's construction](../../develop/beginner/00-app-anatomy.md#constructor). +* `RegisterLegacyAminoCodec(cdc *codec.LegacyAmino)`: Registers the [`codec.LegacyAmino`s](../../learn/advanced/05-encoding.md#amino) of each of the application's `AppModuleBasic`. This function is usually called early on in the [application's construction](../../learn/beginner/00-app-anatomy.md#constructor). * `RegisterInterfaces(registry codectypes.InterfaceRegistry)`: Registers interface types and implementations of each of the application's `AppModuleBasic`. * `DefaultGenesis(cdc codec.JSONCodec)`: Provides default genesis information for modules in the application by calling the [`DefaultGenesis(cdc codec.JSONCodec)`](./08-genesis.md#defaultgenesis) function of each module. It only calls the modules that implements the `HasGenesisBasics` interfaces. * `ValidateGenesis(cdc codec.JSONCodec, txEncCfg client.TxEncodingConfig, genesis map[string]json.RawMessage)`: Validates the genesis information modules by calling the [`ValidateGenesis(codec.JSONCodec, client.TxEncodingConfig, json.RawMessage)`](./08-genesis.md#validategenesis) function of modules implementing the `HasGenesisBasics` interface. * `RegisterGRPCGatewayRoutes(clientCtx client.Context, rtr *runtime.ServeMux)`: Registers gRPC routes for modules. -* `AddTxCommands(rootTxCmd *cobra.Command)`: Adds modules' transaction commands (defined as `GetTxCmd() *cobra.Command`) to the application's [`rootTxCommand`](../../develop/advanced/07-cli.md#transaction-commands). This function is usually called function from the `main.go` function of the [application's command-line interface](../../develop/advanced/07-cli.md). -* `AddQueryCommands(rootQueryCmd *cobra.Command)`: Adds modules' query commands (defined as `GetQueryCmd() *cobra.Command`) to the application's [`rootQueryCommand`](../../develop/advanced/07-cli.md#query-commands). This function is usually called function from the `main.go` function of the [application's command-line interface](../../develop/advanced/07-cli.md). +* `AddTxCommands(rootTxCmd *cobra.Command)`: Adds modules' transaction commands (defined as `GetTxCmd() *cobra.Command`) to the application's [`rootTxCommand`](../../learn/advanced/07-cli.md#transaction-commands). This function is usually called function from the `main.go` function of the [application's command-line interface](../../learn/advanced/07-cli.md). +* `AddQueryCommands(rootQueryCmd *cobra.Command)`: Adds modules' query commands (defined as `GetQueryCmd() *cobra.Command`) to the application's [`rootQueryCommand`](../../learn/advanced/07-cli.md#query-commands). This function is usually called function from the `main.go` function of the [application's command-line interface](../../learn/advanced/07-cli.md). ### `Manager` @@ -292,26 +292,26 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/module/module.go The module manager is used throughout the application whenever an action on a collection of modules is required. It implements the following methods: -* `NewManager(modules ...AppModule)`: Constructor function. It takes a list of the application's `AppModule`s and builds a new `Manager`. It is generally called from the application's main [constructor function](../../develop/beginner/00-app-anatomy.md#constructor-function). -* `SetOrderInitGenesis(moduleNames ...string)`: Sets the order in which the [`InitGenesis`](./08-genesis.md#initgenesis) function of each module will be called when the application is first started. This function is generally called from the application's main [constructor function](../../develop/beginner/00-app-anatomy.md#constructor-function). +* `NewManager(modules ...AppModule)`: Constructor function. It takes a list of the application's `AppModule`s and builds a new `Manager`. It is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderInitGenesis(moduleNames ...string)`: Sets the order in which the [`InitGenesis`](./08-genesis.md#initgenesis) function of each module will be called when the application is first started. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). To initialize modules successfully, module dependencies should be considered. For example, the `genutil` module must occur after `staking` module so that the pools are properly initialized with tokens from genesis accounts, the `genutils` module must also occur after `auth` so that it can access the params from auth, IBC's `capability` module should be initialized before all other modules so that it can initialize any capabilities. -* `SetOrderExportGenesis(moduleNames ...string)`: Sets the order in which the [`ExportGenesis`](./08-genesis.md#exportgenesis) function of each module will be called in case of an export. This function is generally called from the application's main [constructor function](../../develop/beginner/00-app-anatomy.md#constructor-function). -* `SetOrderPreBlockers(moduleNames ...string)`: Sets the order in which the `PreBlock()` function of each module will be called before `BeginBlock()` of all modules. This function is generally called from the application's main [constructor function](../../develop/beginner/00-app-anatomy.md#constructor-function). -* `SetOrderBeginBlockers(moduleNames ...string)`: Sets the order in which the `BeginBlock()` function of each module will be called at the beginning of each block. This function is generally called from the application's main [constructor function](../../develop/beginner/00-app-anatomy.md#constructor-function). -* `SetOrderEndBlockers(moduleNames ...string)`: Sets the order in which the `EndBlock()` function of each module will be called at the end of each block. This function is generally called from the application's main [constructor function](../../develop/beginner/00-app-anatomy.md#constructor-function). -* `SetOrderPrecommiters(moduleNames ...string)`: Sets the order in which the `Precommit()` function of each module will be called during commit of each block. This function is generally called from the application's main [constructor function](../../develop/beginner/00-app-anatomy.md#constructor-function). -* `SetOrderPrepareCheckStaters(moduleNames ...string)`: Sets the order in which the `PrepareCheckState()` function of each module will be called during commit of each block. This function is generally called from the application's main [constructor function](../../develop/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderExportGenesis(moduleNames ...string)`: Sets the order in which the [`ExportGenesis`](./08-genesis.md#exportgenesis) function of each module will be called in case of an export. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderPreBlockers(moduleNames ...string)`: Sets the order in which the `PreBlock()` function of each module will be called before `BeginBlock()` of all modules. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderBeginBlockers(moduleNames ...string)`: Sets the order in which the `BeginBlock()` function of each module will be called at the beginning of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderEndBlockers(moduleNames ...string)`: Sets the order in which the `EndBlock()` function of each module will be called at the end of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderPrecommiters(moduleNames ...string)`: Sets the order in which the `Precommit()` function of each module will be called during commit of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderPrepareCheckStaters(moduleNames ...string)`: Sets the order in which the `PrepareCheckState()` function of each module will be called during commit of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). * `SetOrderMigrations(moduleNames ...string)`: Sets the order of migrations to be run. If not set then migrations will be run with an order defined in `DefaultMigrationsOrder`. * `RegisterInvariants(ir sdk.InvariantRegistry)`: Registers the [invariants](./07-invariants.md) of module implementing the `HasInvariants` interface. * `RegisterServices(cfg Configurator)`: Registers the services of modules implementing the `HasServices` interface. * `InitGenesis(ctx context.Context, cdc codec.JSONCodec, genesisData map[string]json.RawMessage)`: Calls the [`InitGenesis`](./08-genesis.md#initgenesis) function of each module when the application is first started, in the order defined in `OrderInitGenesis`. Returns an `abci.ResponseInitChain` to the underlying consensus engine, which can contain validator updates. * `ExportGenesis(ctx context.Context, cdc codec.JSONCodec)`: Calls the [`ExportGenesis`](./08-genesis.md#exportgenesis) function of each module, in the order defined in `OrderExportGenesis`. The export constructs a genesis file from a previously existing state, and is mainly used when a hard-fork upgrade of the chain is required. * `ExportGenesisForModules(ctx context.Context, cdc codec.JSONCodec, modulesToExport []string)`: Behaves the same as `ExportGenesis`, except takes a list of modules to export. -* `BeginBlock(ctx context.Context) error`: At the beginning of each block, this function is called from [`BaseApp`](../../develop/advanced/00-baseapp.md#beginblock) and, in turn, calls the [`BeginBlock`](./06-beginblock-endblock.md) function of each modules implementing the `appmodule.HasBeginBlocker` interface, in the order defined in `OrderBeginBlockers`. It creates a child [context](../../develop/advanced/02-context.md) with an event manager to aggregate [events](../../develop/advanced/08-events.md) emitted from each modules. -* `EndBlock(ctx context.Context) error`: At the end of each block, this function is called from [`BaseApp`](../../develop/advanced/00-baseapp.md#endblock) and, in turn, calls the [`EndBlock`](./06-beginblock-endblock.md) function of each modules implementing the `appmodule.HasEndBlocker` interface, in the order defined in `OrderEndBlockers`. It creates a child [context](../../develop/advanced/02-context.md) with an event manager to aggregate [events](../../develop/advanced/08-events.md) emitted from all modules. The function returns an `abci` which contains the aforementioned events, as well as validator set updates (if any). -* `EndBlock(context.Context) ([]abci.ValidatorUpdate, error)`: At the end of each block, this function is called from [`BaseApp`](../../develop/advanced/00-baseapp.md#endblock) and, in turn, calls the [`EndBlock`](./06-beginblock-endblock.md) function of each modules implementing the `module.HasABCIEndBlock` interface, in the order defined in `OrderEndBlockers`. It creates a child [context](../../develop/advanced/02-context.md) with an event manager to aggregate [events](../../develop/advanced/08-events.md) emitted from all modules. The function returns an `abci` which contains the aforementioned events, as well as validator set updates (if any). -* `Precommit(ctx context.Context)`: During [`Commit`](../../develop/advanced/00-baseapp.md#commit), this function is called from `BaseApp` immediately before the [`deliverState`](../../develop/advanced/00-baseapp.md#state-updates) is written to the underlying [`rootMultiStore`](../../develop/advanced/04-store.md#commitmultistore) and, in turn calls the `Precommit` function of each modules implementing the `HasPrecommit` interface, in the order defined in `OrderPrecommiters`. It creates a child [context](../../develop/advanced/02-context.md) where the underlying `CacheMultiStore` is that of the newly committed block's [`finalizeblockstate`](../../develop/advanced/00-baseapp.md#state-updates). -* `PrepareCheckState(ctx context.Context)`: During [`Commit`](../../develop/advanced/00-baseapp.md#commit), this function is called from `BaseApp` immediately after the [`deliverState`](../../develop/advanced/00-baseapp.md#state-updates) is written to the underlying [`rootMultiStore`](../../develop/advanced/04-store.md#commitmultistore) and, in turn calls the `PrepareCheckState` function of each module implementing the `HasPrepareCheckState` interface, in the order defined in `OrderPrepareCheckStaters`. It creates a child [context](../../develop/advanced/02-context.md) where the underlying `CacheMultiStore` is that of the next block's [`checkState`](../../develop/advanced/00-baseapp.md#state-updates). Writes to this state will be present in the [`checkState`](../../develop/advanced/00-baseapp.md#state-updates) of the next block, and therefore this method can be used to prepare the `checkState` for the next block. +* `BeginBlock(ctx context.Context) error`: At the beginning of each block, this function is called from [`BaseApp`](../../learn/advanced/00-baseapp.md#beginblock) and, in turn, calls the [`BeginBlock`](./06-beginblock-endblock.md) function of each modules implementing the `appmodule.HasBeginBlocker` interface, in the order defined in `OrderBeginBlockers`. It creates a child [context](../../learn/advanced/02-context.md) with an event manager to aggregate [events](../../learn/advanced/08-events.md) emitted from each modules. +* `EndBlock(ctx context.Context) error`: At the end of each block, this function is called from [`BaseApp`](../../learn/advanced/00-baseapp.md#endblock) and, in turn, calls the [`EndBlock`](./06-beginblock-endblock.md) function of each modules implementing the `appmodule.HasEndBlocker` interface, in the order defined in `OrderEndBlockers`. It creates a child [context](../../learn/advanced/02-context.md) with an event manager to aggregate [events](../../learn/advanced/08-events.md) emitted from all modules. The function returns an `abci` which contains the aforementioned events, as well as validator set updates (if any). +* `EndBlock(context.Context) ([]abci.ValidatorUpdate, error)`: At the end of each block, this function is called from [`BaseApp`](../../learn/advanced/00-baseapp.md#endblock) and, in turn, calls the [`EndBlock`](./06-beginblock-endblock.md) function of each modules implementing the `module.HasABCIEndBlock` interface, in the order defined in `OrderEndBlockers`. It creates a child [context](../../learn/advanced/02-context.md) with an event manager to aggregate [events](../../learn/advanced/08-events.md) emitted from all modules. The function returns an `abci` which contains the aforementioned events, as well as validator set updates (if any). +* `Precommit(ctx context.Context)`: During [`Commit`](../../learn/advanced/00-baseapp.md#commit), this function is called from `BaseApp` immediately before the [`deliverState`](../../learn/advanced/00-baseapp.md#state-updates) is written to the underlying [`rootMultiStore`](../../learn/advanced/04-store.md#commitmultistore) and, in turn calls the `Precommit` function of each modules implementing the `HasPrecommit` interface, in the order defined in `OrderPrecommiters`. It creates a child [context](../../learn/advanced/02-context.md) where the underlying `CacheMultiStore` is that of the newly committed block's [`finalizeblockstate`](../../learn/advanced/00-baseapp.md#state-updates). +* `PrepareCheckState(ctx context.Context)`: During [`Commit`](../../learn/advanced/00-baseapp.md#commit), this function is called from `BaseApp` immediately after the [`deliverState`](../../learn/advanced/00-baseapp.md#state-updates) is written to the underlying [`rootMultiStore`](../../learn/advanced/04-store.md#commitmultistore) and, in turn calls the `PrepareCheckState` function of each module implementing the `HasPrepareCheckState` interface, in the order defined in `OrderPrepareCheckStaters`. It creates a child [context](../../learn/advanced/02-context.md) where the underlying `CacheMultiStore` is that of the next block's [`checkState`](../../learn/advanced/00-baseapp.md#state-updates). Writes to this state will be present in the [`checkState`](../../learn/advanced/00-baseapp.md#state-updates) of the next block, and therefore this method can be used to prepare the `checkState` for the next block. Here's an example of a concrete integration within an `simapp`: diff --git a/docs/build/building-modules/02-messages-and-queries.md b/docs/build/building-modules/02-messages-and-queries.md index 3d92cd82a..573c35cd7 100644 --- a/docs/build/building-modules/02-messages-and-queries.md +++ b/docs/build/building-modules/02-messages-and-queries.md @@ -16,13 +16,13 @@ sidebar_position: 1 ## Messages -`Msg`s are objects whose end-goal is to trigger state-transitions. They are wrapped in [transactions](../../develop/advanced/01-transactions.md), which may contain one or more of them. +`Msg`s are objects whose end-goal is to trigger state-transitions. They are wrapped in [transactions](../../learn/advanced/01-transactions.md), which may contain one or more of them. -When a transaction is relayed from the underlying consensus engine to the Cosmos SDK application, it is first decoded by [`BaseApp`](../../develop/advanced/00-baseapp.md). Then, each message contained in the transaction is extracted and routed to the appropriate module via `BaseApp`'s `MsgServiceRouter` so that it can be processed by the module's [`Msg` service](./03-msg-services.md). For a more detailed explanation of the lifecycle of a transaction, click [here](../../develop/beginner/01-tx-lifecycle.md). +When a transaction is relayed from the underlying consensus engine to the Cosmos SDK application, it is first decoded by [`BaseApp`](../../learn/advanced/00-baseapp.md). Then, each message contained in the transaction is extracted and routed to the appropriate module via `BaseApp`'s `MsgServiceRouter` so that it can be processed by the module's [`Msg` service](./03-msg-services.md). For a more detailed explanation of the lifecycle of a transaction, click [here](../../learn/beginner/01-tx-lifecycle.md). ### `Msg` Services -Defining Protobuf `Msg` services is the recommended way to handle messages. A Protobuf `Msg` service should be created for each module, typically in `tx.proto` (see more info about [conventions and naming](../../develop/advanced/05-encoding.md#faq)). It must have an RPC service method defined for each message in the module. +Defining Protobuf `Msg` services is the recommended way to handle messages. A Protobuf `Msg` service should be created for each module, typically in `tx.proto` (see more info about [conventions and naming](../../learn/advanced/05-encoding.md#faq)). It must have an RPC service method defined for each message in the module. Each `Msg` service method must have exactly one argument, which must implement the `sdk.Msg` interface, and a Protobuf response. The naming convention is to call the RPC argument `Msg` and the RPC response `MsgResponse`. For example: @@ -89,7 +89,7 @@ In order for clients (CLI and grpc-gateway) to have these URLs registered, the C ## Queries -A `query` is a request for information made by end-users of applications through an interface and processed by a full-node. A `query` is received by a full-node through its consensus engine and relayed to the application via the ABCI. It is then routed to the appropriate module via `BaseApp`'s `QueryRouter` so that it can be processed by the module's query service (./04-query-services.md). For a deeper look at the lifecycle of a `query`, click [here](../../develop/beginner/02-query-lifecycle.md). +A `query` is a request for information made by end-users of applications through an interface and processed by a full-node. A `query` is received by a full-node through its consensus engine and relayed to the application via the ABCI. It is then routed to the appropriate module via `BaseApp`'s `QueryRouter` so that it can be processed by the module's query service (./04-query-services.md). For a deeper look at the lifecycle of a `query`, click [here](../../learn/beginner/02-query-lifecycle.md). ### gRPC Queries @@ -115,14 +115,14 @@ queryCategory/queryRoute/queryType/arg1/arg2/... where: -* `queryCategory` is the category of the `query`, typically `custom` for module queries. It is used to differentiate between different kinds of queries within `BaseApp`'s [`Query` method](../../develop/advanced/00-baseapp.md#query). -* `queryRoute` is used by `BaseApp`'s [`queryRouter`](../../develop/advanced/00-baseapp.md#query-routing) to map the `query` to its module. Usually, `queryRoute` should be the name of the module. +* `queryCategory` is the category of the `query`, typically `custom` for module queries. It is used to differentiate between different kinds of queries within `BaseApp`'s [`Query` method](../../learn/advanced/00-baseapp.md#query). +* `queryRoute` is used by `BaseApp`'s [`queryRouter`](../../learn/advanced/00-baseapp.md#query-routing) to map the `query` to its module. Usually, `queryRoute` should be the name of the module. * `queryType` is used by the module's [`querier`](./04-query-services.md#legacy-queriers) to map the `query` to the appropriate `querier function` within the module. * `args` are the actual arguments needed to process the `query`. They are filled out by the end-user. Note that for bigger queries, you might prefer passing arguments in the `Data` field of the request `req` instead of the `path`. The `path` for each `query` must be defined by the module developer in the module's [command-line interface file](./09-module-interfaces.md#query-commands).Overall, there are 3 mains components module developers need to implement in order to make the subset of the state defined by their module queryable: -* A [`querier`](./04-query-services.md#legacy-queriers), to process the `query` once it has been [routed to the module](../../develop/advanced/00-baseapp.md#query-routing). +* A [`querier`](./04-query-services.md#legacy-queriers), to process the `query` once it has been [routed to the module](../../learn/advanced/00-baseapp.md#query-routing). * [Query commands](./09-module-interfaces.md#query-commands) in the module's CLI file, where the `path` for each `query` is specified. * `query` return types. Typically defined in a file `types/querier.go`, they specify the result type of each of the module's `queries`. These custom types must implement the `String()` method of [`fmt.Stringer`](https://pkg.go.dev/fmt#Stringer). diff --git a/docs/build/building-modules/03-msg-services.md b/docs/build/building-modules/03-msg-services.md index b64eaa45f..4d2bf6ee3 100644 --- a/docs/build/building-modules/03-msg-services.md +++ b/docs/build/building-modules/03-msg-services.md @@ -5,7 +5,7 @@ sidebar_position: 1 # `Msg` Services :::note Synopsis -A Protobuf `Msg` service processes [messages](./02-messages-and-queries.md#messages). Protobuf `Msg` services are specific to the module in which they are defined, and only process messages defined within the said module. They are called from `BaseApp` during [`DeliverTx`](../../develop/advanced/00-baseapp.md#delivertx). +A Protobuf `Msg` service processes [messages](./02-messages-and-queries.md#messages). Protobuf `Msg` services are specific to the module in which they are defined, and only process messages defined within the said module. They are called from `BaseApp` during [`DeliverTx`](../../learn/advanced/00-baseapp.md#delivertx). ::: :::note Pre-requisite Readings @@ -71,7 +71,7 @@ After the validation is successful, the `msgServer` method uses the [`keeper`](. ### Events -Before returning, `msgServer` methods generally emit one or more [events](../../develop/advanced/08-events.md) by using the `EventManager` held in the `ctx`. Use the new `EmitTypedEvent` function that uses protobuf-based event types: +Before returning, `msgServer` methods generally emit one or more [events](../../learn/advanced/08-events.md) by using the `EventManager` held in the `ctx`. Use the new `EmitTypedEvent` function that uses protobuf-based event types: ```go ctx.EventManager().EmitTypedEvent( @@ -90,7 +90,7 @@ ctx.EventManager().EmitEvent( ) ``` -These events are relayed back to the underlying consensus engine and can be used by service providers to implement services around the application. Click [here](../../develop/advanced/08-events.md) to learn more about events. +These events are relayed back to the underlying consensus engine and can be used by service providers to implement services around the application. Click [here](../../learn/advanced/08-events.md) to learn more about events. The invoked `msgServer` method returns a `proto.Message` response and an `error`. These return values are then wrapped into an `*sdk.Result` or an `error`: @@ -110,7 +110,7 @@ This diagram shows a typical structure of a Protobuf `Msg` service, and how the ## Telemetry -New [telemetry metrics](../../develop/advanced/09-telemetry.md) can be created from `msgServer` methods when handling messages. +New [telemetry metrics](../../learn/advanced/09-telemetry.md) can be created from `msgServer` methods when handling messages. This is an example from the `x/auth/vesting` module: diff --git a/docs/build/building-modules/04-query-services.md b/docs/build/building-modules/04-query-services.md index 29542d86c..a787a0c22 100644 --- a/docs/build/building-modules/04-query-services.md +++ b/docs/build/building-modules/04-query-services.md @@ -5,7 +5,7 @@ sidebar_position: 1 # Query Services :::note Synopsis -A Protobuf Query service processes [`queries`](./02-messages-and-queries.md#queries). Query services are specific to the module in which they are defined, and only process `queries` defined within said module. They are called from `BaseApp`'s [`Query` method](../../develop/advanced/00-baseapp.md#query). +A Protobuf Query service processes [`queries`](./02-messages-and-queries.md#queries). Query services are specific to the module in which they are defined, and only process `queries` defined within said module. They are called from `BaseApp`'s [`Query` method](../../learn/advanced/00-baseapp.md#query). ::: :::note Pre-requisite Readings diff --git a/docs/build/building-modules/05-beginblock-endblock.md b/docs/build/building-modules/05-beginblock-endblock.md deleted file mode 100644 index 657f1d6d9..000000000 --- a/docs/build/building-modules/05-beginblock-endblock.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -sidebar_position: 1 ---- - -# BeginBlocker and EndBlocker - -:::note Synopsis -`BeginBlocker` and `EndBlocker` are optional methods module developers can implement in their module. They will be triggered at the beginning and at the end of each block respectively, when the [`BeginBlock`](../../develop/advanced/00-baseapp.md#beginblock) and [`EndBlock`](../../develop/advanced/00-baseapp.md#endblock) ABCI messages are received from the underlying consensus engine. -::: - -:::note - -### Pre-requisite Readings - -* [Module Manager](01-module-manager.md) - -::: - -## BeginBlocker and EndBlocker - -`BeginBlocker` and `EndBlocker` are a way for module developers to add automatic execution of logic to their module. This is a powerful tool that should be used carefully, as complex automatic functions can slow down or even halt the chain. - -When needed, `BeginBlocker` and `EndBlocker` are implemented as part of the [`BeginBlockAppModule` and `BeginBlockAppModule` interfaces](01-module-manager.md#appmodule). This means either can be left-out if not required. The `BeginBlock` and `EndBlock` methods of the interface implemented in `module.go` generally defer to `BeginBlocker` and `EndBlocker` methods respectively, which are usually implemented in `abci.go`. - -The actual implementation of `BeginBlocker` and `EndBlocker` in `abci.go` are very similar to that of a [`Msg` service](03-msg-services.md): - -* They generally use the [`keeper`](06-keeper.md) and [`ctx`](../../develop/advanced/02-context.md) to retrieve information about the latest state. -* If needed, they use the `keeper` and `ctx` to trigger state-transitions. -* If needed, they can emit [`events`](../../develop/advanced/08-events.md) via the `ctx`'s `EventManager`. - -A specificity of the `EndBlocker` is that it can return validator updates to the underlying consensus engine in the form of an [`[]abci.ValidatorUpdates`](https://docs.cometbft.com/v0.37/spec/abci/abci++_methods#endblock). This is the preferred way to implement custom validator changes. - -It is possible for developers to define the order of execution between the `BeginBlocker`/`EndBlocker` functions of each of their application's modules via the module's manager `SetOrderBeginBlocker`/`SetOrderEndBlocker` methods. For more on the module manager, click [here](01-module-manager.md#manager). - -See an example implementation of `BeginBlocker` from the `distribution` module: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/distribution/abci.go#L14-L38 -``` - -and an example implementation of `EndBlocker` from the `staking` module: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/staking/abci.go#L22-L27 -``` - - diff --git a/docs/build/building-modules/05-protobuf-annotations.md b/docs/build/building-modules/05-protobuf-annotations.md new file mode 100644 index 000000000..9a0f971ac --- /dev/null +++ b/docs/build/building-modules/05-protobuf-annotations.md @@ -0,0 +1,121 @@ +--- +sidebar_position: 1 +--- + +# ProtocolBuffer Annotations + +This document explains the various protobuf scalars that have been added to make working with protobuf easier for Cosmos SDK application developers + +## Signer + +Signer specifies which field should be used to determine the signer of a message for the Cosmos SDK. This field can be used for clients as well to infer which field should be used to determine the signer of a message. + +Read more about the signer field [here](./02-messages-and-queries.md). + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/e6848d99b55a65d014375b295bdd7f9641aac95e/proto/cosmos/bank/v1beta1/tx.proto#L40 +``` + +```proto +option (cosmos.msg.v1.signer) = "from_address"; +``` + +## Scalar + +The scalar type defines a way for clients to understand how to construct protobuf messages according to what is expected by the module and sdk. + +```proto +(cosmos_proto.scalar) = "cosmos.AddressString" +``` + +Example of account address string scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e6848d99b55a65d014375b295bdd7f9641aac95e/proto/cosmos/bank/v1beta1/tx.proto#L46 +``` + +Example of validator address string scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/distribution/v1beta1/query.proto#L87 +``` + +Example of Decimals scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/distribution/v1beta1/distribution.proto#L26 +``` + +Example of Int scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/gov/v1/gov.proto#L137 +``` + +There are a few options for what can be provided as a scalar: cosmos.AddressString, cosmos.ValidatorAddressString, cosmos.ConsensusAddressString, cosmos.Int, cosmos.Dec. + +## Implements_Interface + +Implement interface is used to provide information to client tooling like [telescope](https://github.com/cosmology-tech/telescope) on how to encode and decode protobuf messages. + +```proto +option (cosmos_proto.implements_interface) = "cosmos.auth.v1beta1.AccountI"; +``` + +## Amino + +The amino codec was removed in 0.50.0, this means there is not a need register `legacyAminoCodec`. To replace the amino codec, Amino protobuf annotations are used to provide information to the amino codec on how to encode and decode protobuf messages. + +:::note +Amino annotations are only used for backwards compatibility with amino. New modules are not required use amino annotations. +::: + +The below annotations are used to provide information to the amino codec on how to encode and decode protobuf messages in a backwards compatible manner. + +### Name + +Name specifies the amino name that would show up for the user in order for them see which message they are signing. + +```proto +option (amino.name) = "cosmos-sdk/BaseAccount"; +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/bank/v1beta1/tx.proto#L41 +``` + +### Field_Name + +Field name specifies the amino name that would show up for the user in order for them see which field they are signing. + +```proto +uint64 height = 1 [(amino.field_name) = "public_key"]; +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/distribution/v1beta1/distribution.proto#L166 +``` + +### Dont_OmitEmpty + +Dont omitempty specifies that the field should not be omitted when encoding to amino. + +```proto +repeated cosmos.base.v1beta1.Coin amount = 3 [(amino.dont_omitempty) = true]; +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/bank/v1beta1/bank.proto#L56 +``` + +### Encoding + +Encoding specifies the amino encoding that should be used when encoding to amino. + +```proto +(amino.encoding) = "legacy_coins", +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/bank/v1beta1/genesis.proto#L23 +``` diff --git a/docs/build/building-modules/06-beginblock-endblock.md b/docs/build/building-modules/06-beginblock-endblock.md new file mode 100644 index 000000000..a8eafdf67 --- /dev/null +++ b/docs/build/building-modules/06-beginblock-endblock.md @@ -0,0 +1,47 @@ +--- +sidebar_position: 1 +--- + +# BeginBlocker and EndBlocker + +:::note Synopsis +`BeginBlocker` and `EndBlocker` are optional methods module developers can implement in their module. They will be triggered at the beginning and at the end of each block respectively, when the [`BeginBlock`](../../learn/advanced/00-baseapp.md#beginblock) and [`EndBlock`](../../learn/advanced/00-baseapp.md#endblock) ABCI messages are received from the underlying consensus engine. +::: + +:::note Pre-requisite Readings + +* [Module Manager](./01-module-manager.md) + +::: + +## BeginBlocker and EndBlocker + +`BeginBlocker` and `EndBlocker` are a way for module developers to add automatic execution of logic to their module. This is a powerful tool that should be used carefully, as complex automatic functions can slow down or even halt the chain. + +In 0.47.0, Prepare and Process Proposal were added that allow app developers to do arbitrary work at those phases, but they do not influence the work that will be done in BeginBlock. If an application required `BeginBlock` to execute prior to any sort of work is done then this is not possible today (0.50.0). + +When needed, `BeginBlocker` and `EndBlocker` are implemented as part of the [`HasBeginBlocker`, `HasABCIEndBlocker` and `EndBlocker` interfaces](./01-module-manager.md#appmodule). This means either can be left-out if not required. The `BeginBlock` and `EndBlock` methods of the interface implemented in `module.go` generally defer to `BeginBlocker` and `EndBlocker` methods respectively, which are usually implemented in `abci.go`. + +The actual implementation of `BeginBlocker` and `EndBlocker` in `abci.go` are very similar to that of a [`Msg` service](./03-msg-services.md): + +* They generally use the [`keeper`](./06-keeper.md) and [`ctx`](../../learn/advanced/02-context.md) to retrieve information about the latest state. +* If needed, they use the `keeper` and `ctx` to trigger state-transitions. +* If needed, they can emit [`events`](../../learn/advanced/08-events.md) via the `ctx`'s `EventManager`. + +A specific type of `EndBlocker` is available to return validator updates to the underlying consensus engine in the form of an [`[]abci.ValidatorUpdates`](https://docs.cometbft.com/v0.37/spec/abci/abci++_methods#endblock). This is the preferred way to implement custom validator changes. + +It is possible for developers to define the order of execution between the `BeginBlocker`/`EndBlocker` functions of each of their application's modules via the module's manager `SetOrderBeginBlocker`/`SetOrderEndBlocker` methods. For more on the module manager, click [here](./01-module-manager.md#manager). + +See an example implementation of `BeginBlocker` from the `distribution` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/distribution/abci.go#L14-L38 +``` + +and an example implementation of `EndBlocker` from the `staking` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/keeper/abci.go#L22-L27 +``` + + diff --git a/docs/build/building-modules/06-keeper.md b/docs/build/building-modules/06-keeper.md index 761171439..399ec648c 100644 --- a/docs/build/building-modules/06-keeper.md +++ b/docs/build/building-modules/06-keeper.md @@ -18,9 +18,9 @@ sidebar_position: 1 The Cosmos SDK is a framework that makes it easy for developers to build complex decentralized applications from scratch, mainly by composing modules together. As the ecosystem of open-source modules for the Cosmos SDK expands, it will become increasingly likely that some of these modules contain vulnerabilities, as a result of the negligence or malice of their developer. -The Cosmos SDK adopts an [object-capabilities-based approach](../../develop/advanced/10-ocap.md) to help developers better protect their application from unwanted inter-module interactions, and `keeper`s are at the core of this approach. A `keeper` can be considered quite literally to be the gatekeeper of a module's store(s). Each store (typically an [`IAVL` Store](../../develop/advanced/04-store.md#iavl-store)) defined within a module comes with a `storeKey`, which grants unlimited access to it. The module's `keeper` holds this `storeKey` (which should otherwise remain unexposed), and defines [methods](#implementing-methods) for reading and writing to the store(s). +The Cosmos SDK adopts an [object-capabilities-based approach](../../learn/advanced/10-ocap.md) to help developers better protect their application from unwanted inter-module interactions, and `keeper`s are at the core of this approach. A `keeper` can be considered quite literally to be the gatekeeper of a module's store(s). Each store (typically an [`IAVL` Store](../../learn/advanced/04-store.md#iavl-store)) defined within a module comes with a `storeKey`, which grants unlimited access to it. The module's `keeper` holds this `storeKey` (which should otherwise remain unexposed), and defines [methods](#implementing-methods) for reading and writing to the store(s). -The core idea behind the object-capabilities approach is to only reveal what is necessary to get the work done. In practice, this means that instead of handling permissions of modules through access-control lists, module `keeper`s are passed a reference to the specific instance of the other modules' `keeper`s that they need to access (this is done in the [application's constructor function](../../develop/beginner/00-app-anatomy.md#constructor-function)). As a consequence, a module can only interact with the subset of state defined in another module via the methods exposed by the instance of the other module's `keeper`. This is a great way for developers to control the interactions that their own module can have with modules developed by external developers. +The core idea behind the object-capabilities approach is to only reveal what is necessary to get the work done. In practice, this means that instead of handling permissions of modules through access-control lists, module `keeper`s are passed a reference to the specific instance of the other modules' `keeper`s that they need to access (this is done in the [application's constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function)). As a consequence, a module can only interact with the subset of state defined in another module via the methods exposed by the instance of the other module's `keeper`. This is a great way for developers to control the interactions that their own module can have with modules developed by external developers. ## Type Definition @@ -47,11 +47,11 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/keeper/keepe Let us go through the different parameters: * An expected `keeper` is a `keeper` external to a module that is required by the internal `keeper` of said module. External `keeper`s are listed in the internal `keeper`'s type definition as interfaces. These interfaces are themselves defined in an `expected_keepers.go` file in the root of the module's folder. In this context, interfaces are used to reduce the number of dependencies, as well as to facilitate the maintenance of the module itself. -* `storeKey`s grant access to the store(s) of the [multistore](../../develop/advanced/04-store.md) managed by the module. They should always remain unexposed to external modules. -* `cdc` is the [codec](../../develop/advanced/05-encoding.md) used to marshall and unmarshall structs to/from `[]byte`. The `cdc` can be any of `codec.BinaryCodec`, `codec.JSONCodec` or `codec.Codec` based on your requirements. It can be either a proto or amino codec as long as they implement these interfaces. +* `storeKey`s grant access to the store(s) of the [multistore](../../learn/advanced/04-store.md) managed by the module. They should always remain unexposed to external modules. +* `cdc` is the [codec](../../learn/advanced/05-encoding.md) used to marshall and unmarshall structs to/from `[]byte`. The `cdc` can be any of `codec.BinaryCodec`, `codec.JSONCodec` or `codec.Codec` based on your requirements. It can be either a proto or amino codec as long as they implement these interfaces. * The authority listed is a module account or user account that has the right to change module level parameters. Previously this was handled by the param module, which has been deprecated. -Of course, it is possible to define different types of internal `keeper`s for the same module (e.g. a read-only `keeper`). Each type of `keeper` comes with its own constructor function, which is called from the [application's constructor function](../../develop/beginner/00-app-anatomy.md). This is where `keeper`s are instantiated, and where developers make sure to pass correct instances of modules' `keeper`s to other modules that require them. +Of course, it is possible to define different types of internal `keeper`s for the same module (e.g. a read-only `keeper`). Each type of `keeper` comes with its own constructor function, which is called from the [application's constructor function](../../learn/beginner/00-app-anatomy.md). This is where `keeper`s are instantiated, and where developers make sure to pass correct instances of modules' `keeper`s to other modules that require them. ## Implementing Methods @@ -83,7 +83,7 @@ and the method will go through the following steps: For more, see an example of `keeper`'s [methods implementation from the `staking` module](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/keeper/keeper.go). -The [module `KVStore`](../../develop/advanced/04-store.md#kvstore-and-commitkvstore-interfaces) also provides an `Iterator()` method which returns an `Iterator` object to iterate over a domain of keys. +The [module `KVStore`](../../learn/advanced/04-store.md#kvstore-and-commitkvstore-interfaces) also provides an `Iterator()` method which returns an `Iterator` object to iterate over a domain of keys. This is an example from the `auth` module to iterate accounts: diff --git a/docs/build/building-modules/07-invariants.md b/docs/build/building-modules/07-invariants.md index cd85864bf..018796f71 100644 --- a/docs/build/building-modules/07-invariants.md +++ b/docs/build/building-modules/07-invariants.md @@ -79,7 +79,7 @@ Typically, this interface is implemented in the `keeper` of a specific module. T https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/crisis/keeper/keeper.go#L48-L50 ``` -The `InvariantRegistry` is therefore typically instantiated by instantiating the `keeper` of the `crisis` module in the [application's constructor function](../../develop/beginner/00-app-anatomy.md#constructor-function). +The `InvariantRegistry` is therefore typically instantiated by instantiating the `keeper` of the `crisis` module in the [application's constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). `Invariant`s can be checked manually via [`message`s](./02-messages-and-queries.md), but most often they are checked automatically at the end of each block. Here is an example from the `crisis` module: diff --git a/docs/build/building-modules/08-genesis.md b/docs/build/building-modules/08-genesis.md index 546eb5fb9..7abb21fb6 100644 --- a/docs/build/building-modules/08-genesis.md +++ b/docs/build/building-modules/08-genesis.md @@ -17,7 +17,7 @@ Modules generally handle a subset of the state and, as such, they need to define ## Type Definition -The subset of the genesis state defined from a given module is generally defined in a `genesis.proto` file ([more info](../../develop/advanced/05-encoding.md#gogoproto) on how to define protobuf messages). The struct defining the module's subset of the genesis state is usually called `GenesisState` and contains all the module-related values that need to be initialized during the genesis process. +The subset of the genesis state defined from a given module is generally defined in a `genesis.proto` file ([more info](../../learn/advanced/05-encoding.md#gogoproto) on how to define protobuf messages). The struct defining the module's subset of the genesis state is usually called `GenesisState` and contains all the module-related values that need to be initialized during the genesis process. See an example of `GenesisState` protobuf message definition from the `auth` module: @@ -49,9 +49,9 @@ Other than the methods related directly to `GenesisState`, module developers are ### `InitGenesis` -The `InitGenesis` method is executed during [`InitChain`](../../develop/advanced/00-baseapp.md#initchain) when the application is first started. Given a `GenesisState`, it initializes the subset of the state managed by the module by using the module's [`keeper`](./06-keeper.md) setter function on each parameter within the `GenesisState`. +The `InitGenesis` method is executed during [`InitChain`](../../learn/advanced/00-baseapp.md#initchain) when the application is first started. Given a `GenesisState`, it initializes the subset of the state managed by the module by using the module's [`keeper`](./06-keeper.md) setter function on each parameter within the `GenesisState`. -The [module manager](./01-module-manager.md#manager) of the application is responsible for calling the `InitGenesis` method of each of the application's modules in order. This order is set by the application developer via the manager's `SetOrderGenesisMethod`, which is called in the [application's constructor function](../../develop/beginner/00-app-anatomy.md#constructor-function). +The [module manager](./01-module-manager.md#manager) of the application is responsible for calling the `InitGenesis` method of each of the application's modules in order. This order is set by the application developer via the manager's `SetOrderGenesisMethod`, which is called in the [application's constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). See an example of `InitGenesis` from the `auth` module: diff --git a/docs/build/building-modules/09-module-interfaces.md b/docs/build/building-modules/09-module-interfaces.md index 262442661..61dd78ae6 100644 --- a/docs/build/building-modules/09-module-interfaces.md +++ b/docs/build/building-modules/09-module-interfaces.md @@ -16,11 +16,11 @@ This document details how to build CLI and REST interfaces for a module. Example ## CLI -One of the main interfaces for an application is the [command-line interface](../../develop/advanced/07-cli.md). This entrypoint adds commands from the application's modules enabling end-users to create [**messages**](./02-messages-and-queries.md#messages) wrapped in transactions and [**queries**](./02-messages-and-queries.md#queries). The CLI files are typically found in the module's `./client/cli` folder. +One of the main interfaces for an application is the [command-line interface](../../learn/advanced/07-cli.md). This entrypoint adds commands from the application's modules enabling end-users to create [**messages**](./02-messages-and-queries.md#messages) wrapped in transactions and [**queries**](./02-messages-and-queries.md#queries). The CLI files are typically found in the module's `./client/cli` folder. ### Transaction Commands -In order to create messages that trigger state changes, end-users must create [transactions](../../develop/advanced/01-transactions.md) that wrap and deliver the messages. A transaction command creates a transaction that includes one or more messages. +In order to create messages that trigger state changes, end-users must create [transactions](../../learn/advanced/01-transactions.md) that wrap and deliver the messages. A transaction command creates a transaction that includes one or more messages. Transaction commands typically have their own `tx.go` file that lives within the module's `./client/cli` folder. The commands are specified in getter functions and the name of the function should include the name of the command. @@ -43,7 +43,7 @@ In general, the getter function does the following: * If applicable, the command's arguments are parsed. In this example, the arguments `[to_address]` and `[amount]` are both parsed. * A [message](./02-messages-and-queries.md) is created using the parsed arguments and information from the `clientCtx`. The constructor function of the message type is called directly. In this case, `types.NewMsgSend(fromAddr, toAddr, amount)`. Its good practice to call, if possible, the necessary [message validation methods](../building-modules/03-msg-services.md#Validation) before broadcasting the message. * Depending on what the user wants, the transaction is either generated offline or signed and broadcasted to the preconfigured node using `tx.GenerateOrBroadcastTxCLI(clientCtx, flags, msg)`. -* **Adds transaction flags:** All transaction commands must add a set of transaction [flags](#flags). The transaction flags are used to collect additional information from the user (e.g. the amount of fees the user is willing to pay). The transaction flags are added to the constructed command using `AddTxFlagsToCmd(cmd)`. +* **Adds transaction flags:** All transaction commands must add a set of transaction flags. The transaction flags are used to collect additional information from the user (e.g. the amount of fees the user is willing to pay). The transaction flags are added to the constructed command using `AddTxFlagsToCmd(cmd)`. * **Returns the command:** Finally, the transaction command is returned. Each module can implement `NewTxCmd()`, which aggregates all of the transaction commands of the module. Here is an example from the `x/bank` module: @@ -101,7 +101,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/module.go#L84-L ### Flags -[Flags](../../develop/advanced/07-cli.md#flags) allow users to customize commands. `--fees` and `--gas-prices` are examples of flags that allow users to set the [fees](../../develop/beginner/04-gas-fees.md) and gas prices for their transactions. +[Flags](../../learn/advanced/07-cli.md#flags) allow users to customize commands. `--fees` and `--gas-prices` are examples of flags that allow users to set the [fees](../../learn/beginner/04-gas-fees.md) and gas prices for their transactions. Flags that are specific to a module are typically created in a `flags.go` file in the module's `./client/cli` folder. When creating a flag, developers set the value type, the name of the flag, the default value, and a description about the flag. Developers also have the option to mark flags as _required_ so that an error is thrown if the user does not include a value for the flag. @@ -159,6 +159,7 @@ Modules that want to expose REST queries should add `google.api.http` annotation https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/auth/v1beta1/query.proto#L14-L89 ``` + gRPC gateway is started in-process along with the application and CometBFT. It can be enabled or disabled by setting gRPC Configuration `enable` in [`app.toml`](../run-node/01-run-node.md#configuring-the-node-using-apptoml-and-configtoml). The Cosmos SDK provides a command for generating [Swagger](https://swagger.io/) documentation (`protoc-gen-swagger`). Setting `swagger` in [`app.toml`](../run-node/01-run-node.md#configuring-the-node-using-apptoml-and-configtoml) defines if swagger documentation should be automatically registered. diff --git a/docs/build/building-modules/13-upgrade.md b/docs/build/building-modules/13-upgrade.md index 703b4cd28..908a6a06e 100644 --- a/docs/build/building-modules/13-upgrade.md +++ b/docs/build/building-modules/13-upgrade.md @@ -5,12 +5,12 @@ sidebar_position: 1 # Upgrading Modules :::note Synopsis -[In-Place Store Migrations](../../develop/advanced/15-upgrade.md) allow your modules to upgrade to new versions that include breaking changes. This document outlines how to build modules to take advantage of this functionality. +[In-Place Store Migrations](../../learn/advanced/15-upgrade.md) allow your modules to upgrade to new versions that include breaking changes. This document outlines how to build modules to take advantage of this functionality. ::: :::note Pre-requisite Readings -* [In-Place Store Migration](../../develop/advanced/15-upgrade.md) +* [In-Place Store Migration](../../learn/advanced/15-upgrade.md) ::: diff --git a/docs/build/building-modules/14-simulator.md b/docs/build/building-modules/14-simulator.md index 5360709f6..14d3271b3 100644 --- a/docs/build/building-modules/14-simulator.md +++ b/docs/build/building-modules/14-simulator.md @@ -6,7 +6,7 @@ sidebar_position: 1 :::note Pre-requisite Readings -* [Cosmos Blockchain Simulator](../../develop/advanced/12-simulation.md) +* [Cosmos Blockchain Simulator](../../learn/advanced/12-simulation.md) ::: ## Synopsis @@ -37,7 +37,7 @@ for the key-value pairs from the stores to be decoded (_i.e_ unmarshalled) to their corresponding types. In particular, it matches the key to a concrete type and then unmarshals the value from the `KVPair` to the type provided. -You can use the example [here](https://github.com/cosmos/cosmos-sdk/blob/v/x/distribution/simulation/decoder.go) from the distribution module to implement your store decoders. +You can use the example [here](https://github.com/cosmos/cosmos-sdk/blob/main/x/distribution/simulation/decoder.go) from the distribution module to implement your store decoders. ### Randomized genesis @@ -48,13 +48,7 @@ Once the module genesis parameter are generated randomly (or with the key and values defined in a `params` file), they are marshaled to JSON format and added to the app genesis JSON to use it on the simulations. -You can check an example on how to create the randomized genesis [here](https://github.com/cosmos/cosmos-sdk/blob/v/x/staking/simulation/genesis.go). - -### Randomized parameter changes - -The simulator is able to test parameter changes at random. The simulator package from each module must contain a `RandomizedParams` func that will simulate parameter changes of the module throughout the simulations lifespan. - -You can see how an example of what is needed to fully test parameter changes [here](https://github.com/cosmos/cosmos-sdk/blob/v/x/staking/simulation/params.go) +You can check an example on how to create the randomized genesis [here](https://github.com/cosmos/cosmos-sdk/blob/main/x/staking/simulation/genesis.go). ### Random weighted operations @@ -62,7 +56,7 @@ Operations are one of the crucial parts of the Cosmos SDK simulation. They are t (`Msg`) that are simulated with random field values. The sender of the operation is also assigned randomly. -Operations on the simulation are simulated using the full [transaction cycle](../../develop/advanced/01-transactions.md) of a +Operations on the simulation are simulated using the full [transaction cycle](../../learn/advanced/01-transactions.md) of a `ABCI` application that exposes the `BaseApp`. Shown below is how weights are set: diff --git a/docs/build/building-modules/17-preblock.md b/docs/build/building-modules/17-preblock.md new file mode 100644 index 000000000..a79646bd4 --- /dev/null +++ b/docs/build/building-modules/17-preblock.md @@ -0,0 +1,31 @@ +--- +sidebar_position: 1 +--- + +# PreBlocker + +:::note Synopsis +`PreBlocker` is optional method module developers can implement in their module. They will be triggered before [`BeginBlock`](../../learn/advanced/00-baseapp.md#beginblock). +::: + +:::note Pre-requisite Readings + +* [Module Manager](./01-module-manager.md) + +::: + +## PreBlocker + +There are two semantics around the new lifecycle method: + +- It runs before the `BeginBlocker` of all modules +- It can modify consensus parameters in storage, and signal the caller through the return value. + +When it returns `ConsensusParamsChanged=true`, the caller must refresh the consensus parameter in the deliver context: +``` +app.finalizeBlockState.ctx = app.finalizeBlockState.ctx.WithConsensusParams(app.GetConsensusParams()) +``` + +The new ctx must be passed to all the other lifecycle methods. + + diff --git a/docs/build/migrations/02-upgrading.md b/docs/build/migrations/02-upgrading.md index 19ef4bb3c..792e7f826 100644 --- a/docs/build/migrations/02-upgrading.md +++ b/docs/build/migrations/02-upgrading.md @@ -5,6 +5,45 @@ Note, always read the **SimApp** section for more information on application wir ## [Unreleased] +### Params + +Params Migrations were removed. It is required to migrate to 0.50 prior to upgrading to .51. + +### SimApp + +In this section we describe the changes made in Cosmos SDK' SimApp. +**These changes are directly applicable to your application wiring.** + +#### Client (`root.go`) + +The `client` package has been refactored to make use of the address codecs (address, validator address, consensus address, etc.). +This is part of the work of abstracting the SDK from the global bech32 config. + +This means the address codecs must be provided in the `client.Context` in the application client (usually `root.go`). + +```diff +clientCtx = clientCtx. ++ WithAddressCodec(addressCodec). ++ WithValidatorAddressCodec(validatorAddressCodec). ++ WithConsensusAddressCodec(consensusAddressCodec) +``` + +**When using `depinject` / `app v2`, the client codecs can be provided directly from application config.** + +Refer to SimApp `root_v2.go` and `root.go` for an example with an app v2 and a legacy app. + +### Modules + +#### `**all**` + +##### Migration to Collections + +Most of Cosmos SDK modules have migrated to [collections](https://docs.cosmos.network/main/packages/collections). +Many functions have been removed due to this changes as the API can be smaller thanks to collections. +For modules that have migrated, verify you are checking against `collections.ErrNotFound` when applicable. + +## [v0.50.x](https://github.com/cosmos/cosmos-sdk/releases/tag/v0.50.0-alpha.0) + ### Migration to CometBFT (Part 2) The Cosmos SDK has migrated in its previous versions, to CometBFT. @@ -18,42 +57,135 @@ Following an exhaustive list: * Package `client/grpc/tmservice` -> `client/grpc/cmtservice` Additionally, the commands and flags mentioning `tendermint` have been renamed to `comet`. -However, these commands and flags is still supported for backward compatibility. +These commands and flags are still supported for backward compatibility. For backward compatibility, the `**/tendermint/**` gRPC services are still supported. Additionally, the SDK is starting its abstraction from CometBFT Go types thorought the codebase: -* The usage of CometBFT have been replaced to use the Cosmos SDK logger interface (`cosmossdk.io/log.Logger`). -* The usage of `github.com/cometbft/cometbft/libs/bytes.HexByte` have been replaced by `[]byte`. +* The usage of the CometBFT logger has been replaced by the Cosmos SDK logger interface (`cosmossdk.io/log.Logger`). +* The usage of `github.com/cometbft/cometbft/libs/bytes.HexByte` has been replaced by `[]byte`. +* Usage of an application genesis (see [genutil](#xgenutil)). -### Configuration +#### Enable Vote Extensions -A new tool have been created for migrating configuration of the SDK. Use the following command to migrate your configuration: +:::tip +This is an optional feature that is disabled by default. +::: -```bash -simd config migrate v0.48 +Once all the code changes required to implement Vote Extensions are in place, +they can be enabled by setting the consensus param `Abci.VoteExtensionsEnableHeight` +to a value greater than zero. + +In a new chain, this can be done in the `genesis.json` file. + +For existing chains this can be done in two ways: + +* During an upgrade the value is set in an upgrade handler. +* A governance proposal that changes the consensus param **after a coordinated upgrade has taken place**. + +### BaseApp + +All ABCI methods now accept a pointer to the request and response types defined +by CometBFT. In addition, they also return errors. An ABCI method should only +return errors in cases where a catastrophic failure has occurred and the application +should halt. However, this is abstracted away from the application developer. Any +handler that an application can define or set that returns an error, will gracefully +by handled by `BaseApp` on behalf of the application. + +BaseApp calls of `BeginBlock` & `Endblock` are now private but are still exposed +to the application to define via the `Manager` type. `FinalizeBlock` is public +and should be used in order to test and run operations. This means that although +`BeginBlock` & `Endblock` no longer exist in the ABCI interface, they are automatically +called by `BaseApp` during `FinalizeBlock`. Specifically, the order of operations +is `BeginBlock` -> `DeliverTx` (for all txs) -> `EndBlock`. + +ABCI++ 2.0 also brings `ExtendVote` and `VerifyVoteExtension` ABCI methods. These +methods allow applications to extend and verify pre-commit votes. The Cosmos SDK +allows an application to define handlers for these methods via `ExtendVoteHandler` +and `VerifyVoteExtensionHandler` respectively. Please see [here](https://docs.cosmos.network/v0.50/building-apps/vote-extensions) +for more info. + +#### Set PreBlocker + +A `SetPreBlocker` method has been added to BaseApp. This is essential for BaseApp to run `PreBlock` which runs before begin blocker other modules, and allows to modify consensus parameters, and the changes are visible to the following state machine logics. +Read more about other use cases [here](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-068-preblock.md). + +`depinject` / app v2 users need to add `x/upgrade` in their `app_config.go` / `app.yml`: + +```diff ++ PreBlockers: []string{ ++ upgradetypes.ModuleName, ++ }, +BeginBlockers: []string{ +- upgradetypes.ModuleName, + minttypes.ModuleName, +} ``` -More information about [confix](https://docs.cosmos.network/main/tooling/confix). +When using (legacy) application wiring, the following must be added to `app.go`: + +```diff ++app.ModuleManager.SetOrderPreBlockers( ++ upgradetypes.ModuleName, ++) + +app.ModuleManager.SetOrderBeginBlockers( +- upgradetypes.ModuleName, +) + ++ app.SetPreBlocker(app.PreBlocker) + +// ... // + ++func (app *SimApp) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeBlock) (*sdk.ResponsePreBlock, error) { ++ return app.ModuleManager.PreBlock(ctx, req) ++} +``` #### Events -The log section of abci.TxResult is not populated in the case of successful msg(s) execution. Instead a new attribute is added to all messages indicating the `msg_index` which identifies which events and attributes relate the same transaction +The log section of `abci.TxResult` is not populated in the case of successful +msg(s) execution. Instead a new attribute is added to all messages indicating +the `msg_index` which identifies which events and attributes relate the same +transaction. + +`BeginBlock` & `EndBlock` Events are now emitted through `FinalizeBlock` but have +an added attribute, `mode=BeginBlock|EndBlock`, to identify if the event belongs +to `BeginBlock` or `EndBlock`. + +### Config files + +Confix is a new SDK tool for modifying and migrating configuration of the SDK. +It is the replacement of the `config.Cmd` command from the `client/config` package. + +Use the following command to migrate your configuration: + +```bash +simd config migrate v0.50 +``` + +If you were using ` config [key]` or ` config [key] [value]` to set and get values from the `client.toml`, replace it with ` config get client [key]` and ` config set client [key] [value]`. The extra verbosity is due to the extra functionalities added in config. + +More information about [confix](https://docs.cosmos.network/main/tooling/confix) and how to add it in your application binary in the [documentation](https://docs.cosmos.network/main/tooling/confix). #### gRPC-Web -gRPC-Web is now listening to the same address as the gRPC Gateway API server (default: `localhost:1317`). +gRPC-Web is now listening to the same address and port as the gRPC Gateway API server (default: `localhost:1317`). The possibility to listen to a different address has been removed, as well as its settings. Use `confix` to clean-up your `app.toml`. A nginx (or alike) reverse-proxy can be set to keep the previous behavior. #### Database Support -ClevelDB, BoltDB and BadgerDB are not supported anymore. To migrate from a unsupported database to a supported database please use the database migration tool. +ClevelDB, BoltDB and BadgerDB are not supported anymore. To migrate from a unsupported database to a supported database please use a database migration tool. ### Protobuf -The SDK is in the process of removing all `gogoproto` annotations. +With the deprecation of the Amino JSON codec defined in [cosmos/gogoproto](https://github.com/cosmos/gogoproto) in favor of the protoreflect powered x/tx/aminojson codec, module developers are encouraged verify that their messages have the correct protobuf annotations to deterministically produce identical output from both codecs. + +For core SDK types equivalence is asserted by generative testing of [SignableTypes](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-beta.0/tests/integration/rapidgen/rapidgen.go#L102) in [TestAminoJSON_Equivalence](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-beta.0/tests/integration/tx/aminojson/aminojson_test.go#L94). + +**TODO: summarize proto annotation requirements.** #### Stringer @@ -62,14 +194,14 @@ The `gogoproto.goproto_stringer = false` annotation has been removed from most p ### SimApp - +In this section we describe the changes made in Cosmos SDK' SimApp. +**These changes are directly applicable to your application wiring.** #### Module Assertions -Previously, all modules were required to be set in `OrderBeginBlockers`, `OrderEndBlockers` and `OrderInitGenesis / OrderExportGenesis` in `app.go` / `app_config.go`. -This is no longer the case, the assertion has been loosened to only require modules implementing, respectively, the `module.BeginBlockAppModule`, `module.EndBlockAppModule` and `module.HasGenesis` interfaces. +Previously, all modules were required to be set in `OrderBeginBlockers`, `OrderEndBlockers` and `OrderInitGenesis / OrderExportGenesis` in `app.go` / `app_config.go`. This is no longer the case, the assertion has been loosened to only require modules implementing, respectively, the `appmodule.HasBeginBlocker`, `appmodule.HasEndBlocker` and `appmodule.HasGenesis` / `module.HasGenesis` interfaces. -#### Modules Keepers +#### Module wiring The following modules `NewKeeper` function now take a `KVStoreService` instead of a `StoreKey`: @@ -77,11 +209,19 @@ The following modules `NewKeeper` function now take a `KVStoreService` instead o * `x/authz` * `x/bank` * `x/consensus` +* `x/crisis` * `x/distribution` +* `x/evidence` * `x/feegrant` +* `x/gov` +* `x/mint` * `x/nft` +* `x/slashing` +* `x/upgrade` + +**Users using `depinject` / app v2 do not need any changes, this is abstracted for them.** -User manually wiring their chain need to use the `runtime.NewKVStoreService` method to create a `KVStoreService` from a `StoreKey`: +Users manually wiring their chain need to use the `runtime.NewKVStoreService` method to create a `KVStoreService` from a `StoreKey`: ```diff app.ConsensusParamsKeeper = consensusparamkeeper.NewKeeper( @@ -92,21 +232,11 @@ app.ConsensusParamsKeeper = consensusparamkeeper.NewKeeper( ) ``` -The following modules' `Keeper` methods now take in a `context.Context` instead of `sdk.Context`. Any module that has an interfaces for them (like "expected keepers") will need to update and re-generate mocks if needed: - -* `x/authz` -* `x/bank` -* `x/distribution` - -**Users using depinject do not need any changes, this is automatically done for them.** - #### Logger -The following modules `NewKeeper` function now take a `log.Logger`: +Replace all your CometBFT logger imports by `cosmossdk.io/log`. -* `x/bank` - -`depinject` users must now supply the logger through the main `depinject.Supply` function instead of passing it to `appBuilder.Build`. +Additionally, `depinject` / app v2 users must now supply a logger through the main `depinject.Supply` function instead of passing it to `appBuilder.Build`. ```diff appConfig = depinject.Configs( @@ -123,14 +253,14 @@ appConfig = depinject.Configs( + app.App = appBuilder.Build(db, traceStore, baseAppOptions...) ``` -User manually wiring their chain need to add the logger argument when creating the keeper. +User manually wiring their chain need to add the logger argument when creating the `x/bank` keeper. #### Module Basics Previously, the `ModuleBasics` was a global variable that was used to register all modules's `AppModuleBasic` implementation. The global variable has been removed and the basic module manager can be now created from the module manager. -This is automatically done for depinject users, however for supplying different app module implementation, pass them via `depinject.Supply` in the main `AppConfig` (`app_config.go`): +This is automatically done for `depinject` / app v2 users, however for supplying different app module implementation, pass them via `depinject.Supply` in the main `AppConfig` (`app_config.go`): ```go depinject.Supply( @@ -148,66 +278,196 @@ depinject.Supply( Users manually wiring their chain need to use the new `module.NewBasicManagerFromManager` function, after the module manager creation, and pass a `map[string]module.AppModuleBasic` as argument for optionally overridding some module's `AppModuleBasic`. +#### AutoCLI + +[`AutoCLI`](https://docs.cosmos.network/main/core/autocli) has been implemented by the SDK for all its module CLI queries. This means chains must add the following in their `root.go` to enable `AutoCLI` in their application: + +```go +if err := autoCliOpts.EnhanceRootCommand(rootCmd); err != nil { + panic(err) +} +``` + +Where `autoCliOpts` is the autocli options of the app, containing all modules and codecs. +That value can injected by depinject ([see root_v2.go](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-beta.0/simapp/simd/cmd/root_v2.go#L49-L67)) or manually provided by the app ([see legacy app.go](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-beta.0/simapp/app.go#L636-L655)). + +:::warning +Not doing this will result in all core SDK modules queries not to be included in the binary. +::: + +Additionally `AutoCLI` automatically adds the custom modules commands to the root command for all modules implementing the [`appmodule.AppModule`](https://pkg.go.dev/cosmossdk.io/core/appmodule#AppModule) interface. +This means, after ensuring all the used modules implement this interface, the following can be removed from your `root.go`: + +```diff +func txCommand() *cobra.Command { + .... +- appd.ModuleBasics.AddTxCommands(cmd) +} +``` + +```diff +func queryCommand() *cobra.Command { + .... +- appd.ModuleBasics.AddQueryCommands(cmd) +} +``` + ### Packages +#### Math + +References to `types/math.go` which contained aliases for math types aliasing the `cosmossdk.io/math` package have been removed. +Import directly the `cosmossdk.io/math` package instead. + #### Store -References to `types/store.go` which contained aliases for store types have been remapped to point to appropriate store/types, hence the `types/store.go` file is no longer needed and has been removed. +References to `types/store.go` which contained aliases for store types have been remapped to point to appropriate `store/types`, hence the `types/store.go` file is no longer needed and has been removed. ##### Extract Store to a standalone module -The `store` module is extracted to have a separate go.mod file which allows it be a standalone module. +The `store` module is extracted to have a separate go.mod file which allows it be a standalone module. All the store imports are now renamed to use `cosmossdk.io/store` instead of `github.com/cosmos/cosmos-sdk/store` across the SDK. +##### Streaming + +[ADR-38](https://docs.cosmos.network/main/architecture/adr-038-state-listening) has been implemented in the SDK. + +To continue using state streaming, replace `streaming.LoadStreamingServices` by the following in your `app.go`: + +```go +if err := app.RegisterStreamingServices(appOpts, app.kvStoreKeys()); err != nil { + panic(err) +} +``` + #### Client -The return type of the interface method `TxConfig.SignModeHandler()` has been changed from `x/auth/signing.SignModeHandler` to `x/tx/signing.HandlerMap`. This change is transparent to most users as the `TxConfig` interface is typically implemented by private `x/auth/tx.config` struct (as returned by `auth.NewTxConfig`) which has been updated to return the new type. If users have implemented their own `TxConfig` interface, they will need to update their implementation to return the new type. +The return type of the interface method `TxConfig.SignModeHandler()` has been changed from `x/auth/signing.SignModeHandler` to `x/tx/signing.HandlerMap`. This change is transparent to most users as the `TxConfig` interface is typically implemented by private `x/auth/tx.config` struct (as returned by `auth.NewTxConfig`) which has been updated to return the new type. If users have implemented their own `TxConfig` interface, they will need to update their implementation to return the new type. ### Modules #### `**all**` -[RFC 001](https://docs.cosmos.network/main/rfc/rfc-001-tx-validation) has defined a simplification of the message validation process for modules. -The `sdk.Msg` interface has been updated to not require the implementation of the `ValidateBasic` method. -It is now recommended to validate message directly in the message server. When the validation is performed in the message server, the `ValidateBasic` method on a message is no longer required and can be removed. +* [RFC 001](https://docs.cosmos.network/main/rfc/rfc-001-tx-validation) has defined a simplification of the message validation process for modules. + The `sdk.Msg` interface has been updated to not require the implementation of the `ValidateBasic` method. + It is now recommended to validate message directly in the message server. When the validation is performed in the message server, the `ValidateBasic` method on a message is no longer required and can be removed. + +* Messages no longer need to implement the `LegacyMsg` interface and implementations of `GetSignBytes` can be deleted. Because of this change, global legacy Amino codec definitions and their registration in `init()` can safely be removed as well. + +* The `AppModuleBasic` interface has been simplifed. Defining `GetTxCmd() *cobra.Command` and `GetQueryCmd() *cobra.Command` is no longer required. The module manager detects when module commands are defined. If AutoCLI is enabled, `EnhanceRootCommand()` will add the auto-generated commands to the root command, unless a custom module command is defined and register that one instead. + +* The following modules' `Keeper` methods now take in a `context.Context` instead of `sdk.Context`. Any module that has an interfaces for them (like "expected keepers") will need to update and re-generate mocks if needed: + + * `x/authz` + * `x/bank` + * `x/mint` + * `x/crisis` + * `x/distribution` + * `x/evidence` + * `x/gov` + * `x/slashing` + * `x/upgrade` + +* `BeginBlock` and `EndBlock` have changed their signature, so it is important that any module implementing them are updated accordingly. + +```diff +- BeginBlock(sdk.Context, abci.RequestBeginBlock) ++ BeginBlock(context.Context) error +``` + +```diff +- EndBlock(sdk.Context, abci.RequestEndBlock) []abci.ValidatorUpdate ++ EndBlock(context.Context) error +``` + +In case a module requires to return `abci.ValidatorUpdate` from `EndBlock`, it can use the `HasABCIEndBlock` interface instead. + +```diff +- EndBlock(sdk.Context, abci.RequestEndBlock) []abci.ValidatorUpdate ++ EndBlock(context.Context) ([]abci.ValidatorUpdate, error) +``` + +:::tip +It is possible to ensure that a module implements the correct interfaces by using compiler assertions in your `x/{moduleName}/module.go`: + +```go +var ( + _ module.AppModuleBasic = (*AppModule)(nil) + _ module.AppModuleSimulation = (*AppModule)(nil) + _ module.HasGenesis = (*AppModule)(nil) + + _ appmodule.AppModule = (*AppModule)(nil) + _ appmodule.HasBeginBlocker = (*AppModule)(nil) + _ appmodule.HasEndBlocker = (*AppModule)(nil) + ... +) +``` + +Read more on those interfaces [here](https://docs.cosmos.network/v0.50/building-modules/module-manager#application-module-interfaces). + +::: + +* `GetSigners()` is no longer required to be implemented on `Msg` types. The SDK will automatically infer the signers from the `Signer` field on the message. The signer field is required on all messages unless using a custom signer function. + +To find out more please read the [signer field](../../build/building-modules/05-protobuf-annotations.md#signer) & [here](https://github.com/cosmos/cosmos-sdk/blob/7352d0bce8e72121e824297df453eb1059c28da8/docs/docs/build/building-modules/02-messages-and-queries.md#L40) documentation. + #### `x/auth` -For ante handler construction via `ante.NewAnteHandler`, the field `ante.HandlerOptions.SignModeHandler` has been updated to `x/tx/signing/HandlerMap` from `x/auth/signing/SignModeHandler`. Callers typically fetch this value from `client.TxConfig.SignModeHandler()` (which is also changed) so this change should be transparent to most users. +For ante handler construction via `ante.NewAnteHandler`, the field `ante.HandlerOptions.SignModeHandler` has been updated to `x/tx/signing/HandlerMap` from `x/auth/signing/SignModeHandler`. Callers typically fetch this value from `client.TxConfig.SignModeHandler()` (which is also changed) so this change should be transparent to most users. #### `x/capability` -Capability was moved to [IBC-GO](https://github.com/cosmos/ibc-go). IBC V8 will contain the necessary changes to incorporate the new module location +Capability has been moved to [IBC Go](https://github.com/cosmos/ibc-go). IBC v8 will contain the necessary changes to incorporate the new module location. + +#### `x/genutil` + +The Cosmos SDK has migrated from a CometBFT genesis to a application managed genesis file. +The genesis is now fully handled by `x/genutil`. This has no consequences for running chains: + +* Importing a CometBFT genesis is still supported. +* Exporting a genesis now exports the genesis as an application genesis. + +When needing to read an application genesis, use the following helpers from the `x/genutil/types` package: + +```go +// AppGenesisFromReader reads the AppGenesis from the reader. +func AppGenesisFromReader(reader io.Reader) (*AppGenesis, error) + +// AppGenesisFromFile reads the AppGenesis from the provided file. +func AppGenesisFromFile(genFile string) (*AppGenesis, error) +``` #### `x/gov` ##### Expedited Proposals -The `gov` v1 module has been updated to support the ability to expedite governance proposals. When a proposal is expedited, the voting period will be shortened to `ExpeditedVotingPeriod` parameter. An expedited proposal must have an higher voting threshold than a classic proposal, that threshold is defined with the `ExpeditedThreshold` parameter. +The `gov` v1 module now supports expedited governance proposals. When a proposal is expedited, the voting period will be shortened to `ExpeditedVotingPeriod` parameter. An expedited proposal must have an higher voting threshold than a classic proposal, that threshold is defined with the `ExpeditedThreshold` parameter. ##### Cancelling Proposals -The `gov` module has been updated to support the ability to cancel governance proposals. When a proposal is canceled, all the deposits of the proposal are either burnt or sent to `ProposalCancelDest` address. The deposits burn rate will be determined by a new parameter called `ProposalCancelRatio` parameter. +The `gov` module now supports cancelling governance proposals. When a proposal is canceled, all the deposits of the proposal are either burnt or sent to `ProposalCancelDest` address. The deposits burn rate will be determined by a new parameter called `ProposalCancelRatio` parameter. ```text - 1. deposits * proposal_cancel_ratio will be burned or sent to `ProposalCancelDest` address , if `ProposalCancelDest` is empty then deposits will be burned. - 2. deposits * (1 - proposal_cancel_ratio) will be sent to depositors. +1. deposits * proposal_cancel_ratio will be burned or sent to `ProposalCancelDest` address , if `ProposalCancelDest` is empty then deposits will be burned. +2. deposits * (1 - proposal_cancel_ratio) will be sent to depositors. ``` -By default, the new `ProposalCancelRatio` parameter is set to 0.5 during migration and `ProposalCancelDest` is set to empty string (i.e. burnt). +By default, the new `ProposalCancelRatio` parameter is set to `0.5` during migration and `ProposalCancelDest` is set to empty string (i.e. burnt). #### `x/evidence` ##### Extract evidence to a standalone module -The `x/evidence` module is extracted to have a separate go.mod file which allows it be a standalone module. +The `x/evidence` module is extracted to have a separate go.mod file which allows it be a standalone module. All the evidence imports are now renamed to use `cosmossdk.io/x/evidence` instead of `github.com/cosmos/cosmos-sdk/x/evidence` across the SDK. #### `x/nft` ##### Extract nft to a standalone module -The `x/nft` module is extracted to have a separate go.mod file which allows it to be a standalone module. +The `x/nft` module is extracted to have a separate go.mod file which allows it to be a standalone module. +All the evidence imports are now renamed to use `cosmossdk.io/x/nft` instead of `github.com/cosmos/cosmos-sdk/x/nft` across the SDK. #### x/feegrant @@ -220,9 +480,17 @@ All the feegrant imports are now renamed to use `cosmossdk.io/x/feegrant` instea ##### Extract upgrade to a standalone module -The `x/upgrade` module is extracted to have a separate go.mod file which allows it to be a standalone module. +The `x/upgrade` module is extracted to have a separate go.mod file which allows it to be a standalone module. All the upgrade imports are now renamed to use `cosmossdk.io/x/upgrade` instead of `github.com/cosmos/cosmos-sdk/x/upgrade` across the SDK. +### Tooling + +#### Rosetta + +Rosetta has moved to it's own [repo](https://github.com/cosmos/rosetta) and not imported by the Cosmos SDK SimApp by default. +Any user who is interested on using the tool can connect it standalone to any node without the need to add it as part of the node binary. +The rosetta tool also allows multi chain connections. + ## [v0.47.x](https://github.com/cosmos/cosmos-sdk/releases/tag/v0.47.0) ### Migration to CometBFT (Part 1) @@ -237,7 +505,7 @@ Due to the import changes, this is a breaking change. Chains need to remove **en * Run `make proto-gen` Other than that, the migration should be seamless. -On the SDK side, clean-up of variables, functions to reflect the new name will only happen from v0.48 (part 2). +On the SDK side, clean-up of variables, functions to reflect the new name will only happen from v0.50 (part 2). Note: It is possible that these steps must first be performed by your dependencies before you can perform them on your own codebase. @@ -323,7 +591,7 @@ This allows you to remove the replace directive `replace github.com/gogo/protobu Please use the `ghcr.io/cosmos/proto-builder` image (version >= `0.11.5`) for generating protobuf files. -See which buf commit for `cosmos/cosmos-sdk` to pin in your `buf.yaml` file [here](../tooling). +See which buf commit for `cosmos/cosmos-sdk` to pin in your `buf.yaml` file [here](https://github.com/cosmos/cosmos-sdk/blob/main/proto/README.md). #### Gogoproto Import Paths @@ -358,7 +626,7 @@ Here are the following replacements that you need to perform on your proto files Please also check that in your own app's proto files that there are no single-word names for those two proto annotations. If so, then replace them with fully-qualified names, even though those names don't actually resolve to an actual protobuf entity. -For more information, see the [encoding guide](../../develop/advanced/06-encoding.md). +For more information, see the [encoding guide](https://github.com/cosmos/cosmos-sdk/blob/main/docs/learn/advanced/05-encoding.md). ### Transactions @@ -394,11 +662,24 @@ In case a module does not follow the standard message path, (e.g. IBC), it is ad The `params` module was deprecated since v0.46. The Cosmos SDK has migrated away from `x/params` for its own modules. Cosmos SDK modules now store their parameters directly in its repective modules. -The `params` module will be removed in `v0.48`, as mentioned [in v0.46 release](https://github.com/cosmos/cosmos-sdk/blob/v0.46.1/UPGRADING.md#xparams). It is strongly encouraged to migrate away from `x/params` before `v0.48`. +The `params` module will be removed in `v0.50`, as mentioned [in v0.46 release](https://github.com/cosmos/cosmos-sdk/blob/v0.46.1/UPGRADING.md#xparams). It is strongly encouraged to migrate away from `x/params` before `v0.50`. When performing a chain migration, the params table must be initizalied manually. This was done in the modules keepers in previous versions. Have a look at `simapp.RegisterUpgradeHandlers()` for an example. +#### `x/crisis` + +With the migrations of all modules away from `x/params`, the crisis module now has a store. +The store must be created during a chain upgrade to v0.47.x. + +```go +storetypes.StoreUpgrades{ + Added: []string{ + crisistypes.ModuleName, + }, +} +``` + #### `x/gov` ##### Minimum Proposal Deposit At Time of Submission @@ -407,11 +688,11 @@ The `gov` module has been updated to support a minimum proposal deposit at submi parameter called `MinInitialDepositRatio`. When multiplied by the existing `MinDeposit` parameter, it produces the necessary proportion of coins needed at the proposal submission time. The motivation for this change is to prevent proposal spamming. -By default, the new `MinInitialDepositRatio` parameter is set to zero during migration. The value of zero signifies that this -feature is disabled. If chains wish to utilize the minimum proposal deposits at time of submission, the migration logic needs to be +By default, the new `MinInitialDepositRatio` parameter is set to zero during migration. The value of zero signifies that this +feature is disabled. If chains wish to utilize the minimum proposal deposits at time of submission, the migration logic needs to be modified to set the new parameter to the desired value. -##### New Proposal.Proposer field +##### New `Proposal.Proposer` field The `Proposal` proto has been updated with proposer field. For proposal state migraton developers can call `v4.AddProposerAddressToProposal` in their upgrade handler to update all existing proposal and make them compatible and **this migration is optional**. @@ -467,7 +748,17 @@ func (app SimApp) RegisterUpgradeHandlers() { } ``` -The old params module is required to still be imported in your app.go in order to handle this migration. +The `x/params` module should still be imported in your app.go in order to handle this migration. + +Because the `x/consensus` module is a new module, its store must be added while upgrading to v0.47.x: + +```go +storetypes.StoreUpgrades{ + Added: []string{ + consensustypes.ModuleName, + }, +} +``` ##### `app.go` changes @@ -539,13 +830,11 @@ The keyring has been refactored in v0.46. * The keys' implementation has been refactored to be serialized as proto. * `keyring.NewInMemory` and `keyring.New` takes now a `codec.Codec`. * Take `keyring.Record` instead of `Info` as first argument in: - * `MkConsKeyOutput` - * `MkValKeyOutput` - * `MkAccKeyOutput` + _ `MkConsKeyOutput` + _ `MkValKeyOutput` \* `MkAccKeyOutput` * Rename: - * `SavePubKey` to `SaveOfflineKey` and remove the `algo` argument. - * `NewMultiInfo`, `NewLedgerInfo` to `NewLegacyMultiInfo`, `newLegacyLedgerInfo` respectively. - * `NewOfflineInfo` to `newLegacyOfflineInfo` and move it to `migration_test.go`. + _ `SavePubKey` to `SaveOfflineKey` and remove the `algo` argument. + _ `NewMultiInfo`, `NewLedgerInfo` to `NewLegacyMultiInfo`, `newLegacyLedgerInfo` respectively. \* `NewOfflineInfo` to `newLegacyOfflineInfo` and move it to `migration_test.go`. ### PostHandler @@ -572,7 +861,7 @@ mistakes. #### `x/params` -* The `x/params` module has been depreacted in favour of each module housing and providing way to modify their parameters. Each module that has parameters that are changable during runtime have an authority, the authority can be a module or user account. The Cosmos SDK team recommends migrating modules away from using the param module. An example of how this could look like can be found [here](https://github.com/cosmos/cosmos-sdk/pull/12363). +* The `x/params` module has been depreacted in favour of each module housing and providing way to modify their parameters. Each module that has parameters that are changable during runtime have an authority, the authority can be a module or user account. The Cosmos SDK team recommends migrating modules away from using the param module. An example of how this could look like can be found [here](https://github.com/cosmos/cosmos-sdk/pull/12363). * The Param module will be maintained until April 18, 2023. At this point the module will reach end of life and be removed from the Cosmos SDK. #### `x/gov` @@ -595,7 +884,7 @@ Instead, the SDK uses [`buf`](https://buf.build). Clients should have their own The protos can as well be downloaded using `buf export buf.build/cosmos/cosmos-sdk:8cb30a2c4de74dc9bd8d260b1e75e176 --output `. -Cosmos message protobufs should be extended with `cosmos.msg.v1.signer`: +Cosmos message protobufs should be extended with `cosmos.msg.v1.signer`: ```protobuf message MsgSetWithdrawAddress { diff --git a/docs/build/migrations/_category_.json b/docs/build/migrations/_category_.json index 258dc810c..5a06c3ebb 100644 --- a/docs/build/migrations/_category_.json +++ b/docs/build/migrations/_category_.json @@ -1,5 +1,5 @@ { - "label": "SDK Migrations", + "label": "Migrations", "position": 3, "link": null -} \ No newline at end of file +} diff --git a/docs/build/modules/README.md b/docs/build/modules/README.md index 5638335f1..157836977 100644 --- a/docs/build/modules/README.md +++ b/docs/build/modules/README.md @@ -1,28 +1,29 @@ --- sidebar_position: 0 +slug : /modules --- -# Module Summary +# List of Modules Here are some production-grade modules that can be used in Cosmos SDK applications, along with their respective documentation: -* [Auth](auth/README.md) - Authentication of accounts and transactions for Cosmos SDK applications. -* [Authz](authz/README.md) - Authorization for accounts to perform actions on behalf of other accounts. -* [Bank](bank/README.md) - Token transfer functionalities. -* [Crisis](crisis/README.md) - Halting the blockchain under certain circumstances (e.g. if an invariant is broken). -* [Distribution](distribution/README.md) - Fee distribution, and staking token provision distribution. -* [Evidence](evidence/README.md) - Evidence handling for double signing, misbehaviour, etc. -* [Feegrant](feegrant/README.md) - Grant fee allowances for executing transactions. -* [Governance](gov/README.md) - On-chain proposals and voting. -* [Mint](mint/README.md) - Creation of new units of staking token. -* [Params](params/README.md) - Globally available parameter store. -* [Slashing](slashing/README.md) - Validator punishment mechanisms. -* [Staking](staking/README.md) - Proof-of-Stake layer for public blockchains. -* [Upgrade](upgrade/README.md) - Software upgrades handling and coordination. -* [NFT](nft/README.md) - NFT module implemented based on [ADR43](https://docs.cosmos.network/main/architecture/adr-043-nft-module.html). -* [Consensus](consensus/README.md) - Consensus module for modifying CometBFT's ABCI consensus params. -* [Circuit](circuit/README.md) - Circuit breaker module for pausing messages. -* [Genutil](genutil/README.md) - Genesis utilities for the Cosmos SDK. +* [Auth](./auth/README.md) - Authentication of accounts and transactions for Cosmos SDK applications. +* [Authz](./authz/README.md) - Authorization for accounts to perform actions on behalf of other accounts. +* [Bank](./bank/README.md) - Token transfer functionalities. +* [Crisis](./crisis/README.md) - Halting the blockchain under certain circumstances (e.g. if an invariant is broken). +* [Distribution](./distribution/README.md) - Fee distribution, and staking token provision distribution. +* [Evidence](./evidence/README.md) - Evidence handling for double signing, misbehaviour, etc. +* [Feegrant](./feegrant/README.md) - Grant fee allowances for executing transactions. +* [Governance](./gov/README.md) - On-chain proposals and voting. +* [Mint](./mint/README.md) - Creation of new units of staking token. +* [Params](./params/README.md) - Globally available parameter store. +* [Slashing](./slashing/README.md) - Validator punishment mechanisms. +* [Staking](./staking/README.md) - Proof-of-Stake layer for public blockchains. +* [Upgrade](./upgrade/README.md) - Software upgrades handling and coordination. +* [NFT](./nft/README.md) - NFT module implemented based on [ADR43](https://docs.cosmos.network/main/architecture/adr-043-nft-module.html). +* [Consensus](./consensus/README.md) - Consensus module for modifying CometBFT's ABCI consensus params. +* [Circuit](./circuit/README.md) - Circuit breaker module for pausing messages. +* [Genutil](./genutil/README.md) - Genesis utilities for the Cosmos SDK. To learn more about the process of building modules, visit the [building modules reference documentation](https://docs.cosmos.network/main/building-modules/intro). @@ -30,7 +31,7 @@ To learn more about the process of building modules, visit the [building modules The IBC module for the SDK is maintained by the IBC Go team in its [own repository](https://github.com/cosmos/ibc-go). -Additionally, the [capability module](https://github.com/cosmos/ibc-go/tree/fdd664698d79864f1e00e147f9879e58497b5ef1/modules/capability) is from v0.48+ maintained by the IBC Go team in its [own repository](https://github.com/cosmos/ibc-go/tree/fdd664698d79864f1e00e147f9879e58497b5ef1/modules/capability). +Additionally, the [capability module](https://github.com/cosmos/ibc-go/tree/fdd664698d79864f1e00e147f9879e58497b5ef1/modules/capability) is from v0.50+ maintained by the IBC Go team in its [own repository](https://github.com/cosmos/ibc-go/tree/fdd664698d79864f1e00e147f9879e58497b5ef1/modules/capability). ## CosmWasm diff --git a/docs/build/modules/_category_.json b/docs/build/modules/_category_.json index 9aac4ff6e..72d229c0b 100644 --- a/docs/build/modules/_category_.json +++ b/docs/build/modules/_category_.json @@ -1,5 +1,5 @@ { - "label": "List of SDK Modules", + "label": "Modules", "position": 2, "link": null -} \ No newline at end of file +} diff --git a/docs/build/modules/accounts/README.md b/docs/build/modules/accounts/README.md new file mode 100644 index 000000000..9ffcbed6e --- /dev/null +++ b/docs/build/modules/accounts/README.md @@ -0,0 +1,3 @@ +# x/accounts + +The x/accounts module provides module and facilities for writing smart cosmos-sdk accounts. \ No newline at end of file diff --git a/docs/build/modules/auth/1-vesting.md b/docs/build/modules/auth/1-vesting.md index 62619ede0..5c04614f7 100644 --- a/docs/build/modules/auth/1-vesting.md +++ b/docs/build/modules/auth/1-vesting.md @@ -256,7 +256,7 @@ func (k Keeper) LockedCoins(ctx Context, addr AccAddress) Coins { acc := k.GetAccount(ctx, addr) if acc != nil { if acc.IsVesting() { - return acc.LockedCoins(ctx.BlockTime()) + return acc.LockedCoins(ctx.HeaderInfo().Time) } } diff --git a/docs/build/modules/auth/2-tx.md b/docs/build/modules/auth/2-tx.md index 0460f69d8..1706690d3 100644 --- a/docs/build/modules/auth/2-tx.md +++ b/docs/build/modules/auth/2-tx.md @@ -4,9 +4,7 @@ sidebar_position: 1 # `x/auth/tx` -:::note - -### Pre-requisite Readings +:::note Pre-requisite Readings * [Transactions](https://docs.cosmos.network/main/core/transactions#transaction-generation) * [Encoding](https://docs.cosmos.network/main/core/encoding#transaction-encoding) diff --git a/docs/build/modules/auth/README.md b/docs/build/modules/auth/README.md index 0c6364bf7..1d5e22b5d 100644 --- a/docs/build/modules/auth/README.md +++ b/docs/build/modules/auth/README.md @@ -32,7 +32,7 @@ This module is used in the Cosmos Hub. ## Concepts -**Note:** The auth module is different from the [authz module](../authz/README.md). +**Note:** The auth module is different from the [authz module](../modules/authz/). The differences are: @@ -481,17 +481,6 @@ simd tx broadcast tx.signed.json More information about the `broadcast` command can be found running `simd tx broadcast --help`. -#### `aux-to-fee` - -The `aux-to-fee` comamnds includes the aux signer data in the tx, broadcast the tx, and sends the tip amount to the broadcaster. -[Learn more about tip transaction](https://docs.cosmos.network/main/core/tips). - -```bash -# simd tx bank send --aux (optional: --tip --tipper ) -simd tx aux-to-fee tx.aux.signed.json -``` - -More information about the `aux-to-fee` command can be found running `simd tx aux-to-fee --help`. ### gRPC diff --git a/docs/build/modules/authz/README.md b/docs/build/modules/authz/README.md index a641693df..2ab2dcba7 100644 --- a/docs/build/modules/authz/README.md +++ b/docs/build/modules/authz/README.md @@ -38,7 +38,7 @@ on behalf of one account to other accounts. The design is defined in the [ADR 03 A *grant* is an allowance to execute a Msg by the grantee on behalf of the granter. Authorization is an interface that must be implemented by a concrete authorization logic to validate and execute grants. Authorizations are extensible and can be defined for any Msg service method even outside of the module where the Msg method is defined. See the `SendAuthorization` example in the next section for more details. -**Note:** The authz module is different from the [auth (authentication)](../auth/README.md) module that is responsible for specifying the base transaction and account types. +**Note:** The authz module is different from the [auth (authentication)](../modules/auth/) module that is responsible for specifying the base transaction and account types. ```go reference https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/authz/authorizations.go#L11-L25 @@ -82,7 +82,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/bank/types/send_authoriz #### StakeAuthorization -`StakeAuthorization` implements the `Authorization` interface for messages in the [staking module](../staking/README.md). It takes an `AuthorizationType` to specify whether you want to authorise delegating, undelegating or redelegating (i.e. these have to be authorised seperately). It also takes a required `MaxTokens` that keeps track of a limit to the amount of tokens that can be delegated/undelegated/redelegated. If left empty, the amount is unlimited. Additionally, this Msg takes an `AllowList` or a `DenyList`, which allows you to select which validators you allow or deny grantees to stake with. +`StakeAuthorization` implements the `Authorization` interface for messages in the [staking module](https://docs.cosmos.network/v0.44/modules/staking/). It takes an `AuthorizationType` to specify whether you want to authorise delegating, undelegating or redelegating (i.e. these have to be authorised seperately). It also takes a required `MaxTokens` that keeps track of a limit to the amount of tokens that can be delegated/undelegated/redelegated. If left empty, the amount is unlimited. Additionally, this Msg takes an `AllowList` or a `DenyList`, which allows you to select which validators you allow or deny grantees to stake with. ```protobuf reference https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/authz.proto#L11-L35 diff --git a/docs/build/modules/bank/README.md b/docs/build/modules/bank/README.md index 340822aa3..217060fb4 100644 --- a/docs/build/modules/bank/README.md +++ b/docs/build/modules/bank/README.md @@ -236,8 +236,12 @@ accounts. The send keeper does not alter the total supply (mint or burn coins). type SendKeeper interface { ViewKeeper - InputOutputCoins(ctx context.Context, inputs types.Input, outputs []types.Output) error - SendCoins(ctx context.Context, fromAddr sdk.AccAddress, toAddr sdk.AccAddress, amt sdk.Coins) error + AppendSendRestriction(restriction SendRestrictionFn) + PrependSendRestriction(restriction SendRestrictionFn) + ClearSendRestriction() + + InputOutputCoins(ctx context.Context, input types.Input, outputs []types.Output) error + SendCoins(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) error GetParams(ctx context.Context) types.Params SetParams(ctx context.Context, params types.Params) error @@ -256,6 +260,86 @@ type SendKeeper interface { } ``` +#### Send Restrictions + +The `SendKeeper` applies a `SendRestrictionFn` before each transfer of funds. + +```golang +// A SendRestrictionFn can restrict sends and/or provide a new receiver address. +type SendRestrictionFn func(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) (newToAddr sdk.AccAddress, err error) +``` + +After the `SendKeeper` (or `BaseKeeper`) has been created, send restrictions can be added to it using the `AppendSendRestriction` or `PrependSendRestriction` functions. +Both functions compose the provided restriction with any previously provided restrictions. +`AppendSendRestriction` adds the provided restriction to be run after any previously provided send restrictions. +`PrependSendRestriction` adds the restriction to be run before any previously provided send restrictions. +The composition will short-circuit when an error is encountered. I.e. if the first one returns an error, the second is not run. + +During `SendCoins`, the send restriction is applied after coins are removed from the from address, but before adding them to the to address. +During `InputOutputCoins`, the send restriction is applied after the input coins are removed and once for each output before the funds are added. + +A send restriction function should make use of a custom value in the context to allow bypassing that specific restriction. + +For example, in your module's keeper package, you'd define the send restriction function: + +```golang +var _ banktypes.SendRestrictionFn = Keeper{}.SendRestrictionFn + +func (k Keeper) SendRestrictionFn(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) (sdk.AccAddress, error) { + // Bypass if the context says to. + if mymodule.HasBypass(ctx) { + return toAddr, nil + } + + // Your custom send restriction logic goes here. + return nil, errors.New("not implemented") +} +``` + +The bank keeper should be provided to your keeper's constructor so the send restriction can be added to it: + +```golang +func NewKeeper(cdc codec.BinaryCodec, storeKey storetypes.StoreKey, bankKeeper mymodule.BankKeeper) Keeper { + rv := Keeper{/*...*/} + bankKeeper.AppendSendRestriction(rv.SendRestrictionFn) + return rv +} +``` + +Then, in the `mymodule` package, define the context helpers: + +```golang +const bypassKey = "bypass-mymodule-restriction" + +// WithBypass returns a new context that will cause the mymodule bank send restriction to be skipped. +func WithBypass(ctx context.Context) context.Context { + return sdk.UnwrapSDKContext(ctx).WithValue(bypassKey, true) +} + +// WithoutBypass returns a new context that will cause the mymodule bank send restriction to not be skipped. +func WithoutBypass(ctx context.Context) context.Context { + return sdk.UnwrapSDKContext(ctx).WithValue(bypassKey, false) +} + +// HasBypass checks the context to see if the mymodule bank send restriction should be skipped. +func HasBypass(ctx context.Context) bool { + bypassValue := ctx.Value(bypassKey) + if bypassValue == nil { + return false + } + bypass, isBool := bypassValue.(bool) + return isBool && bypass +} +``` + +Now, anywhere where you want to use `SendCoins` or `InputOutputCoins`, but you don't want your send restriction applied: + +```golang +func (k Keeper) DoThing(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) error { + return k.bankKeeper.SendCoins(mymodule.WithBypass(ctx), fromAddr, toAddr, amt) +} +``` + ### ViewKeeper The view keeper provides read-only access to account balances. The view keeper does not have balance alteration functionality. All balance lookups are `O(1)`. @@ -331,11 +415,26 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/ The message will fail under the following conditions: -* The authority is not a bech32 address. +* The authority is not a decodable address. * The authority is not x/gov module's address. * There are multiple SendEnabled entries with the same Denom. * One or more SendEnabled entries has an invalid Denom. +### MsgBurn + +Used to burn coins from an account. The coins are removed from the account and the total supply is reduced. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/1af000b3ef6296f9928caf494fe5bb812990f22d/proto/cosmos/bank/v1beta1/tx.proto#L131-L148 +``` + +This message will fail under the following conditions: + +* The signer is not present +* The coins are not spendable +* The coins are not positive +* The coins are not valid + ## Events The bank module emits the following events: diff --git a/docs/build/modules/circuit/README.md b/docs/build/modules/circuit/README.md index 332ebf214..7386680e3 100644 --- a/docs/build/modules/circuit/README.md +++ b/docs/build/modules/circuit/README.md @@ -59,7 +59,7 @@ Authorize, is called by the module authority (default governance module account) ### Trip -Trip, is called by an account to disable message execution for a specific msgURL. +Trip, is called by an authorized account to disable message execution for a specific msgURL. If empty, all the msgs will be disabled. ```protobuf // TripCircuitBreaker pauses processing of Msg's in the state machine. @@ -68,7 +68,7 @@ Trip, is called by an account to disable message execution for a specific msgURL ### Reset -Reset is called to enable execution of a previously disabled message. +Reset is called by an authorized account to enable execution for a specific msgURL of previously disabled message. If empty, all the disabled messages will be enabled. ```protobuf // ResetCircuitBreaker resumes processing of Msg's in the state machine that @@ -87,7 +87,6 @@ https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto# This message is expected to fail if: * the granter is not an account with permission level `LEVEL_SUPER_ADMIN` or the module authority -* if the type urls does not exist ### MsgTripCircuitBreaker @@ -98,7 +97,6 @@ https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto# This message is expected to fail if: * if the signer does not have a permission level with the ability to disable the specified type url message -* if the type urls does not exist ### MsgResetCircuitBreaker @@ -108,8 +106,46 @@ https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto# This message is expected to fail if: -* if the type urls does not exist * if the type url is not disabled -* `## Events` - list and describe event tags used -* `## Client` - list and describe CLI commands and gRPC and REST endpoints +## Events - list and describe event tags + +The circuit module emits the following events: + +### Message Events + +#### MsgAuthorizeCircuitBreaker + +| Type | Attribute Key | Attribute Value | +|---------|---------------|---------------------------| +| string | granter | {granterAddress} | +| string | grantee | {granteeAddress} | +| string | permission | {granteePermissions} | +| message | module | circuit | +| message | action | authorize_circuit_breaker | + +#### MsgTripCircuitBreaker + +| Type | Attribute Key | Attribute Value | +|----------|---------------|--------------------| +| string | authority | {authorityAddress} | +| []string | msg_urls | []string{msg_urls} | +| message | module | circuit | +| message | action | trip_circuit_breaker | + +#### ResetCircuitBreaker + +| Type | Attribute Key | Attribute Value | +|----------|---------------|--------------------| +| string | authority | {authorityAddress} | +| []string | msg_urls | []string{msg_urls} | +| message | module | circuit | +| message | action | reset_circuit_breaker | + + +## Keys - list of key prefixes used by the circuit module + +* `AccountPermissionPrefix` - `0x01` +* `DisableListPrefix` - `0x02` + +## Client - list and describe CLI commands and gRPC and REST endpoints diff --git a/docs/build/modules/distribution/README.md b/docs/build/modules/distribution/README.md index e46dab3fc..32858fd69 100644 --- a/docs/build/modules/distribution/README.md +++ b/docs/build/modules/distribution/README.md @@ -352,15 +352,22 @@ The transaction fails if the amount cannot be transferred from the sender to the ```go func (k Keeper) FundCommunityPool(ctx context.Context, amount sdk.Coins, sender sdk.AccAddress) error { - if err := k.bankKeeper.SendCoinsFromAccountToModule(ctx, sender, types.ModuleName, amount); err != nil { - return err - } + if err := k.bankKeeper.SendCoinsFromAccountToModule(ctx, sender, types.ModuleName, amount); err != nil { + return err + } + + feePool, err := k.FeePool.Get(ctx) + if err != nil { + return err + } - feePool := k.GetFeePool(ctx) - feePool.CommunityPool = feePool.CommunityPool.Add(sdk.NewDecCoinsFromCoins(amount...)...) - k.SetFeePool(ctx, feePool) + feePool.CommunityPool = feePool.CommunityPool.Add(sdk.NewDecCoinsFromCoins(amount...)...) + + if err := k.FeePool.Set(ctx, feePool); err != nil { + return err + } - return nil + return nil } ``` @@ -681,7 +688,7 @@ rewards: The `validator-distribution-info` command allows users to query validator commission and self-delegation rewards for validator. -```shell +````shell simd query distribution validator-distribution-info cosmosvaloper1... ``` diff --git a/docs/build/modules/evidence/README.md b/docs/build/modules/evidence/README.md index 263481b0c..82cd03bac 100644 --- a/docs/build/modules/evidence/README.md +++ b/docs/build/modules/evidence/README.md @@ -107,7 +107,7 @@ by the `Handler` should be persisted. // for executing all corresponding business logic necessary for verifying the // evidence as valid. In addition, the Handler may execute any necessary // slashing and potential jailing. -type Handler func(sdk.Context, Evidence) error +type Handler func(context.Context, Evidence) error ``` @@ -152,7 +152,7 @@ as follows: ```go func SubmitEvidence(ctx Context, evidence Evidence) error { - if _, ok := GetEvidence(ctx, evidence.Hash()); ok { + if _, err := GetEvidence(ctx, evidence.Hash()); err == nil { return errorsmod.Wrap(types.ErrEvidenceExists, strings.ToUpper(hex.EncodeToString(evidence.Hash()))) } if !router.HasRoute(evidence.Route()) { @@ -271,7 +271,7 @@ The `evidence` command allows users to list all evidence or evidence by hash. Usage: ```bash -simd query evidence [flags] +simd query evidence evidence [flags] ``` To query evidence by hash @@ -279,7 +279,7 @@ To query evidence by hash Example: ```bash -simd query evidence "DF0C23E8634E480F84B9D5674A7CDC9816466DEC28A3358F73260F68D28D7660" +simd query evidence evidence "DF0C23E8634E480F84B9D5674A7CDC9816466DEC28A3358F73260F68D28D7660" ``` Example Output: @@ -297,7 +297,7 @@ To get all evidence Example: ```bash -simd query evidence +simd query evidence list ``` Example Output: diff --git a/docs/build/modules/genutil/README.md b/docs/build/modules/genutil/README.md index c534b8b0d..45cb45355 100644 --- a/docs/build/modules/genutil/README.md +++ b/docs/build/modules/genutil/README.md @@ -2,16 +2,36 @@ ## Concepts -The `genutil` package contains a variaety of genesis utility functionalities for usage within a blockchain application. Namely: +The `genutil` package contains a variety of genesis utility functionalities for usage within a blockchain application. Namely: * Genesis transactions related (gentx) * Commands for collection and creation of gentxs * `InitChain` processing of gentxs +* Genesis file creation * Genesis file validation * Genesis file migration * CometBFT related initialization * Translation of an app genesis to a CometBFT genesis +## Genesis + +Genutil contains the data structure that defines an application genesis. +An application genesis consist of a consensus genesis (g.e. CometBFT genesis) and application related genesis data. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-rc.0/x/genutil/types/genesis.go#L24-L34 +``` + +The application genesis can then be translated to the consensus engine to the right format: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-rc.0/x/genutil/types/genesis.go#L126-L136 +``` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-rc.0/server/start.go#L397-L407 +``` + ## Client ### CLI diff --git a/docs/build/modules/gov/README.md b/docs/build/modules/gov/README.md index 8dac16a3e..a50832749 100644 --- a/docs/build/modules/gov/README.md +++ b/docs/build/modules/gov/README.md @@ -16,14 +16,14 @@ on proposals on a 1 token 1 vote basis. Next is a list of features the module currently supports: * **Proposal submission:** Users can submit proposals with a deposit. Once the -minimum deposit is reached, the proposal enters voting period. -* **Vote:** Participants can vote on proposals that reached MinDeposit +minimum deposit is reached, the proposal enters voting period. The minimum deposit can be reached by collecting deposits from different users (including proposer) within deposit period. +* **Vote:** Participants can vote on proposals that reached MinDeposit and entered voting period. * **Inheritance and penalties:** Delegators inherit their validator's vote if they don't vote themselves. * **Claiming deposit:** Users that deposited on proposals can recover their -deposits if the proposal was accepted or rejected. If the proposal was vetoed, or never entered voting period, the deposit is burned. +deposits if the proposal was accepted or rejected. If the proposal was vetoed, or never entered voting period (minimum deposit not reached within deposit period), the deposit is burned. -This module will be used in the Cosmos Hub, the first Hub in the Cosmos network. +This module is in use on the Cosmos Hub (a.k.a [gaia](https://github.com/cosmos/gaia)). Features that may be added in the future are described in [Future Improvements](#future-improvements). ## Contents @@ -36,6 +36,7 @@ staking token of the chain. * [Proposal submission](#proposal-submission) * [Deposit](#deposit) * [Vote](#vote) + * [Software Upgrade](#software-upgrade) * [State](#state) * [Proposals](#proposals) * [Parameters and base types](#parameters-and-base-types) @@ -141,9 +142,7 @@ Note that when *participants* have bonded and unbonded Atoms, their voting power Once a proposal reaches `MinDeposit`, it immediately enters `Voting period`. We define `Voting period` as the interval between the moment the vote opens and -the moment the vote closes. `Voting period` should always be shorter than -`Unbonding period` to prevent double voting. The initial value of -`Voting period` is 2 weeks. +the moment the vote closes. The initial value of `Voting period` is 2 weeks. #### Option set @@ -495,11 +494,13 @@ And the pseudocode for the `ProposalProcessingQueue`: ### Legacy Proposal +:::warning +Legacy proposals are deprecated. Use the new proposal flow by granting the governance module the right to execute the message. +::: + A legacy proposal is the old implementation of governance proposal. Contrary to proposal that can contain any messages, a legacy proposal allows to submit a set of pre-defined proposals. -These proposal are defined by their types. - -While proposals should use the new implementation of the governance proposal, we need still to use legacy proposal in order to submit a `software-upgrade` and a `cancel-software-upgrade` proposal. +These proposals are defined by their types and handled by handlers that are registered in the gov v1beta1 router. More information on how to submit proposals in the [client section](#client). @@ -1182,6 +1183,10 @@ where `proposal.json` contains: By default the metadata, summary and title are both limited by 255 characters, this can be overridden by the application developer. ::: +:::tip +When metadata is not specified, the title is limited to 255 characters and the summary 40x the title length. +::: + ##### submit-legacy-proposal The `submit-legacy-proposal` command allows users to submit a governance legacy proposal along with an initial deposit. diff --git a/docs/build/modules/group/README.md b/docs/build/modules/group/README.md index b94fec728..71d91ccb4 100644 --- a/docs/build/modules/group/README.md +++ b/docs/build/modules/group/README.md @@ -49,6 +49,7 @@ This module allows the creation and management of on-chain multisig accounts and * [EventVote](#eventvote) * [EventExec](#eventexec) * [EventLeaveGroup](#eventleavegroup) + * [EventProposalPruned](#eventproposalpruned) * [Client](#client) * [CLI](#cli) * [gRPC](#grpc) @@ -578,6 +579,15 @@ The group module emits the following events: | cosmos.group.v1.EventLeaveGroup | proposal_id | {proposalId} | | cosmos.group.v1.EventLeaveGroup | address | {address} | +### EventProposalPruned + +| Type | Attribute Key | Attribute Value | +|-------------------------------------|---------------|---------------------------------| +| message | action | /cosmos.group.v1.Msg/LeaveGroup | +| cosmos.group.v1.EventProposalPruned | proposal_id | {proposalId} | +| cosmos.group.v1.EventProposalPruned | status | {ProposalStatus} | +| cosmos.group.v1.EventProposalPruned | tally_result | {TallyResult} | + ## Client @@ -1118,18 +1128,18 @@ Example: simd tx group update-group-policy-decision-policy cosmos1.. cosmos1.. '{"@type":"/cosmos.group.v1.ThresholdDecisionPolicy", "threshold":"2", "windows": {"voting_period": "120h", "min_execution_period": "0s"}}' ``` -#### create-proposal +#### submit-proposal -The `create-proposal` command allows users to submit a new proposal. +The `submit-proposal` command allows users to submit a new proposal. ```bash -simd tx group create-proposal [group-policy-account] [proposer[,proposer]*] [msg_tx_json_file] [metadata] [flags] +simd tx group submit-proposal [group-policy-account] [proposer[,proposer]*] [msg_tx_json_file] [metadata] [flags] ``` Example: ```bash -simd tx group create-proposal cosmos1.. cosmos1.. msg_tx.json "AQ==" +simd tx group submit-proposal cosmos1.. cosmos1.. msg_tx.json "AQ==" ``` #### withdraw-proposal @@ -2099,7 +2109,7 @@ Example Output: ## Metadata -The group module has four locations for metadata where users can provide further context about the on-chain actions they are taking. By default all metadata fields have a 255 character length field where metadata can be stored in json format, either on-chain or off-chain depending on the amount of data required. Here we provide a recommendation for the json structure and where the data should be stored. There are two important factors in making these recommendations. First, that the group and gov modules are consistent with one another, note the number of proposals made by all groups may be quite large. Second, that client applications such as block explorers and governance interfaces have confidence in the consistency of metadata structure accross chains. +The group module has four locations for metadata where users can provide further context about the on-chain actions they are taking. By default all metadata fields have a 255 character length field where metadata can be stored in json format, either on-chain or off-chain depending on the amount of data required. Here we provide a recommendation for the json structure and where the data should be stored. There are two important factors in making these recommendations. First, that the group and gov modules are consistent with one another, note the number of proposals made by all groups may be quite large. Second, that client applications such as block explorers and governance interfaces have confidence in the consistency of metadata structure across chains. ### Proposal diff --git a/docs/build/modules/upgrade/README.md b/docs/build/modules/upgrade/README.md index d1be1b2c8..0d98c1601 100644 --- a/docs/build/modules/upgrade/README.md +++ b/docs/build/modules/upgrade/README.md @@ -8,7 +8,7 @@ sidebar_position: 1 `x/upgrade` is an implementation of a Cosmos SDK module that facilitates smoothly upgrading a live Cosmos chain to a new (breaking) software version. It accomplishes this by -providing a `BeginBlocker` hook that prevents the blockchain state machine from +providing a `PreBlocker` hook that prevents the blockchain state machine from proceeding once a pre-defined upgrade block height has been reached. The module does not prescribe anything regarding how governance decides to do an @@ -41,12 +41,6 @@ may contain various metadata about the upgrade, typically application specific upgrade info to be included on-chain such as a git commit that validators could automatically upgrade to. -#### Sidecar Process - -If an operator running the application binary also runs a sidecar process to assist -in the automatic download and upgrade of a binary, the `Info` allows this process to -be seamless. This tool is [Cosmovisor](https://github.com/cosmos/cosmos-sdk/tree/main/tools/cosmovisor#readme). - ```go type Plan struct { Name string @@ -55,6 +49,12 @@ type Plan struct { } ``` +#### Sidecar Process + +If an operator running the application binary also runs a sidecar process to assist +in the automatic download and upgrade of a binary, the `Info` allows this process to +be seamless. This tool is [Cosmovisor](https://github.com/cosmos/cosmos-sdk/tree/main/tools/cosmovisor#readme). + ### Handler The `x/upgrade` module facilitates upgrading from major version X to major version Y. To diff --git a/docs/build/packages/01-depinject.md b/docs/build/packages/01-depinject.md index 258e1e0b6..405a9d4b7 100644 --- a/docs/build/packages/01-depinject.md +++ b/docs/build/packages/01-depinject.md @@ -45,17 +45,20 @@ import ( type AnotherInt int +func GetInt() int { return 1 } +func GetAnotherInt() AnotherInt { return 2 } + func main() { var ( - x int - y AnotherInt + x int + y AnotherInt ) fmt.Printf("Before (%v, %v)\n", x, y) depinject.Inject( depinject.Provide( - func() int { return 1 }, - func() AnotherInt { return AnotherInt(2) }, + GetInt, + GetAnotherInt, ), &x, &y, @@ -66,7 +69,7 @@ func main() { In this example, `depinject.Provide` registers two provider functions that return `int` and `AnotherInt` values. The `depinject.Inject` function is then used to inject these values into the variables `x` and `y`. -Provider functions serve as the basis for the dependency tree. They are analysed to identify their inputs as dependencies and their outputs as dependents. These dependents can either be used by another provider function or be stored outside the DI container (e.g., `&x` and `&y` in the example above). +Provider functions serve as the basis for the dependency tree. They are analysed to identify their inputs as dependencies and their outputs as dependents. These dependents can either be used by another provider function or be stored outside the DI container (e.g., `&x` and `&y` in the example above). Provider functions must be exported. ### Interface type resolution @@ -98,6 +101,22 @@ type Pond struct { } ``` +And the following provider functions: + +```go +func GetMallard() duck.Mallard { + return Mallard{} +} + +func GetPond(duck Duck) Pond { + return Pond{Duck: duck} +} + +func GetCanvasback() Canvasback { + return Canvasback{} +} +``` + In this example, there's a `Pond` struct that has a `Duck` field of type `AlsoDuck`. The `depinject` framework can automatically resolve the appropriate implementation when there's only one available, as shown below: ```go @@ -105,10 +124,9 @@ var pond Pond depinject.Inject( depinject.Provide( - func() Mallard { return Mallard{} }, - func(duck Duck) Pond { - return Pond{Duck: duck} - }), + GetMallard, + GetPond, + ), &pond) ``` @@ -120,13 +138,12 @@ However, if there are multiple implementations of the `Duck` interface, as in th var pond Pond depinject.Inject( - depinject.Provide( - func() Mallard { return Mallard{} }, - func() Canvasback { return Canvasback{} }, - func(duck Duck) Pond { - return Pond{Duck: duck} - }), - &pond) + depinject.Provide( + GetMallard, + GetCanvasback, + GetPond, + ), + &pond) ``` A specific binding preference for `Duck` is required. @@ -137,17 +154,18 @@ In the above situation registering a binding for a given interface binding may l ```go depinject.Inject( - depinject.Configs( - depinject.BindInterface( - "duck.Duck", - "duck.Mallard"), - depinject.Provide( - func() Mallard { return Mallard{} }, - func() Canvasback { return Canvasback{} }, - func(duck Duck) APond { - return Pond{Duck: duck} - })), - &pond) + depinject.Configs( + depinject.BindInterface( + "duck/duck.Duck", + "duck/duck.Mallard", + ), + depinject.Provide( + GetMallard, + GetCanvasback, + GetPond, + ), + ), + &pond) ``` Now `depinject` has enough information to provide `Mallard` as an input to `APond`. diff --git a/docs/build/packages/02-collections.md b/docs/build/packages/02-collections.md index fc74594ec..72d5f7172 100644 --- a/docs/build/packages/02-collections.md +++ b/docs/build/packages/02-collections.md @@ -533,7 +533,7 @@ func (k Keeper) GetAllAccounts(ctx sdk.Context) ([]authtypes.BaseAccount, error) } func (k Keeper) IterateAccountsBetween(ctx sdk.Context, start, end uint64) ([]authtypes.BaseAccount, error) { - // The collections.Range API offers a lot of capability + // The collections.Range API offers a lot of capabilities // like defining where the iteration starts or ends. rng := new(collections.Range[uint64]). StartInclusive(start). @@ -1117,3 +1117,82 @@ func (k Keeper) GetAccount(ctx sdk.context, addr sdk.AccAddress) (sdk.AccountI, return k.Accounts.Get(ctx, addr) } ``` + +## Triple key + +The `collections.Triple` is a special type of key composed of three keys, it's identical to `collections.Pair`. + +Let's see an example. + +```go +package example + +import ( + "context" + + "cosmossdk.io/collections" + storetypes "cosmossdk.io/store/types" + "github.com/cosmos/cosmos-sdk/codec" +) + +type AccAddress = string +type ValAddress = string + +type Keeper struct { + // let's simulate we have redelegations which are stored as a triple key composed of + // the delegator, the source validator and the destination validator. + Redelegations collections.KeySet[collections.Triple[AccAddress, ValAddress, ValAddress]] +} + +func NewKeeper(storeKey *storetypes.KVStoreKey) Keeper { + sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey)) + return Keeper{ + Redelegations: collections.NewKeySet(sb, collections.NewPrefix(0), "redelegations", collections.TripleKeyCodec(collections.StringKey, collections.StringKey, collections.StringKey) + } +} + +// RedelegationsByDelegator iterates over all the redelegations of a given delegator and calls onResult providing +// each redelegation from source validator towards the destination validator. +func (k Keeper) RedelegationsByDelegator(ctx context.Context, delegator AccAddress, onResult func(src, dst ValAddress) (stop bool, err error)) error { + rng := collections.NewPrefixedTripleRange[AccAddress, ValAddress, ValAddress](delegator) + return k.Redelegations.Walk(ctx, rng, func(key collections.Triple[AccAddress, ValAddress, ValAddress]) (stop bool, err error) { + return onResult(key.K2(), key.K3()) + }) +} + +// RedelegationsByDelegatorAndValidator iterates over all the redelegations of a given delegator and its source validator and calls onResult for each +// destination validator. +func (k Keeper) RedelegationsByDelegatorAndValidator(ctx context.Context, delegator AccAddress, validator ValAddress, onResult func(dst ValAddress) (stop bool, err error)) error { + rng := collections.NewSuperPrefixedTripleRange[AccAddress, ValAddress, ValAddress](delegator, validator) + return k.Redelegations.Walk(ctx, rng, func(key collections.Triple[AccAddress, ValAddress, ValAddress]) (stop bool, err error) { + return onResult(key.K3()) + }) +} +``` + +## Advanced Usages + +### Alternative Value Codec + +The `codec.AltValueCodec` allows a collection to decode values using a different codec than the one used to encode them. +Basically it enables to decode two different byte representations of the same concrete value. +It can be used to lazily migrate values from one bytes representation to another, as long as the new representation is +not able to decode the old one. + +A concrete example can be found in `x/bank` where the balance was initially stored as `Coin` and then migrated to `Int`. + +```go + +var BankBalanceValueCodec = codec.NewAltValueCodec(sdk.IntValue, func(b []byte) (sdk.Int, error) { + coin := sdk.Coin{} + err := coin.Unmarshal(b) + if err != nil { + return sdk.Int{}, err + } + return coin.Amount, nil +}) +``` + +The above example shows how to create an `AltValueCodec` that can decode both `sdk.Int` and `sdk.Coin` values. The provided +decoder function will be used as a fallback in case the default decoder fails. When the value will be encoded back into state +it will use the default encoder. This allows to lazily migrate values to a new bytes representation. \ No newline at end of file diff --git a/docs/build/packages/README.md b/docs/build/packages/README.md index 7132b4138..d7a115b26 100644 --- a/docs/build/packages/README.md +++ b/docs/build/packages/README.md @@ -20,13 +20,13 @@ For more information on SDK tooling, see the [Tooling](https://docs.cosmos.netwo ## State Management -* [Collections](02-collections.md) - State management library -* [ORM](03-orm.md) - State management library +* [Collections](./02-collections.md) - State management library +* [ORM](./03-orm.md) - State management library ## Automation -* [Depinject](01-depinject.md) - Dependency injection framework -* [Client/v2](https://pkg.go.dev/cosmossdk.io/client/v2) - Library powering [AutoCLI](https://docs.cosmos.network/main/building-modules/autocli) +* [Depinject](./01-depinject.md) - Dependency injection framework +* [Client/v2](https://pkg.go.dev/cosmossdk.io/client/v2) - Library powering [AutoCLI](https://docs.cosmos.network/main/core/autocli) ## Utilities diff --git a/docs/build/packages/_category_.json b/docs/build/packages/_category_.json index e91118d38..5ed885eb2 100644 --- a/docs/build/packages/_category_.json +++ b/docs/build/packages/_category_.json @@ -1,5 +1,5 @@ { "label": "Packages", - "position": 9, + "position": 4, "link": null } \ No newline at end of file diff --git a/docs/build/rfc/PROCESS.md b/docs/build/rfc/PROCESS.md index 71b0c7ce7..a34af2269 100644 --- a/docs/build/rfc/PROCESS.md +++ b/docs/build/rfc/PROCESS.md @@ -3,7 +3,7 @@ 1. Copy the `rfc-template.md` file. Use the following filename pattern: `rfc-next_number-title.md` 2. Create a draft Pull Request if you want to get an early feedback. 3. Make sure the context and a solution is clear and well documented. -4. Add an entry to a list in the [README](README.md) file. +4. Add an entry to a list in the [README](./README.md) file. 5. Create a Pull Request to propose a new ADR. ## What is an RFC? diff --git a/docs/build/rfc/README.md b/docs/build/rfc/README.md index 40559b626..61ec12775 100644 --- a/docs/build/rfc/README.md +++ b/docs/build/rfc/README.md @@ -13,7 +13,7 @@ discussion that might otherwise only be recorded in an ad-hoc way (for example, via gists or Google docs) that are difficult to discover for someone after the fact. An RFC _may_ give rise to more specific architectural _decisions_ for the Cosmos SDK, but those decisions must be recorded separately in -[Architecture Decision Records (ADR)](../architecture/README.md). +[Architecture Decision Records (ADR)](../architecture). As a rule of thumb, if you can articulate a specific question that needs to be answered, write an ADR. If you need to explore the topic and get input from @@ -30,5 +30,9 @@ An RFC should provide: substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](rfc-template.md) file includes placeholders for these +The [rfc-template.md](./rfc-template.md) file includes placeholders for these sections. + +## Table of Contents + +* [RFC-002: Zero Copy Encoding](./rfc-002-zero-copy-encoding.md) diff --git a/docs/build/rfc/_category_.json b/docs/build/rfc/_category_.json index 40fac9822..a5712bdae 100644 --- a/docs/build/rfc/_category_.json +++ b/docs/build/rfc/_category_.json @@ -1,5 +1,5 @@ { - "label": "RFCs", - "position": 12, - "link": null + "label": "RFC", + "position": 7, + "link": null } \ No newline at end of file diff --git a/docs/build/rfc/rfc-002-zero-copy-encoding.md b/docs/build/rfc/rfc-002-zero-copy-encoding.md new file mode 100644 index 000000000..aeef7308d --- /dev/null +++ b/docs/build/rfc/rfc-002-zero-copy-encoding.md @@ -0,0 +1,406 @@ +# RFC 002: Zero-Copy Encoding + +## Changelog + +* 2022-03-08: Initial draft + +## Background + +When the SDK originally migrated to [protobuf state encoding](./../architecture/adr-019-protobuf-state-encoding.md), +zero-copy encodings such as [Cap'n Proto](https://capnproto.org/) +and [FlatBuffers](https://google.github.io/flatbuffers/) +were considered. We considered how a zero-copy encoding could be beneficial for interoperability with modules +and scripts in other languages and VMs. However, protobuf was still chosen because the maturity of its ecosystem and +tooling was much higher and the client experience and performance were considered the highest priorities. + +In [ADR 033: Protobuf-based Inter-Module Communication](./../architecture/adr-033-protobuf-inter-module-comm.md), the +idea of cross-language/VM inter-module +communication was considered again. And in the discussions surrounding [ADR 054: Semver Compatible SDK Modules](./../architecture/adr-054-semver-compatible-modules.md), +it was determined that multi-language/VM support in the SDK is a near term priority. + +While we could do cross-language/VM inter-module communication with protobuf binary or even JSON, the performance +overhead is deemed to be too high because: +* we are proposing replacing keeper calls with inter-module message calls and the overhead of even the inter-module +routing checks has come into question by some SDK users without even considering the possible overhead of encoding. +Effectively we would be replacing function calls with encoding. One of the SDK's primary objectives currently is +improving performance, and we want to avoid inter-module calls from becoming a big step backward. +* we want Rust code to be able to operate in highly resource constrained virtual machines so whatever we can do to +reduce performance overhead as well as the size of generated code will make it easier and more feasible to deploy +first-class integrations with these virtual machines. + +Thus, the agreement when the [ADR 054](./../architecture/adr-054-semver-compatible-modules.md) working group concluded +was to pursue a performant zero-copy encoding which is suitable for usage in highly resource constrained environments. + +## Proposal + +This RFC proposes a zero-copy encoding that is derived from the schema definitions defined in .proto files in the SDK +and all app chains. This would result in a new code generator for that supports both this zero-copy encoding as well as +the existing protobuf binary and JSON encodings as well as the google.golang.org/protobuf API. To make this zero-copy +encoding work, a number of changes are needed to how we manage the versioning of protobuf messages that should +address other concerns raised in [ADR 054](./../architecture/adr-054-semver-compatible-modules.md). The API for using +protobuf in golang would also change and this will be described in the [code generation](#generated-code) section +along with a proposed Rust code generator. + +An alternative approach to building a zero-copy encoding based on protobuf schemas would be to switch to FlatBuffers +or Cap'n Proto directly. However, this would require a complete rewrite of the SDK and all app chains. Places this +burden on the ecosystem would not be a wise choice when creating a zero-copy encoding compatible with all our +existing types and schemas is feasible. In the future, we may consider a native schema language for this encoding +that is more natural and succinct for its rules, but for now we are assuming that it is best to continue supporting +the existing protobuf based workflow. + +Also, we are not proposing a new encoding for transactions or gRPC query servers. From a client API perspective nothing +would change. The SDK would be capable of marshaling any message to and from protobuf binary and this zero-copy encoding +as needed. + +Furthermore, migrating to the **new golang generated code would be 100% opt-in** because the inter-module router will +simply marshal existing gogo proto generated types to/from the zero-copy encoding when needed. So migrating to the new +code generator would provide a performance benefit, but would not be required. + +In addition to supporting first-class Cosmos SDK modules defined in other languages and VMs, this encoding is intended +to be useful for user-defined code executing in a VM. To satisfy this, this encoding is designed to enable proper bounds +checking on all memory access at the expense of introducing some error return values in generated code. + +### New Protobuf Linting and Breaking Change Rules + +This zero-copy encoding places some additional requirements on the definition and maintenance of protobuf schemas. + +#### No New Fields Can Be Added To Existing Messages + +The biggest change is that it will be invalid to add a new field to an existing message and a breaking change detector +will need to be created which augments [buf breaking](https://docs.buf.build/breaking/overview) to detect this. + +The reasons for this are two-fold: + +1) from an API compatibility perspective, adding a new field to an existing message is actually a state machine breaking + change which in [ADR 020](../architecture/adr-020-protobuf-transaction-encoding.md) required us to add an unknown + field detector. Furthermore, in [ADR 054](../architecture/adr-054-semver-compatible-modules.md) this "feature" of protobuf + poses one of the biggest problems for correct forward compatibility between different versions of the same module. +2) not allowing new fields in existing messages makes the generated code in languages like Rust (which is currently our + highest priority target), much simpler and more performant because we can assume a fixed size struct gets allocated. + If new fields can be added to existing messages, we need to encode the number of fields into the message and then + do runtime checks. So this both increases memory layers and requires another layout of indirection. With the encoding + proposed below, "plain old Rust structs" (used with some special field types) can be used. + +Instead of adding new fields to existing messages, APIs can add new messages to existing packages or create new packages +with new versions of the messages. Also, we are not restricting the addition of cases to `oneof`s or values to `enum`s. +All of these cases are easier to detect at runtime with standard `switch` statements than the addition of new fields. + +#### Additional Linting Rules + +The following additional rules will be enforced by a linter that +complements [buf lint](https://docs.buf.build/lint/overview): + +* all message fields must be specified in continuous ascending order starting from `1` +* all enums must be specified in continuous ascending order starting from `0` - otherwise it is too complex to check at + runtime whether an enum value is unknown. An alternative would be to make adding new values to existing enums breaking +* all enum values must be `<= 255`. Any enum in a blockchain application which needs more than 256 values is probably + doing something very wrong. +* all oneof's must be the *only* element in their containing message and must start at field number `1` and be added in + continuous ascending order - this makes it possible to quickly check for unknown values +* all `oneof` field numbers must be `<= 255`. Any `oneof` which needs more field cases is probably doing something very + wrong. + +These requirements make the encoding and generated code simpler. + +### Encoding + +#### Buffers and Memory Management + +By default, this encoding attempts to use a single fixed size encoding buffer of 64kb. This imposes a limit on the +maximum size of a message that can be encoded. In the context of a message passing protocol for blockchains, this +is generally a reasonable limit and the only known valid use case for exceeding it is to store user-uploaded byte +code for execution in VMs. To accommodate this, large `string` and `bytes` values can be encoded in additional +standalone buffers if needed. Still, the body of a message included all scalar and message fields +must fit inside the 64kb buffer. + +While this design decision greatly simplifies the encoding and decoding logic, as well as the complexity of +generated code, it does mean that APIs will need to do proper bounds checking when writing data that is not fixed +size and return errors. + +The term `Root` is used to refer to the main 64kb buffer plus any additional large `string`/`bytes` buffers that are +allocated. + +#### Scalar Encoding + +* `bool`s are encoded as 1 byte - `0` or `1` +* `uint32`, `int32`, `sint32`, `fixed32`, `sfixed32` are encoded as 4 bytes by default +* `uint64`, `int64`, `sint64`, `fixed64`, `sfixed64` are encoded as 8 bytes by default +* `enum`s are encoded as 1 byte and values *MUST* be in the range of `0` to `255`. +* all scalars declared as `optional` are prefixed with 1 additional byte whose value is `0` or `1` to indicate presence + +All multibyte integers are encoded as little-endian which is by far the most common native byte order for modern +CPUs. Signed integers always use two's complement encoding. + +#### Message Encoding + +By default, messages field are encoded inline as structs. Meaning that if a message struct takes 8 bytes then its inline +field in another struct will add 8 bytes to that struct size. + +`optional` message fields will be prefixed by 1 byte to indicate presence. (Alternatively, we could encode optional +message fields as pointers (see below) if the desire is to save memory when they are rarely used needed.) + +#### Oneof’s + +`oneof`s are encoded as a combination of a `uint8` discriminant field and memory that is as large as the largest member +field. `oneof` field numbers *MUST* be between `1` and `255`. + +```protobuf +message Foo { + oneof sum { + bool x = 1; + int32 y = 2; + } +} +``` + +A discriminant of `0` indicates that the field is not set. + +#### Pointer Types: Bytes and Strings and Repeated fields + +A pointer is an 16-bit unsigned integer that points to an offset in the current memory buffer or to another memory +buffer. If the bit mask `0xFF00` on the is unset, then the pointer points to an offset in the main 64kb memory buffer. +If that bit mask is set, then the pointer points to a large `string` or `bytes` buffer. Up to 256 such buffers +can be referenced in a single `Root`. The pointer `0` indicates that a field is not defined. + +`bytes`, `string`s and repeated fields are encoded as pointers to a memory location that is prefixed with the +length of the `bytes`, `string` or repeated field value. If the referenced memory location is in the main 64kb memory +buffer, then this length prefix will be a 16-bit unsigned integer. If the referenced memory location is a large +`string` or `bytes` buffer, then this length prefix will be a 32-bit unsigned integer. + +#### `Any`s + +`Any`s are encoded as a pointer to the type URL string and a pointer to the start of the message +specified by the type URL. + +#### Maps + +Maps are not supported. + +#### Extended Encoding Options + +We may choose to allow customizing the encoding of fields so that they take up less space. + +For example, we could allow 8-bit or 16-bit integers: +`int32 x = 1 [(cosmos_proto.int16) = true]` would indicate that the field only needs 2 bytes + +Or we could allow `string`, `bytes` or `repeated` fields to have a fixed size rather than being encoding as +pointers to a variable-length value: +`string y = 2 [(cosmos_proto.fixed_size) = 3]` could indicate that this is a fixed width 3 byte string + +If we choose to enable these encoding options, changing these options would be a breaking change that needs to be +prevented by the breaking change detector. + +### Generated Code + +We will describe the generated Go and Rust code using this example protobuf file: + +```protobuf +message Foo { + int32 x = 1; + optional uint32 y = 2; + string z = 3; + Bar bar = 4; + repeated Bar bars = 5; +} + + +message Bar { + ABC abc = 1; + Baz baz = 2; + repeated uint32 xs = 3; +} + +message Baz { + oneof sum { + uint32 x = 1; + string y = 2; + } +} + +enum ABC { + A = 0; + B = 1; + C = 2; + D = 3; +} +``` + +#### Go + +In golang, the generated code would not expose any exported struct fields, but rather getters and setters as an +interface +or struct methods, ex: + +```go +type Foo interface { + X() int32 + SetX(int32) + Y() zpb.Option[uint32] + SetY(zpb.Option[uint32]) + Z() (string, error) + SetZ(string) error + Bar() Bar + Bars() (zpb.Array[Bar], error) +} + +type Bar interface { + Abc() ABC + SetAbc(ABC) Bar + Baz() Baz + Xs() (zpb.ScalarArray[uint32], error) +} + +type Baz interface { + Case() Baz_case + GetX() uint32 + SetX(uint32) + GetY() (string, error) + SetY(string) +} + +type Baz_case int32 +const ( + Baz_X Baz_case = 0 + Baz_Y Baz_case = 1 +) + +type ABC int32 +const ( + ABC_A ABC = 0 + ABC_B ABC = 1 + ABC_C ABC = 2 + ABC_D ABC = 3 +) +``` + +Special types `zpb.Option`, `zpb.Array` and `zpb.ScalarArray` are used to represent `optional` and repeated fields +respectively. These types would be included in the runtime library (called `zpb` here for zero-copy protobuf) and would +have an API like this: + +```go +type Option[T] interface { + IsSet() bool + Value() T +} + +type Array[T] interface { + InitWithLength(int) error + Len() int + Get(int) T +} + +type ScalarArray[T] interface { + Array[T] + Set(int, T) +} +``` + +Arrays in particular would not be resizable, but would be initialized with a fixed length. This is to ensure that arrays +can be written to the underlying buffer in a linear way. + +In golang, buffers would be managed transparently under the hood by the first message initialized, and usage of this +generated code might look like this: + +```go +foo := NewFoo() +foo.SetX(1) +foo.SetY(zpb.NewOption[uint32](2)) +err := foo.SetZ("hello") +if err != nil { + panic(err) +} + +bar := foo.Bar() +bar.Baz().SetX(3) + +xs, err = bar.Xs() +if err != nil { + panic(err) +} +xs.InitWithLength(2) +xs.Set(0, 0) +xs.Set(1, 2) + +bars, err = foo.Bars() +if err != nil { + panic(err) +} +bars.InitWithLength(3) +bars.Get(0).Baz().SetY("hello") +bars.Get(1).SetAbc(ABC_B) +bars.Get(2).Baz().SetX(4) +``` + +Under the hood the generated code would manage memory buffers on its own. The usage of `oneof`s is a bit easier than +the existing go generated code (as with `bar.Baz()` above). And rather than using setters on embedded messages, we +simply get the field (already allocated) and set its fields (as in the case of `foo.Bar()` above or the repeated +field `foo.Bars()`). Whenever a field is stored with a pointer (`string`, `bytes`, and `repeated` fields), there is +always an error returned on the getter to do proper bounds checking on the buffer. + +#### Rust + +This encoding should allow generating native structs in Rust that are annotated with `#[repr(C, align(1))]`. It should +be fairly natural to use from Rust with a key difference that memory buffers (called `Root`s) must be manually allocated +and passed into any pointer type. + +Here is some example code that uses library types `Option`, `Enum`, `String`, `OneOf` and `Repeated` +as well as little-endian integer types from [rend](https://lib.rs/crates/rend): + +```rust! +#[repr(C, align(1))] +struct Foo { + x: rend:i32_le, + y: cosmos_proto::Option, + z: cosmos_proto::String, // String wraps a pointer to a string + bar: Bar +} + +#[repr(C, align(1))] +struct Bar { + abc: cosmos_proto::Enum, // the Enum wrapper allows us to distinguish undefined and defined values of ABC at runtime. 3 is specified as the max value of ABC. + baz: cosmos_proto::OneOf, // the OneOf wrapper allows distinguished undefined values of Baz at runtime. 2 is specified as the max field value of Baz. + xs: cosmos_proto::Repeated // Repeated wraps a pointer to repeated fields +} + +#[repr(u8)] +enum ABC { + A = 0, + B = 1, + C = 2, + D = 3, +} + +#[repr(C, u8)] +enum Baz { + Empty, // all oneof's have a case for Empty if they are unset + X(rend::u32_le), + Y(cosmos_proto::String) +} +``` + +Example usage (which does the exact same thing as the go example above) would be: + +```rust! +let mut root = Root::new(); +let mut foo = root.get_mut(); +foo.x = 1.into(); +foo.y = Some(2.into()); +foo.z.set(root.new_string("hello")?); // could return an allocation error + +foo.bar.baz = Baz::X(3.into()); + +foo.bar.xs.init_with_size(&mut root, 2)?; // could return an allocation error +foo.bar.xs[0] = 0.into(); +foo.bar.xs[1] = 2.into(); + +foo.bars.init_with_size(&mut root, 3)?; // could return an allocation error +foo.bars[0].baz = Baz::Y(root.new_string("hello")?); // could return an allocation error +foo.bars[1].abc = ABC::B; +foo.bars[2].baz = Baz::X(4.into()); +``` + +## Abandoned Ideas (Optional) + +## References + +## Discussion diff --git a/docs/build/rfc/rfc-004-accounts.md b/docs/build/rfc/rfc-004-accounts.md new file mode 100644 index 000000000..c465508ef --- /dev/null +++ b/docs/build/rfc/rfc-004-accounts.md @@ -0,0 +1,330 @@ +# RFC 004: Accounts + +## Changelog + +* 17/03/2023: DRAFT +* 09/05/2023: DRAFT 2 + +## Context + +The current implementation of accounts in the Cosmos SDK is limiting in terms of functionality, extensibility, and overall +architecture. This RFC aims to address the following issues with the current account system: + +### 1. Accounts Representation and Authentication Mechanism + +The current SDK accounts are represented as `google.Protobuf.Any`, which are then encapsulated into the account interface. +This account interface essentially represents the authentication mechanism, as it implements methods such as `GetNumber` +and `GetSequence` that serve as abstractions over the authentication system. However, this approach restricts the scope and +functionality of accounts within the SDK. + +### 2. Limited Account Interface + +The account interface in its current form is not versatile enough to accommodate more advanced account functionalities, +such as implementing vesting capabilities or more complex authentication and authorization systems. + +### 3. Multiple Implementations of the Account Interface + +There are several implementations of the account interface, like `ModuleAccount`, but the existing abstraction does not +allow for meaningful differentiation between them. This hinders the ability to create specialized accounts that cater to +specific use cases. + +### 4. Primitive Authorization System + +The authorization system in the `x/auth` module is basic and defines authorizations solely for the functionalities of the +`x/bank` module. Consequently, although the state transition authorization system is defined in `x/auth`, it only covers the +use cases of `x/bank`, limiting the system's overall scope and adaptability. + +### 5. Cyclic Dependencies and Abstraction Leaks + +The current account system leads to cyclic dependencies and abstraction leaks throughout the Cosmos SDK. For instance, +the `Vesting` functionality belongs to the `x/auth` module, which depends on the `x/bank` module. However, +the `x/bank` module depends on the `x/auth` module again to identify the account type (either `Vesting` or `Base`) during +a coin transfer. This dependency structure creates architectural issues and complicates the overall design of the SDK. + +## Proposal + +This proposal aims to transform the way accounts are managed within the Cosmos SDK by introducing significant changes to +their structure and functionality. + +### Rethinking Account Representation and Business Logic + +Instead of representing accounts as simple `google.Protobuf.Any` structures stored in state with no business logic +attached, this proposal suggests a more sophisticated account representation that is closer to module entities. +In fact, accounts should be able to receive messages and process them in the same way modules do, and be capable of storing +state in a isolated (prefixed) portion of state belonging only to them, in the same way as modules do. + +### Account Message Reception + +We propose that accounts should be able to receive messages in the same way modules can, allowing them to manage their +own state modifications without relying on other modules. This change would enable more advanced account functionality, such as the +`VestingAccount` example, where the x/bank module previously needed to change the vestingState by casting the abstracted +account to `VestingAccount` and triggering the `TrackDelegation` call. Accounts are already capable of sending messages when +a state transition, originating from a transaction, is executed. + +When accounts receive messages, they will be able to identify the sender of the message and decide how to process the +state transition, if at all. + +### Consequences + +These changes would have significant implications for the Cosmos SDK, resulting in a system of actors that are equal from +the runtime perspective. The runtime would only be responsible for propagating messages between actors and would not +manage the authorization system. Instead, actors would manage their own authorizations. For instance, there would be no +need for the `x/auth` module to manage minting or burning of coins permissions, as it would fall within the scope of the +`x/bank` module. + +The key difference between accounts and modules would lie in the origin of the message (state transition). Accounts +(ExternallyOwnedAccount), which have credentials (e.g., a public/private key pairing), originate state transitions from +transactions. In contrast, module state transitions do not have authentication credentials backing them and can be +caused by two factors: either as a consequence of a state transition coming from a transaction or triggered by a scheduler +(e.g., the runtime's Begin/EndBlock). + +By implementing these proposed changes, the Cosmos SDK will benefit from a more extensible, versatile, and efficient account +management system that is better suited to address the requirements of the Cosmos ecosystem. + +#### Standardization + +With `x/accounts` allowing a modular api there becomes a need for standardization of accounts or the interfaces wallets and other clients should expect to use. For this reason we will be using the [`CIP` repo](https://github.com/cosmos/cips) in order to standardize interfaces in order for wallets to know what to expect when interacting with accounts. + +## Implementation + +### Account Definition + +We define the new `Account` type, which is what an account needs to implement to be treated as such. +An `Account` type is defined at APP level, so it cannot be dynamically loaded as the chain is running without upgrading the +node code, unless we create something like a `CosmWasmAccount` which is an account backed by an `x/wasm` contract. + +```go +// Account is what the developer implements to define an account. +type Account[InitMsg proto.Message] interface { + // Init is the function that initialises an account instance of a given kind. + // InitMsg is used to initialise the initial state of an account. + Init(ctx *Context, msg InitMsg) error + // RegisterExecuteHandlers registers an account's execution messages. + RegisterExecuteHandlers(executeRouter *ExecuteRouter) + // RegisterQueryHandlers registers an account's query messages. + RegisterQueryHandlers(queryRouter *QueryRouter) + // RegisterMigrationHandlers registers an account's migration messages. + RegisterMigrationHandlers(migrationRouter *MigrationRouter) +} +``` + +### The InternalAccount definition + +The public `Account` interface implementation is then converted by the runtime into an `InternalAccount` implementation, +which contains all the information and business logic needed to operate the account. + +```go +type Schema struct { + state StateSchema // represents the state of an account + init InitSchema // represents the init msg schema + exec ExecSchema // represents the multiple execution msg schemas, containing also responses + query QuerySchema // represents the multiple query msg schemas, containing also responses + migrate *MigrateSchema // represents the multiple migrate msg schemas, containing also responses, it's optional +} + +type InternalAccount struct { + init func(ctx *Context, msg proto.Message) (*InitResponse, error) + execute func(ctx *Context, msg proto.Message) (*ExecuteResponse, error) + query func(ctx *Context, msg proto.Message) (proto.Message, error) + schema func() *Schema + migrate func(ctx *Context, msg proto.Message) (*MigrateResponse, error) +} +``` + +This is an internal view of the account as intended by the system. It is not meant to be what developers implement. An +example implementation of the `InternalAccount` type can be found in [this](https://github.com/testinginprod/accounts-poc/blob/main/examples/recover/recover.go) +example of account whose credentials can be recovered. In fact, even if the `Internal` implementation is untyped (with +respect to `proto.Message`), the concrete implementation is fully typed. + +During any of the execution methods of `InternalAccount`, `schema` excluded, the account is given a `Context` which provides: + +* A namespaced `KVStore` for the account, which isolates the account state from others (NOTE: no `store keys` needed, + the account address serves as `store key`). +* Information regarding itself (its address) +* Information regarding the sender. +* ... + +#### Init + +Init defines the entrypoint that allows for a new account instance of a given kind to be initialised. +The account is passed some opaque protobuf message which is then interpreted and contains the instructions that +constitute the initial state of an account once it is deployed. + +An `Account` code can be deployed multiple times through the `Init` function, similar to how a `CosmWasm` contract code +can be deployed (Instantiated) multiple times. + +#### Execute + +Execute defines the entrypoint that allows an `Account` to process a state transition, the account can decide then how to +process the state transition based on the message provided and the sender of the transition. + +#### Query + +Query defines a read-only entrypoint that provides a stable interface that links an account with its state. The reason for +which `Query` is still being preferred as an addition to raw state reflection is to: + +* Provide a stable interface for querying (state can be optimised and change more frequently than a query) +* Provide a way to define an account `Interface` with respect to its `Read/Write` paths. +* Provide a way to query information that cannot be processed from raw state reflection, ex: compute information from lazy + state that has not been yet concretely processed (eg: balances with respect to lazy inputs/outputs) + +#### Schema + +Schema provides the definition of an account from `API` perspective, and it's the only thing that should be taken into account +when interacting with an account from another account or module, for example: an account is an `authz-interface` account if +it has the following message in its execution messages `MsgProxyStateTransition{ state_transition: google.Protobuf.Any }`. + +### Migrate + +Migrate defines the entrypoint that allows an `Account` to migrate its state from a previous version to a new one. Migrations +can be initiated only by the account itself, concretely this means that the migrate action sender can only be the account address +itself, if the account wants to allow another address to migrate it on its behalf then it could create an execution message +that makes the account migrate itself. + +### x/accounts module + +In order to create accounts we define a new module `x/accounts`, note that `x/accounts` deploys account with no authentication +credentials attached to it which means no action of an account can be incepted from a TX, we will later explore how the +`x/authn` module uses `x/accounts` to deploy authenticated accounts. + +This also has another important implication for which account addresses are now fully decoupled from the authentication mechanism +which makes in turn off-chain operations a little more complex, as the chain becomes the real link between account identifier +and credentials. + +We could also introduce a way to deterministically compute the account address. + +Note, from the transaction point of view, the `init_message` and `execute_message` are opaque `google.Protobuf.Any`. + +The module protobuf definition for `x/accounts` are the following: + +```protobuf +// Msg defines the Msg service. +service Msg { + rpc Deploy(MsgDeploy) returns (MsgDeployResponse); + rpc Execute(MsgExecute) returns (MsgExecuteResponse); + rpc Migrate(MsgMigrate) returns (MsgMigrateResponse); +} + +message MsgDeploy { + string sender = 1; + string kind = 2; + google.Protobuf.Any init_message = 3; + repeated google.Protobuf.Any authorize_messages = 4 [(gogoproto.nullable) = false]; +} + +message MsgDeployResponse { + string address = 1; + uint64 id = 2; + google.Protobuf.Any data = 3; +} + +message MsgExecute { + string sender = 1; + string address = 2; + google.Protobuf.Any message = 3; + repeated google.Protobuf.Any authorize_messages = 4 [(gogoproto.nullable) = false]; +} + +message MsgExecuteResponse { + google.Protobuf.Any data = 1; +} + +message MsgMigrate { + string sender = 1; + string new_account_kind = 2; + google.Protobuf.Any migrate_message = 3; +} + +message MsgMigrateResponse { + google.Protobuf.Any data = 1; +} + +``` + +#### MsgDeploy + +Deploys a new instance of the given account `kind` with initial settings represented by the `init_message` which is a `google.Protobuf.Any`. +Of course the `init_message` can be empty. A response is returned containing the account ID and humanised address, alongside some response +that the account instantiation might produce. + +#### Address derivation + +In order to decouple public keys from account addresses, we introduce a new address derivation mechanism which is + + +#### MsgExecute + +Sends a `StateTransition` execution request, where the state transition is represented by the `message` which is a `google.Protobuf.Any`. +The account can then decide if to process it or not based on the `sender`. + +### MsgMigrate + +Migrates an account to a new version of itself, the new version is represented by the `new_account_kind`. The state transition +can only be incepted by the account itself, which means that the `sender` must be the account address itself. During the migration +the account current state is given to the new version of the account, which then executes the migration logic using the `migrate_message`, +it might change state or not, it's up to the account to decide. The response contains possible data that the account might produce +after the migration. + +#### Authorize Messages + +The `Deploy` and `Execute` messages have a field in common called `authorize_messages`, these messages are messages that the account +can execute on behalf of the sender. For example, in case an account is expecting some funds to be sent from the sender, +the sender can attach a `MsgSend` that the account can execute on the sender's behalf. These authorizations are short-lived, +they live only for the duration of the `Deploy` or `Execute` message execution, or until they are consumed. + +An alternative would have been to add a `funds` field, like it happens in cosmwasm, which guarantees the called contract that +the funds are available and sent in the context of the message execution. This would have been a simpler approach, but it would +have been limited to the context of `MsgSend` only, where the asset is `sdk.Coins`. The proposed generic way, instead, allows +the account to execute any message on behalf of the sender, which is more flexible, it could include NFT send execution, or +more complex things like `MsgMultiSend` or `MsgDelegate`, etc. + + +### Further discussion + +#### Sub-accounts + +We could provide a way to link accounts to other accounts. Maybe during deployment the sender could decide to link the +newly created to its own account, although there might be use-cases for which the deployer is different from the account +that needs to be linked, in this case a handshake protocol on linking would need to be defined. + +#### Predictable address creation + +We need to provide a way to create an account with a predictable address, this might serve a lot of purposes, like accounts +wanting to generate an address that: + +* nobody else can claim besides the account used to generate the new account +* is predictable + +For example: + +```protobuf + +message MsgDeployPredictable { + string sender = 1; + uint32 nonce = 2; + ... +} +``` + +And then the address becomes `bechify(concat(sender, nonce))` + +`x/accounts` would still use the monotonically increasing sequence as account number. + +#### Joining Multiple Accounts + +As developers are building new kinds of accounts, it becomes necessary to provide a default way to combine the +functionalities of different account types. This allows developers to avoid duplicating code and enables end-users to +create or migrate to accounts with multiple functionalities without requiring custom development. + +To address this need, we propose the inclusion of a default account type called "MultiAccount". The MultiAccount type is +designed to merge the functionalities of other accounts by combining their execution, query, and migration APIs. +The account joining process would only fail in the case of API (intended as non-state Schema APIs) conflicts, ensuring +compatibility and consistency. + +With the introduction of the MultiAccount type, users would have the option to either migrate their existing accounts to +a MultiAccount type or extend an existing MultiAccount with newer APIs. This flexibility empowers users to leverage +various account functionalities without compromising compatibility or resorting to manual code duplication. + +The MultiAccount type serves as a standardized solution for combining different account functionalities within the +cosmos-sdk ecosystem. By adopting this approach, developers can streamline the development process and users can benefit +from a modular and extensible account system. diff --git a/docs/build/rfc/rfc-005-optimistic-execution.md b/docs/build/rfc/rfc-005-optimistic-execution.md new file mode 100644 index 000000000..997cc35aa --- /dev/null +++ b/docs/build/rfc/rfc-005-optimistic-execution.md @@ -0,0 +1,124 @@ +# RFC 005: Optimistic Execution + +## Changelog + +- 2023-06-07: Refactor for Cosmos SDK (@facundomedica) +- 2022-08-16: Initial draft by Sei Network + +## Background + +Before ABCI++, the first and only time a CometBFT blockchain's application layer would know about a block proposal is after the voting period, at which point CometBFT would invoke `BeginBlock`, `DeliverTx`, `EndBlock`, and `Commit` ABCI methods of the application, with the block proposal contents passed in. + +With the introduction of ABCI++, the application layer now receives the block proposal before the voting period commences. This can be used to optimistically execute the block proposal in parallel with the voting process, thus reducing the block time. + +## Proposal + +Given that the application receives the block proposal in an earlier stage (`ProcessProposal`), it can be executed in the background so when `FinalizeBlock` is called the response can returned instantly. + +## Decision + +The newly introduced ABCI method `ProcessProposal` is called after a node receives the full block proposal of the current height but before prevote starts. CometBFT states that preemptively executing the block proposal is a potential use case for it: + +> - **Usage**: +> - Contains all information on the proposed block needed to fully execute it. +> - The Application may fully execute the block as though it was handling +> `RequestFinalizeBlock`. +> - However, any resulting state changes must be kept as _candidate state_, +> and the Application should be ready to discard it in case another block is decided. +> - The Application MAY fully execute the block — immediate execution + +Nevertheless, synchronously executing the proposal preemptively would not improve block time because it would just change the order of events (so the time we would like to save will be spent at `ProcessProposal` instead of `FinalizeBlock`). + +Instead, we need to make block execution asynchronous by starting a goroutine in `ProcessProposal` (whose termination signal is kept in the application context) and returning a response immediately. That way, the actual block execution would happen at the same time as voting. When voting finishes and `FinalizeBlock` is called, the application handler can wait for the previously started goroutine to finish, and commit the resulting cache store if the block hash matches. + +Assuming average voting period takes `P` and average block execution takes `Q`, this would reduce the average block time by `P + Q - max(P, Q)`. + +Sei Network reported `P=~600ms` and `Q=~300ms` during a load test, meaning that optimistic execution could cut the block time by ~300ms. + +The following diagram illustrates the intended flow: + +```mermaid +flowchart LR; + Propose-->ProcessProposal; + ProcessProposal-->Prevote; + Precommit-->FinalizeBlock; + FinalizeBlock-->Commit; + subgraph CometBFT + Propose + Prevote-->Precommit; + Commit + end + subgraph CosmosSDK + OEG[Optimistic Execution Goroutine]-.->FinalizeBlock + ProcessProposal-.->OEG + FinalizeBlock + end +``` + +Some considerations: + +- In the case that a proposal is being rejected by the local node, this node won't attempt to execute the proposal. +- The app must drop any previous Optimistic Execution data if `ProcessProposal` is called, in other words, abort and restart. + +## Implementation + +The execution context needs to have the following information: + +- The block proposal (`abci.RequestFinalizeBlock`) +- Termination and completion signal for the OE goroutine + +The OE goroutine would run on top of a cached branch of the root multi-store (which is the default behavior for `FinalizeBlock` as we only write to the underlying store once we've reached the end). + +The OE goroutine would periodically check if a termination signal has been sent to it, and stops if so. Once the OE goroutine finishes the execution it will set the completion signal. + +The application developers will have the option to enable or disable Optimistic Execution, being disabled by default. To enable it, the application will need to pass the `SetOptimisticExecution` option to `NewBaseApp`. This is because this feature will be considered experimental until it's proven to be stable. Note that individual nodes should not enable/disable this feature on their own, as it could lead to inconsistent behavior. + +Upon receiving a `ProcessProposal` call, the SDK would adopt the following procedure if OE is enabled: + +``` +abort any running OE goroutine and wait for goroutine exit +if height > initial height: + set OE fields + kick off an OP goroutine that optimistically process the proposal +else: + do nothing +respond to CometBFT +``` + +OE won't be enabled during the first block of the chain, regardless of the configuration. This is because during the first block `InitChain` writes to `finalizeBlockState` so `FinalizeBlock` starts with an already initialized state. In the case that an abort would be needed it would mean we need to run `InitChain` again or discard/recover the state; either way complicates the implementation too much just for a single block. + +Upon receiving a `FinalizeBlock` call, the SDK would adopt the following procedure: + +``` +if OE is enabled: + abort if the currently executing OE's hash doesn't match the proposal's hash + if aborted: + process the proposal synchronously + else: + wait for the OE goroutine to finish + respond to CometBFT +``` + +## Consequences + +### Backwards Compatibility + +This can only be implemented on SDK versions using ABCI++, meaning v0.50.x and above. + +### Positive + +- Shorter block times for the same amount of transactions + +### Negative + +- Adds some complexity to the SDK +- Multiple rounds on a block that contains long running transactions could be problematic + +### Neutral + +- Each node can decide whether to use this feature or not. + +### References + +- [Original RFC by Sei Network](https://github.com/sei-protocol/sei-chain/blob/81b8af7980df722a63a910cc35ff96e60a94cbfe/docs/rfc/rfc-000-optimistic-proposal-processing.md) +- [ABCI++ methods in CometBFT](https://github.com/cometbft/cometbft/blob/a09f5d33ecd8846369b93cae9063291eb8abc3a0/spec/abci/abci%2B%2B_methods.md) diff --git a/docs/build/spec/README.md b/docs/build/spec/README.md new file mode 100644 index 000000000..91f347a8e --- /dev/null +++ b/docs/build/spec/README.md @@ -0,0 +1,25 @@ +--- +sidebar_position: 1 +--- + +# Specifications + +This directory contains specifications for the modules of the Cosmos SDK as well as Interchain Standards (ICS) and other specifications. + +Cosmos SDK applications hold this state in a Merkle store. Updates to +the store may be made during transactions and at the beginning and end of every +block. + +## Cosmos SDK specifications + +* [Store](./store) - The core Merkle store that holds the state. +* [Bech32](./addresses/bech32.md) - Address format for Cosmos SDK applications. + +## Modules specifications + +Go the [module directory](https://docs.cosmos.network/main/modules) + +## CometBFT + +For details on the underlying blockchain and p2p protocols, see +the [CometBFT specification](https://github.com/cometbft/cometbft/tree/main/spec). diff --git a/docs/build/spec/_category_.json b/docs/build/spec/_category_.json index 0f1092344..5c2ccf7d4 100644 --- a/docs/build/spec/_category_.json +++ b/docs/build/spec/_category_.json @@ -1,5 +1,5 @@ { "label": "Specifications", - "position": 6, + "position": 8, "link": null } \ No newline at end of file diff --git a/docs/build/spec/_ics/README.md b/docs/build/spec/_ics/README.md new file mode 100644 index 000000000..803e0c89c --- /dev/null +++ b/docs/build/spec/_ics/README.md @@ -0,0 +1,3 @@ +# Cosmos ICS + +* [ICS030 - Signed Messages](./ics-030-signed-messages.md) diff --git a/docs/build/spec/_ics/ics-030-signed-messages.md b/docs/build/spec/_ics/ics-030-signed-messages.md new file mode 100644 index 000000000..ce4e273e6 --- /dev/null +++ b/docs/build/spec/_ics/ics-030-signed-messages.md @@ -0,0 +1,192 @@ +# ICS 030: Cosmos Signed Messages + +>TODO: Replace with valid ICS number and possibly move to new location. + +* [Changelog](#changelog) +* [Abstract](#abstract) +* [Preliminary](#preliminary) +* [Specification](#specification) +* [Future Adaptations](#future-adaptations) +* [API](#api) +* [References](#references) + +## Status + +Proposed. + +## Changelog + +## Abstract + +Having the ability to sign messages off-chain has proven to be a fundamental aspect +of nearly any blockchain. The notion of signing messages off-chain has many +added benefits such as saving on computational costs and reducing transaction +throughput and overhead. Within the context of the Cosmos, some of the major +applications of signing such data includes, but is not limited to, providing a +cryptographic secure and verifiable means of proving validator identity and +possibly associating it with some other framework or organization. In addition, +having the ability to sign Cosmos messages with a Ledger or similar HSM device. + +A standardized protocol for hashing, signing, and verifying messages that can be +implemented by the Cosmos SDK and other third-party organizations is needed. Such a +standardized protocol subscribes to the following: + +* Contains a specification of human-readable and machine-verifiable typed structured data +* Contains a framework for deterministic and injective encoding of structured data +* Utilizes cryptographic secure hashing and signing algorithms +* A framework for supporting extensions and domain separation +* Is invulnerable to chosen ciphertext attacks +* Has protection against potentially signing transactions a user did not intend to + +This specification is only concerned with the rationale and the standardized +implementation of Cosmos signed messages. It does **not** concern itself with the +concept of replay attacks as that will be left up to the higher-level application +implementation. If you view signed messages in the means of authorizing some +action or data, then such an application would have to either treat this as +idempotent or have mechanisms in place to reject known signed messages. + +## Preliminary + +The Cosmos message signing protocol will be parameterized with a cryptographic +secure hashing algorithm `SHA-256` and a signing algorithm `S` that contains +the operations `sign` and `verify` which provide a digital signature over a set +of bytes and verification of a signature respectively. + +Note, our goal here is not to provide context and reasoning about why necessarily +these algorithms were chosen apart from the fact they are the defacto algorithms +used in CometBFT and the Cosmos SDK and that they satisfy our needs for such +cryptographic algorithms such as having resistance to collision and second +pre-image attacks, as well as being [deterministic](https://en.wikipedia.org/wiki/Hash_function#Determinism) and [uniform](https://en.wikipedia.org/wiki/Hash_function#Uniformity). + +## Specification + +CometBFT has a well established protocol for signing messages using a canonical +JSON representation as defined [here](https://github.com/cometbft/cometbft/blob/master/types/canonical.go). + +An example of such a canonical JSON structure is CometBFT's vote structure: + +```go +type CanonicalJSONVote struct { + ChainID string `json:"@chain_id"` + Type string `json:"@type"` + BlockID CanonicalJSONBlockID `json:"block_id"` + Height int64 `json:"height"` + Round int `json:"round"` + Timestamp string `json:"timestamp"` + VoteType byte `json:"type"` +} +``` + +With such canonical JSON structures, the specification requires that they include +meta fields: `@chain_id` and `@type`. These meta fields are reserved and must be +included. They are both of type `string`. In addition, fields must be ordered +in lexicographically ascending order. + +For the purposes of signing Cosmos messages, the `@chain_id` field must correspond +to the Cosmos chain identifier. The user-agent should **refuse** signing if the +`@chain_id` field does not match the currently active chain! The `@type` field +must equal the constant `"message"`. The `@type` field corresponds to the type of +structure the user will be signing in an application. For now, a user is only +allowed to sign bytes of valid ASCII text ([see here](https://github.com/cometbft/cometbft/blob/v0.37.0/libs/strings/string.go#L35-L64)). +However, this will change and evolve to support additional application-specific +structures that are human-readable and machine-verifiable. + +Thus, we can have a canonical JSON structure for signing Cosmos messages using +the [JSON schema](http://json-schema.org/) specification as such: + +```json +{ + "$schema": "http://json-schema.org/draft-04/schema#", + "$id": "cosmos/signing/typeData/schema", + "title": "The Cosmos signed message typed data schema.", + "type": "object", + "properties": { + "@chain_id": { + "type": "string", + "description": "The corresponding Cosmos chain identifier.", + "minLength": 1 + }, + "@type": { + "type": "string", + "description": "The message type. It must be 'message'.", + "enum": [ + "message" + ] + }, + "text": { + "type": "string", + "description": "The valid ASCII text to sign.", + "pattern": "^[\\x20-\\x7E]+$", + "minLength": 1 + } + }, + "required": [ + "@chain_id", + "@type", + "text" + ] +} +``` + +e.g. + +```json +{ + "@chain_id": "1", + "@type": "message", + "text": "Hello, you can identify me as XYZ on keybase." +} +``` + +## Future Adaptations + +As applications can vary greatly in domain, it will be vital to support both +domain separation and human-readable and machine-verifiable structures. + +Domain separation will allow for application developers to prevent collisions of +otherwise identical structures. It should be designed to be unique per application +use and should directly be used in the signature encoding itself. + +Human-readable and machine-verifiable structures will allow end users to sign +more complex structures, apart from just string messages, and still be able to +know exactly what they are signing (opposed to signing a bunch of arbitrary bytes). + +Thus, in the future, the Cosmos signing message specification will be expected +to expand upon it's canonical JSON structure to include such functionality. + +## API + +Application developers and designers should formalize a standard set of APIs that +adhere to the following specification: + +----- + +### **cosmosSignBytes** + +Params: + +* `data`: the Cosmos signed message canonical JSON structure +* `address`: the Bech32 Cosmos account address to sign data with + +Returns: + +* `signature`: the Cosmos signature derived using signing algorithm `S` + +----- + +### Examples + +Using the `secp256k1` as the DSA, `S`: + +```javascript +data = { + "@chain_id": "1", + "@type": "message", + "text": "I hereby claim I am ABC on Keybase!" +} + +cosmosSignBytes(data, "cosmos1pvsch6cddahhrn5e8ekw0us50dpnugwnlfngt3") +> "0x7fc4a495473045022100dec81a9820df0102381cdbf7e8b0f1e2cb64c58e0ecda1324543742e0388e41a02200df37905a6505c1b56a404e23b7473d2c0bc5bcda96771d2dda59df6ed2b98f8" +``` + +## References diff --git a/docs/build/spec/addresses/README.md b/docs/build/spec/addresses/README.md index af63429d5..61db3aa93 100644 --- a/docs/build/spec/addresses/README.md +++ b/docs/build/spec/addresses/README.md @@ -1,3 +1,3 @@ # Addresses spec -* [Bech32](bech32.md) +* [Bech32](./bech32.md) diff --git a/docs/build/spec/addresses/bech32.md b/docs/build/spec/addresses/bech32.md index f996f9016..2c15bac68 100644 --- a/docs/build/spec/addresses/bech32.md +++ b/docs/build/spec/addresses/bech32.md @@ -18,4 +18,4 @@ While all user facing interfaces to Cosmos software should exposed Bech32 interf To covert between other binary representation of addresses and keys, it is important to first apply the Amino encoding process before Bech32 encoding. -A complete implementation of the Amino serialization format is unnecessary in most cases. Simply prepending bytes from this [table](https://github.com/cometbft/cometbft/blob/main/spec/blockchain/05-encoding.md) to the byte string payload before Bech32 encoding will sufficient for compatible representation. +A complete implementation of the Amino serialization format is unnecessary in most cases. Simply prepending bytes from this [table](https://github.com/cometbft/cometbft/blob/main/spec/blockchain/encoding.md) to the byte string payload before Bech32 encoding will sufficient for compatible representation. diff --git a/docs/build/spec/store/README.md b/docs/build/spec/store/README.md new file mode 100644 index 000000000..c53d69c67 --- /dev/null +++ b/docs/build/spec/store/README.md @@ -0,0 +1,235 @@ +# Store + +The store package defines the interfaces, types and abstractions for Cosmos SDK +modules to read and write to Merkleized state within a Cosmos SDK application. +The store package provides many primitives for developers to use in order to +work with both state storage and state commitment. Below we describe the various +abstractions. + +## Types + +### `Store` + +The bulk of the store interfaces are defined [here](https://github.com/cosmos/cosmos-sdk/blob/main/store/types/store.go), +where the base primitive interface, for which other interfaces build off of, is +the `Store` type. The `Store` interface defines the ability to tell the type of +the implementing store and the ability to cache wrap via the `CacheWrapper` interface. + +### `CacheWrapper` & `CacheWrap` + +One of the most important features a store has the ability to perform is the +ability to cache wrap. Cache wrapping is essentially the underlying store wrapping +itself within another store type that performs caching for both reads and writes +with the ability to flush writes via `Write()`. + +### `KVStore` & `CacheKVStore` + +One of the most important interfaces that both developers and modules interface +with, which also provides the basis of most state storage and commitment operations, +is the `KVStore`. The `KVStore` interface provides basic CRUD abilities and +prefix-based iteration, including reverse iteration. + +Typically, each module has it's own dedicated `KVStore` instance, which it can +get access to via the `sdk.Context` and the use of a pointer-based named key -- +`KVStoreKey`. The `KVStoreKey` provides pseudo-OCAP. How a exactly a `KVStoreKey` +maps to a `KVStore` will be illustrated below through the `CommitMultiStore`. + +Note, a `KVStore` cannot directly commit state. Instead, a `KVStore` can be wrapped +by a `CacheKVStore` which extends a `KVStore` and provides the ability for the +caller to execute `Write()` which commits state to the underlying state storage. +Note, this doesn't actually flush writes to disk as writes are held in memory +until `Commit()` is called on the `CommitMultiStore`. + +### `CommitMultiStore` + +The `CommitMultiStore` interface exposes the the top-level interface that is used +to manage state commitment and storage by an SDK application and abstracts the +concept of multiple `KVStore`s which are used by multiple modules. Specifically, +it supports the following high-level primitives: + +* Allows for a caller to retrieve a `KVStore` by providing a `KVStoreKey`. +* Exposes pruning mechanisms to remove state pinned against a specific height/version + in the past. +* Allows for loading state storage at a particular height/version in the past to + provide current head and historical queries. +* Provides the ability to rollback state to a previous height/version. +* Provides the ability to to load state storage at a particular height/version + while also performing store upgrades, which are used during live hard-fork + application state migrations. +* Provides the ability to commit all current accumulated state to disk and performs + Merkle commitment. + +## Implementation Details + +While there are many interfaces that the `store` package provides, there is +typically a core implementation for each main interface that modules and +developers interact with that are defined in the Cosmos SDK. + +### `iavl.Store` + +The `iavl.Store` provides the core implementation for state storage and commitment +by implementing the following interfaces: + +* `KVStore` +* `CommitStore` +* `CommitKVStore` +* `Queryable` +* `StoreWithInitialVersion` + +It allows for all CRUD operations to be performed along with allowing current +and historical state queries, prefix iteration, and state commitment along with +Merkle proof operations. The `iavl.Store` also provides the ability to remove +historical state from the state commitment layer. + +An overview of the IAVL implementation can be found [here](https://github.com/cosmos/iavl/blob/master/docs/overview.md). +It is important to note that the IAVL store provides both state commitment and +logical storage operations, which comes with drawbacks as there are various +performance impacts, some of which are very drastic, when it comes to the +operations mentioned above. + +When dealing with state management in modules and clients, the Cosmos SDK provides +various layers of abstractions or "store wrapping", where the `iavl.Store` is the +bottom most layer. When requesting a store to perform reads or writes in a module, +the typical abstraction layer in order is defined as follows: + +```text +iavl.Store <- cachekv.Store <- gaskv.Store <- cachemulti.Store <- rootmulti.Store +``` + +### Concurrent use of IAVL store + +The tree under `iavl.Store` is not safe for concurrent use. It is the +responsibility of the caller to ensure that concurrent access to the store is +not performed. + +The main issue with concurrent use is when data is written at the same time as +it's being iterated over. Doing so will cause a irrecoverable fatal error because +of concurrent reads and writes to an internal map. + +Although it's not recommended, you can iterate through values while writing to +it by disabling "FastNode" **without guarantees that the values being written will +be returned during the iteration** (if you need this, you might want to reconsider +the design of your application). This is done by setting `iavl-disable-fastnode` +to `true` in the config TOML file. + +### `cachekv.Store` + +The `cachekv.Store` store wraps an underlying `KVStore`, typically a `iavl.Store` +and contains an in-memory cache for storing pending writes to underlying `KVStore`. +`Set` and `Delete` calls are executed on the in-memory cache, whereas `Has` calls +are proxied to the underlying `KVStore`. + +One of the most important calls to a `cachekv.Store` is `Write()`, which ensures +that key-value pairs are written to the underlying `KVStore` in a deterministic +and ordered manner by sorting the keys first. The store keeps track of "dirty" +keys and uses these to determine what keys to sort. In addition, it also keeps +track of deleted keys and ensures these are also removed from the underlying +`KVStore`. + +The `cachekv.Store` also provides the ability to perform iteration and reverse +iteration. Iteration is performed through the `cacheMergeIterator` type and uses +both the dirty cache and underlying `KVStore` to iterate over key-value pairs. + +Note, all calls to CRUD and iteration operations on a `cachekv.Store` are thread-safe. + +### `gaskv.Store` + +The `gaskv.Store` store provides a simple implementation of a `KVStore`. +Specifically, it just wraps an existing `KVStore`, such as a cache-wrapped +`iavl.Store`, and incurs configurable gas costs for CRUD operations via +`ConsumeGas()` calls defined on the `GasMeter` which exists in a `sdk.Context` +and then proxies the underlying CRUD call to the underlying store. Note, the +`GasMeter` is reset on each block. + +### `cachemulti.Store` & `rootmulti.Store` + +The `rootmulti.Store` acts as an abstraction around a series of stores. Namely, +it implements the `CommitMultiStore` an `Queryable` interfaces. Through the +`rootmulti.Store`, an SDK module can request access to a `KVStore` to perform +state CRUD operations and queries by holding access to a unique `KVStoreKey`. + +The `rootmulti.Store` ensures these queries and state operations are performed +through cached-wrapped instances of `cachekv.Store` which is described above. The +`rootmulti.Store` implementation is also responsible for committing all accumulated +state from each `KVStore` to disk and returning an application state Merkle root. + +Queries can be performed to return state data along with associated state +commitment proofs for both previous heights/versions and the current state root. +Queries are routed based on store name, i.e. a module, along with other parameters +which are defined in `abci.RequestQuery`. + +The `rootmulti.Store` also provides primitives for pruning data at a given +height/version from state storage. When a height is committed, the `rootmulti.Store` +will determine if other previous heights should be considered for removal based +on the operator's pruning settings defined by `PruningOptions`, which defines +how many recent versions to keep on disk and the interval at which to remove +"staged" pruned heights from disk. During each interval, the staged heights are +removed from each `KVStore`. Note, it is up to the underlying `KVStore` +implementation to determine how pruning is actually performed. The `PruningOptions` +are defined as follows: + +```go +type PruningOptions struct { + // KeepRecent defines how many recent heights to keep on disk. + KeepRecent uint64 + + // Interval defines when the pruned heights are removed from disk. + Interval uint64 + + // Strategy defines the kind of pruning strategy. See below for more information on each. + Strategy PruningStrategy +} +``` + +The Cosmos SDK defines a preset number of pruning "strategies": `default`, `everything` +`nothing`, and `custom`. + +It is important to note that the `rootmulti.Store` considers each `KVStore` as a +separate logical store. In other words, they do not share a Merkle tree or +comparable data structure. This means that when state is committed via +`rootmulti.Store`, each store is committed in sequence and thus is not atomic. + +In terms of store construction and wiring, each Cosmos SDK application contains +a `BaseApp` instance which internally has a reference to a `CommitMultiStore` +that is implemented by a `rootmulti.Store`. The application then registers one or +more `KVStoreKey` that pertain to a unique module and thus a `KVStore`. Through +the use of an `sdk.Context` and a `KVStoreKey`, each module can get direct access +to it's respective `KVStore` instance. + +Example: + +```go +func NewApp(...) Application { + // ... + + bApp := baseapp.NewBaseApp(appName, logger, db, txConfig.TxDecoder(), baseAppOptions...) + bApp.SetCommitMultiStoreTracer(traceStore) + bApp.SetVersion(version.Version) + bApp.SetInterfaceRegistry(interfaceRegistry) + + // ... + + keys := sdk.NewKVStoreKeys(...) + transientKeys := sdk.NewTransientStoreKeys(...) + memKeys := sdk.NewMemoryStoreKeys(...) + + // ... + + // initialize stores + app.MountKVStores(keys) + app.MountTransientStores(transientKeys) + app.MountMemoryStores(memKeys) + + // ... +} +``` + +The `rootmulti.Store` itself can be cache-wrapped which returns an instance of a +`cachemulti.Store`. For each block, `BaseApp` ensures that the proper abstractions +are created on the `CommitMultiStore`, i.e. ensuring that the `rootmulti.Store` +is cached-wrapped and uses the resulting `cachemulti.Store` to be set on the +`sdk.Context` which is then used for block and transaction execution. As a result, +all state mutations due to block and transaction execution are actually held +ephemerally until `Commit()` is called by the ABCI client. This concept is further +expanded upon when the AnteHandler is executed per transaction to ensure state +is not committed for transactions that failed CheckTx. diff --git a/docs/develop/advanced/05-interblock-cache.md b/docs/build/spec/store/interblock-cache.md similarity index 99% rename from docs/develop/advanced/05-interblock-cache.md rename to docs/build/spec/store/interblock-cache.md index 9914c5b55..8741dad03 100644 --- a/docs/develop/advanced/05-interblock-cache.md +++ b/docs/build/spec/store/interblock-cache.md @@ -1,7 +1,3 @@ ---- -sidebar_position: 1 ---- - # Inter-block Cache * [Inter-block Cache](#inter-block-cache) @@ -281,9 +277,9 @@ func CacheWrap( ``` ### Implementation details - + The inter-block cache implementation uses a fixed-sized adaptive replacement cache (ARC) as cache. [The ARC implementation](https://github.com/hashicorp/golang-lru/blob/master/arc.go) is thread-safe. ARC is an enhancement over the standard LRU cache in that tracks both frequency and recency of use. This avoids a burst in access to new entries from evicting the frequently used older entries. It adds some additional tracking overhead to a standard LRU cache, computationally it is roughly `2x` the cost, and the extra memory overhead is linear with the size of the cache. The default cache size is `1000`. - + ## History Dec 20, 2022 - Initial draft finished and submitted as a PR diff --git a/docs/build/tooling/01-cosmovisor.md b/docs/build/tooling/01-cosmovisor.md index dae11c4b8..49dda0b35 100644 --- a/docs/build/tooling/01-cosmovisor.md +++ b/docs/build/tooling/01-cosmovisor.md @@ -4,7 +4,7 @@ sidebar_position: 1 # Cosmovisor -`cosmovisor` is a small process manager for Cosmos SDK application binaries that monitors the governance module for incoming chain upgrade proposals. If it sees a proposal that gets approved, `cosmovisor` can automatically download the new binary, stop the current binary, switch from the old binary to the new one, and finally restart the node with the new binary. +`cosmovisor` is a process manager for Cosmos SDK application binaries that monitors the governance module for incoming chain upgrade proposals. If it sees a proposal that gets approved, `cosmovisor` can automatically download the new binary, stop the current binary, switch from the old binary to the new one, and finally restart the node with the new binary. * [Design](#design) * [Contributing](#contributing) @@ -32,7 +32,11 @@ Cosmovisor is designed to be used as a wrapper for a `Cosmos SDK` app: *Note: If new versions of the application are not set up to run in-place store migrations, migrations will need to be run manually before restarting `cosmovisor` with the new binary. For this reason, we recommend applications adopt in-place store migrations.* -*Note: If validators would like to enable the auto-download option (which [we don't recommend](#auto-download)), and they are currently running an application using Cosmos SDK `v0.42`, they will need to use Cosmovisor [`v0.1`](https://github.com/cosmos/cosmos-sdk/releases/tag/cosmovisor%2Fv0.1.0). Later versions of Cosmovisor do not support Cosmos SDK `v0.44.3` or earlier if the auto-download option is enabled.* +*Note: Only the last version of cosmovisor is actively developed/maintained.* + +:::warning +Versions prior to v1.0.0 have a vulnerability that could lead to a DOS. Please upgrade to the latest version. +::: ## Contributing @@ -44,7 +48,7 @@ Release branches have the following format `release/cosmovisor/vA.B.x`, where A ### Installation -You can download Cosmovisor from the [GitHub releases](https://github.com/cosmos/cosmos-sdk/releases/tag/cosmovisor%2Fv1.3.0). +You can download Cosmovisor from the [GitHub releases](https://github.com/cosmos/cosmos-sdk/releases/tag/cosmovisor%2Fv1.5.0). To install the latest version of `cosmovisor`, run the following command: @@ -52,10 +56,10 @@ To install the latest version of `cosmovisor`, run the following command: go install cosmossdk.io/tools/cosmovisor/cmd/cosmovisor@latest ``` -To install a previous version, you can specify the version. IMPORTANT: Chains that use Cosmos SDK v0.44.3 or earlier (eg v0.44.2) and want to use auto-download feature MUST use `cosmovisor v0.1.0` +To install a specific version, you can specify the version: ```shell -go install github.com/cosmos/cosmos-sdk/cosmovisor/cmd/cosmovisor@v0.1.0 +go install cosmossdk.io/tools/cosmovisor/cmd/cosmovisor@v1.5.0 ``` Run `cosmovisor version` to check the cosmovisor version. @@ -63,7 +67,8 @@ Run `cosmovisor version` to check the cosmovisor version. Alternatively, for building from source, simply run `make cosmovisor`. The binary will be located in `tools/cosmovisor`. :::warning -Building from source using `make cosmovisor` won't display the correct `cosmovisor` version. +Installing cosmovisor using `go install` will display the correct `cosmovisor` version. +Building from source (`make cosmovisor`) or installing `cosmovisor` by other means won't display the correct version. ::: ### Command Line Arguments And Environment Variables @@ -73,23 +78,33 @@ The first argument passed to `cosmovisor` is the action for `cosmovisor` to take * `help`, `--help`, or `-h` - Output `cosmovisor` help information and check your `cosmovisor` configuration. * `run` - Run the configured binary using the rest of the provided arguments. * `version` - Output the `cosmovisor` version and also run the binary with the `version` argument. +* `config` - Display the current `cosmovisor` configuration, that means displaying the environment variables value that `cosmovisor` is using. +* `add-upgrade` - Add an upgrade manually to `cosmovisor`. This command allow you to easily add the binary corresponding to an upgrade in cosmovisor. All arguments passed to `cosmovisor run` will be passed to the application binary (as a subprocess). `cosmovisor` will return `/dev/stdout` and `/dev/stderr` of the subprocess as its own. For this reason, `cosmovisor run` cannot accept any command-line arguments other than those available to the application binary. -*Note: Use of `cosmovisor` without one of the action arguments is deprecated. For backwards compatibility, if the first argument is not an action argument, `run` is assumed. However, this fallback might be removed in future versions, so it is recommended that you always provide `run`. +:::warning +Use of `cosmovisor` without one of the action arguments is deprecated. For backwards compatibility, if the first argument is not an action argument, `run` is assumed. However, this fallback might be removed in future versions, so it is recommended that you always provide `run`. +::: `cosmovisor` reads its configuration from environment variables: * `DAEMON_HOME` is the location where the `cosmovisor/` directory is kept that contains the genesis binary, the upgrade binaries, and any additional auxiliary files associated with each binary (e.g. `$HOME/.gaiad`, `$HOME/.regend`, `$HOME/.simd`, etc.). * `DAEMON_NAME` is the name of the binary itself (e.g. `gaiad`, `regend`, `simd`, etc.). * `DAEMON_ALLOW_DOWNLOAD_BINARIES` (*optional*), if set to `true`, will enable auto-downloading of new binaries (for security reasons, this is intended for full nodes rather than validators). By default, `cosmovisor` will not auto-download new binaries. +* `DAEMON_DOWNLOAD_MUST_HAVE_CHECKSUM` (*optional*, default = `false`), if `true` cosmovisor will require that a checksum is provided in the upgrade plan for the binary to be downloaded. If `false`, cosmovisor will not require a checksum to be provided, but still check the checksum if one is provided. * `DAEMON_RESTART_AFTER_UPGRADE` (*optional*, default = `true`), if `true`, restarts the subprocess with the same command-line arguments and flags (but with the new binary) after a successful upgrade. Otherwise (`false`), `cosmovisor` stops running after an upgrade and requires the system administrator to manually restart it. Note restart is only after the upgrade and does not auto-restart the subprocess after an error occurs. * `DAEMON_RESTART_DELAY` (*optional*, default none), allow a node operator to define a delay between the node halt (for upgrade) and backup by the specified time. The value must be a duration (e.g. `1s`). +* `DAEMON_SHUTDOWN_GRACE` (*optional*, default none), if set, send interrupt to binary and wait the specified time to allow for cleanup/cache flush to disk before sending the kill signal. The value must be a duration (e.g. `1s`). * `DAEMON_POLL_INTERVAL` (*optional*, default 300 milliseconds), is the interval length for polling the upgrade plan file. The value must be a duration (e.g. `1s`). * `DAEMON_DATA_BACKUP_DIR` option to set a custom backup directory. If not set, `DAEMON_HOME` is used. * `UNSAFE_SKIP_BACKUP` (defaults to `false`), if set to `true`, upgrades directly without performing a backup. Otherwise (`false`, default) backs up the data before trying the upgrade. The default value of false is useful and recommended in case of failures and when a backup needed to rollback. We recommend using the default backup option `UNSAFE_SKIP_BACKUP=false`. -* `DAEMON_PREUPGRADE_MAX_RETRIES` (defaults to `0`). The maximum number of times to call `pre-upgrade` in the application after exit status of `31`. After the maximum number of retries, Cosmovisor fails the upgrade. +* `DAEMON_PREUPGRADE_MAX_RETRIES` (defaults to `0`). The maximum number of times to call [`pre-upgrade`](https://docs.cosmos.network/main/building-apps/app-upgrade#pre-upgrade-handling) in the application after exit status of `31`. After the maximum number of retries, Cosmovisor fails the upgrade. * `COSMOVISOR_DISABLE_LOGS` (defaults to `false`). If set to true, this will disable Cosmovisor logs (but not the underlying process) completely. This may be useful, for example, when a Cosmovisor subcommand you are executing returns a valid JSON you are then parsing, as logs added by Cosmovisor make this output not a valid JSON. +* `COSMOVISOR_COLOR_LOGS` (defaults to `true`). If set to true, this will colorise Cosmovisor logs (but not the underlying process). +* `COSMOVISOR_TIMEFORMAT_LOGS` (defaults to `kitchen`). If set to a value (`layout|ansic|unixdate|rubydate|rfc822|rfc822z|rfc850|rfc1123|rfc1123z|rfc3339|rfc3339nano|kitchen`), this will add timestamp prefix to Cosmovisor logs (but not the underlying process). +* `COSMOVISOR_CUSTOM_PREUPGRADE` (defaults to ``). If set, this will run $DAEMON_HOME/cosmovisor/$COSMOVISOR_CUSTOM_PREUPGRADE prior to upgrade with the arguments [ upgrade.Name, upgrade.Height ]. Executes a custom script (separate and prior to the chain daemon pre-upgrade command) +* `COSMOVISOR_DISABLE_RECASE` (defaults to `false`). If set to true, the upgrade directory will expected to match the upgrade plan name without any case changes ### Folder Layout @@ -102,10 +117,11 @@ All arguments passed to `cosmovisor run` will be passed to the application binar │   └── bin │   └── $DAEMON_NAME └── upgrades - └── - ├── bin - │   └── $DAEMON_NAME - └── upgrade-info.json +│ └── +│ ├── bin +│ │   └── $DAEMON_NAME +│ └── upgrade-info.json +└── preupgrade.sh (optional) ``` The `cosmovisor/` directory incudes a subdirectory for each version of the application (i.e. `genesis` or `upgrades/`). Within each subdirectory is the application binary (i.e. `bin/$DAEMON_NAME`) and any additional auxiliary files associated with each binary. `current` is a symbolic link to the currently active directory (i.e. `genesis` or `upgrades/`). The `name` variable in `upgrades/` is the lowercased URI-encoded name of the upgrade as specified in the upgrade module plan. Note that the upgrade name path are normalized to be lowercased: for instance, `MyUpgrade` is normalized to `myupgrade`, and its path is `upgrades/myupgrade`. @@ -167,6 +183,18 @@ When the upgrade mechanism is triggered, `cosmovisor` will: 1. if `DAEMON_ALLOW_DOWNLOAD_BINARIES` is enabled, start by auto-downloading a new binary into `cosmovisor//bin` (where `` is the `upgrade-info.json:name` attribute); 2. update the `current` symbolic link to point to the new directory and save `data/upgrade-info.json` to `cosmovisor/current/upgrade-info.json`. +### Adding Upgrade Binary + +`cosmovisor` has an `add-upgrade` command that allows to easily link a binary to an upgrade. It creates a new folder in `cosmovisor/upgrades/` and copies the provided executable file to `cosmovisor/upgrades//bin/`. + +Using the `--upgrade-height` flag allows to specify at which height the binary should be switched, without going via a gorvernance proposal. +This enables support for an emergency coordinated upgrades where the binary must be switched at a specific height, but there is no time to go through a governance proposal. + +:::warning +`--upgrade-height` creates an `upgrade-info.json` file. This means if a chain upgrade via governance proposal is executed before the specified height with `--upgrade-height`, the governance proposal will overwrite the `upgrade-info.json` plan created by `add-upgrade --upgrade-height `. +Take this into consideration when using `--upgrade-height`. +::: + ### Auto-Download Generally, `cosmovisor` requires that the system administrator place all relevant binaries on disk before the upgrade happens. However, for people who don't need such control and want an automated setup (maybe they are syncing a non-validating fullnode and want to do little maintenance), there is another option. @@ -240,25 +268,25 @@ The following instructions provide a demonstration of `cosmovisor` using the sim ### Chain Setup -Let's create a new chain using the `v0.44` version of simapp (the Cosmos SDK demo app): +Let's create a new chain using the `v0.47.4` version of simapp (the Cosmos SDK demo app): ```shell -git checkout v0.44.6 +git checkout v0.47.4 make build ``` Clean `~/.simapp` (never do this in a production environment): ```shell -./build/simd unsafe-reset-all +./build/simd tendermint unsafe-reset-all ``` Set up app config: ```shell -./build/simd config set client chain-id test -./build/simd config set client keyring-backend test -./build/simd config set client broadcast-mode sync +./build/simd config chain-id test +./build/simd config keyring-backend test +./build/simd config broadcast-mode sync ``` Initialize the node and overwrite any previous genesis file (never do this in a production environment): @@ -269,16 +297,10 @@ Initialize the node and overwrite any previous genesis file (never do this in a ./build/simd init test --chain-id test --overwrite ``` -Set the minimum gas price to `0stake` in `~/.simapp/config/app.toml`: - -```shell -minimum-gas-prices = "0stake" -``` - For the sake of this demonstration, amend `voting_period` in `genesis.json` to a reduced time of 20 seconds (`20s`): ```shell -cat <<< $(jq '.app_state.gov.voting_params.voting_period = "20s"' $HOME/.simapp/config/genesis.json) > $HOME/.simapp/config/genesis.json +cat <<< $(jq '.app_state.gov.params.voting_period = "20s"' $HOME/.simapp/config/genesis.json) > $HOME/.simapp/config/genesis.json ``` Create a validator, and setup genesis transaction: @@ -305,60 +327,52 @@ Set the optional environment variable to trigger an automatic app restart: export DAEMON_RESTART_AFTER_UPGRADE=true ``` -Create the folder for the genesis binary and copy the `simd` binary: +Initialize cosmovisor with the current binary: ```shell -mkdir -p $DAEMON_HOME/cosmovisor/genesis/bin -cp ./build/simd $DAEMON_HOME/cosmovisor/genesis/bin +cosmovisor init ./build/simd ``` -Now you can run cosmovisor with simapp v0.44: +Now you can run cosmovisor with simapp v0.47.4: ```shell cosmovisor run start ``` -#### Update App +### Update App -Update app to the latest version (e.g. v0.45). +Update app to the latest version (e.g. v0.50.0). -Next, we can add a migration - which is defined using `x/upgrade` [upgrade plan](https://github.com/cosmos/cosmos-sdk/blob/main/docs/advanced/13-upgrade.md) (you may refer to a past version if you are using an older Cosmos SDK release). In a migration we can do any deterministic state change. +:::note -Build the new version `simd` binary: +Migration plans are defined using the `x/upgrade` module and described in [In-Place Store Migrations](https://github.com/cosmos/cosmos-sdk/blob/main/docs/docs/core/15-upgrade.md). Migrations can perform any deterministic state change. -```shell -make build -``` +The migration plan to upgrade the simapp from v0.47 to v0.50 is defined in `simapp/upgrade.go`. + +::: -Create the folder for the upgrade binary and copy the `simd` binary: +Build the new version `simd` binary: ```shell -mkdir -p $DAEMON_HOME/cosmovisor/upgrades/test1/bin -cp ./build/simd $DAEMON_HOME/cosmovisor/upgrades/test1/bin +make build ``` -Open a new terminal window and submit an upgrade proposal along with a deposit and a vote (these commands must be run within 20 seconds of each other): +Add the new `simd` binary and the upgrade name: -**<= v0.45**: +:::warning -```shell -./build/simd tx gov submit-proposal software-upgrade test1 --title upgrade --description upgrade --upgrade-height 200 --from validator --yes -./build/simd tx gov deposit 1 10000000stake --from validator --yes -./build/simd tx gov vote 1 yes --from validator --yes -``` +The migration name must match the one defined in the migration plan. -**v0.46, v0.47**: +::: ```shell -./build/simd tx gov submit-legacy-proposal software-upgrade test1 --title upgrade --description upgrade --upgrade-height 200 --from validator --yes -./build/simd tx gov deposit 1 10000000stake --from validator --yes -./build/simd tx gov vote 1 yes --from validator --yes +cosmovisor add-upgrade v047-to-v050 ./build/simd ``` -**>= v0.48+**: +Open a new terminal window and submit an upgrade proposal along with a deposit and a vote (these commands must be run within 20 seconds of each other): ```shell -./build/simd tx upgrade software-upgrade test1 --title upgrade --summary upgrade --upgrade-height 200 --from validator --yes +./build/simd tx upgrade software-upgrade v047-to-v050 --title upgrade --summary upgrade --upgrade-height 200 --upgrade-info "{}" --no-validate --from validator --yes ./build/simd tx gov deposit 1 10000000stake --from validator --yes ./build/simd tx gov vote 1 yes --from validator --yes ``` diff --git a/docs/build/tooling/02-confix.md b/docs/build/tooling/02-confix.md index 19c637ba9..a990ed98d 100644 --- a/docs/build/tooling/02-confix.md +++ b/docs/build/tooling/02-confix.md @@ -12,7 +12,7 @@ It is based on the [CometBFT RFC 019](https://github.com/cometbft/cometbft/blob/ ### Add Config Command -To add the confix tool, it's required to add the `ConfigCommand` to your application's root command file (e.g. `simd/cmd/root.go`). +To add the confix tool, it's required to add the `ConfigCommand` to your application's root command file (e.g. `/cmd/root.go`). Import the `confixCmd` package: diff --git a/docs/build/tooling/04-hubl.md b/docs/build/tooling/03-hubl.md similarity index 100% rename from docs/build/tooling/04-hubl.md rename to docs/build/tooling/03-hubl.md diff --git a/docs/build/tooling/05-depinject.md b/docs/build/tooling/05-depinject.md deleted file mode 100644 index 17eb62872..000000000 --- a/docs/build/tooling/05-depinject.md +++ /dev/null @@ -1,169 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Depinject - -> **DISCLAIMER**: This is a **beta** package. The SDK team is actively working on this feature and we are looking for feedback from the community. Please try it out and let us know what you think. - -## Overview - -`depinject` is a dependency injection framework for the Cosmos SDK. This module together with `core/appconfig` are meant to simplify the definition of a blockchain by replacing most of `app.go`'s boilerplate code with a configuration file (Go, YAML or JSON). - -* [Go Doc](https://pkg.go.dev/cosmossdk.io/depinject) - -## Usage - -`depinject` includes an expressive and composable [Configuration API](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/depinject#Config). -A core configuration function is `Provide`. The example below demonstrates the registration of free **provider functions** via the `Provide` API. - - -```go -package main - -import ( - "fmt" - - "cosmossdk.io/depinject" -) - -type AnotherInt int - -func main() { - var ( - x int - y AnotherInt - ) - - fmt.Printf("Before (%v, %v)\n", x, y) - depinject.Inject( - depinject.Provide( - func() int { return 1 }, - func() AnotherInt { return AnotherInt(2) }, - ), - &x, - &y, - ) - fmt.Printf("After (%v, %v)\n", x, y) -} -``` - -Provider functions form the basis of the dependency tree, they are introspected then their inputs identified as dependencies and outputs as dependants, either for another provider function or state stored outside the DI container, as is the case of `&x` and `&y` above. - -### Interface type resolution - -`depinject` supports interface types as inputs to provider functions. In the SDK's case this pattern is used to decouple -`Keeper` dependencies between modules. For example `x/bank` expects an [AccountKeeper](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/x/bank/types#AccountKeeper) interface as [input to ProvideModule](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/bank/module.go#L208-L260). - -Concretely `SimApp` uses the implementation in `x/auth`, but this design allows for this loose coupling to change. - -Given the following types: - -```go -package duck - -type Duck interface { - quack() -} - -type AlsoDuck interface { - quack() -} - -type Mallard struct{} -type Canvasback struct{} - -func (duck Mallard) quack() {} -func (duck Canvasback) quack() {} - -type Pond struct { - Duck AlsoDuck -} -``` - -This usage - -```go -var pond Pond - -depinject.Inject( - depinject.Provide( - func() Mallard { return Mallard{} }, - func(duck Duck) Pond { - return Pond{Duck: duck} - }), - &pond) -``` - -results in an *implicit* binding of `Duck` to `Mallard`. This works because there is only one implementation of `Duck` in the container. -However, adding a second provider of `Duck` will result in an error: - -```go -var pond Pond - -depinject.Inject( - depinject.Provide( - func() Mallard { return Mallard{} }, - func() Canvasback { return Canvasback{} }, - func(duck Duck) Pond { - return Pond{Duck: duck} - }), - &pond) -``` - -A specific binding preference for `Duck` is required. - -#### `BindInterface` API - -In the above situation registering a binding for a given interface binding may look like - -```go -depinject.Inject( - depinject.Configs( - depinject.BindInterface( - "duck.Duck", - "duck.Mallard"), - depinject.Provide( - func() Mallard { return Mallard{} }, - func() Canvasback { return Canvasback{} }, - func(duck Duck) APond { - return Pond{Duck: duck} - })), - &pond) -``` - -Now `depinject` has enough information to provide `Mallard` as an input to `APond`. - -### Full example in real app - -:::warning -When using `depinject.Inject`, the injected types must be pointers. -::: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/app_v2.go#L219-L244 -``` - -## Debugging - -Issues with resolving dependencies in the container can be done with logs and [Graphviz](https://graphviz.org) renderings of the container tree. -By default, whenever there is an error, logs will be printed to stderr and a rendering of the dependency graph in Graphviz DOT format will be saved to `debug_container.dot`. - -Here is an example Graphviz rendering of a successful build of a dependency graph: -![Graphviz Example](https://raw.githubusercontent.com/cosmos/cosmos-sdk/ff39d243d421442b400befcd959ec3ccd2525154/depinject/testdata/example.svg) - -Rectangles represent functions, ovals represent types, rounded rectangles represent modules and the single hexagon -represents the function which called `Build`. Black-colored shapes mark functions and types that were called/resolved -without an error. Gray-colored nodes mark functions and types that could have been called/resolved in the container but -were left unused. - -Here is an example Graphviz rendering of a dependency graph build which failed: -![Graphviz Error Example](https://raw.githubusercontent.com/cosmos/cosmos-sdk/ff39d243d421442b400befcd959ec3ccd2525154/depinject/testdata/example_error.svg) - -Graphviz DOT files can be converted into SVG's for viewing in a web browser using the `dot` command-line tool, ex: - -```txt -dot -Tsvg debug_container.dot > debug_container.svg -``` - -Many other tools including some IDEs support working with DOT files. diff --git a/docs/build/tooling/README.md b/docs/build/tooling/README.md index 1fa6ea2cc..27bc94e24 100644 --- a/docs/build/tooling/README.md +++ b/docs/build/tooling/README.md @@ -16,4 +16,4 @@ This includes tools for development, operating a node, and ease of use of a Cosm ## Other Tools -* [Protocol Buffers](./00-protobuf.md) \ No newline at end of file +* [Protocol Buffers](./00-protobuf.md) diff --git a/docs/build/tooling/_category_.json b/docs/build/tooling/_category_.json index a01a4fcc4..eb57cb8a5 100644 --- a/docs/build/tooling/_category_.json +++ b/docs/build/tooling/_category_.json @@ -1,5 +1,5 @@ { "label": "Tooling", - "position": 10, + "position": 5, "link": null } \ No newline at end of file diff --git a/docs/develop/_category_.json b/docs/develop/_category_.json deleted file mode 100644 index f40637f4d..000000000 --- a/docs/develop/_category_.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "label": "Develop", - "position": 0, - "link": null -} \ No newline at end of file diff --git a/docs/develop/advanced/15-tips.md b/docs/develop/advanced/15-tips.md deleted file mode 100644 index 7a3261233..000000000 --- a/docs/develop/advanced/15-tips.md +++ /dev/null @@ -1,214 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Transaction Tips - -:::note Synopsis -Transaction tips are a mechanism to pay for transaction fees using another denom than the native fee denom of the chain. They are still in beta, and are not included by default in the SDK. -::: - -## Context - -In a Cosmos ecosystem where more and more chains are connected via [IBC](https://ibc.cosmos.network/), it happens that users want to perform actions on chains where they don't have native tokens yet. An example would be an Osmosis user who wants to vote on a proposal on the Cosmos Hub, but they don't have ATOMs in their wallet. A solution would be to swap OSMO for ATOM just for voting on this proposal, but that is cumbersome. Cross-chain DeFi project [Emeris](https://emeris.com/) is another use case. - -Transaction tips is a new solution for cross-chain transaction fees payment, whereby the transaction initiator signs a transaction without specifying fees, but uses a new `Tip` field. They send this signed transaction to a fee relayer who will choose the transaction fees and broadcast the final transaction, and the SDK provides a mechanism that will transfer the pre-defined `Tip` to the fee payer, to cover for fees. - -Assuming we have two chains, A and B, we define the following terms: - -* **the tipper**: this is the initiator of the transaction, who wants to execute a `Msg` on chain A, but doesn't have any native chain A tokens, only chain B tokens. In our example above, the tipper is the Osmosis (chain B) user wanting to vote on a Cosmos Hub (chain A) proposal. -* **the fee payer**: this is the party that will relay and broadcast the final transaction on chain A, and has chain A tokens. The tipper doesn't need to trust the feepayer. -* **the target chain**: the chain where the `Msg` is executed, chain A in this case. - -## Transaction Tips Flow - -The transaction tips flow happens in multiple steps. - -1. The tipper sends via IBC some chain B tokens to chain A. These tokens will cover for fees on the target chain A. This means that chain A's bank module holds some IBC tokens under the tipper's address. - -2. The tipper drafts a transaction to be executed on the chain A. It can include chain A `Msg`s. However, instead of creating a normal transaction, they create the following `AuxSignerData` document: - - ```protobuf reference - https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/tx/v1beta1/tx.proto#L237-L256 - ``` - - where we have defined `SignDocDirectAux` as: - - ```protobuf reference - https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/tx/v1beta1/tx.proto#L67-L97 - ``` - - where `Tip` is defined as - - ```protobuf reference - https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/tx/v1beta1/tx.proto#L226-L235 - ``` - - Notice that this document doesn't sign over the final chain A fees. Instead, it includes a `Tip` field. It also doesn't include the whole `AuthInfo` object as in `SIGN_MODE_DIRECT`, only the minimum information needed by the tipper - -3. The tipper signs the `SignDocDirectAux` document and attaches the signature to the `AuxSignerData`, then sends the signed `AuxSignerData` to the fee payer. - -4. From the signed `AuxSignerData` document, the fee payer constructs a transaction, using the following algorithm: - -* use as `TxBody` the exact `AuxSignerData.SignDocDirectAux.body_bytes`, to not alter the original intent of the tipper, -* create an `AuthInfo` with: - * `AuthInfo.Tip` copied from `AuxSignerData.SignDocDirectAux.Tip`, - * `AuthInfo.Fee` chosen by the fee payer, which should cover for the transaction gas, but also be small enough so that the tip/fee exchange rate is economically interesting for the fee payer, - * `AuthInfo.SignerInfos` has two signers: the first signer is the tipper, using the public key, sequence and sign mode specified in `AuxSignerData`; and the second signer is the fee payer, using their favorite sign mode, -* a `Signatures` array with two items: the tipper's signature from `AuxSignerData.Sig`, and the final fee payer's signature. - -5. Broadcast the final transaction signed by the two parties to the target chain. Once included, the Cosmos SDK will trigger a transfer of the `Tip` specified in the transaction from the tipper address to the fee payer address. - -### Fee Payers Market - -The benefit of transaction tips for the tipper is clear: there is no need to swap tokens before executing a cross-chain message. - -For the fee payer, the benefit is in the tip v.s. fee exchange. Put simply, the fee payer pays the fees of an unknown tipper's transaction, and gets in exchange the tip that the tipper chose. There is an economic incentive for the fee payer to do so only when the tip is greater than the transaction fees, given the exchange rates between the two tokens. - -In the future, we imagine a market where fee payers will compete to include transactions from tippers, who on their side will optimize by specifying the lowest tip possible. A number of automated services might spin up to perform transaction gas simulation and exchange rate monitoring to optimize both the tip and fee values in real-time. - -### Tipper and Fee Payer Sign Modes - -As we mentioned in the flow above, the tipper signs over the `SignDocDirectAux`, and the fee payer signs over the whole final transaction. As such, both parties might use different sign modes. - -* The tipper MUST use `SIGN_MODE_DIRECT_AUX` or `SIGN_MODE_LEGACY_AMINO_JSON`. That is because the tipper needs to sign over the body, the tip, but not the other signers' information and not over the fee (which is unknown to the tipper). -* The fee payer MUST use `SIGN_MODE_DIRECT` or `SIGN_MODE_LEGACY_AMINO_JSON`. The fee payer signs over the whole transaction. - -For example, if the fee payer signs the whole transaction with `SIGN_MODE_DIRECT_AUX`, it will be rejected by the node, as that would introduce malleability issues (`SIGN_MODE_DIRECT_AUX` doesn't sign over fees). - -In both cases, using `SIGN_MODE_LEGACY_AMINO_JSON` is recommended only if hardware wallet signing is needed. - -## Enabling Tips on your Chain - -The transaction tips functionality is introduced in Cosmos SDK v0.46, so earlier versions do not have support for tips. It is however not included by default in a v0.46 app. Sending a transaction with tips to a chain which didn't enable tips will result in a no-op, i.e. the `tip` field in the transaction will be ignored. - -Enabling tips on a chain is done by adding the `TipDecorator` in the posthandler chain: - -```go -// HandlerOptions are the options required for constructing a SDK PostHandler which supports tips. -type HandlerOptions struct { - BankKeeper types.BankKeeper -} - -// MyPostHandler returns a posthandler chain with the TipDecorator. -func MyPostHandler(options HandlerOptions) (sdk.AnteHandler, error) { - if options.BankKeeper == nil { - return nil, errorsmod.Wrap(sdkerrors.ErrLogic, "bank keeper is required for posthandler") - } - - postDecorators := []sdk.AnteDecorator{ - posthandler.NewTipDecorator(options.bankKeeper), - } - - return sdk.ChainAnteDecorators(postDecorators...), nil -} - -func (app *SimApp) setPostHandler() { - postHandler, err := MyPostHandler( - HandlerOptions{ - BankKeeper: app.BankKeeper, - }, - ) - if err != nil { - panic(err) - } - - app.SetPostHandler(postHandler) -} -``` - -Notice that `NewTipDecorator` needs a reference to the BankKeeper, for transferring the tip to the fee payer. - -## CLI Usage - -The Cosmos SDK also provides some CLI tooling for the transaction tips flow, both for the tipper and for the feepayer. - -For the tipper, the CLI `tx` subcommand has two new flags: `--aux` and `--tip`. The `--aux` flag is used to denote that we are creating an `AuxSignerData` instead of a `Tx`, and the `--tip` is used to populate its `Tip` field. - -```bash -$ simd tx gov vote 16 yes --from --aux --tip 50ibcdenom - - -### Prints the AuxSignerData as JSON: -### {"address":"cosmos1q0ayf5vq6fd2xxrwh30upg05hxdnyw2h5249a2","sign_doc":{"body_bytes":"CosBChwvY29zbW9zLmJhbmsudjFiZXRhMS5Nc2dTZW5kEmsKLWNvc21vczFxMGF5ZjV2cTZmZDJ4eHJ3aDMwdXBnMDVoeGRueXcyaDUyNDlhMhItY29zbW9zMXdlNWoyZXI2MHV5OXF3YzBta3ptdGdtdHA5Z3F5NXY2bjhnZGdlGgsKBXN0YWtlEgIxMA==","public_key":{"@type":"/cosmos.crypto.secp256k1.PubKey","key":"AojOF/1luQ5H/nZDSrE1w3CyzGJhJdQuS7hFX5wAA6uJ"},"chain_id":"","account_number":"0","sequence":"1","tip":{"amount":[{"denom":"ibcdenom","amount":"50"}],"tipper":"cosmos1q0ayf5vq6fd2xxrwh30upg05hxdnyw2h5249a2"}},"mode":"SIGN_MODE_DIRECT_AUX","sig":"v/d/bGq9FGdecs6faMG2t//nRirFTiqwFtUB65M6kh0QdUeM6jg3r8oJX1o17xkoDxJ09EyJiSyvo6fbU7vUxg=="} -``` - -It is useful to pipe the JSON output to a file, `> aux_signed_tx.json` - -For the fee payer, the Cosmos SDK added a `tx aux-to-fee` subcommand to include an `AuxSignerData` into a transaction, add fees to it, and broadcast it. - -```bash -$ simd tx aux-to-fee aux_signed_tx.json --from --fees 30atom - -### Prints the broadcasted tx response: -### code: 0 -### codespace: sdk -### data: "" -### events: [] -### gas_used: "0" -### gas_wanted: "0" -### height: "0" -### info: "" -### logs: [] -### timestamp: "" -### tx: null -``` - -Upon completion of the second command, the fee payer's balance will be down the `30atom` fees, and up the `50ibcdenom` tip. - -For both commands, the flag `--sign-mode=amino-json` is still available for hardware wallet signing. - -## Programmatic Usage - -For the tipper, the SDK exposes a new transaction builder, the `AuxTxBuilder`, for generating an `AuxSignerData`. The API of `AuxTxBuilder` is defined [in `client/tx`](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/client/tx/aux_builder.go#L16), and can be used as follows: - -```go -// Note: there's no need to use clientCtx.TxConfig anymore. - -bldr := clienttx.NewAuxTxBuilder() -err := bldr.SetMsgs(msgs...) -bldr.SetAddress("cosmos1...") -bldr.SetMemo(...) -bldr.SetTip(...) -bldr.SetPubKey(...) -err := bldr.SetSignMode(...) // DIRECT_AUX or AMINO, or else error -// ... other setters are also available - -// Get the bytes to sign. -signBz, err := bldr.GetSignBytes() - -// Sign the bz using your favorite method. -sig, err := privKey.sign(signBz) - -// Set the signature -bldr.SetSig(sig) - -// Get the final auxSignerData to be sent to the fee payer -auxSignerData, err:= bldr.GetAuxSignerData() -``` - -For the fee payer, the SDK added a new method on the existing `TxBuilder` to import data from an `AuxSignerData`: - -```go -// get `auxSignerData` from tipper, see code snippet above. - -txBuilder := clientCtx.TxConfig.NewTxBuilder() -err := txBuilder.AddAuxSignerData(auxSignerData) -if err != nil { - return err -} - -// A lot of fields will be populated in txBuilder, such as its Msgs, tip -// memo, etc... - -// The fee payer choses the fee to set on the transaction. -txBuilder.SetFeePayer() -txBuilder.SetFeeAmount(...) -txBuilder.SetGasLimit(...) - -// Usual signing code -err = authclient.SignTx(...) -if err != nil { - return err -} -``` diff --git a/docs/develop/advanced/00-baseapp.md b/docs/learn/advanced/00-baseapp.md similarity index 98% rename from docs/develop/advanced/00-baseapp.md rename to docs/learn/advanced/00-baseapp.md index 293a82928..90183a155 100644 --- a/docs/develop/advanced/00-baseapp.md +++ b/docs/learn/advanced/00-baseapp.md @@ -187,7 +187,7 @@ During `FinalizeBlock`, the `finalizeBlockState` is set for use during transacti `finalizeBlockState` is based off of the last committed state from the root store and is branched. Note, the `finalizeBlockState` is set to `nil` on [`Commit`](#commit). -The state flow for transcation execution is nearly identical to `CheckTx` except state transitions occur on +The state flow for transaction execution is nearly identical to `CheckTx` except state transitions occur on the `finalizeBlockState` and messages in a transaction are executed. Similarly to `CheckTx`, state transitions occur on a doubly branched state -- `finalizeBlockState`. Successful message execution results in writes being committed to `finalizeBlockState`. Note, if message execution fails, state transitions from @@ -324,7 +324,7 @@ to do the following checks: with the transaction is superior to a minimum reference gas amount based on the raw transaction size, in order to avoid spam with transactions that provide 0 gas. -`CheckTx` does **not** process `sdk.Msg`s - they only need to be processed when the canonical state need to be updated, which happens during `FinalizeBlock`. +`CheckTx` does **not** process `sdk.Msg`s - they only need to be processed when the canonical state needs to be updated, which happens during `FinalizeBlock`. Steps 2. and 3. are performed by the [`AnteHandler`](../beginner/04-gas-fees.md#antehandler) in the [`RunTx()`](#runtx-antehandler-and-runmsgs) function, which `CheckTx()` calls with the `runTxModeCheck` mode. During each step of `CheckTx()`, a @@ -410,9 +410,10 @@ First, it retrieves the `sdk.Msg`'s fully-qualified type name, by checking the ` ### PostHandler -`PostHandler` is similar to `AnteHandler`, but it, as the name suggests, executes custom post tx processing logic after [`RunMsgs`](#runmsgs) is called. `PostHandler` receives the `Result` of the the `RunMsgs` in order to enable this customizable behavior. +`PostHandler` is similar to `AnteHandler`, but it, as the name suggests, executes custom post tx processing logic after [`RunMsgs`](#runmsgs) is called. `PostHandler` receives the `Result` of the `RunMsgs` in order to enable this customizable behavior. + +Like `AnteHandler`s, `PostHandler`s are theoretically optional. -Like `AnteHandler`s, `PostHandler`s are theoretically optional, one use case for `PostHandler`s is transaction tips (enabled by default in simapp). Other use cases like unused gas refund can also be enabled by `PostHandler`s. ```go reference @@ -527,7 +528,7 @@ Each CometBFT `query` comes with a `path`, which is a `string` which denotes wha ### ExtendVote -`ExtendVote` allows an application to extend a pre-commit vote with arbitrary data. This process does NOT have be deterministic and the data returned can be unique to the validator process. +`ExtendVote` allows an application to extend a pre-commit vote with arbitrary data. This process does NOT have to be deterministic and the data returned can be unique to the validator process. In the Cosmos-SDK this is implemented as a NoOp: @@ -537,7 +538,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci_utils.go# ### VerifyVoteExtension -`VerifyVoteExtension` allows an application to verify that the data returned by `ExtendVote` is valid. This process does NOT have be deterministic and the data returned can be unique to the validator process. +`VerifyVoteExtension` allows an application to verify that the data returned by `ExtendVote` is valid. This process does NOT have to be deterministic and the data returned can be unique to the validator process. In the Cosmos-SDK this is implemented as a NoOp: diff --git a/docs/develop/advanced/01-transactions.md b/docs/learn/advanced/01-transactions.md similarity index 93% rename from docs/develop/advanced/01-transactions.md rename to docs/learn/advanced/01-transactions.md index eee4fae37..13d3727b6 100644 --- a/docs/develop/advanced/01-transactions.md +++ b/docs/learn/advanced/01-transactions.md @@ -91,7 +91,6 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/v1beta The use case is a multi-signer transaction, where one of the signers is appointed to gather all signatures, broadcast the signature and pay for fees, and the others only care about the transaction body. This generally allows for a better multi-signing UX. If Alice, Bob and Charlie are part of a 3-signer transaction, then Alice and Bob can both use `SIGN_MODE_DIRECT_AUX` to sign over the `TxBody` and their own signer info (no need an additional step to gather other signers' ones, like in `SIGN_MODE_DIRECT`), without specifying a fee in their SignDoc. Charlie can then gather both signatures from Alice and Bob, and create the final transaction by appending a fee. Note that the fee payer of the transaction (in our case Charlie) must sign over the fees, so must use `SIGN_MODE_DIRECT` or `SIGN_MODE_LEGACY_AMINO_JSON`. -A concrete use case is implemented in [transaction tips](./14-tips.md): the tipper may use `SIGN_MODE_DIRECT_AUX` to specify a tip in the transaction, without signing over the actual transaction fees. Then, the fee payer appends fees inside the tipper's desired `TxBody`, and as an exchange for paying the fees and broadcasting the transaction, receives the tipper's transaction tips as payment. #### `SIGN_MODE_TEXTUAL` @@ -99,7 +98,7 @@ A concrete use case is implemented in [transaction tips](./14-tips.md): the tipp #### Custom Sign modes -There is the the opportunity to add your own custom sign mode to the Cosmos-SDK. While we can not accept the implementation of the sign mode to the repository, we can accept a pull request to add the custom signmode to the SignMode enum located [here](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/signing/v1beta1/signing.proto#L17) +There is the opportunity to add your own custom sign mode to the Cosmos-SDK. While we can not accept the implementation of the sign mode to the repository, we can accept a pull request to add the custom signmode to the SignMode enum located [here](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/signing/v1beta1/signing.proto#L17) ## Transaction Process @@ -146,7 +145,7 @@ As there are currently two sign modes for signing transactions, there are also t * [wrapper](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/tx/builder.go#L26-L43) for creating transactions for `SIGN_MODE_DIRECT`, * [StdTxBuilder](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/migrations/legacytx/stdtx_builder.go#L14-L17) for `SIGN_MODE_LEGACY_AMINO_JSON`. -However, the two implementation of `TxBuilder` should be hidden away from end-users, as they should prefer using the overarching `TxConfig` interface: +However, the two implementations of `TxBuilder` should be hidden away from end-users, as they should prefer using the overarching `TxConfig` interface: ```go reference https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/client/tx_config.go#L24-L34 diff --git a/docs/develop/advanced/02-context.md b/docs/learn/advanced/02-context.md similarity index 99% rename from docs/develop/advanced/02-context.md rename to docs/learn/advanced/02-context.md index 4d9ecdf69..0056ec10c 100644 --- a/docs/develop/advanced/02-context.md +++ b/docs/learn/advanced/02-context.md @@ -62,7 +62,7 @@ explicitly pass a context `ctx` as the first argument of a process. ## Store branching -The `Context` contains a `MultiStore`, which allows for branchinig and caching functionality using `CacheMultiStore` +The `Context` contains a `MultiStore`, which allows for branching and caching functionality using `CacheMultiStore` (queries in `CacheMultiStore` are cached to avoid future round trips). Each `KVStore` is branched in a safe and isolated ephemeral storage. Processes are free to write changes to the `CacheMultiStore`. If a state-transition sequence is performed without issue, the store branch can diff --git a/docs/develop/advanced/03-node.md b/docs/learn/advanced/03-node.md similarity index 100% rename from docs/develop/advanced/03-node.md rename to docs/learn/advanced/03-node.md diff --git a/docs/develop/advanced/04-store.md b/docs/learn/advanced/04-store.md similarity index 98% rename from docs/develop/advanced/04-store.md rename to docs/learn/advanced/04-store.md index 35f676657..1419b54ae 100644 --- a/docs/develop/advanced/04-store.md +++ b/docs/learn/advanced/04-store.md @@ -168,7 +168,7 @@ The documentation on the IAVL Tree is located [here](https://github.com/cosmos/i ### `DbAdapter` Store -`dbadapter.Store` is a adapter for `dbm.DB` making it fulfilling the `KVStore` interface. +`dbadapter.Store` is an adapter for `dbm.DB` making it fulfilling the `KVStore` interface. ```go reference https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/dbadapter/store.go#L13-L16 @@ -269,7 +269,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/prefix/store.go# When `Store.{Get, Set}()` is called, the store forwards the call to its parent, with the key prefixed with the `Store.prefix`. -When `Store.Iterator()` is called, it does not simply prefix the `Store.prefix`, since it does not work as intended. In that case, some of the elements are traversed even they are not starting with the prefix. +When `Store.Iterator()` is called, it does not simply prefix the `Store.prefix`, since it does not work as intended. In that case, some of the elements are traversed even if they are not starting with the prefix. ### `ListenKv` Store diff --git a/versioned_docs/version-0.50/develop/advanced/05-encoding.md b/docs/learn/advanced/05-encoding.md similarity index 98% rename from versioned_docs/version-0.50/develop/advanced/05-encoding.md rename to docs/learn/advanced/05-encoding.md index f1aca5835..0178884b2 100644 --- a/versioned_docs/version-0.50/develop/advanced/05-encoding.md +++ b/docs/learn/advanced/05-encoding.md @@ -10,7 +10,7 @@ While encoding in the Cosmos SDK used to be mainly handled by `go-amino` codec, :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK application](../beginner/00-overview-app.md) +* [Anatomy of a Cosmos SDK application](../beginner/00-app-anatomy.md) ::: @@ -164,7 +164,7 @@ err := cdc.Unmarshal(profilebz, &myProfile) fmt.Printf("%T\n", myProfile.Account) // Prints "Any" fmt.Printf("%T\n", myProfile.Account.GetCachedValue()) // Prints "BaseAccount", "ContinuousVestingAccount" or whatever was initially packed in the Any. -// Get the address of the accountt. +// Get the address of the account. accAddr := myProfile.Account.GetCachedValue().(AccountI).GetAddress() ``` @@ -191,8 +191,8 @@ The above `Profile` example is a fictive example used for educational purposes. * the `cryptotypes.PubKey` interface for encoding different types of public keys, * the `sdk.Msg` interface for encoding different `Msg`s in a transaction, -* the `AccountI` interface for encodinig different types of accounts (similar to the above example) in the x/auth query responses, -* the `Evidencei` interface for encoding different types of evidences in the x/evidence module, +* the `AccountI` interface for encoding different types of accounts (similar to the above example) in the x/auth query responses, +* the `EvidenceI` interface for encoding different types of evidences in the x/evidence module, * the `AuthorizationI` interface for encoding different types of x/authz authorizations, * the [`Validator`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/types/staking.pb.go#L340-L377) struct that contains information about a validator. diff --git a/versioned_docs/version-0.50/develop/advanced/06-grpc_rest.md b/docs/learn/advanced/06-grpc_rest.md similarity index 96% rename from versioned_docs/version-0.50/develop/advanced/06-grpc_rest.md rename to docs/learn/advanced/06-grpc_rest.md index f9ef118a3..f20a76750 100644 --- a/versioned_docs/version-0.50/develop/advanced/06-grpc_rest.md +++ b/docs/learn/advanced/06-grpc_rest.md @@ -94,7 +94,7 @@ Some CometBFT RPC endpoints are directly related to the Cosmos SDK: * `/store/{storeName}/subspace`: this will directly query the named store for key/value pairs in which the key has the value of the `data` parameter as a prefix. * `/p2p/filter/addr/{port}`: this will return a filtered list of the node's P2P peers by address port. * `/p2p/filter/id/{id}`: this will return a filtered list of the node's P2P peers by ID. -* `/broadcast_tx_{aync,async,commit}`: these 3 endpoint will broadcast a transaction to other peers. CLI, gRPC and REST expose [a way to broadcast transations](./01-transactions.md#broadcasting-the-transaction), but they all use these 3 CometBFT RPCs under the hood. +* `/broadcast_tx_{aync,async,commit}`: these 3 endpoints will broadcast a transaction to other peers. CLI, gRPC and REST expose [a way to broadcast transactions](./01-transactions.md#broadcasting-the-transaction), but they all use these 3 CometBFT RPCs under the hood. ## Comparison Table diff --git a/docs/develop/advanced/07-cli.md b/docs/learn/advanced/07-cli.md similarity index 95% rename from docs/develop/advanced/07-cli.md rename to docs/learn/advanced/07-cli.md index 37f90efed..efc3b638c 100644 --- a/docs/develop/advanced/07-cli.md +++ b/docs/learn/advanced/07-cli.md @@ -172,7 +172,7 @@ Flags are added to commands directly (generally in the [module's CLI file](../.. ## Environment variables -Each flag is bound to it's respecteve named environment variable. Then name of the environment variable consist of two parts - capital case `basename` followed by flag name of the flag. `-` must be substituted with `_`. For example flag `--home` for application with basename `GAIA` is bound to `GAIA_HOME`. It allows reducing the amount of flags typed for routine operations. For example instead of: +Each flag is bound to its respective named environment variable. Then name of the environment variable consist of two parts - capital case `basename` followed by flag name of the flag. `-` must be substituted with `_`. For example flag `--home` for application with basename `GAIA` is bound to `GAIA_HOME`. It allows reducing the amount of flags typed for routine operations. For example instead of: ```shell gaia --home=./ --node= --chain-id="testchain-1" --keyring-backend=test tx ... --from= @@ -193,7 +193,7 @@ gaia tx ... --from= ## Configurations -It is vital that the root command of an application uses `PersistentPreRun()` cobra command property for executing the command, so all child commands have access to the server and client contexts. These contexts are set as their default values initially and maybe modified, scoped to the command, in their respective `PersistentPreRun()` functions. Note that the `client.Context` is typically pre-populated with "default" values that may be useful for all commands to inherit and override if necessary. +It is vital that the root command of an application uses `PersistentPreRun()` cobra command property for executing the command, so all child commands have access to the server and client contexts. These contexts are set as their default values initially and may be modified, scoped to the command, in their respective `PersistentPreRun()` functions. Note that the `client.Context` is typically pre-populated with "default" values that may be useful for all commands to inherit and override if necessary. Here is an example of an `PersistentPreRun()` function from `simapp`: @@ -206,7 +206,7 @@ The `SetCmdClientContextHandler` call reads persistent flags via `ReadPersistent The `InterceptConfigsPreRunHandler` call creates a viper literal, default `server.Context`, and a logger and sets that on the root command's `Context`. The `server.Context` will be modified and saved to disk. The internal `interceptConfigs` call reads or creates a CometBFT configuration based on the home path provided. In addition, `interceptConfigs` also reads and loads the application configuration, `app.toml`, and binds that to the `server.Context` viper literal. This is vital so the application can get access to not only the CLI flags, but also to the application configuration values provided by this file. :::tip -When willing to configure which logger is used, do not to use `InterceptConfigsPreRunHandler`, which sets the default SDK logger, but instead use `InterceptConfigsAndCreateContext` and set the server context and the logger manually: +When willing to configure which logger is used, do not use `InterceptConfigsPreRunHandler`, which sets the default SDK logger, but instead use `InterceptConfigsAndCreateContext` and set the server context and the logger manually: ```diff -return server.InterceptConfigsPreRunHandler(cmd, customAppTemplate, customAppConfig, customCMTConfig) diff --git a/docs/develop/advanced/08-events.md b/docs/learn/advanced/08-events.md similarity index 100% rename from docs/develop/advanced/08-events.md rename to docs/learn/advanced/08-events.md diff --git a/versioned_docs/version-0.50/develop/advanced/09-telemetry.md b/docs/learn/advanced/09-telemetry.md similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/09-telemetry.md rename to docs/learn/advanced/09-telemetry.md diff --git a/docs/develop/advanced/10-ocap.md b/docs/learn/advanced/10-ocap.md similarity index 100% rename from docs/develop/advanced/10-ocap.md rename to docs/learn/advanced/10-ocap.md diff --git a/versioned_docs/version-0.50/develop/advanced/11-runtx_middleware.md b/docs/learn/advanced/11-runtx_middleware.md similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/11-runtx_middleware.md rename to docs/learn/advanced/11-runtx_middleware.md diff --git a/versioned_docs/version-0.50/develop/advanced/12-simulation.md b/docs/learn/advanced/12-simulation.md similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/12-simulation.md rename to docs/learn/advanced/12-simulation.md diff --git a/versioned_docs/version-0.50/develop/advanced/13-proto-docs.md b/docs/learn/advanced/13-proto-docs.md similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/13-proto-docs.md rename to docs/learn/advanced/13-proto-docs.md diff --git a/versioned_docs/version-0.50/develop/advanced/15-upgrade.md b/docs/learn/advanced/15-upgrade.md similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/15-upgrade.md rename to docs/learn/advanced/15-upgrade.md diff --git a/versioned_docs/version-0.50/develop/advanced/16-config.md b/docs/learn/advanced/16-config.md similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/16-config.md rename to docs/learn/advanced/16-config.md diff --git a/versioned_docs/version-0.50/develop/advanced/17-autocli.md b/docs/learn/advanced/17-autocli.md similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/17-autocli.md rename to docs/learn/advanced/17-autocli.md diff --git a/docs/develop/advanced/_category_.json b/docs/learn/advanced/_category_.json similarity index 100% rename from docs/develop/advanced/_category_.json rename to docs/learn/advanced/_category_.json diff --git a/docs/develop/advanced/baseapp_state-begin_block.png b/docs/learn/advanced/baseapp_state-begin_block.png similarity index 100% rename from docs/develop/advanced/baseapp_state-begin_block.png rename to docs/learn/advanced/baseapp_state-begin_block.png diff --git a/docs/develop/advanced/baseapp_state-checktx.png b/docs/learn/advanced/baseapp_state-checktx.png similarity index 100% rename from docs/develop/advanced/baseapp_state-checktx.png rename to docs/learn/advanced/baseapp_state-checktx.png diff --git a/docs/develop/advanced/baseapp_state-commit.png b/docs/learn/advanced/baseapp_state-commit.png similarity index 100% rename from docs/develop/advanced/baseapp_state-commit.png rename to docs/learn/advanced/baseapp_state-commit.png diff --git a/docs/develop/advanced/baseapp_state-deliver_tx.png b/docs/learn/advanced/baseapp_state-deliver_tx.png similarity index 100% rename from docs/develop/advanced/baseapp_state-deliver_tx.png rename to docs/learn/advanced/baseapp_state-deliver_tx.png diff --git a/docs/develop/advanced/baseapp_state-initchain.png b/docs/learn/advanced/baseapp_state-initchain.png similarity index 100% rename from docs/develop/advanced/baseapp_state-initchain.png rename to docs/learn/advanced/baseapp_state-initchain.png diff --git a/docs/develop/advanced/baseapp_state-prepareproposal.png b/docs/learn/advanced/baseapp_state-prepareproposal.png similarity index 100% rename from docs/develop/advanced/baseapp_state-prepareproposal.png rename to docs/learn/advanced/baseapp_state-prepareproposal.png diff --git a/docs/develop/advanced/baseapp_state-processproposal.png b/docs/learn/advanced/baseapp_state-processproposal.png similarity index 100% rename from docs/develop/advanced/baseapp_state-processproposal.png rename to docs/learn/advanced/baseapp_state-processproposal.png diff --git a/docs/develop/advanced/baseapp_state.png b/docs/learn/advanced/baseapp_state.png similarity index 100% rename from docs/develop/advanced/baseapp_state.png rename to docs/learn/advanced/baseapp_state.png diff --git a/docs/develop/beginner/00-overview-app.md b/docs/learn/beginner/00-app-anatomy.md similarity index 87% rename from docs/develop/beginner/00-overview-app.md rename to docs/learn/beginner/00-app-anatomy.md index d0414233d..d128228ae 100644 --- a/docs/develop/beginner/00-overview-app.md +++ b/docs/learn/beginner/00-app-anatomy.md @@ -1,9 +1,8 @@ --- sidebar_position: 1 - --- -# Overview of a Cosmos SDK Application +# Anatomy of a Cosmos SDK Application :::note Synopsis This document describes the core parts of a Cosmos SDK application, represented throughout the document as a placeholder application named `app`. @@ -54,14 +53,14 @@ The first thing defined in `app.go` is the `type` of the application. It is gene * **A reference to [`baseapp`](../advanced/00-baseapp.md).** The custom application defined in `app.go` is an extension of `baseapp`. When a transaction is relayed by CometBFT to the application, `app` uses `baseapp`'s methods to route them to the appropriate module. `baseapp` implements most of the core logic for the application, including all the [ABCI methods](https://docs.cometbft.com/v0.37/spec/abci/) and the [routing logic](../advanced/00-baseapp.md#routing). * **A list of store keys**. The [store](../advanced/04-store.md), which contains the entire state, is implemented as a [`multistore`](../advanced/04-store.md#multistore) (i.e. a store of stores) in the Cosmos SDK. Each module uses one or multiple stores in the multistore to persist their part of the state. These stores can be accessed with specific keys that are declared in the `app` type. These keys, along with the `keepers`, are at the heart of the [object-capabilities model](../advanced/10-ocap.md) of the Cosmos SDK. * **A list of module's `keeper`s.** Each module defines an abstraction called [`keeper`](../../build/building-modules/06-keeper.md), which handles reads and writes for this module's store(s). The `keeper`'s methods of one module can be called from other modules (if authorized), which is why they are declared in the application's type and exported as interfaces to other modules so that the latter can only access the authorized functions. -* **A reference to an [`appCodec`](../advanced/06-encoding.md).** The application's `appCodec` is used to serialize and deserialize data structures in order to store them, as stores can only persist `[]bytes`. The default codec is [Protocol Buffers](../advanced/06-encoding.md). -* **A reference to a [`legacyAmino`](../advanced/06-encoding.md) codec.** Some parts of the Cosmos SDK have not been migrated to use the `appCodec` above, and are still hardcoded to use Amino. Other parts explicitly use Amino for backwards compatibility. For these reasons, the application still holds a reference to the legacy Amino codec. Please note that the Amino codec will be removed from the SDK in the upcoming releases. -* **A reference to a [module manager](../../build/building-modules/01-module-manager.md#manager)** and a [basic module manager](../../build/building-modules/01-module-manager.md#basicmanager). The module manager is an object that contains a list of the application's modules. It facilitates operations related to these modules, like registering their [`Msg` service](../advanced/00-baseapp.md#msg-services) and [gRPC `Query` service](../advanced/00-baseapp.md#grpc-query-services), or setting the order of execution between modules for various functions like [`InitChainer`](#initchainer), [`BeginBlocker` and `EndBlocker`](#beginblocker-and-endblocker). +* **A reference to an [`appCodec`](../advanced/05-encoding.md).** The application's `appCodec` is used to serialize and deserialize data structures in order to store them, as stores can only persist `[]bytes`. The default codec is [Protocol Buffers](../advanced/05-encoding.md). +* **A reference to a [`legacyAmino`](../advanced/05-encoding.md) codec.** Some parts of the Cosmos SDK have not been migrated to use the `appCodec` above, and are still hardcoded to use Amino. Other parts explicitly use Amino for backwards compatibility. For these reasons, the application still holds a reference to the legacy Amino codec. Please note that the Amino codec will be removed from the SDK in the upcoming releases. +* **A reference to a [module manager](../../build/building-modules/01-module-manager.md#manager)** and a [basic module manager](../../build/building-modules/01-module-manager.md#basicmanager). The module manager is an object that contains a list of the application's modules. It facilitates operations related to these modules, like registering their [`Msg` service](../advanced/00-baseapp.md#msg-services) and [gRPC `Query` service](../advanced/00-baseapp.md#grpc-query-services), or setting the order of execution between modules for various functions like [`InitChainer`](#initchainer), [`PreBlocker`](#preblocker) and [`BeginBlocker` and `EndBlocker`](#beginblocker-and-endblocker). See an example of application type definition from `simapp`, the Cosmos SDK's own app used for demo and testing purposes: ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/app.go#L161-L203 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go#L173-L212 ``` ### Constructor Function @@ -69,21 +68,22 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/app.go#L161-L203 Also defined in `app.go` is the constructor function, which constructs a new application of the type defined in the preceding section. The function must fulfill the `AppCreator` signature in order to be used in the [`start` command](../advanced/03-node.md#start-command) of the application's daemon command. ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/server/types/app.go#L64-L66 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/server/types/app.go#L66-L68 ``` Here are the main actions performed by this function: -* Instantiate a new [`codec`](../advanced/06-encoding.md) and initialize the `codec` of each of the application's modules using the [basic manager](../../build/building-modules/01-module-manager.md#basicmanager). +* Instantiate a new [`codec`](../advanced/05-encoding.md) and initialize the `codec` of each of the application's modules using the [basic manager](../../build/building-modules/01-module-manager.md#basicmanager). * Instantiate a new application with a reference to a `baseapp` instance, a codec, and all the appropriate store keys. * Instantiate all the [`keeper`](#keeper) objects defined in the application's `type` using the `NewKeeper` function of each of the application's modules. Note that keepers must be instantiated in the correct order, as the `NewKeeper` of one module might require a reference to another module's `keeper`. * Instantiate the application's [module manager](../../build/building-modules/01-module-manager.md#manager) with the [`AppModule`](#application-module-interface) object of each of the application's modules. * With the module manager, initialize the application's [`Msg` services](../advanced/00-baseapp.md#msg-services), [gRPC `Query` services](../advanced/00-baseapp.md#grpc-query-services), [legacy `Msg` routes](../advanced/00-baseapp.md#routing), and [legacy query routes](../advanced/00-baseapp.md#query-routing). When a transaction is relayed to the application by CometBFT via the ABCI, it is routed to the appropriate module's [`Msg` service](#msg-services) using the routes defined here. Likewise, when a gRPC query request is received by the application, it is routed to the appropriate module's [`gRPC query service`](#grpc-query-services) using the gRPC routes defined here. The Cosmos SDK still supports legacy `Msg`s and legacy CometBFT queries, which are routed using the legacy `Msg` routes and the legacy query routes, respectively. * With the module manager, register the [application's modules' invariants](../../build/building-modules/07-invariants.md). Invariants are variables (e.g. total supply of a token) that are evaluated at the end of each block. The process of checking invariants is done via a special module called the [`InvariantsRegistry`](../../build/building-modules/07-invariants.md#invariant-registry). The value of the invariant should be equal to a predicted value defined in the module. Should the value be different than the predicted one, special logic defined in the invariant registry is triggered (usually the chain is halted). This is useful to make sure that no critical bug goes unnoticed, producing long-lasting effects that are hard to fix. -* With the module manager, set the order of execution between the `InitGenesis`, `BeginBlocker`, and `EndBlocker` functions of each of the [application's modules](#application-module-interface). Note that not all modules implement these functions. +* With the module manager, set the order of execution between the `InitGenesis`, `PreBlocker`, `BeginBlocker`, and `EndBlocker` functions of each of the [application's modules](#application-module-interface). Note that not all modules implement these functions. * Set the remaining application parameters: * [`InitChainer`](#initchainer): used to initialize the application when it is first started. - * [`BeginBlocker`, `EndBlocker`](#beginblocker-and-endlbocker): called at the beginning and at the end of every block. + * [`PreBlocker`](#preblocker): called before BeginBlock. + * [`BeginBlocker`, `EndBlocker`](#beginblocker-and-endblocker): called at the beginning and at the end of every block. * [`anteHandler`](../advanced/00-baseapp.md#antehandler): used to handle fees and signature verification. * Mount the stores. * Return the application. @@ -93,33 +93,46 @@ Note that the constructor function only creates an instance of the app, while th See an example of application constructor from `simapp`: ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/app.go#L214-L522 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go#L223-L575 ``` ### InitChainer The `InitChainer` is a function that initializes the state of the application from a genesis file (i.e. token balances of genesis accounts). It is called when the application receives the `InitChain` message from the CometBFT engine, which happens when the node is started at `appBlockHeight == 0` (i.e. on genesis). The application must set the `InitChainer` in its [constructor](#constructor-function) via the [`SetInitChainer`](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/baseapp#BaseApp.SetInitChainer) method. -In general, the `InitChainer` is mostly composed of the [`InitGenesis`](../../build/building-modules/08-genesis.md#initgenesis) function of each of the application's modules. This is done by calling the `InitGenesis` function of the module manager, which in turn calls the `InitGenesis` function of each of the modules it contains. Note that the order in which the modules' `InitGenesis` functions must be called has to be set in the module manager using the [module manager's](../../build/building-modules/01-module-manager.md) `SetOrderInitGenesis` method. This is done in the [application's constructor](#constructor-function), and the `SetOrderInitGenesis` has to be called before the `SetInitChainer`. +In general, the `InitChainer` is mostly composed of the [`InitGenesis`](../../build/building-modules/08-genesis.md#initgenesis) function of each of the application's modules. This is done by calling the `InitGenesis` function of the module manager, which in turn calls the `InitGenesis` function of each of the modules it contains. Note that the order in which the modules' `InitGenesis` functions must be called has to be set in the module manager using the [module manager's](../../build/building-modules/01-module-manager.md) `SetOrderInitGenesis` method. This is done in the [application's constructor](#application-constructor), and the `SetOrderInitGenesis` has to be called before the `SetInitChainer`. See an example of an `InitChainer` from `simapp`: ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/app.go#L569-L577 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go#L626-L634 +``` + +### PreBlocker + +There are two semantics around the new lifecycle method: + +- It runs before the `BeginBlocker` of all modules +- It can modify consensus parameters in storage, and signal the caller through the return value. + +When it returns `ConsensusParamsChanged=true`, the caller must refresh the consensus parameter in the finalize context: +``` +app.finalizeBlockState.ctx = app.finalizeBlockState.ctx.WithConsensusParams(app.GetConsensusParams()) ``` +The new ctx must be passed to all the other lifecycle methods. ### BeginBlocker and EndBlocker -The Cosmos SDK offers developers the possibility to implement automatic execution of code as part of their application. This is implemented through two functions called `BeginBlocker` and `EndBlocker`. They are called when the application receives the `BeginBlock` and `EndBlock` messages from the CometBFT engine, which happens respectively at the beginning and at the end of each block. The application must set the `BeginBlocker` and `EndBlocker` in its [constructor](#constructor-function) via the [`SetBeginBlocker`](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/baseapp#BaseApp.SetBeginBlocker) and [`SetEndBlocker`](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/baseapp#BaseApp.SetEndBlocker) methods. +The Cosmos SDK offers developers the possibility to implement automatic execution of code as part of their application. This is implemented through two functions called `BeginBlocker` and `EndBlocker`. They are called when the application receives the `FinalizeBlock` messages from the CometBFT consensus engine, which happens respectively at the beginning and at the end of each block. The application must set the `BeginBlocker` and `EndBlocker` in its [constructor](#constructor-function) via the [`SetBeginBlocker`](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/baseapp#BaseApp.SetBeginBlocker) and [`SetEndBlocker`](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/baseapp#BaseApp.SetEndBlocker) methods. -In general, the `BeginBlocker` and `EndBlocker` functions are mostly composed of the [`BeginBlock` and `EndBlock`](../../build/building-modules/05-beginblock-endblock.md) functions of each of the application's modules. This is done by calling the `BeginBlock` and `EndBlock` functions of the module manager, which in turn calls the `BeginBlock` and `EndBlock` functions of each of the modules it contains. Note that the order in which the modules' `BeginBlock` and `EndBlock` functions must be called has to be set in the module manager using the `SetOrderBeginBlockers` and `SetOrderEndBlockers` methods, respectively. This is done via the [module manager](../../build/building-modules/01-module-manager.md) in the [application's constructor](#constructor-function), and the `SetOrderBeginBlockers` and `SetOrderEndBlockers` methods have to be called before the `SetBeginBlocker` and `SetEndBlocker` functions. +In general, the `BeginBlocker` and `EndBlocker` functions are mostly composed of the [`BeginBlock` and `EndBlock`](../../build/building-modules/06-beginblock-endblock.md) functions of each of the application's modules. This is done by calling the `BeginBlock` and `EndBlock` functions of the module manager, which in turn calls the `BeginBlock` and `EndBlock` functions of each of the modules it contains. Note that the order in which the modules' `BeginBlock` and `EndBlock` functions must be called has to be set in the module manager using the `SetOrderBeginBlockers` and `SetOrderEndBlockers` methods, respectively. This is done via the [module manager](../../build/building-modules/01-module-manager.md) in the [application's constructor](#application-constructor), and the `SetOrderBeginBlockers` and `SetOrderEndBlockers` methods have to be called before the `SetBeginBlocker` and `SetEndBlocker` functions. -As a sidenote, it is important to remember that application-specific blockchains are deterministic. Developers must be careful not to introduce non-determinism in `BeginBlocker` or `EndBlocker`, and must also be careful not to make them too computationally expensive, as [gas](04-gas-fees.md) does not constrain the cost of `BeginBlocker` and `EndBlocker` execution. +As a sidenote, it is important to remember that application-specific blockchains are deterministic. Developers must be careful not to introduce non-determinism in `BeginBlocker` or `EndBlocker`, and must also be careful not to make them too computationally expensive, as [gas](./04-gas-fees.md) does not constrain the cost of `BeginBlocker` and `EndBlocker` execution. See an example of `BeginBlocker` and `EndBlocker` functions from `simapp` ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/app.go#L555-L563 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go#L613-L620 ``` ### Register Codec @@ -127,7 +140,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/app.go#L555-L563 The `EncodingConfig` structure is the last important part of the `app.go` file. The goal of this structure is to define the codecs that will be used throughout the app. ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/params/encoding.go#L9-L16 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/params/encoding.go#L9-L16 ``` Here are descriptions of what each of the four fields means: @@ -143,7 +156,7 @@ An application should create its own encoding config. See an example of a `simappparams.EncodingConfig` from `simapp`: ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/app.go#L731-L738 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/params/encoding.go#L11-L16 ``` ## Modules @@ -165,16 +178,16 @@ Note that `sdk.Msg`s are bundled in [transactions](../advanced/01-transactions.m When a valid block of transactions is received by the full-node, CometBFT relays each one to the application via [`DeliverTx`](https://docs.cometbft.com/v0.37/spec/abci/abci++_app_requirements#specifics-of-responsedelivertx). Then, the application handles the transaction: 1. Upon receiving the transaction, the application first unmarshalls it from `[]byte`. -2. Then, it verifies a few things about the transaction like [fee payment and signatures](04-gas-fees.md#antehandler) before extracting the `Msg`(s) contained in the transaction. +2. Then, it verifies a few things about the transaction like [fee payment and signatures](./04-gas-fees.md#antehandler) before extracting the `Msg`(s) contained in the transaction. 3. `sdk.Msg`s are encoded using Protobuf [`Any`s](#register-codec). By analyzing each `Any`'s `type_url`, baseapp's `msgServiceRouter` routes the `sdk.Msg` to the corresponding module's `Msg` service. 4. If the message is successfully processed, the state is updated. -For more details, see [transaction lifecycle](01-tx-lifecycle.md). +For more details, see [transaction lifecycle](./01-tx-lifecycle.md). Module developers create custom `Msg` services when they build their own module. The general practice is to define the `Msg` Protobuf service in a `tx.proto` file. For example, the `x/bank` module defines a service with two methods to transfer tokens: ```protobuf reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L13-L36 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/bank/v1beta1/tx.proto#L13-L36 ``` Service methods use `keeper` in order to update the module state. @@ -183,7 +196,7 @@ Each module should also implement the `RegisterServices` method as part of the [ ### gRPC `Query` Services -gRPC `Query` services allow users to query the state using [gRPC](https://grpc.io). They are enabled by default, and can be configured under the `grpc.enable` and `grpc.address` fields inside [`app.toml`](../../user/run-node/02-interact-node.md#configuring-the-node-using-apptoml). +gRPC `Query` services allow users to query the state using [gRPC](https://grpc.io). They are enabled by default, and can be configured under the `grpc.enable` and `grpc.address` fields inside [`app.toml`](../../user/run-node/01-run-node.md#configuring-the-node-using-apptoml-and-configtoml). gRPC `Query` services are defined in the module's Protobuf definition files, specifically inside `query.proto`. The `query.proto` definition file exposes a single `Query` [Protobuf service](https://developers.google.com/protocol-buffers/docs/proto#services). Each gRPC query endpoint corresponds to a service method, starting with the `rpc` keyword, inside the `Query` service. @@ -207,14 +220,14 @@ Along with the type definition, the next important component of the `keeper.go` ### Command-Line, gRPC Services and REST Interfaces -Each module defines command-line commands, gRPC services, and REST routes to be exposed to the end-user via the [application's interfaces](#application-interfacev). This enables end-users to create messages of the types defined in the module, or to query the subset of the state managed by the module. +Each module defines command-line commands, gRPC services, and REST routes to be exposed to the end-user via the [application's interfaces](#application-interfaces). This enables end-users to create messages of the types defined in the module, or to query the subset of the state managed by the module. #### CLI Generally, the [commands related to a module](../../build/building-modules/09-module-interfaces.md#cli) are defined in a folder called `client/cli` in the module's folder. The CLI divides commands into two categories, transactions and queries, defined in `client/cli/tx.go` and `client/cli/query.go`, respectively. Both commands are built on top of the [Cobra Library](https://github.com/spf13/cobra): -* Transactions commands let users generate new transactions so that they can be included in a block and eventually update the state. One command should be created for each defined in the module. The command calls the constructor of the message with the parameters provided by the end-user, and wraps it into a transaction. The Cosmos SDK handles signing and the addition of other transaction metadata. -* Queries let users query the subset of the state defined by the module. Query commands forward queries to the [application's query router](../advanced/00-baseapp.md#query-routing), which routes them to the appropriate the `queryRoute` parameter supplied. +* Transactions commands let users generate new transactions so that they can be included in a block and eventually update the state. One command should be created for each [message type](#message-types) defined in the module. The command calls the constructor of the message with the parameters provided by the end-user, and wraps it into a transaction. The Cosmos SDK handles signing and the addition of other transaction metadata. +* Queries let users query the subset of the state defined by the module. Query commands forward queries to the [application's query router](../advanced/00-baseapp.md#query-routing), which routes them to the appropriate [querier](#querier) the `queryRoute` parameter supplied. #### gRPC @@ -231,7 +244,7 @@ Some external clients may not wish to use gRPC. In this case, the Cosmos SDK pro The REST endpoints are defined in the Protobuf files, along with the gRPC services, using Protobuf annotations. Modules that want to expose REST queries should add `google.api.http` annotations to their `rpc` methods. By default, all REST endpoints defined in the SDK have a URL starting with the `/cosmos/` prefix. -The Cosmos SDK also provides a development endpoint to generate [Swagger](https://swagger.io/) definition files for these REST endpoints. This endpoint can be enabled inside the [`app.toml`](../../user/run-node/01-run-node.md#configuring-the-node-using-apptoml) config file, under the `api.swagger` key. +The Cosmos SDK also provides a development endpoint to generate [Swagger](https://swagger.io/) definition files for these REST endpoints. This endpoint can be enabled inside the [`app.toml`](../../user/run-node/01-run-node.md#configuring-the-node-using-apptoml-and-configtoml) config file, under the `api.swagger` key. ## Application Interface diff --git a/docs/develop/beginner/01-tx-lifecycle.md b/docs/learn/beginner/01-tx-lifecycle.md similarity index 100% rename from docs/develop/beginner/01-tx-lifecycle.md rename to docs/learn/beginner/01-tx-lifecycle.md diff --git a/docs/develop/beginner/02-query-lifecycle.md b/docs/learn/beginner/02-query-lifecycle.md similarity index 100% rename from docs/develop/beginner/02-query-lifecycle.md rename to docs/learn/beginner/02-query-lifecycle.md diff --git a/docs/develop/beginner/03-accounts.md b/docs/learn/beginner/03-accounts.md similarity index 100% rename from docs/develop/beginner/03-accounts.md rename to docs/learn/beginner/03-accounts.md diff --git a/docs/develop/beginner/04-gas-fees.md b/docs/learn/beginner/04-gas-fees.md similarity index 100% rename from docs/develop/beginner/04-gas-fees.md rename to docs/learn/beginner/04-gas-fees.md diff --git a/docs/develop/beginner/_category_.json b/docs/learn/beginner/_category_.json similarity index 100% rename from docs/develop/beginner/_category_.json rename to docs/learn/beginner/_category_.json diff --git a/versioned_docs/version-0.47/learn/intro/00-what-is-sdk.md b/docs/learn/intro/00-overview.md similarity index 90% rename from versioned_docs/version-0.47/learn/intro/00-what-is-sdk.md rename to docs/learn/intro/00-overview.md index 3c77e7953..05bc63d83 100644 --- a/versioned_docs/version-0.47/learn/intro/00-what-is-sdk.md +++ b/docs/learn/intro/00-overview.md @@ -1,9 +1,10 @@ --- -sidebar_position: 0 - +sidebar_position: 1 --- -# What is the Cosmos SDK +# High-level Overview + +## What is the Cosmos SDK The [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is an open-source framework for building multi-asset public Proof-of-Stake (PoS) blockchains, like the Cosmos Hub, as well as permissioned Proof-of-Authority (PoA) blockchains. Blockchains built with the Cosmos SDK are generally referred to as **application-specific blockchains**. @@ -15,18 +16,18 @@ One development paradigm in the blockchain world today is that of virtual-machin Application-specific blockchains offer a radically different development paradigm than virtual-machine blockchains. An application-specific blockchain is a blockchain customized to operate a single application: developers have all the freedom to make the design decisions required for the application to run optimally. They can also provide better sovereignty, security and performance. -Learn more about [application-specific blockchains](01-why-app-specific.md). +Learn more about [application-specific blockchains](./01-why-app-specific.md). ## Why the Cosmos SDK The Cosmos SDK is the most advanced framework for building custom application-specific blockchains today. Here are a few reasons why you might want to consider building your decentralized application with the Cosmos SDK: * The default consensus engine available within the Cosmos SDK is [CometBFT](https://github.com/cometbft/cometbft). CometBFT is the most (and only) mature BFT consensus engine in existence. It is widely used across the industry and is considered the gold standard consensus engine for building Proof-of-Stake systems. -* The Cosmos SDK is open-source and designed to make it easy to build blockchains out of composable [modules](../../build/modules). As the ecosystem of open-source Cosmos SDK modules grows, it will become increasingly easier to build complex decentralized platforms with it. +* The Cosmos SDK is open-source and designed to make it easy to build blockchains out of composable [modules](../modules). As the ecosystem of open-source Cosmos SDK modules grows, it will become increasingly easier to build complex decentralized platforms with it. * The Cosmos SDK is inspired by capabilities-based security, and informed by years of wrestling with blockchain state-machines. This makes the Cosmos SDK a very secure environment to build blockchains. * Most importantly, the Cosmos SDK has already been used to build many application-specific blockchains that are already in production. Among others, we can cite [Cosmos Hub](https://hub.cosmos.network), [IRIS Hub](https://irisnet.org), [Binance Chain](https://docs.binance.org/), [Terra](https://terra.money/) or [Kava](https://www.kava.io/). [Many more](https://cosmos.network/ecosystem) are building on the Cosmos SDK. ## Getting started with the Cosmos SDK -* Learn more about the [architecture of a Cosmos SDK application](02-sdk-app-architecture.md) +* Learn more about the [architecture of a Cosmos SDK application](./02-sdk-app-architecture.md) * Learn how to build an application-specific blockchain from scratch with the [Cosmos SDK Tutorial](https://cosmos.network/docs/tutorial) diff --git a/docs/develop/intro/01-why-app-specific.md b/docs/learn/intro/01-why-app-specific.md similarity index 100% rename from docs/develop/intro/01-why-app-specific.md rename to docs/learn/intro/01-why-app-specific.md diff --git a/docs/develop/intro/02-sdk-app-architecture.md b/docs/learn/intro/02-sdk-app-architecture.md similarity index 100% rename from docs/develop/intro/02-sdk-app-architecture.md rename to docs/learn/intro/02-sdk-app-architecture.md diff --git a/docs/develop/intro/03-sdk-design.md b/docs/learn/intro/03-sdk-design.md similarity index 100% rename from docs/develop/intro/03-sdk-design.md rename to docs/learn/intro/03-sdk-design.md diff --git a/docs/develop/intro/_category_.json b/docs/learn/intro/_category_.json similarity index 100% rename from docs/develop/intro/_category_.json rename to docs/learn/intro/_category_.json diff --git a/docs/user/run-node/00-keyring.md b/docs/user/run-node/00-keyring.md index 09d8b0742..0ed06d412 100644 --- a/docs/user/run-node/00-keyring.md +++ b/docs/user/run-node/00-keyring.md @@ -5,7 +5,7 @@ sidebar_position: 1 # Setting up the keyring :::note Synopsis -This document describes how to configure and use the keyring and its various backends for an [**application**](../../develop/beginner/00-app-anatomy.md). +This document describes how to configure and use the keyring and its various backends for an [**application**](../../learn/beginner/00-app-anatomy.md). ::: The keyring holds the private/public keypairs used to interact with a node. For instance, a validator key needs to be set up before running the blockchain node, so that blocks can be correctly signed. The private key can be stored in different locations, called "backends", such as a file or the operating system's own key storage. diff --git a/docs/user/run-node/01-run-node.md b/docs/user/run-node/01-run-node.md index fc539989b..7bd1a9c79 100644 --- a/docs/user/run-node/01-run-node.md +++ b/docs/user/run-node/01-run-node.md @@ -10,7 +10,7 @@ Now that the application is ready and the keyring populated, it's time to see ho :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK Application](../../develop/beginner/00-app-anatomy.md) +* [Anatomy of a Cosmos SDK Application](../../learn/beginner/00-app-anatomy.md) * [Setting up the keyring](./00-keyring.md) ::: @@ -87,7 +87,7 @@ simd genesis add-genesis-account $MY_VALIDATOR_ADDRESS 100000000000stake Recall that `$MY_VALIDATOR_ADDRESS` is a variable that holds the address of the `my_validator` key in the [keyring](./00-keyring.md#adding-keys-to-the-keyring). Also note that the tokens in the Cosmos SDK have the `{amount}{denom}` format: `amount` is is a 18-digit-precision decimal number, and `denom` is the unique token identifier with its denomination key (e.g. `atom` or `uatom`). Here, we are granting `stake` tokens, as `stake` is the token identifier used for staking in [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/main/simapp). For your own chain with its own staking denom, that token identifier should be used instead. -Now that your account has some tokens, you need to add a validator to your chain. Validators are special full-nodes that participate in the consensus process (implemented in the [underlying consensus engine](../../develop/intro/02-sdk-app-architecture.md#cometbft)) in order to add new blocks to the chain. Any account can declare its intention to become a validator operator, but only those with sufficient delegation get to enter the active set (for example, only the top 125 validator candidates with the most delegation get to be validators in the Cosmos Hub). For this guide, you will add your local node (created via the `init` command above) as a validator of your chain. Validators can be declared before a chain is first started via a special transaction included in the genesis file called a `gentx`: +Now that your account has some tokens, you need to add a validator to your chain. Validators are special full-nodes that participate in the consensus process (implemented in the [underlying consensus engine](../../learn/intro/02-sdk-app-architecture.md#cometbft)) in order to add new blocks to the chain. Any account can declare its intention to become a validator operator, but only those with sufficient delegation get to enter the active set (for example, only the top 125 validator candidates with the most delegation get to be validators in the Cosmos Hub). For this guide, you will add your local node (created via the `init` command above) as a validator of your chain. Validators can be declared before a chain is first started via a special transaction included in the genesis file called a `gentx`: ```bash # Create a gentx. diff --git a/docs/user/run-node/02-interact-node.md b/docs/user/run-node/02-interact-node.md index 48193504a..a511aec41 100644 --- a/docs/user/run-node/02-interact-node.md +++ b/docs/user/run-node/02-interact-node.md @@ -10,7 +10,7 @@ There are multiple ways to interact with a node: using the CLI, using gRPC or us :::note Pre-requisite Readings -* [gRPC, REST and CometBFT Endpoints](../../develop/advanced/06-grpc_rest.md) +* [gRPC, REST and CometBFT Endpoints](../../learn/advanced/06-grpc_rest.md) * [Running a Node](./01-run-node.md) ::: @@ -54,7 +54,7 @@ You should see two delegations, the first one made from the `gentx`, and the sec ## Using gRPC -The Protobuf ecosystem developed tools for different use cases, including code-generation from `*.proto` files into various languages. These tools allow the building of clients easily. Often, the client connection (i.e. the transport) can be plugged and replaced very easily. Let's explore one of the most popular transport: [gRPC](../../develop/advanced/06-grpc_rest.md). +The Protobuf ecosystem developed tools for different use cases, including code-generation from `*.proto` files into various languages. These tools allow the building of clients easily. Often, the client connection (i.e. the transport) can be plugged and replaced very easily. Let's explore one of the most popular transport: [gRPC](../../learn/advanced/06-grpc_rest.md). Since the code generation library largely depends on your own tech stack, we will only present three alternatives: @@ -248,7 +248,7 @@ CosmJS documentation can be found at [https://cosmos.github.io/cosmjs](https://c ## Using the REST Endpoints -As described in the [gRPC guide](../../develop/advanced/06-grpc_rest.md), all gRPC services on the Cosmos SDK are made available for more convenient REST-based queries through gRPC-gateway. The format of the URL path is based on the Protobuf service method's full-qualified name, but may contain small customizations so that final URLs look more idiomatic. For example, the REST endpoint for the `cosmos.bank.v1beta1.Query/AllBalances` method is `GET /cosmos/bank/v1beta1/balances/{address}`. Request arguments are passed as query parameters. +As described in the [gRPC guide](../../learn/advanced/06-grpc_rest.md), all gRPC services on the Cosmos SDK are made available for more convenient REST-based queries through gRPC-gateway. The format of the URL path is based on the Protobuf service method's full-qualified name, but may contain small customizations so that final URLs look more idiomatic. For example, the REST endpoint for the `cosmos.bank.v1beta1.Query/AllBalances` method is `GET /cosmos/bank/v1beta1/balances/{address}`. Request arguments are passed as query parameters. Note that the REST endpoints are not enabled by default. To enable them, edit the `api` section of your `~/.simapp/config/app.toml` file: diff --git a/docs/user/run-node/03-txs.md b/docs/user/run-node/03-txs.md index faf1d8b65..106f02e8e 100644 --- a/docs/user/run-node/03-txs.md +++ b/docs/user/run-node/03-txs.md @@ -269,7 +269,7 @@ func sendTx() error { ### Broadcasting a Transaction -The preferred way to broadcast a transaction is to use gRPC, though using REST (via `gRPC-gateway`) or the CometBFT RPC is also posible. An overview of the differences between these methods is exposed [here](../../develop/advanced/06-grpc_rest.md). For this tutorial, we will only describe the gRPC method. +The preferred way to broadcast a transaction is to use gRPC, though using REST (via `gRPC-gateway`) or the CometBFT RPC is also posible. An overview of the differences between these methods is exposed [here](../../learn/advanced/06-grpc_rest.md). For this tutorial, we will only describe the gRPC method. ```go import ( diff --git a/docs/user/run-node/04-rosetta.md b/docs/user/run-node/04-rosetta.md index 3d8467aea..936666fa8 100644 --- a/docs/user/run-node/04-rosetta.md +++ b/docs/user/run-node/04-rosetta.md @@ -1,21 +1,33 @@ ---- -sidebar_position: 1 ---- - # Rosetta -The `rosetta` package implements Coinbase's [Rosetta API](https://www.rosetta-api.org). This document provides instructions on how to use the Rosetta API integration. For information about the motivation and design choices, refer to [ADR 035](https://docs.cosmos.network/main/architecture/adr-035-rosetta-api-support). +The `rosetta` project implements Coinbase's [Rosetta API](https://www.rosetta-api.org). This document provides instructions on how to use the Rosetta API integration. For information about the motivation and design choices, refer to [ADR 035](https://docs.cosmos.network/main/architecture/adr-035-rosetta-api-support). -## Add Rosetta Command +## Installing Rosetta The Rosetta API server is a stand-alone server that connects to a node of a chain developed with Cosmos SDK. -To enable Rosetta API support, it's required to add the `RosettaCommand` to your application's root command file (e.g. `simd/cmd/root.go`). +Rosetta can be added to any cosmos chain node. standalone or natively. + +### Standalone + +Rosetta can be executed as a standalone service, it connects to the node endpoints and expose the required endpoints. + +Install Rosetta standalone server with the following command: + +```bash +go install github.com/cosmos/rosetta +``` + +Alternatively, for building from source, simply run `make rosetta`. The binary will be located in the root folder. + +### Native - As a node command + +To enable Native Rosetta API support, it's required to add the `RosettaCommand` to your application's root command file (e.g. `simd/cmd/root.go`). Import the `rosettaCmd` package: ```go -import "cosmossdk.io/tools/rosetta/cmd" +import "github.com/cosmos/rosetta/cmd" ``` Find the following line: @@ -32,7 +44,7 @@ rootCmd.AddCommand( ) ``` -The `RosettaCommand` function builds the `rosetta` root command and is defined in the `rosettaCmd` package (`cosmossdk.io/tools/rosetta/cmd`). +The `RosettaCommand` function builds the `rosetta` root command and is defined in the `rosettaCmd` package (`github.com/cosmos/rosetta/cmd`). Since we’ve updated the Cosmos SDK to work with the Rosetta API, updating the application's root command file is all you need to do. @@ -42,30 +54,40 @@ An implementation example can be found in `simapp` package. To run Rosetta in your application CLI, use the following command: +> **Note:** if using the native approach, add your node name before any rosetta comand. + ```shell -simd rosetta --help +rosetta --help ``` To test and run Rosetta API endpoints for applications that are running and exposed, use the following command: ```shell -simd rosetta +rosetta --blockchain "your application name (ex: gaia)" --network "your chain identifier (ex: testnet-1)" --tendermint "tendermint endpoint (ex: localhost:26657)" --grpc "gRPC endpoint (ex: localhost:9090)" --addr "rosetta binding address (ex: :8080)" + --grpc-types-server (optional) "gRPC endpoint for message descriptor types" ``` -## Use Rosetta Standalone +## Plugins - Multi chain connections -To use Rosetta standalone, without having to add it in your application, install it with the following command: +Rosetta will try to reflect the node types trough reflection over the node gRPC endpoints, there may be cases were this approach is not enough. It is possible to extend or implement the required types easily trough plugins. -```bash -go install cosmossdk.io/tools/rosetta/cmd/rosetta -``` +To use Rosetta over any chain, it is required to set up prefixes and registering zone specific interfaces through plugins. + +Each plugin is a minimalist implementation of `InitZone` and `RegisterInterfaces` which allow Rosetta to parse chain specific data. There is an example for cosmos-hub chain under `plugins/cosmos-hun/` folder +- **InitZone**: An empty method that is executed first and defines prefixes, parameters and other settings. +- **RegisterInterfaces**: This method receives an interface registry which is were the zone specific types and interfaces will be loaded + +In order to add a new plugin: +1. Create a folder over `plugins` folder with the name of the desired zone +2. Add a `main.go` file with the mentioned methods above. +3. Build the code binary through `go build -buildmode=plugin -o main.so main.go` -Alternatively, for building from source, simply run `make rosetta`. The binary will be located in `tools/rosetta`. +The plugin folder is selected through the cli `--plugin` flag and loaded into the Rosetta server. ## Extensions @@ -87,7 +109,7 @@ import ( "context" "github.com/coinbase/rosetta-sdk-go/types" -"cosmossdk.io/tools/rosetta/lib" +"github.com/cosmos/rosetta/lib" ) // CustomClient embeds the standard cosmos client @@ -113,7 +135,7 @@ Example: ```go package custom_errors -import crgerrs "cosmossdk.io/tools/rosetta/lib/errors" +import crgerrs "github.com/cosmos/rosetta/lib/errors" var customErrRetriable = true var CustomError = crgerrs.RegisterError(100, "custom message", customErrRetriable, "description") diff --git a/docs/user/run-node/05-run-testnet.md b/docs/user/run-node/05-run-testnet.md new file mode 100644 index 000000000..c2b5da598 --- /dev/null +++ b/docs/user/run-node/05-run-testnet.md @@ -0,0 +1,101 @@ +--- +sidebar_position: 1 +--- + +# Running a Testnet + +:::note Synopsis +The `simd testnet` subcommand makes it easy to initialize and start a simulated test network for testing purposes. +::: + +In addition to the commands for [running a node](./01-run-node.md), the `simd` binary also includes a `testnet` command that allows you to start a simulated test network in-process or to initialize files for a simulated test network that runs in a separate process. + +## Initialize Files + +First, let's take a look at the `init-files` subcommand. + +This is similar to the `init` command when initializing a single node, but in this case we are initializing multiple nodes, generating the genesis transactions for each node, and then collecting those transactions. + +The `init-files` subcommand initializes the necessary files to run a test network in a separate process (i.e. using a Docker container). Running this command is not a prerequisite for the `start` subcommand ([see below](#start-testnet)). + +In order to initialize the files for a test network, run the following command: + +```bash +simd testnet init-files +``` + +You should see the following output in your terminal: + +```bash +Successfully initialized 4 node directories +``` + +The default output directory is a relative `.testnets` directory. Let's take a look at the files created within the `.testnets` directory. + +### gentxs + +The `gentxs` directory includes a genesis transaction for each validator node. Each file includes a JSON encoded genesis transaction used to register a validator node at the time of genesis. The genesis transactions are added to the `genesis.json` file within each node directory during the initilization process. + +### nodes + +A node directory is created for each validator node. Within each node directory is a `simd` directory. The `simd` directory is the home directory for each node, which includes the configuration and data files for that node (i.e. the same files included in the default `~/.simapp` directory when running a single node). + +## Start Testnet + +Now, let's take a look at the `start` subcommand. + +The `start` subcommand both initializes and starts an in-process test network. This is the fastest way to spin up a local test network for testing purposes. + +You can start the local test network by running the following command: + +```bash +simd testnet start +``` + +You should see something similar to the following: + +```bash +acquiring test network lock +preparing test network with chain-id "chain-mtoD9v" + + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ THIS MNEMONIC IS FOR TESTING PURPOSES ONLY ++ +++ DO NOT USE IN PRODUCTION ++ +++ ++ +++ sustain know debris minute gate hybrid stereo custom ++ +++ divorce cross spoon machine latin vibrant term oblige ++ +++ moment beauty laundry repeat grab game bronze truly ++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + + +starting test network... +started test network +press the Enter Key to terminate +``` + +The first validator node is now running in-process, which means the test network will terminate once you either close the terminal window or you press the Enter key. In the output, the mnemonic phrase for the first validator node is provided for testing purposes. The validator node is using the same default addresses being used when initializing and starting a single node (no need to provide a `--node` flag). + +Check the status of the first validator node: + +```shell +simd status +``` + +Import the key from the provided mnemonic: + +```shell +simd keys add test --recover --keyring-backend test +``` + +Check the balance of the account address: + +```shell +simd q bank balances [address] +``` + +Use this test account to manually test against the test network. + +## Testnet Options + +You can customize the configuration of the test network with flags. In order to see all flag options, append the `--help` flag to each command. diff --git a/docs/user/run-node/_category_.json b/docs/user/run-node/_category_.json index 9b291ba6f..65e64b945 100644 --- a/docs/user/run-node/_category_.json +++ b/docs/user/run-node/_category_.json @@ -1,5 +1,5 @@ { - "label": "User Guides", + "label": "Running a Node, API and CLI", "position": 0, "link": null } \ No newline at end of file diff --git a/docusaurus.config.js b/docusaurus.config.js index ea0ab7c8d..1984c6008 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -83,8 +83,8 @@ const config = { items: [ { type: "doc", - label: "Develop", - docId: "develop/intro/what-is-sdk", + label: "Learn", + docId: "learn/intro/overview", position: "left", }, { @@ -408,25 +408,25 @@ const config = { ], to: "/main/build/modules/upgrade/", }, - { - from: ["/main/modules/capability", "/main/ecosystem"], - to: "/main/build/modules/", - }, + // { + // from: ["/main/modules/capability", "/main/ecosystem"], + // to: "/main/build/modules/", + // }, ], createRedirects(existingPath) { - if (existingPath.includes('/develop/advanced')) { + if (existingPath.includes('/learn/advanced')) { return [ - existingPath.replace("/develop/advanced", "/core"), + existingPath.replace("/learn/advanced", "/core"), ]; } - if (existingPath.includes('/develop/beginner')) { + if (existingPath.includes('/learn/beginner')) { return [ - existingPath.replace("/develop/beginner", "/basics"), + existingPath.replace("/learn/beginner", "/basics"), ]; } - if (existingPath.includes('/develop/intro')) { + if (existingPath.includes('/learn/intro')) { return [ - existingPath.replace("/develop/intro", "/intro"), + existingPath.replace("/learn/intro", "/intro"), ]; } if (existingPath.includes('build/architecture')) { diff --git a/sidebars.js b/sidebars.js index f50b64254..40c127c58 100644 --- a/sidebars.js +++ b/sidebars.js @@ -14,7 +14,7 @@ /** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */ const sidebars = { // By default, Docusaurus generates a sidebar from the docs folder structure - developSidebar: [{ type: "autogenerated", dirName: "develop" }], + learnSidebar: [{ type: "autogenerated", dirName: "learn" }], buildSidebar: [{ type: "autogenerated", dirName: "build" }], userSidebar: [{ type: "autogenerated", dirName: "user" }], }; diff --git a/src/components/HomepageFeatures/index.js b/src/components/HomepageFeatures/index.js index d75529b17..5354ac5f6 100644 --- a/src/components/HomepageFeatures/index.js +++ b/src/components/HomepageFeatures/index.js @@ -7,7 +7,7 @@ const FeatureList = [ { title: `Learn Basics`, Svg: require("@site/static/img/innovation.svg").default, - to: "/main/develop/intro/what-is-sdk", + to: "/main/learn/intro/overview", description: ( <> Get a quick introduction to the Cosmos SDK and its key features, diff --git a/versioned_docs/version-0.47/build/tooling/README.md b/versioned_docs/version-0.47/build/tooling/README.md index 886fed510..5ecdef85b 100644 --- a/versioned_docs/version-0.47/build/tooling/README.md +++ b/versioned_docs/version-0.47/build/tooling/README.md @@ -8,4 +8,4 @@ This section provides documentation on various tooling used in development of a * [Protocol Buffers](./00-protobuf.md) * [Cosmovisor](./01-cosmovisor.md) -* [Depinject](./02-depinject.md) \ No newline at end of file +* [Confix](./02-confix.md) diff --git a/versioned_docs/version-0.47/learn/advanced/01-transactions.md b/versioned_docs/version-0.47/learn/advanced/01-transactions.md index 074703d61..deee7514e 100644 --- a/versioned_docs/version-0.47/learn/advanced/01-transactions.md +++ b/versioned_docs/version-0.47/learn/advanced/01-transactions.md @@ -89,8 +89,6 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/tx/v1beta1/tx The use case is a multi-signer transaction, where one of the signers is appointed to gather all signatures, broadcast the signature and pay for fees, and the others only care about the transaction body. This generally allows for a better multi-signing UX. If Alice, Bob and Charlie are part of a 3-signer transaction, then Alice and Bob can both use `SIGN_MODE_DIRECT_AUX` to sign over the `TxBody` and their own signer info (no need an additional step to gather other signers' ones, like in `SIGN_MODE_DIRECT`), without specifying a fee in their SignDoc. Charlie can then gather both signatures from Alice and Bob, and create the final transaction by appending a fee. Note that the fee payer of the transaction (in our case Charlie) must sign over the fees, so must use `SIGN_MODE_DIRECT` or `SIGN_MODE_LEGACY_AMINO_JSON`. -A concrete use case is implemented in [transaction tips](15-tips.md): the tipper may use `SIGN_MODE_DIRECT_AUX` to specify a tip in the transaction, without signing over the actual transaction fees. Then, the fee payer appends fees inside the tipper's desired `TxBody`, and as an exchange for paying the fees and broadcasting the transaction, receives the tipper's transaction tips as payment. - #### `SIGN_MODE_TEXTUAL` `SIGN_MODE_TEXTUAL` is a new sign mode for delivering a better signing experience on hardware wallets, it is currently still under implementation. If you wish to learn more, please refer to [ADR-050](https://github.com/cosmos/cosmos-sdk/pull/10701). diff --git a/versioned_docs/version-0.47/learn/advanced/15-tips.md b/versioned_docs/version-0.47/learn/advanced/15-tips.md deleted file mode 100644 index 7a3261233..000000000 --- a/versioned_docs/version-0.47/learn/advanced/15-tips.md +++ /dev/null @@ -1,214 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Transaction Tips - -:::note Synopsis -Transaction tips are a mechanism to pay for transaction fees using another denom than the native fee denom of the chain. They are still in beta, and are not included by default in the SDK. -::: - -## Context - -In a Cosmos ecosystem where more and more chains are connected via [IBC](https://ibc.cosmos.network/), it happens that users want to perform actions on chains where they don't have native tokens yet. An example would be an Osmosis user who wants to vote on a proposal on the Cosmos Hub, but they don't have ATOMs in their wallet. A solution would be to swap OSMO for ATOM just for voting on this proposal, but that is cumbersome. Cross-chain DeFi project [Emeris](https://emeris.com/) is another use case. - -Transaction tips is a new solution for cross-chain transaction fees payment, whereby the transaction initiator signs a transaction without specifying fees, but uses a new `Tip` field. They send this signed transaction to a fee relayer who will choose the transaction fees and broadcast the final transaction, and the SDK provides a mechanism that will transfer the pre-defined `Tip` to the fee payer, to cover for fees. - -Assuming we have two chains, A and B, we define the following terms: - -* **the tipper**: this is the initiator of the transaction, who wants to execute a `Msg` on chain A, but doesn't have any native chain A tokens, only chain B tokens. In our example above, the tipper is the Osmosis (chain B) user wanting to vote on a Cosmos Hub (chain A) proposal. -* **the fee payer**: this is the party that will relay and broadcast the final transaction on chain A, and has chain A tokens. The tipper doesn't need to trust the feepayer. -* **the target chain**: the chain where the `Msg` is executed, chain A in this case. - -## Transaction Tips Flow - -The transaction tips flow happens in multiple steps. - -1. The tipper sends via IBC some chain B tokens to chain A. These tokens will cover for fees on the target chain A. This means that chain A's bank module holds some IBC tokens under the tipper's address. - -2. The tipper drafts a transaction to be executed on the chain A. It can include chain A `Msg`s. However, instead of creating a normal transaction, they create the following `AuxSignerData` document: - - ```protobuf reference - https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/tx/v1beta1/tx.proto#L237-L256 - ``` - - where we have defined `SignDocDirectAux` as: - - ```protobuf reference - https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/tx/v1beta1/tx.proto#L67-L97 - ``` - - where `Tip` is defined as - - ```protobuf reference - https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/tx/v1beta1/tx.proto#L226-L235 - ``` - - Notice that this document doesn't sign over the final chain A fees. Instead, it includes a `Tip` field. It also doesn't include the whole `AuthInfo` object as in `SIGN_MODE_DIRECT`, only the minimum information needed by the tipper - -3. The tipper signs the `SignDocDirectAux` document and attaches the signature to the `AuxSignerData`, then sends the signed `AuxSignerData` to the fee payer. - -4. From the signed `AuxSignerData` document, the fee payer constructs a transaction, using the following algorithm: - -* use as `TxBody` the exact `AuxSignerData.SignDocDirectAux.body_bytes`, to not alter the original intent of the tipper, -* create an `AuthInfo` with: - * `AuthInfo.Tip` copied from `AuxSignerData.SignDocDirectAux.Tip`, - * `AuthInfo.Fee` chosen by the fee payer, which should cover for the transaction gas, but also be small enough so that the tip/fee exchange rate is economically interesting for the fee payer, - * `AuthInfo.SignerInfos` has two signers: the first signer is the tipper, using the public key, sequence and sign mode specified in `AuxSignerData`; and the second signer is the fee payer, using their favorite sign mode, -* a `Signatures` array with two items: the tipper's signature from `AuxSignerData.Sig`, and the final fee payer's signature. - -5. Broadcast the final transaction signed by the two parties to the target chain. Once included, the Cosmos SDK will trigger a transfer of the `Tip` specified in the transaction from the tipper address to the fee payer address. - -### Fee Payers Market - -The benefit of transaction tips for the tipper is clear: there is no need to swap tokens before executing a cross-chain message. - -For the fee payer, the benefit is in the tip v.s. fee exchange. Put simply, the fee payer pays the fees of an unknown tipper's transaction, and gets in exchange the tip that the tipper chose. There is an economic incentive for the fee payer to do so only when the tip is greater than the transaction fees, given the exchange rates between the two tokens. - -In the future, we imagine a market where fee payers will compete to include transactions from tippers, who on their side will optimize by specifying the lowest tip possible. A number of automated services might spin up to perform transaction gas simulation and exchange rate monitoring to optimize both the tip and fee values in real-time. - -### Tipper and Fee Payer Sign Modes - -As we mentioned in the flow above, the tipper signs over the `SignDocDirectAux`, and the fee payer signs over the whole final transaction. As such, both parties might use different sign modes. - -* The tipper MUST use `SIGN_MODE_DIRECT_AUX` or `SIGN_MODE_LEGACY_AMINO_JSON`. That is because the tipper needs to sign over the body, the tip, but not the other signers' information and not over the fee (which is unknown to the tipper). -* The fee payer MUST use `SIGN_MODE_DIRECT` or `SIGN_MODE_LEGACY_AMINO_JSON`. The fee payer signs over the whole transaction. - -For example, if the fee payer signs the whole transaction with `SIGN_MODE_DIRECT_AUX`, it will be rejected by the node, as that would introduce malleability issues (`SIGN_MODE_DIRECT_AUX` doesn't sign over fees). - -In both cases, using `SIGN_MODE_LEGACY_AMINO_JSON` is recommended only if hardware wallet signing is needed. - -## Enabling Tips on your Chain - -The transaction tips functionality is introduced in Cosmos SDK v0.46, so earlier versions do not have support for tips. It is however not included by default in a v0.46 app. Sending a transaction with tips to a chain which didn't enable tips will result in a no-op, i.e. the `tip` field in the transaction will be ignored. - -Enabling tips on a chain is done by adding the `TipDecorator` in the posthandler chain: - -```go -// HandlerOptions are the options required for constructing a SDK PostHandler which supports tips. -type HandlerOptions struct { - BankKeeper types.BankKeeper -} - -// MyPostHandler returns a posthandler chain with the TipDecorator. -func MyPostHandler(options HandlerOptions) (sdk.AnteHandler, error) { - if options.BankKeeper == nil { - return nil, errorsmod.Wrap(sdkerrors.ErrLogic, "bank keeper is required for posthandler") - } - - postDecorators := []sdk.AnteDecorator{ - posthandler.NewTipDecorator(options.bankKeeper), - } - - return sdk.ChainAnteDecorators(postDecorators...), nil -} - -func (app *SimApp) setPostHandler() { - postHandler, err := MyPostHandler( - HandlerOptions{ - BankKeeper: app.BankKeeper, - }, - ) - if err != nil { - panic(err) - } - - app.SetPostHandler(postHandler) -} -``` - -Notice that `NewTipDecorator` needs a reference to the BankKeeper, for transferring the tip to the fee payer. - -## CLI Usage - -The Cosmos SDK also provides some CLI tooling for the transaction tips flow, both for the tipper and for the feepayer. - -For the tipper, the CLI `tx` subcommand has two new flags: `--aux` and `--tip`. The `--aux` flag is used to denote that we are creating an `AuxSignerData` instead of a `Tx`, and the `--tip` is used to populate its `Tip` field. - -```bash -$ simd tx gov vote 16 yes --from --aux --tip 50ibcdenom - - -### Prints the AuxSignerData as JSON: -### {"address":"cosmos1q0ayf5vq6fd2xxrwh30upg05hxdnyw2h5249a2","sign_doc":{"body_bytes":"CosBChwvY29zbW9zLmJhbmsudjFiZXRhMS5Nc2dTZW5kEmsKLWNvc21vczFxMGF5ZjV2cTZmZDJ4eHJ3aDMwdXBnMDVoeGRueXcyaDUyNDlhMhItY29zbW9zMXdlNWoyZXI2MHV5OXF3YzBta3ptdGdtdHA5Z3F5NXY2bjhnZGdlGgsKBXN0YWtlEgIxMA==","public_key":{"@type":"/cosmos.crypto.secp256k1.PubKey","key":"AojOF/1luQ5H/nZDSrE1w3CyzGJhJdQuS7hFX5wAA6uJ"},"chain_id":"","account_number":"0","sequence":"1","tip":{"amount":[{"denom":"ibcdenom","amount":"50"}],"tipper":"cosmos1q0ayf5vq6fd2xxrwh30upg05hxdnyw2h5249a2"}},"mode":"SIGN_MODE_DIRECT_AUX","sig":"v/d/bGq9FGdecs6faMG2t//nRirFTiqwFtUB65M6kh0QdUeM6jg3r8oJX1o17xkoDxJ09EyJiSyvo6fbU7vUxg=="} -``` - -It is useful to pipe the JSON output to a file, `> aux_signed_tx.json` - -For the fee payer, the Cosmos SDK added a `tx aux-to-fee` subcommand to include an `AuxSignerData` into a transaction, add fees to it, and broadcast it. - -```bash -$ simd tx aux-to-fee aux_signed_tx.json --from --fees 30atom - -### Prints the broadcasted tx response: -### code: 0 -### codespace: sdk -### data: "" -### events: [] -### gas_used: "0" -### gas_wanted: "0" -### height: "0" -### info: "" -### logs: [] -### timestamp: "" -### tx: null -``` - -Upon completion of the second command, the fee payer's balance will be down the `30atom` fees, and up the `50ibcdenom` tip. - -For both commands, the flag `--sign-mode=amino-json` is still available for hardware wallet signing. - -## Programmatic Usage - -For the tipper, the SDK exposes a new transaction builder, the `AuxTxBuilder`, for generating an `AuxSignerData`. The API of `AuxTxBuilder` is defined [in `client/tx`](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/client/tx/aux_builder.go#L16), and can be used as follows: - -```go -// Note: there's no need to use clientCtx.TxConfig anymore. - -bldr := clienttx.NewAuxTxBuilder() -err := bldr.SetMsgs(msgs...) -bldr.SetAddress("cosmos1...") -bldr.SetMemo(...) -bldr.SetTip(...) -bldr.SetPubKey(...) -err := bldr.SetSignMode(...) // DIRECT_AUX or AMINO, or else error -// ... other setters are also available - -// Get the bytes to sign. -signBz, err := bldr.GetSignBytes() - -// Sign the bz using your favorite method. -sig, err := privKey.sign(signBz) - -// Set the signature -bldr.SetSig(sig) - -// Get the final auxSignerData to be sent to the fee payer -auxSignerData, err:= bldr.GetAuxSignerData() -``` - -For the fee payer, the SDK added a new method on the existing `TxBuilder` to import data from an `AuxSignerData`: - -```go -// get `auxSignerData` from tipper, see code snippet above. - -txBuilder := clientCtx.TxConfig.NewTxBuilder() -err := txBuilder.AddAuxSignerData(auxSignerData) -if err != nil { - return err -} - -// A lot of fields will be populated in txBuilder, such as its Msgs, tip -// memo, etc... - -// The fee payer choses the fee to set on the transaction. -txBuilder.SetFeePayer() -txBuilder.SetFeeAmount(...) -txBuilder.SetGasLimit(...) - -// Usual signing code -err = authclient.SignTx(...) -if err != nil { - return err -} -``` diff --git a/docs/develop/intro/00-what-is-sdk.md b/versioned_docs/version-0.47/learn/intro/00-overview.md similarity index 100% rename from docs/develop/intro/00-what-is-sdk.md rename to versioned_docs/version-0.47/learn/intro/00-overview.md diff --git a/versioned_docs/version-0.50/build/architecture/README.md b/versioned_docs/version-0.50/build/architecture/README.md index 8a5c5f307..db466599a 100644 --- a/versioned_docs/version-0.50/build/architecture/README.md +++ b/versioned_docs/version-0.50/build/architecture/README.md @@ -92,3 +92,4 @@ When writing ADRs, follow the same best practices for writing RFCs. When writing * [ADR 044: Guidelines for Updating Protobuf Definitions](./adr-044-protobuf-updates-guidelines.md) * [ADR 047: Extend Upgrade Plan](./adr-047-extend-upgrade-plan.md) * [ADR 053: Go Module Refactoring](./adr-053-go-module-refactoring.md) +* [ADR 068: Preblock](./adr-068-preblock.md) diff --git a/versioned_docs/version-0.50/build/architecture/_category_.json b/versioned_docs/version-0.50/build/architecture/_category_.json index 87ceb9374..e0b1907a9 100644 --- a/versioned_docs/version-0.50/build/architecture/_category_.json +++ b/versioned_docs/version-0.50/build/architecture/_category_.json @@ -1,5 +1,5 @@ { "label": "ADRs", - "position": 11, + "position": 6, "link": null } \ No newline at end of file diff --git a/versioned_docs/version-0.50/build/architecture/adr-033-protobuf-inter-module-comm.md b/versioned_docs/version-0.50/build/architecture/adr-033-protobuf-inter-module-comm.md index 672c906d7..28c69a910 100644 --- a/versioned_docs/version-0.50/build/architecture/adr-033-protobuf-inter-module-comm.md +++ b/versioned_docs/version-0.50/build/architecture/adr-033-protobuf-inter-module-comm.md @@ -20,7 +20,7 @@ service definitions defined in [ADR 021](./adr-021-protobuf-query-encoding.md) a ## Context -In the current Cosmos SDK documentation on the [Object-Capability Model](../../develop/advanced/10-ocap.md), it is stated that: +In the current Cosmos SDK documentation on the [Object-Capability Model](../../learn/advanced/10-ocap.md), it is stated that: > We assume that a thriving ecosystem of Cosmos SDK modules that are easy to compose into a blockchain application will contain faulty or malicious modules. diff --git a/versioned_docs/version-0.50/build/architecture/adr-063-core-module-api.md b/versioned_docs/version-0.50/build/architecture/adr-063-core-module-api.md index 622e29435..743dd5fee 100644 --- a/versioned_docs/version-0.50/build/architecture/adr-063-core-module-api.md +++ b/versioned_docs/version-0.50/build/architecture/adr-063-core-module-api.md @@ -280,6 +280,17 @@ type HasGenesis interface { } ``` +#### Pre Blockers + +Modules that have functionality that runs before BeginBlock and should implement the has `HasPreBlocker` interfaces: + +```go +type HasPreBlocker interface { + AppModule + PreBlock(context.Context) error +} +``` + #### Begin and End Blockers Modules that have functionality that runs before transactions (begin blockers) or after transactions diff --git a/versioned_docs/version-0.50/build/architecture/adr-064-abci-2.0.md b/versioned_docs/version-0.50/build/architecture/adr-064-abci-2.0.md index 594965167..c0dc7f746 100644 --- a/versioned_docs/version-0.50/build/architecture/adr-064-abci-2.0.md +++ b/versioned_docs/version-0.50/build/architecture/adr-064-abci-2.0.md @@ -292,32 +292,22 @@ decision based on the vote extensions. #### Vote Extension Persistence In certain contexts, it may be useful or necessary for applications to persist -data derived from vote extensions. In order to facilitate this use case, we -propose to allow application developers to manually retrieve the `finalizeState` -context (see [`FinalizeBlock`](#finalizeblock-1) below). Using this context, -state can be directly written to `finalizeState`, which will be used during -`FinalizeBlock` and eventually committed to the application state. Note, since -`ProcessProposal` can timeout and thus require another round of consensus, we -will reset `finalizeState` in the beginning of `ProcessProposal`. +data derived from vote extensions. In order to facilitate this use case, we propose +to allow app developers to define a pre-Blocker hook which will be called +at the very beginning of `FinalizeBlock`, i.e. before `BeginBlock` (see below). -A `ProcessProposal` handler could look like the following: +Note, we cannot allow applications to directly write to the application state +during `ProcessProposal` because during replay, CometBFT will NOT call `ProcessProposal`, +which would result in an incomplete state view. ```go -func (h MyHandler) ProcessProposalHandler() sdk.ProcessProposalHandler { - return func(ctx sdk.Context, req abci.RequestProcessProposal) abci.ResponseProcessProposal { - for _, txBytes := range req.Txs { - _, err := h.app.ProcessProposalVerifyTx(txBytes) - if err != nil { - return abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT} - } - } - - fCtx := h.app.GetFinalizeState() - - // Any state changes that occur on the provided fCtx WILL be written to state! - h.myKeeper.SetVoteExtResult(fCtx, ...) +func (a MyApp) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeBlock) error { + voteExts := GetVoteExtensions(ctx, req.Txs) - return abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_ACCEPT} + // Process and perform some compute on vote extensions, storing any resulting + // state. + if err a.processVoteExtensions(ctx, voteExts); if err != nil { + return err } } ``` @@ -360,11 +350,20 @@ legacy ABCI types, e.g. `LegacyBeginBlockRequest` and `LegacyEndBlockRequest`. O we can come up with new types and names altogether. ```go -func (app *BaseApp) FinalizeBlock(req abci.RequestFinalizeBlock) abci.ResponseFinalizeBlock { - // merge any state changes from ProcessProposal into the FinalizeBlock state - app.MergeProcessProposalState() +func (app *BaseApp) FinalizeBlock(req abci.RequestFinalizeBlock) (*abci.ResponseFinalizeBlock, error) { + ctx := ... - beginBlockResp := app.beginBlock(ctx, req) + if app.preBlocker != nil { + ctx := app.finalizeBlockState.ctx + rsp, err := app.preBlocker(ctx, req) + if err != nil { + return nil, err + } + if rsp.ConsensusParamsChanged { + app.finalizeBlockState.ctx = ctx.WithConsensusParams(app.GetConsensusParams(ctx)) + } + } + beginBlockResp, err := app.beginBlock(req) appendBlockEventAttr(beginBlockResp.Events, "begin_block") txExecResults := make([]abci.ExecTxResult, 0, len(req.Txs)) @@ -373,7 +372,7 @@ func (app *BaseApp) FinalizeBlock(req abci.RequestFinalizeBlock) abci.ResponseFi txExecResults = append(txExecResults, result) } - endBlockResp := app.endBlock(ctx, req) + endBlockResp, err := app.endBlock(app.finalizeBlockState.ctx) appendBlockEventAttr(beginBlockResp.Events, "end_block") return abci.ResponseFinalizeBlock{ diff --git a/versioned_docs/version-0.50/build/architecture/adr-068-preblock.md b/versioned_docs/version-0.50/build/architecture/adr-068-preblock.md new file mode 100644 index 000000000..86692c412 --- /dev/null +++ b/versioned_docs/version-0.50/build/architecture/adr-068-preblock.md @@ -0,0 +1,61 @@ +# ADR 068: Preblock + +## Changelog + +* Sept 13, 2023: Initial Draft + +## Status + +DRAFT + +## Abstract + +Introduce `PreBlock`, which runs before begin blocker other modules, and allows to modify consensus parameters, and the changes are visible to the following state machine logics. + +## Context + +When upgrading to sdk 0.47, the storage format for consensus parameters changed, but in the migration block, `ctx.ConsensusParams()` is always `nil`, because it fails to load the old format using new code, it's supposed to be migrated by the `x/upgrade` module first, but unfortunately, the migration happens in `BeginBlocker` handler, which runs after the `ctx` is initialized. +When we try to solve this, we find the `x/upgrade` module can't modify the context to make the consensus parameters visible for the other modules, the context is passed by value, and sdk team want to keep it that way, that's good for isolations between modules. + +## Alternatives + +The first alternative solution introduced a `MigrateModuleManager`, which only includes the `x/upgrade` module right now, and baseapp will run their `BeginBlocker`s before the other modules, and reload context's consensus parameters in between. + +## Decision + +Suggested this new lifecycle method. + +### `PreBlocker` + +There are two semantics around the new lifecycle method: + +- It runs before the `BeginBlocker` of all modules +- It can modify consensus parameters in storage, and signal the caller through the return value. + +When it returns `ConsensusParamsChanged=true`, the caller must refresh the consensus parameter in the finalize context: +``` +app.finalizeBlockState.ctx = app.finalizeBlockState.ctx.WithConsensusParams(app.GetConsensusParams()) +``` + +The new ctx must be passed to all the other lifecycle methods. + + +## Consequences + +### Backwards Compatibility + +### Positive + +### Negative + +### Neutral + +## Further Discussions + +## Test Cases + +## References +* [1] https://github.com/cosmos/cosmos-sdk/issues/16494 +* [2] https://github.com/cosmos/cosmos-sdk/pull/16583 +* [3] https://github.com/cosmos/cosmos-sdk/pull/17421 +* [4] https://github.com/cosmos/cosmos-sdk/pull/17713 diff --git a/versioned_docs/version-0.50/build/building-apps/02-app-mempool.md b/versioned_docs/version-0.50/build/building-apps/02-app-mempool.md index 1f591fc6d..630b94ba7 100644 --- a/versioned_docs/version-0.50/build/building-apps/02-app-mempool.md +++ b/versioned_docs/version-0.50/build/building-apps/02-app-mempool.md @@ -15,7 +15,7 @@ Notably it introduces the `PrepareProposal` and `ProcessProposal` steps of ABCI+ :::note Pre-requisite Readings -* [BaseApp](../../develop/advanced/00-baseapp.md) +* [BaseApp](../../learn/advanced/00-baseapp.md) ::: diff --git a/versioned_docs/version-0.50/build/building-apps/_category_.json b/versioned_docs/version-0.50/build/building-apps/_category_.json index de6617ddc..342732cce 100644 --- a/versioned_docs/version-0.50/build/building-apps/_category_.json +++ b/versioned_docs/version-0.50/build/building-apps/_category_.json @@ -1,5 +1,5 @@ { "label": "Building Apps", - "position": 4, + "position": 0, "link": null } \ No newline at end of file diff --git a/versioned_docs/version-0.50/build/building-modules/00-intro.md b/versioned_docs/version-0.50/build/building-modules/00-intro.md index 6f5303150..6850f0b67 100644 --- a/versioned_docs/version-0.50/build/building-modules/00-intro.md +++ b/versioned_docs/version-0.50/build/building-modules/00-intro.md @@ -10,18 +10,18 @@ Modules define most of the logic of Cosmos SDK applications. Developers compose :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK application](../../develop/beginner/00-overview-app.md) -* [Lifecycle of a Cosmos SDK transaction](../../develop/beginner/01-tx-lifecycle.md) +* [Anatomy of a Cosmos SDK application](../../learn/beginner/00-app-anatomy.md) +* [Lifecycle of a Cosmos SDK transaction](../../learn/beginner/01-tx-lifecycle.md) ::: ## Role of Modules in a Cosmos SDK Application -The Cosmos SDK can be thought of as the Ruby-on-Rails of blockchain development. It comes with a core that provides the basic functionalities every blockchain application needs, like a [boilerplate implementation of the ABCI](../../develop/advanced/00-baseapp.md) to communicate with the underlying consensus engine, a [`multistore`](../../develop/advanced/04-store.md#multistore) to persist state, a [server](../../develop/advanced/03-node.md) to form a full-node and [interfaces](./09-module-interfaces.md) to handle queries. +The Cosmos SDK can be thought of as the Ruby-on-Rails of blockchain development. It comes with a core that provides the basic functionalities every blockchain application needs, like a [boilerplate implementation of the ABCI](../../learn/advanced/00-baseapp.md) to communicate with the underlying consensus engine, a [`multistore`](../../learn/advanced/04-store.md#multistore) to persist state, a [server](../../learn/advanced/03-node.md) to form a full-node and [interfaces](./09-module-interfaces.md) to handle queries. On top of this core, the Cosmos SDK enables developers to build modules that implement the business logic of their application. In other words, SDK modules implement the bulk of the logic of applications, while the core does the wiring and enables modules to be composed together. The end goal is to build a robust ecosystem of open-source Cosmos SDK modules, making it increasingly easier to build complex blockchain applications. -Cosmos SDK modules can be seen as little state-machines within the state-machine. They generally define a subset of the state using one or more `KVStore`s in the [main multistore](../../develop/advanced/04-store.md), as well as a subset of [message types](./02-messages-and-queries.md#messages). These messages are routed by one of the main components of Cosmos SDK core, [`BaseApp`](../../develop/advanced/00-baseapp.md), to a module Protobuf [`Msg` service](./03-msg-services.md) that defines them. +Cosmos SDK modules can be seen as little state-machines within the state-machine. They generally define a subset of the state using one or more `KVStore`s in the [main multistore](../../learn/advanced/04-store.md), as well as a subset of [message types](./02-messages-and-queries.md#messages). These messages are routed by one of the main components of Cosmos SDK core, [`BaseApp`](../../learn/advanced/00-baseapp.md), to a module Protobuf [`Msg` service](./03-msg-services.md) that defines them. ```text + @@ -75,7 +75,7 @@ As a result of this architecture, building a Cosmos SDK application usually revo While there are no definitive guidelines for writing modules, here are some important design principles developers should keep in mind when building them: * **Composability**: Cosmos SDK applications are almost always composed of multiple modules. This means developers need to carefully consider the integration of their module not only with the core of the Cosmos SDK, but also with other modules. The former is achieved by following standard design patterns outlined [here](#main-components-of-sdk-modules), while the latter is achieved by properly exposing the store(s) of the module via the [`keeper`](./06-keeper.md). -* **Specialization**: A direct consequence of the **composability** feature is that modules should be **specialized**. Developers should carefully establish the scope of their module and not batch multiple functionalities into the same module. This separation of concerns enables modules to be re-used in other projects and improves the upgradability of the application. **Specialization** also plays an important role in the [object-capabilities model](../../develop/advanced/10-ocap.md) of the Cosmos SDK. +* **Specialization**: A direct consequence of the **composability** feature is that modules should be **specialized**. Developers should carefully establish the scope of their module and not batch multiple functionalities into the same module. This separation of concerns enables modules to be re-used in other projects and improves the upgradability of the application. **Specialization** also plays an important role in the [object-capabilities model](../../learn/advanced/10-ocap.md) of the Cosmos SDK. * **Capabilities**: Most modules need to read and/or write to the store(s) of other modules. However, in an open-source environment, it is possible for some modules to be malicious. That is why module developers need to carefully think not only about how their module interacts with other modules, but also about how to give access to the module's store(s). The Cosmos SDK takes a capabilities-oriented approach to inter-module security. This means that each store defined by a module is accessed by a `key`, which is held by the module's [`keeper`](./06-keeper.md). This `keeper` defines how to access the store(s) and under what conditions. Access to the module's store(s) is done by passing a reference to the module's `keeper`. ## Main Components of Cosmos SDK Modules @@ -83,8 +83,8 @@ While there are no definitive guidelines for writing modules, here are some impo Modules are by convention defined in the `./x/` subfolder (e.g. the `bank` module will be defined in the `./x/bank` folder). They generally share the same core components: * A [`keeper`](./06-keeper.md), used to access the module's store(s) and update the state. -* A [`Msg` service](./02-messages-and-queries.md#messages), used to process messages when they are routed to the module by [`BaseApp`](../../develop/advanced/00-baseapp.md#message-routing) and trigger state-transitions. -* A [query service](./04-query-services.md), used to process user queries when they are routed to the module by [`BaseApp`](../../develop/advanced/00-baseapp.md#query-routing). +* A [`Msg` service](./02-messages-and-queries.md#messages), used to process messages when they are routed to the module by [`BaseApp`](../../learn/advanced/00-baseapp.md#message-routing) and trigger state-transitions. +* A [query service](./04-query-services.md), used to process user queries when they are routed to the module by [`BaseApp`](../../learn/advanced/00-baseapp.md#query-routing). * Interfaces, for end users to query the subset of the state defined by the module and create `message`s of the custom types defined in the module. In addition to these components, modules implement the `AppModule` interface in order to be managed by the [`module manager`](./01-module-manager.md). diff --git a/versioned_docs/version-0.50/build/building-modules/01-module-manager.md b/versioned_docs/version-0.50/build/building-modules/01-module-manager.md index cbad8367a..dfd17d5f5 100644 --- a/versioned_docs/version-0.50/build/building-modules/01-module-manager.md +++ b/versioned_docs/version-0.50/build/building-modules/01-module-manager.md @@ -5,7 +5,7 @@ sidebar_position: 1 # Module Manager :::note Synopsis -Cosmos SDK modules need to implement the [`AppModule` interfaces](#application-module-interfaces), in order to be managed by the application's [module manager](#module-manager). The module manager plays an important role in [`message` and `query` routing](../../develop/advanced/00-baseapp.md#routing), and allows application developers to set the order of execution of a variety of functions like [`PreBlocker`](../../develop/beginner/00-app-anatomy#preblocker) and [`BeginBlocker` and `EndBlocker`](../../develop/beginner/00-overview-app.md#begingblocker-and-endblocker). +Cosmos SDK modules need to implement the [`AppModule` interfaces](#application-module-interfaces), in order to be managed by the application's [module manager](#module-manager). The module manager plays an important role in [`message` and `query` routing](../../learn/advanced/00-baseapp.md#routing), and allows application developers to set the order of execution of a variety of functions like [`PreBlocker`](../../learn/beginner/00-app-anatomy#preblocker) and [`BeginBlocker` and `EndBlocker`](../../learn/beginner/00-app-anatomy.md#begingblocker-and-endblocker). ::: :::note Pre-requisite Readings @@ -46,7 +46,7 @@ The above interfaces are mostly embedding smaller interfaces (extension interfac * (legacy) [`module.HasInvariants`](#hasinvariants): The extension interface for registering invariants. * (legacy) [`module.HasConsensusVersion`](#hasconsensusversion): The extension interface for declaring a module consensus version. -The `AppModuleBasic` interface exists to define independent methods of the module, i.e. those that do not depend on other modules in the application. This allows for the construction of the basic application structure early in the application definition, generally in the `init()` function of the [main application file](../../develop/beginner/00-overview-app.md#core-application-file). +The `AppModuleBasic` interface exists to define independent methods of the module, i.e. those that do not depend on other modules in the application. This allows for the construction of the basic application structure early in the application definition, generally in the `init()` function of the [main application file](../../learn/beginner/00-app-anatomy.md#core-application-file). The `AppModule` interface exists to define inter-dependent module methods. Many modules need to interact with other modules, typically through [`keeper`s](./06-keeper.md), which means there is a need for an interface where modules list their `keeper`s and other methods that require a reference to another module's object. `AppModule` interface extension, such as `HasBeginBlocker` and `HasEndBlocker`, also enables the module manager to set the order of execution between module's methods like `BeginBlock` and `EndBlock`, which is important in cases where the order of execution between modules matters in the context of the application. @@ -224,7 +224,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/module/module.go https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/core/appmodule/module.go#L49-L52 ``` -* `Precommit(context.Context)`: This method gives module developers the option to implement logic that is automatically triggered during [`Commit'](../../develop/advanced/00-baseapp.md#commit) of each block using the [`finalizeblockstate`](../../develop/advanced/00-baseapp.md#state-updates) of the block to be committed. Implement empty if no logic needs to be triggered during `Commit` of each block for this module. +* `Precommit(context.Context)`: This method gives module developers the option to implement logic that is automatically triggered during [`Commit'](../../learn/advanced/00-baseapp.md#commit) of each block using the [`finalizeblockstate`](../../learn/advanced/00-baseapp.md#state-updates) of the block to be committed. Implement empty if no logic needs to be triggered during `Commit` of each block for this module. ### `HasPrepareCheckState` @@ -234,7 +234,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/core/appmodule/module. https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/core/appmodule/module.go#L49-L52 ``` -* `PrepareCheckState(context.Context)`: This method gives module developers the option to implement logic that is automatically triggered during [`Commit'](../../develop/advanced/00-baseapp.md#commit) of each block using the [`checkState`](../../develop/advanced/00-baseapp.md#state-updates) of the next block. Implement empty if no logic needs to be triggered during `Commit` of each block for this module. +* `PrepareCheckState(context.Context)`: This method gives module developers the option to implement logic that is automatically triggered during [`Commit'](../../learn/advanced/00-baseapp.md#commit) of each block using the [`checkState`](../../learn/advanced/00-baseapp.md#state-updates) of the next block. Implement empty if no logic needs to be triggered during `Commit` of each block for this module. ### Implementing the Application Module Interfaces @@ -272,15 +272,15 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/module/module.go It implements the following methods: -* `NewBasicManager(modules ...AppModuleBasic)`: Constructor function. It takes a list of the application's `AppModuleBasic` and builds a new `BasicManager`. This function is generally called in the `init()` function of [`app.go`](../../develop/beginner/00-overview-app.md#core-application-file) to quickly initialize the independent elements of the application's modules (click [here](https://github.com/cosmos/gaia/blob/main/app/app.go#L59-L74) to see an example). +* `NewBasicManager(modules ...AppModuleBasic)`: Constructor function. It takes a list of the application's `AppModuleBasic` and builds a new `BasicManager`. This function is generally called in the `init()` function of [`app.go`](../../learn/beginner/00-app-anatomy.md#core-application-file) to quickly initialize the independent elements of the application's modules (click [here](https://github.com/cosmos/gaia/blob/main/app/app.go#L59-L74) to see an example). * `NewBasicManagerFromManager(manager *Manager, customModuleBasics map[string]AppModuleBasic)`: Contructor function. It creates a new `BasicManager` from a `Manager`. The `BasicManager` will contain all `AppModuleBasic` from the `AppModule` manager using `CoreAppModuleBasicAdaptor` whenever possible. Module's `AppModuleBasic` can be overridden by passing a custom AppModuleBasic map -* `RegisterLegacyAminoCodec(cdc *codec.LegacyAmino)`: Registers the [`codec.LegacyAmino`s](../../develop/advanced/05-encoding.md#amino) of each of the application's `AppModuleBasic`. This function is usually called early on in the [application's construction](../../develop/beginner/00-overview-app.md#constructor). +* `RegisterLegacyAminoCodec(cdc *codec.LegacyAmino)`: Registers the [`codec.LegacyAmino`s](../../learn/advanced/05-encoding.md#amino) of each of the application's `AppModuleBasic`. This function is usually called early on in the [application's construction](../../learn/beginner/00-app-anatomy.md#constructor). * `RegisterInterfaces(registry codectypes.InterfaceRegistry)`: Registers interface types and implementations of each of the application's `AppModuleBasic`. * `DefaultGenesis(cdc codec.JSONCodec)`: Provides default genesis information for modules in the application by calling the [`DefaultGenesis(cdc codec.JSONCodec)`](./08-genesis.md#defaultgenesis) function of each module. It only calls the modules that implements the `HasGenesisBasics` interfaces. * `ValidateGenesis(cdc codec.JSONCodec, txEncCfg client.TxEncodingConfig, genesis map[string]json.RawMessage)`: Validates the genesis information modules by calling the [`ValidateGenesis(codec.JSONCodec, client.TxEncodingConfig, json.RawMessage)`](./08-genesis.md#validategenesis) function of modules implementing the `HasGenesisBasics` interface. * `RegisterGRPCGatewayRoutes(clientCtx client.Context, rtr *runtime.ServeMux)`: Registers gRPC routes for modules. -* `AddTxCommands(rootTxCmd *cobra.Command)`: Adds modules' transaction commands (defined as `GetTxCmd() *cobra.Command`) to the application's [`rootTxCommand`](../../develop/advanced/07-cli.md#transaction-commands). This function is usually called function from the `main.go` function of the [application's command-line interface](../../develop/advanced/07-cli.md). -* `AddQueryCommands(rootQueryCmd *cobra.Command)`: Adds modules' query commands (defined as `GetQueryCmd() *cobra.Command`) to the application's [`rootQueryCommand`](../../develop/advanced/07-cli.md#query-commands). This function is usually called function from the `main.go` function of the [application's command-line interface](../../develop/advanced/07-cli.md). +* `AddTxCommands(rootTxCmd *cobra.Command)`: Adds modules' transaction commands (defined as `GetTxCmd() *cobra.Command`) to the application's [`rootTxCommand`](../../learn/advanced/07-cli.md#transaction-commands). This function is usually called function from the `main.go` function of the [application's command-line interface](../../learn/advanced/07-cli.md). +* `AddQueryCommands(rootQueryCmd *cobra.Command)`: Adds modules' query commands (defined as `GetQueryCmd() *cobra.Command`) to the application's [`rootQueryCommand`](../../learn/advanced/07-cli.md#query-commands). This function is usually called function from the `main.go` function of the [application's command-line interface](../../learn/advanced/07-cli.md). ### `Manager` @@ -292,26 +292,26 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/module/module.go The module manager is used throughout the application whenever an action on a collection of modules is required. It implements the following methods: -* `NewManager(modules ...AppModule)`: Constructor function. It takes a list of the application's `AppModule`s and builds a new `Manager`. It is generally called from the application's main [constructor function](../../develop/beginner/00-overview-app.md#constructor-function). -* `SetOrderInitGenesis(moduleNames ...string)`: Sets the order in which the [`InitGenesis`](./08-genesis.md#initgenesis) function of each module will be called when the application is first started. This function is generally called from the application's main [constructor function](../../develop/beginner/00-overview-app.md#constructor-function). +* `NewManager(modules ...AppModule)`: Constructor function. It takes a list of the application's `AppModule`s and builds a new `Manager`. It is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderInitGenesis(moduleNames ...string)`: Sets the order in which the [`InitGenesis`](./08-genesis.md#initgenesis) function of each module will be called when the application is first started. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). To initialize modules successfully, module dependencies should be considered. For example, the `genutil` module must occur after `staking` module so that the pools are properly initialized with tokens from genesis accounts, the `genutils` module must also occur after `auth` so that it can access the params from auth, IBC's `capability` module should be initialized before all other modules so that it can initialize any capabilities. -* `SetOrderExportGenesis(moduleNames ...string)`: Sets the order in which the [`ExportGenesis`](./08-genesis.md#exportgenesis) function of each module will be called in case of an export. This function is generally called from the application's main [constructor function](../../develop/beginner/00-overview-app.md#constructor-function). -* `SetOrderPreBlockers(moduleNames ...string)`: Sets the order in which the `PreBlock()` function of each module will be called before `BeginBlock()` of all modules. This function is generally called from the application's main [constructor function](../../develop/beginner/00-overview-app.md#constructor-function). -* `SetOrderBeginBlockers(moduleNames ...string)`: Sets the order in which the `BeginBlock()` function of each module will be called at the beginning of each block. This function is generally called from the application's main [constructor function](../../develop/beginner/00-overview-app.md#constructor-function). -* `SetOrderEndBlockers(moduleNames ...string)`: Sets the order in which the `EndBlock()` function of each module will be called at the end of each block. This function is generally called from the application's main [constructor function](../../develop/beginner/00-overview-app.md#constructor-function). -* `SetOrderPrecommiters(moduleNames ...string)`: Sets the order in which the `Precommit()` function of each module will be called during commit of each block. This function is generally called from the application's main [constructor function](../../develop/beginner/00-overview-app.md#constructor-function). -* `SetOrderPrepareCheckStaters(moduleNames ...string)`: Sets the order in which the `PrepareCheckState()` function of each module will be called during commit of each block. This function is generally called from the application's main [constructor function](../../develop/beginner/00-overview-app.md#constructor-function). +* `SetOrderExportGenesis(moduleNames ...string)`: Sets the order in which the [`ExportGenesis`](./08-genesis.md#exportgenesis) function of each module will be called in case of an export. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderPreBlockers(moduleNames ...string)`: Sets the order in which the `PreBlock()` function of each module will be called before `BeginBlock()` of all modules. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderBeginBlockers(moduleNames ...string)`: Sets the order in which the `BeginBlock()` function of each module will be called at the beginning of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderEndBlockers(moduleNames ...string)`: Sets the order in which the `EndBlock()` function of each module will be called at the end of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderPrecommiters(moduleNames ...string)`: Sets the order in which the `Precommit()` function of each module will be called during commit of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderPrepareCheckStaters(moduleNames ...string)`: Sets the order in which the `PrepareCheckState()` function of each module will be called during commit of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). * `SetOrderMigrations(moduleNames ...string)`: Sets the order of migrations to be run. If not set then migrations will be run with an order defined in `DefaultMigrationsOrder`. * `RegisterInvariants(ir sdk.InvariantRegistry)`: Registers the [invariants](./07-invariants.md) of module implementing the `HasInvariants` interface. * `RegisterServices(cfg Configurator)`: Registers the services of modules implementing the `HasServices` interface. * `InitGenesis(ctx context.Context, cdc codec.JSONCodec, genesisData map[string]json.RawMessage)`: Calls the [`InitGenesis`](./08-genesis.md#initgenesis) function of each module when the application is first started, in the order defined in `OrderInitGenesis`. Returns an `abci.ResponseInitChain` to the underlying consensus engine, which can contain validator updates. * `ExportGenesis(ctx context.Context, cdc codec.JSONCodec)`: Calls the [`ExportGenesis`](./08-genesis.md#exportgenesis) function of each module, in the order defined in `OrderExportGenesis`. The export constructs a genesis file from a previously existing state, and is mainly used when a hard-fork upgrade of the chain is required. * `ExportGenesisForModules(ctx context.Context, cdc codec.JSONCodec, modulesToExport []string)`: Behaves the same as `ExportGenesis`, except takes a list of modules to export. -* `BeginBlock(ctx context.Context) error`: At the beginning of each block, this function is called from [`BaseApp`](../../develop/advanced/00-baseapp.md#beginblock) and, in turn, calls the [`BeginBlock`](./06-beginblock-endblock.md) function of each modules implementing the `appmodule.HasBeginBlocker` interface, in the order defined in `OrderBeginBlockers`. It creates a child [context](../../develop/advanced/02-context.md) with an event manager to aggregate [events](../../develop/advanced/08-events.md) emitted from each modules. -* `EndBlock(ctx context.Context) error`: At the end of each block, this function is called from [`BaseApp`](../../develop/advanced/00-baseapp.md#endblock) and, in turn, calls the [`EndBlock`](./06-beginblock-endblock.md) function of each modules implementing the `appmodule.HasEndBlocker` interface, in the order defined in `OrderEndBlockers`. It creates a child [context](../../develop/advanced/02-context.md) with an event manager to aggregate [events](../../develop/advanced/08-events.md) emitted from all modules. The function returns an `abci` which contains the aforementioned events, as well as validator set updates (if any). -* `EndBlock(context.Context) ([]abci.ValidatorUpdate, error)`: At the end of each block, this function is called from [`BaseApp`](../../develop/advanced/00-baseapp.md#endblock) and, in turn, calls the [`EndBlock`](./06-beginblock-endblock.md) function of each modules implementing the `module.HasABCIEndBlock` interface, in the order defined in `OrderEndBlockers`. It creates a child [context](../../develop/advanced/02-context.md) with an event manager to aggregate [events](../../develop/advanced/08-events.md) emitted from all modules. The function returns an `abci` which contains the aforementioned events, as well as validator set updates (if any). -* `Precommit(ctx context.Context)`: During [`Commit`](../../develop/advanced/00-baseapp.md#commit), this function is called from `BaseApp` immediately before the [`deliverState`](../../develop/advanced/00-baseapp.md#state-updates) is written to the underlying [`rootMultiStore`](../../develop/advanced/04-store.md#commitmultistore) and, in turn calls the `Precommit` function of each modules implementing the `HasPrecommit` interface, in the order defined in `OrderPrecommiters`. It creates a child [context](../../develop/advanced/02-context.md) where the underlying `CacheMultiStore` is that of the newly committed block's [`finalizeblockstate`](../../develop/advanced/00-baseapp.md#state-updates). -* `PrepareCheckState(ctx context.Context)`: During [`Commit`](../../develop/advanced/00-baseapp.md#commit), this function is called from `BaseApp` immediately after the [`deliverState`](../../develop/advanced/00-baseapp.md#state-updates) is written to the underlying [`rootMultiStore`](../../develop/advanced/04-store.md#commitmultistore) and, in turn calls the `PrepareCheckState` function of each module implementing the `HasPrepareCheckState` interface, in the order defined in `OrderPrepareCheckStaters`. It creates a child [context](../../develop/advanced/02-context.md) where the underlying `CacheMultiStore` is that of the next block's [`checkState`](../../develop/advanced/00-baseapp.md#state-updates). Writes to this state will be present in the [`checkState`](../../develop/advanced/00-baseapp.md#state-updates) of the next block, and therefore this method can be used to prepare the `checkState` for the next block. +* `BeginBlock(ctx context.Context) error`: At the beginning of each block, this function is called from [`BaseApp`](../../learn/advanced/00-baseapp.md#beginblock) and, in turn, calls the [`BeginBlock`](./06-beginblock-endblock.md) function of each modules implementing the `appmodule.HasBeginBlocker` interface, in the order defined in `OrderBeginBlockers`. It creates a child [context](../../learn/advanced/02-context.md) with an event manager to aggregate [events](../../learn/advanced/08-events.md) emitted from each modules. +* `EndBlock(ctx context.Context) error`: At the end of each block, this function is called from [`BaseApp`](../../learn/advanced/00-baseapp.md#endblock) and, in turn, calls the [`EndBlock`](./06-beginblock-endblock.md) function of each modules implementing the `appmodule.HasEndBlocker` interface, in the order defined in `OrderEndBlockers`. It creates a child [context](../../learn/advanced/02-context.md) with an event manager to aggregate [events](../../learn/advanced/08-events.md) emitted from all modules. The function returns an `abci` which contains the aforementioned events, as well as validator set updates (if any). +* `EndBlock(context.Context) ([]abci.ValidatorUpdate, error)`: At the end of each block, this function is called from [`BaseApp`](../../learn/advanced/00-baseapp.md#endblock) and, in turn, calls the [`EndBlock`](./06-beginblock-endblock.md) function of each modules implementing the `module.HasABCIEndBlock` interface, in the order defined in `OrderEndBlockers`. It creates a child [context](../../learn/advanced/02-context.md) with an event manager to aggregate [events](../../learn/advanced/08-events.md) emitted from all modules. The function returns an `abci` which contains the aforementioned events, as well as validator set updates (if any). +* `Precommit(ctx context.Context)`: During [`Commit`](../../learn/advanced/00-baseapp.md#commit), this function is called from `BaseApp` immediately before the [`deliverState`](../../learn/advanced/00-baseapp.md#state-updates) is written to the underlying [`rootMultiStore`](../../learn/advanced/04-store.md#commitmultistore) and, in turn calls the `Precommit` function of each modules implementing the `HasPrecommit` interface, in the order defined in `OrderPrecommiters`. It creates a child [context](../../learn/advanced/02-context.md) where the underlying `CacheMultiStore` is that of the newly committed block's [`finalizeblockstate`](../../learn/advanced/00-baseapp.md#state-updates). +* `PrepareCheckState(ctx context.Context)`: During [`Commit`](../../learn/advanced/00-baseapp.md#commit), this function is called from `BaseApp` immediately after the [`deliverState`](../../learn/advanced/00-baseapp.md#state-updates) is written to the underlying [`rootMultiStore`](../../learn/advanced/04-store.md#commitmultistore) and, in turn calls the `PrepareCheckState` function of each module implementing the `HasPrepareCheckState` interface, in the order defined in `OrderPrepareCheckStaters`. It creates a child [context](../../learn/advanced/02-context.md) where the underlying `CacheMultiStore` is that of the next block's [`checkState`](../../learn/advanced/00-baseapp.md#state-updates). Writes to this state will be present in the [`checkState`](../../learn/advanced/00-baseapp.md#state-updates) of the next block, and therefore this method can be used to prepare the `checkState` for the next block. Here's an example of a concrete integration within an `simapp`: diff --git a/versioned_docs/version-0.50/build/building-modules/02-messages-and-queries.md b/versioned_docs/version-0.50/build/building-modules/02-messages-and-queries.md index 3d92cd82a..573c35cd7 100644 --- a/versioned_docs/version-0.50/build/building-modules/02-messages-and-queries.md +++ b/versioned_docs/version-0.50/build/building-modules/02-messages-and-queries.md @@ -16,13 +16,13 @@ sidebar_position: 1 ## Messages -`Msg`s are objects whose end-goal is to trigger state-transitions. They are wrapped in [transactions](../../develop/advanced/01-transactions.md), which may contain one or more of them. +`Msg`s are objects whose end-goal is to trigger state-transitions. They are wrapped in [transactions](../../learn/advanced/01-transactions.md), which may contain one or more of them. -When a transaction is relayed from the underlying consensus engine to the Cosmos SDK application, it is first decoded by [`BaseApp`](../../develop/advanced/00-baseapp.md). Then, each message contained in the transaction is extracted and routed to the appropriate module via `BaseApp`'s `MsgServiceRouter` so that it can be processed by the module's [`Msg` service](./03-msg-services.md). For a more detailed explanation of the lifecycle of a transaction, click [here](../../develop/beginner/01-tx-lifecycle.md). +When a transaction is relayed from the underlying consensus engine to the Cosmos SDK application, it is first decoded by [`BaseApp`](../../learn/advanced/00-baseapp.md). Then, each message contained in the transaction is extracted and routed to the appropriate module via `BaseApp`'s `MsgServiceRouter` so that it can be processed by the module's [`Msg` service](./03-msg-services.md). For a more detailed explanation of the lifecycle of a transaction, click [here](../../learn/beginner/01-tx-lifecycle.md). ### `Msg` Services -Defining Protobuf `Msg` services is the recommended way to handle messages. A Protobuf `Msg` service should be created for each module, typically in `tx.proto` (see more info about [conventions and naming](../../develop/advanced/05-encoding.md#faq)). It must have an RPC service method defined for each message in the module. +Defining Protobuf `Msg` services is the recommended way to handle messages. A Protobuf `Msg` service should be created for each module, typically in `tx.proto` (see more info about [conventions and naming](../../learn/advanced/05-encoding.md#faq)). It must have an RPC service method defined for each message in the module. Each `Msg` service method must have exactly one argument, which must implement the `sdk.Msg` interface, and a Protobuf response. The naming convention is to call the RPC argument `Msg` and the RPC response `MsgResponse`. For example: @@ -89,7 +89,7 @@ In order for clients (CLI and grpc-gateway) to have these URLs registered, the C ## Queries -A `query` is a request for information made by end-users of applications through an interface and processed by a full-node. A `query` is received by a full-node through its consensus engine and relayed to the application via the ABCI. It is then routed to the appropriate module via `BaseApp`'s `QueryRouter` so that it can be processed by the module's query service (./04-query-services.md). For a deeper look at the lifecycle of a `query`, click [here](../../develop/beginner/02-query-lifecycle.md). +A `query` is a request for information made by end-users of applications through an interface and processed by a full-node. A `query` is received by a full-node through its consensus engine and relayed to the application via the ABCI. It is then routed to the appropriate module via `BaseApp`'s `QueryRouter` so that it can be processed by the module's query service (./04-query-services.md). For a deeper look at the lifecycle of a `query`, click [here](../../learn/beginner/02-query-lifecycle.md). ### gRPC Queries @@ -115,14 +115,14 @@ queryCategory/queryRoute/queryType/arg1/arg2/... where: -* `queryCategory` is the category of the `query`, typically `custom` for module queries. It is used to differentiate between different kinds of queries within `BaseApp`'s [`Query` method](../../develop/advanced/00-baseapp.md#query). -* `queryRoute` is used by `BaseApp`'s [`queryRouter`](../../develop/advanced/00-baseapp.md#query-routing) to map the `query` to its module. Usually, `queryRoute` should be the name of the module. +* `queryCategory` is the category of the `query`, typically `custom` for module queries. It is used to differentiate between different kinds of queries within `BaseApp`'s [`Query` method](../../learn/advanced/00-baseapp.md#query). +* `queryRoute` is used by `BaseApp`'s [`queryRouter`](../../learn/advanced/00-baseapp.md#query-routing) to map the `query` to its module. Usually, `queryRoute` should be the name of the module. * `queryType` is used by the module's [`querier`](./04-query-services.md#legacy-queriers) to map the `query` to the appropriate `querier function` within the module. * `args` are the actual arguments needed to process the `query`. They are filled out by the end-user. Note that for bigger queries, you might prefer passing arguments in the `Data` field of the request `req` instead of the `path`. The `path` for each `query` must be defined by the module developer in the module's [command-line interface file](./09-module-interfaces.md#query-commands).Overall, there are 3 mains components module developers need to implement in order to make the subset of the state defined by their module queryable: -* A [`querier`](./04-query-services.md#legacy-queriers), to process the `query` once it has been [routed to the module](../../develop/advanced/00-baseapp.md#query-routing). +* A [`querier`](./04-query-services.md#legacy-queriers), to process the `query` once it has been [routed to the module](../../learn/advanced/00-baseapp.md#query-routing). * [Query commands](./09-module-interfaces.md#query-commands) in the module's CLI file, where the `path` for each `query` is specified. * `query` return types. Typically defined in a file `types/querier.go`, they specify the result type of each of the module's `queries`. These custom types must implement the `String()` method of [`fmt.Stringer`](https://pkg.go.dev/fmt#Stringer). diff --git a/versioned_docs/version-0.50/build/building-modules/03-msg-services.md b/versioned_docs/version-0.50/build/building-modules/03-msg-services.md index ead334108..421e53dec 100644 --- a/versioned_docs/version-0.50/build/building-modules/03-msg-services.md +++ b/versioned_docs/version-0.50/build/building-modules/03-msg-services.md @@ -5,7 +5,7 @@ sidebar_position: 1 # `Msg` Services :::note Synopsis -A Protobuf `Msg` service processes [messages](./02-messages-and-queries.md#messages). Protobuf `Msg` services are specific to the module in which they are defined, and only process messages defined within the said module. They are called from `BaseApp` during [`DeliverTx`](../../develop/advanced/00-baseapp.md#delivertx). +A Protobuf `Msg` service processes [messages](./02-messages-and-queries.md#messages). Protobuf `Msg` services are specific to the module in which they are defined, and only process messages defined within the said module. They are called from `BaseApp` during [`DeliverTx`](../../learn/advanced/00-baseapp.md#delivertx). ::: :::note Pre-requisite Readings @@ -71,7 +71,7 @@ After the validation is successful, the `msgServer` method uses the [`keeper`](. ### Events -Before returning, `msgServer` methods generally emit one or more [events](../../develop/advanced/08-events.md) by using the `EventManager` held in the `ctx`. Use the new `EmitTypedEvent` function that uses protobuf-based event types: +Before returning, `msgServer` methods generally emit one or more [events](../../learn/advanced/08-events.md) by using the `EventManager` held in the `ctx`. Use the new `EmitTypedEvent` function that uses protobuf-based event types: ```go ctx.EventManager().EmitTypedEvent( @@ -90,7 +90,7 @@ ctx.EventManager().EmitEvent( ) ``` -These events are relayed back to the underlying consensus engine and can be used by service providers to implement services around the application. Click [here](../../develop/advanced/08-events.md) to learn more about events. +These events are relayed back to the underlying consensus engine and can be used by service providers to implement services around the application. Click [here](../../learn/advanced/08-events.md) to learn more about events. The invoked `msgServer` method returns a `proto.Message` response and an `error`. These return values are then wrapped into an `*sdk.Result` or an `error` using `sdk.WrapServiceResult(ctx context.Context, res proto.Message, err error)`: @@ -110,7 +110,7 @@ This diagram shows a typical structure of a Protobuf `Msg` service, and how the ## Telemetry -New [telemetry metrics](../../develop/advanced/09-telemetry.md) can be created from `msgServer` methods when handling messages. +New [telemetry metrics](../../learn/advanced/09-telemetry.md) can be created from `msgServer` methods when handling messages. This is an example from the `x/auth/vesting` module: diff --git a/versioned_docs/version-0.50/build/building-modules/04-query-services.md b/versioned_docs/version-0.50/build/building-modules/04-query-services.md index 29542d86c..a787a0c22 100644 --- a/versioned_docs/version-0.50/build/building-modules/04-query-services.md +++ b/versioned_docs/version-0.50/build/building-modules/04-query-services.md @@ -5,7 +5,7 @@ sidebar_position: 1 # Query Services :::note Synopsis -A Protobuf Query service processes [`queries`](./02-messages-and-queries.md#queries). Query services are specific to the module in which they are defined, and only process `queries` defined within said module. They are called from `BaseApp`'s [`Query` method](../../develop/advanced/00-baseapp.md#query). +A Protobuf Query service processes [`queries`](./02-messages-and-queries.md#queries). Query services are specific to the module in which they are defined, and only process `queries` defined within said module. They are called from `BaseApp`'s [`Query` method](../../learn/advanced/00-baseapp.md#query). ::: :::note Pre-requisite Readings diff --git a/versioned_docs/version-0.50/build/building-modules/06-beginblock-endblock.md b/versioned_docs/version-0.50/build/building-modules/06-beginblock-endblock.md index 35b5bb350..a8eafdf67 100644 --- a/versioned_docs/version-0.50/build/building-modules/06-beginblock-endblock.md +++ b/versioned_docs/version-0.50/build/building-modules/06-beginblock-endblock.md @@ -5,7 +5,7 @@ sidebar_position: 1 # BeginBlocker and EndBlocker :::note Synopsis -`BeginBlocker` and `EndBlocker` are optional methods module developers can implement in their module. They will be triggered at the beginning and at the end of each block respectively, when the [`BeginBlock`](../../develop/advanced/00-baseapp.md#beginblock) and [`EndBlock`](../../develop/advanced/00-baseapp.md#endblock) ABCI messages are received from the underlying consensus engine. +`BeginBlocker` and `EndBlocker` are optional methods module developers can implement in their module. They will be triggered at the beginning and at the end of each block respectively, when the [`BeginBlock`](../../learn/advanced/00-baseapp.md#beginblock) and [`EndBlock`](../../learn/advanced/00-baseapp.md#endblock) ABCI messages are received from the underlying consensus engine. ::: :::note Pre-requisite Readings @@ -24,9 +24,9 @@ When needed, `BeginBlocker` and `EndBlocker` are implemented as part of the [`Ha The actual implementation of `BeginBlocker` and `EndBlocker` in `abci.go` are very similar to that of a [`Msg` service](./03-msg-services.md): -* They generally use the [`keeper`](./06-keeper.md) and [`ctx`](../../develop/advanced/02-context.md) to retrieve information about the latest state. +* They generally use the [`keeper`](./06-keeper.md) and [`ctx`](../../learn/advanced/02-context.md) to retrieve information about the latest state. * If needed, they use the `keeper` and `ctx` to trigger state-transitions. -* If needed, they can emit [`events`](../../develop/advanced/08-events.md) via the `ctx`'s `EventManager`. +* If needed, they can emit [`events`](../../learn/advanced/08-events.md) via the `ctx`'s `EventManager`. A specific type of `EndBlocker` is available to return validator updates to the underlying consensus engine in the form of an [`[]abci.ValidatorUpdates`](https://docs.cometbft.com/v0.37/spec/abci/abci++_methods#endblock). This is the preferred way to implement custom validator changes. diff --git a/versioned_docs/version-0.50/build/building-modules/06-keeper.md b/versioned_docs/version-0.50/build/building-modules/06-keeper.md index a2a77e0ba..399ec648c 100644 --- a/versioned_docs/version-0.50/build/building-modules/06-keeper.md +++ b/versioned_docs/version-0.50/build/building-modules/06-keeper.md @@ -18,9 +18,9 @@ sidebar_position: 1 The Cosmos SDK is a framework that makes it easy for developers to build complex decentralized applications from scratch, mainly by composing modules together. As the ecosystem of open-source modules for the Cosmos SDK expands, it will become increasingly likely that some of these modules contain vulnerabilities, as a result of the negligence or malice of their developer. -The Cosmos SDK adopts an [object-capabilities-based approach](../../develop/advanced/10-ocap.md) to help developers better protect their application from unwanted inter-module interactions, and `keeper`s are at the core of this approach. A `keeper` can be considered quite literally to be the gatekeeper of a module's store(s). Each store (typically an [`IAVL` Store](../../develop/advanced/04-store.md#iavl-store)) defined within a module comes with a `storeKey`, which grants unlimited access to it. The module's `keeper` holds this `storeKey` (which should otherwise remain unexposed), and defines [methods](#implementing-methods) for reading and writing to the store(s). +The Cosmos SDK adopts an [object-capabilities-based approach](../../learn/advanced/10-ocap.md) to help developers better protect their application from unwanted inter-module interactions, and `keeper`s are at the core of this approach. A `keeper` can be considered quite literally to be the gatekeeper of a module's store(s). Each store (typically an [`IAVL` Store](../../learn/advanced/04-store.md#iavl-store)) defined within a module comes with a `storeKey`, which grants unlimited access to it. The module's `keeper` holds this `storeKey` (which should otherwise remain unexposed), and defines [methods](#implementing-methods) for reading and writing to the store(s). -The core idea behind the object-capabilities approach is to only reveal what is necessary to get the work done. In practice, this means that instead of handling permissions of modules through access-control lists, module `keeper`s are passed a reference to the specific instance of the other modules' `keeper`s that they need to access (this is done in the [application's constructor function](../../develop/beginner/00-overview-app.md#constructor-function)). As a consequence, a module can only interact with the subset of state defined in another module via the methods exposed by the instance of the other module's `keeper`. This is a great way for developers to control the interactions that their own module can have with modules developed by external developers. +The core idea behind the object-capabilities approach is to only reveal what is necessary to get the work done. In practice, this means that instead of handling permissions of modules through access-control lists, module `keeper`s are passed a reference to the specific instance of the other modules' `keeper`s that they need to access (this is done in the [application's constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function)). As a consequence, a module can only interact with the subset of state defined in another module via the methods exposed by the instance of the other module's `keeper`. This is a great way for developers to control the interactions that their own module can have with modules developed by external developers. ## Type Definition @@ -47,10 +47,11 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/keeper/keepe Let us go through the different parameters: * An expected `keeper` is a `keeper` external to a module that is required by the internal `keeper` of said module. External `keeper`s are listed in the internal `keeper`'s type definition as interfaces. These interfaces are themselves defined in an `expected_keepers.go` file in the root of the module's folder. In this context, interfaces are used to reduce the number of dependencies, as well as to facilitate the maintenance of the module itself. -* `storeKey`s grant access to the store(s) of the [multistore](../../develop/advanced/04-store.md) managed by the module. They should always remain unexposed to external modules. -* `cdc` is the [codec](../develop/advanced/05-encoding.md) used to marshall and unmarshall structs to/from `[]byte`. The `cdc` can be any of `codec.BinaryCodec`, `codec.JSONCodec` or `codec.Codec` based on your requirements. It can be either a proto or amino codec as long as they implement these interfaces. The authority listed is a module account or user account that has the right to change module level parameters. Previously this was handled by the param module, which has been deprecated. +* `storeKey`s grant access to the store(s) of the [multistore](../../learn/advanced/04-store.md) managed by the module. They should always remain unexposed to external modules. +* `cdc` is the [codec](../../learn/advanced/05-encoding.md) used to marshall and unmarshall structs to/from `[]byte`. The `cdc` can be any of `codec.BinaryCodec`, `codec.JSONCodec` or `codec.Codec` based on your requirements. It can be either a proto or amino codec as long as they implement these interfaces. +* The authority listed is a module account or user account that has the right to change module level parameters. Previously this was handled by the param module, which has been deprecated. -Of course, it is possible to define different types of internal `keeper`s for the same module (e.g. a read-only `keeper`). Each type of `keeper` comes with its own constructor function, which is called from the [application's constructor function](../../develop/beginner/00-overview-app.md). This is where `keeper`s are instantiated, and where developers make sure to pass correct instances of modules' `keeper`s to other modules that require them. +Of course, it is possible to define different types of internal `keeper`s for the same module (e.g. a read-only `keeper`). Each type of `keeper` comes with its own constructor function, which is called from the [application's constructor function](../../learn/beginner/00-app-anatomy.md). This is where `keeper`s are instantiated, and where developers make sure to pass correct instances of modules' `keeper`s to other modules that require them. ## Implementing Methods @@ -82,7 +83,7 @@ and the method will go through the following steps: For more, see an example of `keeper`'s [methods implementation from the `staking` module](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/keeper/keeper.go). -The [module `KVStore`](../../develop/advanced/04-store.md#kvstore-and-commitkvstore-interfaces) also provides an `Iterator()` method which returns an `Iterator` object to iterate over a domain of keys. +The [module `KVStore`](../../learn/advanced/04-store.md#kvstore-and-commitkvstore-interfaces) also provides an `Iterator()` method which returns an `Iterator` object to iterate over a domain of keys. This is an example from the `auth` module to iterate accounts: diff --git a/versioned_docs/version-0.50/build/building-modules/07-invariants.md b/versioned_docs/version-0.50/build/building-modules/07-invariants.md index fe18ba6e1..018796f71 100644 --- a/versioned_docs/version-0.50/build/building-modules/07-invariants.md +++ b/versioned_docs/version-0.50/build/building-modules/07-invariants.md @@ -79,7 +79,7 @@ Typically, this interface is implemented in the `keeper` of a specific module. T https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/crisis/keeper/keeper.go#L48-L50 ``` -The `InvariantRegistry` is therefore typically instantiated by instantiating the `keeper` of the `crisis` module in the [application's constructor function](../../develop/beginner/00-overview-app.md#constructor-function). +The `InvariantRegistry` is therefore typically instantiated by instantiating the `keeper` of the `crisis` module in the [application's constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). `Invariant`s can be checked manually via [`message`s](./02-messages-and-queries.md), but most often they are checked automatically at the end of each block. Here is an example from the `crisis` module: diff --git a/versioned_docs/version-0.50/build/building-modules/08-genesis.md b/versioned_docs/version-0.50/build/building-modules/08-genesis.md index 9fdb2cef1..7abb21fb6 100644 --- a/versioned_docs/version-0.50/build/building-modules/08-genesis.md +++ b/versioned_docs/version-0.50/build/building-modules/08-genesis.md @@ -17,7 +17,7 @@ Modules generally handle a subset of the state and, as such, they need to define ## Type Definition -The subset of the genesis state defined from a given module is generally defined in a `genesis.proto` file ([more info](../../develop/advanced/05-encoding.md#gogoproto) on how to define protobuf messages). The struct defining the module's subset of the genesis state is usually called `GenesisState` and contains all the module-related values that need to be initialized during the genesis process. +The subset of the genesis state defined from a given module is generally defined in a `genesis.proto` file ([more info](../../learn/advanced/05-encoding.md#gogoproto) on how to define protobuf messages). The struct defining the module's subset of the genesis state is usually called `GenesisState` and contains all the module-related values that need to be initialized during the genesis process. See an example of `GenesisState` protobuf message definition from the `auth` module: @@ -49,9 +49,9 @@ Other than the methods related directly to `GenesisState`, module developers are ### `InitGenesis` -The `InitGenesis` method is executed during [`InitChain`](../../develop/advanced/00-baseapp.md#initchain) when the application is first started. Given a `GenesisState`, it initializes the subset of the state managed by the module by using the module's [`keeper`](./06-keeper.md) setter function on each parameter within the `GenesisState`. +The `InitGenesis` method is executed during [`InitChain`](../../learn/advanced/00-baseapp.md#initchain) when the application is first started. Given a `GenesisState`, it initializes the subset of the state managed by the module by using the module's [`keeper`](./06-keeper.md) setter function on each parameter within the `GenesisState`. -The [module manager](./01-module-manager.md#manager) of the application is responsible for calling the `InitGenesis` method of each of the application's modules in order. This order is set by the application developer via the manager's `SetOrderGenesisMethod`, which is called in the [application's constructor function](../../develop/beginner/00-overview-app.md#constructor-function). +The [module manager](./01-module-manager.md#manager) of the application is responsible for calling the `InitGenesis` method of each of the application's modules in order. This order is set by the application developer via the manager's `SetOrderGenesisMethod`, which is called in the [application's constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). See an example of `InitGenesis` from the `auth` module: diff --git a/versioned_docs/version-0.50/build/building-modules/09-module-interfaces.md b/versioned_docs/version-0.50/build/building-modules/09-module-interfaces.md index 262442661..4552baef2 100644 --- a/versioned_docs/version-0.50/build/building-modules/09-module-interfaces.md +++ b/versioned_docs/version-0.50/build/building-modules/09-module-interfaces.md @@ -16,11 +16,11 @@ This document details how to build CLI and REST interfaces for a module. Example ## CLI -One of the main interfaces for an application is the [command-line interface](../../develop/advanced/07-cli.md). This entrypoint adds commands from the application's modules enabling end-users to create [**messages**](./02-messages-and-queries.md#messages) wrapped in transactions and [**queries**](./02-messages-and-queries.md#queries). The CLI files are typically found in the module's `./client/cli` folder. +One of the main interfaces for an application is the [command-line interface](../../learn/advanced/07-cli.md). This entrypoint adds commands from the application's modules enabling end-users to create [**messages**](./02-messages-and-queries.md#messages) wrapped in transactions and [**queries**](./02-messages-and-queries.md#queries). The CLI files are typically found in the module's `./client/cli` folder. ### Transaction Commands -In order to create messages that trigger state changes, end-users must create [transactions](../../develop/advanced/01-transactions.md) that wrap and deliver the messages. A transaction command creates a transaction that includes one or more messages. +In order to create messages that trigger state changes, end-users must create [transactions](../../learn/advanced/01-transactions.md) that wrap and deliver the messages. A transaction command creates a transaction that includes one or more messages. Transaction commands typically have their own `tx.go` file that lives within the module's `./client/cli` folder. The commands are specified in getter functions and the name of the function should include the name of the command. @@ -101,7 +101,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/module.go#L84-L ### Flags -[Flags](../../develop/advanced/07-cli.md#flags) allow users to customize commands. `--fees` and `--gas-prices` are examples of flags that allow users to set the [fees](../../develop/beginner/04-gas-fees.md) and gas prices for their transactions. +[Flags](../../learn/advanced/07-cli.md#flags) allow users to customize commands. `--fees` and `--gas-prices` are examples of flags that allow users to set the [fees](../../learn/beginner/04-gas-fees.md) and gas prices for their transactions. Flags that are specific to a module are typically created in a `flags.go` file in the module's `./client/cli` folder. When creating a flag, developers set the value type, the name of the flag, the default value, and a description about the flag. Developers also have the option to mark flags as _required_ so that an error is thrown if the user does not include a value for the flag. diff --git a/versioned_docs/version-0.50/build/building-modules/13-upgrade.md b/versioned_docs/version-0.50/build/building-modules/13-upgrade.md index 703b4cd28..908a6a06e 100644 --- a/versioned_docs/version-0.50/build/building-modules/13-upgrade.md +++ b/versioned_docs/version-0.50/build/building-modules/13-upgrade.md @@ -5,12 +5,12 @@ sidebar_position: 1 # Upgrading Modules :::note Synopsis -[In-Place Store Migrations](../../develop/advanced/15-upgrade.md) allow your modules to upgrade to new versions that include breaking changes. This document outlines how to build modules to take advantage of this functionality. +[In-Place Store Migrations](../../learn/advanced/15-upgrade.md) allow your modules to upgrade to new versions that include breaking changes. This document outlines how to build modules to take advantage of this functionality. ::: :::note Pre-requisite Readings -* [In-Place Store Migration](../../develop/advanced/15-upgrade.md) +* [In-Place Store Migration](../../learn/advanced/15-upgrade.md) ::: diff --git a/versioned_docs/version-0.50/build/building-modules/14-simulator.md b/versioned_docs/version-0.50/build/building-modules/14-simulator.md index 5360709f6..8fe2d92c3 100644 --- a/versioned_docs/version-0.50/build/building-modules/14-simulator.md +++ b/versioned_docs/version-0.50/build/building-modules/14-simulator.md @@ -6,7 +6,7 @@ sidebar_position: 1 :::note Pre-requisite Readings -* [Cosmos Blockchain Simulator](../../develop/advanced/12-simulation.md) +* [Cosmos Blockchain Simulator](../../learn/advanced/12-simulation.md) ::: ## Synopsis @@ -62,7 +62,7 @@ Operations are one of the crucial parts of the Cosmos SDK simulation. They are t (`Msg`) that are simulated with random field values. The sender of the operation is also assigned randomly. -Operations on the simulation are simulated using the full [transaction cycle](../../develop/advanced/01-transactions.md) of a +Operations on the simulation are simulated using the full [transaction cycle](../../learn/advanced/01-transactions.md) of a `ABCI` application that exposes the `BaseApp`. Shown below is how weights are set: diff --git a/versioned_docs/version-0.50/build/building-modules/17-preblock.md b/versioned_docs/version-0.50/build/building-modules/17-preblock.md new file mode 100644 index 000000000..a79646bd4 --- /dev/null +++ b/versioned_docs/version-0.50/build/building-modules/17-preblock.md @@ -0,0 +1,31 @@ +--- +sidebar_position: 1 +--- + +# PreBlocker + +:::note Synopsis +`PreBlocker` is optional method module developers can implement in their module. They will be triggered before [`BeginBlock`](../../learn/advanced/00-baseapp.md#beginblock). +::: + +:::note Pre-requisite Readings + +* [Module Manager](./01-module-manager.md) + +::: + +## PreBlocker + +There are two semantics around the new lifecycle method: + +- It runs before the `BeginBlocker` of all modules +- It can modify consensus parameters in storage, and signal the caller through the return value. + +When it returns `ConsensusParamsChanged=true`, the caller must refresh the consensus parameter in the deliver context: +``` +app.finalizeBlockState.ctx = app.finalizeBlockState.ctx.WithConsensusParams(app.GetConsensusParams()) +``` + +The new ctx must be passed to all the other lifecycle methods. + + diff --git a/versioned_docs/version-0.50/build/building-modules/_category_.json b/versioned_docs/version-0.50/build/building-modules/_category_.json index 8dc3f9a94..2d50f8b3e 100644 --- a/versioned_docs/version-0.50/build/building-modules/_category_.json +++ b/versioned_docs/version-0.50/build/building-modules/_category_.json @@ -1,5 +1,5 @@ { "label": "Building Modules", - "position": 3, + "position": 1, "link": null } \ No newline at end of file diff --git a/versioned_docs/version-0.50/build/migrations/02-upgrading.md b/versioned_docs/version-0.50/build/migrations/02-upgrading.md index 91d8d7203..f8924cd08 100644 --- a/versioned_docs/version-0.50/build/migrations/02-upgrading.md +++ b/versioned_docs/version-0.50/build/migrations/02-upgrading.md @@ -67,6 +67,43 @@ allows an application to define handlers for these methods via `ExtendVoteHandle and `VerifyVoteExtensionHandler` respectively. Please see [here](https://docs.cosmos.network/v0.50/building-apps/vote-extensions) for more info. +#### Set PreBlocker + +A `SetPreBlocker` method has been added to BaseApp. This is essential for BaseApp to run `PreBlock` which runs before begin blocker other modules, and allows to modify consensus parameters, and the changes are visible to the following state machine logics. +Read more about other use cases [here](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-068-preblock.md). + +`depinject` / app v2 users need to add `x/upgrade` in their `app_config.go` / `app.yml`: + +```diff ++ PreBlockers: []string{ ++ upgradetypes.ModuleName, ++ }, +BeginBlockers: []string{ +- upgradetypes.ModuleName, + minttypes.ModuleName, +} +``` + +When using (legacy) application wiring, the following must be added to `app.go`: + +```diff ++app.ModuleManager.SetOrderPreBlockers( ++ upgradetypes.ModuleName, ++) + +app.ModuleManager.SetOrderBeginBlockers( +- upgradetypes.ModuleName, +) + ++ app.SetPreBlocker(app.PreBlocker) + +// ... // + ++func (app *SimApp) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeBlock) (*sdk.ResponsePreBlock, error) { ++ return app.ModuleManager.PreBlock(ctx, req) ++} +``` + #### Events The log section of `abci.TxResult` is not populated in the case of successful @@ -292,7 +329,7 @@ The return type of the interface method `TxConfig.SignModeHandler()` has been ch * `x/slashing` * `x/upgrade` -* BeginBlock and EndBlock have changed their signature, so it is important that any module implementing them are updated accordingly. +* `BeginBlock` and `EndBlock` have changed their signature, so it is important that any module implementing them are updated accordingly. ```diff - BeginBlock(sdk.Context, abci.RequestBeginBlock) @@ -304,18 +341,38 @@ The return type of the interface method `TxConfig.SignModeHandler()` has been ch + EndBlock(context.Context) error ``` -In case a module requires to return `abci.ValidatorUpdate` from `EndBlock`, it can use the `HasABCIEndblock` interface instead. +In case a module requires to return `abci.ValidatorUpdate` from `EndBlock`, it can use the `HasABCIEndBlock` interface instead. ```diff - EndBlock(sdk.Context, abci.RequestEndBlock) []abci.ValidatorUpdate + EndBlock(context.Context) ([]abci.ValidatorUpdate, error) ``` -`GetSigners()` is no longer required to be implemented on `Msg` types. The SDK will automatically infer the signers from the `Signer` field on the message. The signer field is required on all messages unless using a custom signer function. +:::tip +It is possible to ensure that a module implements the correct interfaces by using compiler assertions in your `x/{moduleName}/module.go`: +```go +var ( + _ module.AppModuleBasic = (*AppModule)(nil) + _ module.AppModuleSimulation = (*AppModule)(nil) + _ module.HasGenesis = (*AppModule)(nil) + + _ appmodule.AppModule = (*AppModule)(nil) + _ appmodule.HasBeginBlocker = (*AppModule)(nil) + _ appmodule.HasEndBlocker = (*AppModule)(nil) + ... +) +``` + +Read more on those interfaces [here](https://docs.cosmos.network/v0.50/building-modules/module-manager#application-module-interfaces). + +::: + +* `GetSigners()` is no longer required to be implemented on `Msg` types. The SDK will automatically infer the signers from the `Signer` field on the message. The signer field is required on all messages unless using a custom signer function. -To find out more please read the [signer field](./05-protobuf-annotations.md#signer) & [here](https://github.com/cosmos/cosmos-sdk/blob/7352d0bce8e72121e824297df453eb1059c28da8/docs/docs/build/building-modules/02-messages-and-queries.md#L40)documentation. +To find out more please read the [signer field](../../build/building-modules/05-protobuf-annotations.md#signer) & [here](https://github.com/cosmos/cosmos-sdk/blob/7352d0bce8e72121e824297df453eb1059c28da8/docs/docs/build/building-modules/02-messages-and-queries.md#L40) documentation. + #### `x/auth` For ante handler construction via `ante.NewAnteHandler`, the field `ante.HandlerOptions.SignModeHandler` has been updated to `x/tx/signing/HandlerMap` from `x/auth/signing/SignModeHandler`. Callers typically fetch this value from `client.TxConfig.SignModeHandler()` (which is also changed) so this change should be transparent to most users. diff --git a/versioned_docs/version-0.50/build/migrations/_category_.json b/versioned_docs/version-0.50/build/migrations/_category_.json index 95d1d4717..5a06c3ebb 100644 --- a/versioned_docs/version-0.50/build/migrations/_category_.json +++ b/versioned_docs/version-0.50/build/migrations/_category_.json @@ -1,5 +1,5 @@ { "label": "Migrations", - "position": 6, + "position": 3, "link": null } diff --git a/versioned_docs/version-0.50/build/modules/_category_.json b/versioned_docs/version-0.50/build/modules/_category_.json index 7202a9479..72d229c0b 100644 --- a/versioned_docs/version-0.50/build/modules/_category_.json +++ b/versioned_docs/version-0.50/build/modules/_category_.json @@ -1,5 +1,5 @@ { "label": "Modules", - "position": 7, + "position": 2, "link": null } diff --git a/versioned_docs/version-0.50/build/modules/auth/README.md b/versioned_docs/version-0.50/build/modules/auth/README.md index 4b5a2ad1f..1d5e22b5d 100644 --- a/versioned_docs/version-0.50/build/modules/auth/README.md +++ b/versioned_docs/version-0.50/build/modules/auth/README.md @@ -481,17 +481,6 @@ simd tx broadcast tx.signed.json More information about the `broadcast` command can be found running `simd tx broadcast --help`. -#### `aux-to-fee` - -The `aux-to-fee` comamnds includes the aux signer data in the tx, broadcast the tx, and sends the tip amount to the broadcaster. -[Learn more about tip transaction](https://docs.cosmos.network/main/core/tips). - -```bash -# simd tx bank send --aux (optional: --tip --tipper ) -simd tx aux-to-fee tx.aux.signed.json -``` - -More information about the `aux-to-fee` command can be found running `simd tx aux-to-fee --help`. ### gRPC diff --git a/versioned_docs/version-0.50/build/modules/bank/README.md b/versioned_docs/version-0.50/build/modules/bank/README.md index 340822aa3..990634b8b 100644 --- a/versioned_docs/version-0.50/build/modules/bank/README.md +++ b/versioned_docs/version-0.50/build/modules/bank/README.md @@ -236,8 +236,12 @@ accounts. The send keeper does not alter the total supply (mint or burn coins). type SendKeeper interface { ViewKeeper - InputOutputCoins(ctx context.Context, inputs types.Input, outputs []types.Output) error - SendCoins(ctx context.Context, fromAddr sdk.AccAddress, toAddr sdk.AccAddress, amt sdk.Coins) error + AppendSendRestriction(restriction SendRestrictionFn) + PrependSendRestriction(restriction SendRestrictionFn) + ClearSendRestriction() + + InputOutputCoins(ctx context.Context, input types.Input, outputs []types.Output) error + SendCoins(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) error GetParams(ctx context.Context) types.Params SetParams(ctx context.Context, params types.Params) error @@ -256,6 +260,86 @@ type SendKeeper interface { } ``` +#### Send Restrictions + +The `SendKeeper` applies a `SendRestrictionFn` before each transfer of funds. + +```golang +// A SendRestrictionFn can restrict sends and/or provide a new receiver address. +type SendRestrictionFn func(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) (newToAddr sdk.AccAddress, err error) +``` + +After the `SendKeeper` (or `BaseKeeper`) has been created, send restrictions can be added to it using the `AppendSendRestriction` or `PrependSendRestriction` functions. +Both functions compose the provided restriction with any previously provided restrictions. +`AppendSendRestriction` adds the provided restriction to be run after any previously provided send restrictions. +`PrependSendRestriction` adds the restriction to be run before any previously provided send restrictions. +The composition will short-circuit when an error is encountered. I.e. if the first one returns an error, the second is not run. + +During `SendCoins`, the send restriction is applied after coins are removed from the from address, but before adding them to the to address. +During `InputOutputCoins`, the send restriction is applied after the input coins are removed and once for each output before the funds are added. + +A send restriction function should make use of a custom value in the context to allow bypassing that specific restriction. + +For example, in your module's keeper package, you'd define the send restriction function: + +```golang +var _ banktypes.SendRestrictionFn = Keeper{}.SendRestrictionFn + +func (k Keeper) SendRestrictionFn(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) (sdk.AccAddress, error) { + // Bypass if the context says to. + if mymodule.HasBypass(ctx) { + return toAddr, nil + } + + // Your custom send restriction logic goes here. + return nil, errors.New("not implemented") +} +``` + +The bank keeper should be provided to your keeper's constructor so the send restriction can be added to it: + +```golang +func NewKeeper(cdc codec.BinaryCodec, storeKey storetypes.StoreKey, bankKeeper mymodule.BankKeeper) Keeper { + rv := Keeper{/*...*/} + bankKeeper.AppendSendRestriction(rv.SendRestrictionFn) + return rv +} +``` + +Then, in the `mymodule` package, define the context helpers: + +```golang +const bypassKey = "bypass-mymodule-restriction" + +// WithBypass returns a new context that will cause the mymodule bank send restriction to be skipped. +func WithBypass(ctx context.Context) context.Context { + return sdk.UnwrapSDKContext(ctx).WithValue(bypassKey, true) +} + +// WithoutBypass returns a new context that will cause the mymodule bank send restriction to not be skipped. +func WithoutBypass(ctx context.Context) context.Context { + return sdk.UnwrapSDKContext(ctx).WithValue(bypassKey, false) +} + +// HasBypass checks the context to see if the mymodule bank send restriction should be skipped. +func HasBypass(ctx context.Context) bool { + bypassValue := ctx.Value(bypassKey) + if bypassValue == nil { + return false + } + bypass, isBool := bypassValue.(bool) + return isBool && bypass +} +``` + +Now, anywhere where you want to use `SendCoins` or `InputOutputCoins`, but you don't want your send restriction applied: + +```golang +func (k Keeper) DoThing(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) error { + return k.bankKeeper.SendCoins(mymodule.WithBypass(ctx), fromAddr, toAddr, amt) +} +``` + ### ViewKeeper The view keeper provides read-only access to account balances. The view keeper does not have balance alteration functionality. All balance lookups are `O(1)`. diff --git a/versioned_docs/version-0.50/build/modules/upgrade/README.md b/versioned_docs/version-0.50/build/modules/upgrade/README.md index 698e822ba..0d98c1601 100644 --- a/versioned_docs/version-0.50/build/modules/upgrade/README.md +++ b/versioned_docs/version-0.50/build/modules/upgrade/README.md @@ -8,7 +8,7 @@ sidebar_position: 1 `x/upgrade` is an implementation of a Cosmos SDK module that facilitates smoothly upgrading a live Cosmos chain to a new (breaking) software version. It accomplishes this by -providing a `BeginBlocker` hook that prevents the blockchain state machine from +providing a `PreBlocker` hook that prevents the blockchain state machine from proceeding once a pre-defined upgrade block height has been reached. The module does not prescribe anything regarding how governance decides to do an diff --git a/versioned_docs/version-0.50/build/packages/README.md b/versioned_docs/version-0.50/build/packages/README.md index 6892dad8e..d7a115b26 100644 --- a/versioned_docs/version-0.50/build/packages/README.md +++ b/versioned_docs/version-0.50/build/packages/README.md @@ -36,4 +36,4 @@ For more information on SDK tooling, see the [Tooling](https://docs.cosmos.netwo ## Example -* [SimApp](https://pkg.go.dev/cosmossdk.io/simapp) - SimApp is **the** sample Cosmos SDK chain. This package should not be imported in your application. \ No newline at end of file +* [SimApp](https://pkg.go.dev/cosmossdk.io/simapp) - SimApp is **the** sample Cosmos SDK chain. This package should not be imported in your application. diff --git a/versioned_docs/version-0.50/build/packages/_category_.json b/versioned_docs/version-0.50/build/packages/_category_.json index e91118d38..5ed885eb2 100644 --- a/versioned_docs/version-0.50/build/packages/_category_.json +++ b/versioned_docs/version-0.50/build/packages/_category_.json @@ -1,5 +1,5 @@ { "label": "Packages", - "position": 9, + "position": 4, "link": null } \ No newline at end of file diff --git a/versioned_docs/version-0.50/build/rfc/README.md b/versioned_docs/version-0.50/build/rfc/README.md index c4254e7af..8b8ead241 100644 --- a/versioned_docs/version-0.50/build/rfc/README.md +++ b/versioned_docs/version-0.50/build/rfc/README.md @@ -35,7 +35,4 @@ sections. ## Table of Contents - - -* [RFC-002: Zero Copy Encoding](./rfc-002-zero-copy-encoding.md) - +* [RFC-001: Tx Validation](./rfc-001-tx-validation.md) diff --git a/versioned_docs/version-0.50/build/rfc/_category_.json b/versioned_docs/version-0.50/build/rfc/_category_.json index 91a788569..a5712bdae 100644 --- a/versioned_docs/version-0.50/build/rfc/_category_.json +++ b/versioned_docs/version-0.50/build/rfc/_category_.json @@ -1,5 +1,5 @@ { "label": "RFC", - "position": 12, + "position": 7, "link": null -} +} \ No newline at end of file diff --git a/versioned_docs/version-0.50/build/spec/_category_.json b/versioned_docs/version-0.50/build/spec/_category_.json index 1143a00a1..5c2ccf7d4 100644 --- a/versioned_docs/version-0.50/build/spec/_category_.json +++ b/versioned_docs/version-0.50/build/spec/_category_.json @@ -1,5 +1,5 @@ { "label": "Specifications", - "position": 13, + "position": 8, "link": null } \ No newline at end of file diff --git a/versioned_docs/version-0.50/build/tooling/00-protobuf.md b/versioned_docs/version-0.50/build/tooling/00-protobuf.md new file mode 100644 index 000000000..332c39f2b --- /dev/null +++ b/versioned_docs/version-0.50/build/tooling/00-protobuf.md @@ -0,0 +1,113 @@ +--- +sidebar_position: 1 +--- + +# Protocol Buffers + +It is known that Cosmos SDK uses protocol buffers extensively, this document is meant to provide a guide on how it is used in the cosmos-sdk. + +To generate the proto file, the Cosmos SDK uses a docker image, this image is provided to all to use as well. The latest version is `ghcr.io/cosmos/proto-builder:0.12.x` + +Below is the example of the Cosmos SDK's commands for generating, linting, and formatting protobuf files that can be reused in any applications makefile. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/Makefile#L411-L432 +``` + +The script used to generate the protobuf files can be found in the `scripts/` directory. + +```shell reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/scripts/protocgen.sh +``` + +## Buf + +[Buf](https://buf.build) is a protobuf tool that abstracts the needs to use the complicated `protoc` toolchain on top of various other things that ensure you are using protobuf in accordance with the majority of the ecosystem. Within the cosmos-sdk repository there are a few files that have a buf prefix. Lets start with the top level and then dive into the various directories. + +### Workspace + +At the root level directory a workspace is defined using [buf workspaces](https://docs.buf.build/configuration/v1/buf-work-yaml). This helps if there are one or more protobuf containing directories in your project. + +Cosmos SDK example: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/buf.work.yaml#L6-L9 +``` + +### Proto Directory + +Next is the `proto/` directory where all of our protobuf files live. In here there are many different buf files defined each serving a different purpose. + +```bash +├── README.md +├── buf.gen.gogo.yaml +├── buf.gen.pulsar.yaml +├── buf.gen.swagger.yaml +├── buf.lock +├── buf.md +├── buf.yaml +├── cosmos +└── tendermint +``` + +The above diagram all the files and directories within the Cosmos SDK `proto/` directory. + +#### `buf.gen.gogo.yaml` + +`buf.gen.gogo.yaml` defines how the protobuf files should be generated for use with in the module. This file uses [gogoproto](https://github.com/gogo/protobuf), a separate generator from the google go-proto generator that makes working with various objects more ergonomic, and it has more performant encode and decode steps + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.gen.gogo.yaml#L1-L9 +``` + +:::tip +Example of how to define `gen` files can be found [here](https://docs.buf.build/tour/generate-go-code) +::: + +#### `buf.gen.pulsar.yaml` + +`buf.gen.pulsar.yaml` defines how protobuf files should be generated using the [new golang apiv2 of protobuf](https://go.dev/blog/protobuf-apiv2). This generator is used instead of the google go-proto generator because it has some extra helpers for Cosmos SDK applications and will have more performant encode and decode than the google go-proto generator. You can follow the development of this generator [here](https://github.com/cosmos/cosmos-proto). + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.gen.pulsar.yaml#L1-L18 +``` + +:::tip +Example of how to define `gen` files can be found [here](https://docs.buf.build/tour/generate-go-code) +::: + +#### `buf.gen.swagger.yaml` + +`buf.gen.swagger.yaml` generates the swagger documentation for the query and messages of the chain. This will only define the REST API end points that were defined in the query and msg servers. You can find examples of this [here](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/bank/v1beta1/query.proto#L19) + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.gen.swagger.yaml#L1-L6 +``` + +:::tip +Example of how to define `gen` files can be found [here](https://docs.buf.build/tour/generate-go-code) +::: + +#### `buf.lock` + +This is a autogenerated file based off the dependencies required by the `.gen` files. There is no need to copy the current one. If you depend on cosmos-sdk proto definitions a new entry for the Cosmos SDK will need to be provided. The dependency you will need to use is `buf.build/cosmos/cosmos-sdk`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.lock#L1-L16 +``` + +#### `buf.yaml` + +`buf.yaml` defines the [name of your package](https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.yaml#L3), which [breakage checker](https://docs.buf.build/tour/detect-breaking-changes) to use and how to [lint your protobuf files](https://docs.buf.build/tour/lint-your-api). + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.yaml#L1-L24 +``` + +We use a variety of linters for the Cosmos SDK protobuf files. The repo also checks this in ci. + +A reference to the github actions can be found [here](https://github.com/cosmos/cosmos-sdk/blob/main/.github/workflows/proto.yml#L1-L32) + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/.github/workflows/proto.yml#L1-L32 +``` diff --git a/versioned_docs/version-0.50/build/tooling/README.md b/versioned_docs/version-0.50/build/tooling/README.md index 1fa6ea2cc..27bc94e24 100644 --- a/versioned_docs/version-0.50/build/tooling/README.md +++ b/versioned_docs/version-0.50/build/tooling/README.md @@ -16,4 +16,4 @@ This includes tools for development, operating a node, and ease of use of a Cosm ## Other Tools -* [Protocol Buffers](./00-protobuf.md) \ No newline at end of file +* [Protocol Buffers](./00-protobuf.md) diff --git a/versioned_docs/version-0.50/build/tooling/_category_.json b/versioned_docs/version-0.50/build/tooling/_category_.json index a01a4fcc4..eb57cb8a5 100644 --- a/versioned_docs/version-0.50/build/tooling/_category_.json +++ b/versioned_docs/version-0.50/build/tooling/_category_.json @@ -1,5 +1,5 @@ { "label": "Tooling", - "position": 10, + "position": 5, "link": null } \ No newline at end of file diff --git a/versioned_docs/version-0.50/develop/advanced/14-tips.md b/versioned_docs/version-0.50/develop/advanced/14-tips.md deleted file mode 100644 index 90f0f2abc..000000000 --- a/versioned_docs/version-0.50/develop/advanced/14-tips.md +++ /dev/null @@ -1,214 +0,0 @@ ---- -sidebar_position: 1 ---- - -# Transaction Tips - -:::note Synopsis -Transaction tips are a mechanism to pay for transaction fees using another denom than the native fee denom of the chain. They are still in beta, and are not included by default in the SDK. -::: - -## Context - -In a Cosmos ecosystem where more and more chains are connected via [IBC](https://ibc.cosmos.network/), it happens that users want to perform actions on chains where they don't have native tokens yet. An example would be an Osmosis user who wants to vote on a proposal on the Cosmos Hub, but they don't have ATOMs in their wallet. A solution would be to swap OSMO for ATOM just for voting on this proposal, but that is cumbersome. Cross-chain DeFi project [Emeris](https://emeris.com/) is another use case. - -Transaction tips is a new solution for cross-chain transaction fees payment, whereby the transaction initiator signs a transaction without specifying fees, but uses a new `Tip` field. They send this signed transaction to a fee relayer who will choose the transaction fees and broadcast the final transaction, and the SDK provides a mechanism that will transfer the pre-defined `Tip` to the fee payer, to cover for fees. - -Assuming we have two chains, A and B, we define the following terms: - -* **the tipper**: this is the initiator of the transaction, who wants to execute a `Msg` on chain A, but doesn't have any native chain A tokens, only chain B tokens. In our example above, the tipper is the Osmosis (chain B) user wanting to vote on a Cosmos Hub (chain A) proposal. -* **the fee payer**: this is the party that will relay and broadcast the final transaction on chain A, and has chain A tokens. The tipper doesn't need to trust the feepayer. -* **the target chain**: the chain where the `Msg` is executed, chain A in this case. - -## Transaction Tips Flow - -The transaction tips flow happens in multiple steps. - -1. The tipper sends via IBC some chain B tokens to chain A. These tokens will cover for fees on the target chain A. This means that chain A's bank module holds some IBC tokens under the tipper's address. - -2. The tipper drafts a transaction to be executed on the chain A. It can include chain A `Msg`s. However, instead of creating a normal transaction, they create the following `AuxSignerData` document: - - ```protobuf reference - https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/v1beta1/tx.proto#L231-L244 - ``` - - where we have defined `SignDocDirectAux` as: - - ```protobuf reference - https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/v1beta1/tx.proto#L68-L98 - ``` - - where `Tip` is defined as - - ```protobuf reference - https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/v1beta1/tx.proto#L233-L244 - ``` - - Notice that this document doesn't sign over the final chain A fees. Instead, it includes a `Tip` field. It also doesn't include the whole `AuthInfo` object as in `SIGN_MODE_DIRECT`, only the minimum information needed by the tipper - -3. The tipper signs the `SignDocDirectAux` document and attaches the signature to the `AuxSignerData`, then sends the signed `AuxSignerData` to the fee payer. - -4. From the signed `AuxSignerData` document, the fee payer constructs a transaction, using the following algorithm: - -* use as `TxBody` the exact `AuxSignerData.SignDocDirectAux.body_bytes`, to not alter the original intent of the tipper, -* create an `AuthInfo` with: - * `AuthInfo.Tip` copied from `AuxSignerData.SignDocDirectAux.Tip`, - * `AuthInfo.Fee` chosen by the fee payer, which should cover for the transaction gas, but also be small enough so that the tip/fee exchange rate is economically interesting for the fee payer, - * `AuthInfo.SignerInfos` has two signers: the first signer is the tipper, using the public key, sequence and sign mode specified in `AuxSignerData`; and the second signer is the fee payer, using their favorite sign mode, -* a `Signatures` array with two items: the tipper's signature from `AuxSignerData.Sig`, and the final fee payer's signature. - -5. Broadcast the final transaction signed by the two parties to the target chain. Once included, the Cosmos SDK will trigger a transfer of the `Tip` specified in the transaction from the tipper address to the fee payer address. - -### Fee Payers Market - -The benefit of transaction tips for the tipper is clear: there is no need to swap tokens before executing a cross-chain message. - -For the fee payer, the benefit is in the tip v.s. fee exchange. Put simply, the fee payer pays the fees of an unknown tipper's transaction, and gets in exchange the tip that the tipper chose. There is an economic incentive for the fee payer to do so only when the tip is greater than the transaction fees, given the exchange rates between the two tokens. - -In the future, we imagine a market where fee payers will compete to include transactions from tippers, who on their side will optimize by specifying the lowest tip possible. A number of automated services might spin up to perform transaction gas simulation and exchange rate monitoring to optimize both the tip and fee values in real-time. - -### Tipper and Fee Payer Sign Modes - -As we mentioned in the flow above, the tipper signs over the `SignDocDirectAux`, and the fee payer signs over the whole final transaction. As such, both parties might use different sign modes. - -* The tipper MUST use `SIGN_MODE_DIRECT_AUX` or `SIGN_MODE_LEGACY_AMINO_JSON`. That is because the tipper needs to sign over the body, the tip, but not the other signers' information and not over the fee (which is unknown to the tipper). -* The fee payer MUST use `SIGN_MODE_DIRECT` or `SIGN_MODE_LEGACY_AMINO_JSON`. The fee payer signs over the whole transaction. - -For example, if the fee payer signs the whole transaction with `SIGN_MODE_DIRECT_AUX`, it will be rejected by the node, as that would introduce malleability issues (`SIGN_MODE_DIRECT_AUX` doesn't sign over fees). - -In both cases, using `SIGN_MODE_LEGACY_AMINO_JSON` is recommended only if hardware wallet signing is needed. - -## Enabling Tips on your Chain - -The transaction tips functionality is introduced in Cosmos SDK v0.46, so earlier versions do not have support for tips. It is however not included by default in a v0.46 app. Sending a transaction with tips to a chain which didn't enable tips will result in a no-op, i.e. the `tip` field in the transaction will be ignored. - -Enabling tips on a chain is done by adding the `TipDecorator` in the posthandler chain: - -```go -// HandlerOptions are the options required for constructing a SDK PostHandler which supports tips. -type HandlerOptions struct { - BankKeeper types.BankKeeper -} - -// MyPostHandler returns a posthandler chain with the TipDecorator. -func MyPostHandler(options HandlerOptions) (sdk.AnteHandler, error) { - if options.BankKeeper == nil { - return nil, errorsmod.Wrap(sdkerrors.ErrLogic, "bank keeper is required for posthandler") - } - - postDecorators := []sdk.AnteDecorator{ - posthandler.NewTipDecorator(options.bankKeeper), - } - - return sdk.ChainAnteDecorators(postDecorators...), nil -} - -func (app *SimApp) setPostHandler() { - postHandler, err := MyPostHandler( - HandlerOptions{ - BankKeeper: app.BankKeeper, - }, - ) - if err != nil { - panic(err) - } - - app.SetPostHandler(postHandler) -} -``` - -Notice that `NewTipDecorator` needs a reference to the BankKeeper, for transferring the tip to the fee payer. - -## CLI Usage - -The Cosmos SDK also provides some CLI tooling for the transaction tips flow, both for the tipper and for the feepayer. - -For the tipper, the CLI `tx` subcommand has two new flags: `--aux` and `--tip`. The `--aux` flag is used to denote that we are creating an `AuxSignerData` instead of a `Tx`, and the `--tip` is used to populate its `Tip` field. - -```bash -$ simd tx gov vote 16 yes --from --aux --tip 50ibcdenom - - -### Prints the AuxSignerData as JSON: -### {"address":"cosmos1q0ayf5vq6fd2xxrwh30upg05hxdnyw2h5249a2","sign_doc":{"body_bytes":"CosBChwvY29zbW9zLmJhbmsudjFiZXRhMS5Nc2dTZW5kEmsKLWNvc21vczFxMGF5ZjV2cTZmZDJ4eHJ3aDMwdXBnMDVoeGRueXcyaDUyNDlhMhItY29zbW9zMXdlNWoyZXI2MHV5OXF3YzBta3ptdGdtdHA5Z3F5NXY2bjhnZGdlGgsKBXN0YWtlEgIxMA==","public_key":{"@type":"/cosmos.crypto.secp256k1.PubKey","key":"AojOF/1luQ5H/nZDSrE1w3CyzGJhJdQuS7hFX5wAA6uJ"},"chain_id":"","account_number":"0","sequence":"1","tip":{"amount":[{"denom":"ibcdenom","amount":"50"}],"tipper":"cosmos1q0ayf5vq6fd2xxrwh30upg05hxdnyw2h5249a2"}},"mode":"SIGN_MODE_DIRECT_AUX","sig":"v/d/bGq9FGdecs6faMG2t//nRirFTiqwFtUB65M6kh0QdUeM6jg3r8oJX1o17xkoDxJ09EyJiSyvo6fbU7vUxg=="} -``` - -It is useful to pipe the JSON output to a file, `> aux_signed_tx.json` - -For the fee payer, the Cosmos SDK added a `tx aux-to-fee` subcommand to include an `AuxSignerData` into a transaction, add fees to it, and broadcast it. - -```bash -$ simd tx aux-to-fee aux_signed_tx.json --from --fees 30atom - -### Prints the broadcasted tx response: -### code: 0 -### codespace: sdk -### data: "" -### events: [] -### gas_used: "0" -### gas_wanted: "0" -### height: "0" -### info: "" -### logs: [] -### timestamp: "" -### tx: null -``` - -Upon completion of the second command, the fee payer's balance will be down the `30atom` fees, and up the `50ibcdenom` tip. - -For both commands, the flag `--sign-mode=amino-json` is still available for hardware wallet signing. - -## Programmatic Usage - -For the tipper, the SDK exposes a new transaction builder, the `AuxTxBuilder`, for generating an `AuxSignerData`. The API of `AuxTxBuilder` is defined [in `client/tx`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/client/tx/aux_builder.go#L22), and can be used as follows: - -```go -// Note: there's no need to use clientCtx.TxConfig anymore. - -bldr := clienttx.NewAuxTxBuilder() -err := bldr.SetMsgs(msgs...) -bldr.SetAddress("cosmos1...") -bldr.SetMemo(...) -bldr.SetTip(...) -bldr.SetPubKey(...) -err := bldr.SetSignMode(...) // DIRECT_AUX or AMINO, or else error -// ... other setters are also available - -// Get the bytes to sign. -signBz, err := bldr.GetSignBytes() - -// Sign the bz using your favorite method. -sig, err := privKey.sign(signBz) - -// Set the signature -bldr.SetSig(sig) - -// Get the final auxSignerData to be sent to the fee payer -auxSignerData, err:= bldr.GetAuxSignerData() -``` - -For the fee payer, the SDK added a new method on the existing `TxBuilder` to import data from an `AuxSignerData`: - -```go -// get `auxSignerData` from tipper, see code snippet above. - -txBuilder := clientCtx.TxConfig.NewTxBuilder() -err := txBuilder.AddAuxSignerData(auxSignerData) -if err != nil { - return err -} - -// A lot of fields will be populated in txBuilder, such as its Msgs, tip -// memo, etc... - -// The fee payer choses the fee to set on the transaction. -txBuilder.SetFeePayer() -txBuilder.SetFeeAmount(...) -txBuilder.SetGasLimit(...) - -// Usual signing code -err = authclient.SignTx(...) -if err != nil { - return err -} -``` diff --git a/versioned_docs/version-0.50/develop/advanced/00-baseapp.md b/versioned_docs/version-0.50/learn/advanced/00-baseapp.md similarity index 95% rename from versioned_docs/version-0.50/develop/advanced/00-baseapp.md rename to versioned_docs/version-0.50/learn/advanced/00-baseapp.md index cc59baced..90183a155 100644 --- a/versioned_docs/version-0.50/develop/advanced/00-baseapp.md +++ b/versioned_docs/version-0.50/learn/advanced/00-baseapp.md @@ -10,7 +10,7 @@ This document describes `BaseApp`, the abstraction that implements the core func :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK application](../beginner/00-overview-app.md) +* [Anatomy of a Cosmos SDK application](../beginner/00-app-anatomy.md) * [Lifecycle of a Cosmos SDK transaction](../beginner/01-tx-lifecycle.md) ::: @@ -78,7 +78,7 @@ First, the important parameters that are initialized during the bootstrapping of * [`AnteHandler`](#antehandler): This handler is used to handle signature verification, fee payment, and other pre-message execution checks when a transaction is received. It's executed during [`CheckTx/RecheckTx`](#checktx) and [`FinalizeBlock`](#finalizeblock). -* [`InitChainer`](../beginner/00-overview-app.md#initchainer), [`PreBlocker`](../beginner/00-overview-app.md#preblocker), [`BeginBlocker` and `EndBlocker`](../beginner/00-overview-app.md#beginblocker-and-endblocker): These are +* [`InitChainer`](../beginner/00-app-anatomy.md#initchainer), [`PreBlocker`](../beginner/00-app-anatomy.md#preblocker), [`BeginBlocker` and `EndBlocker`](../beginner/00-app-anatomy.md#beginblocker-and-endblocker): These are the functions executed when the application receives the `InitChain` and `FinalizeBlock` ABCI messages from the underlying CometBFT engine. @@ -104,7 +104,7 @@ Finally, a few more important parameters: `minGasPrices` (e.g. if `minGasPrices == 1uatom,1photon`, the `gas-price` of the transaction must be greater than `1uatom` OR `1photon`). * `appVersion`: Version of the application. It is set in the - [application's constructor function](../beginner/00-overview-app.md#constructor-function). + [application's constructor function](../beginner/00-app-anatomy.md#constructor-function). ## Constructor @@ -187,7 +187,7 @@ During `FinalizeBlock`, the `finalizeBlockState` is set for use during transacti `finalizeBlockState` is based off of the last committed state from the root store and is branched. Note, the `finalizeBlockState` is set to `nil` on [`Commit`](#commit). -The state flow for transcation execution is nearly identical to `CheckTx` except state transitions occur on +The state flow for transaction execution is nearly identical to `CheckTx` except state transitions occur on the `finalizeBlockState` and messages in a transaction are executed. Similarly to `CheckTx`, state transitions occur on a doubly branched state -- `finalizeBlockState`. Successful message execution results in writes being committed to `finalizeBlockState`. Note, if message execution fails, state transitions from @@ -220,13 +220,13 @@ When messages and queries are received by the application, they must be routed t The [default `msgServiceRouter` included in `BaseApp`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/msg_service_router.go) is stateless. However, some applications may want to make use of more stateful routing mechanisms such as allowing governance to disable certain routes or point them to new modules for upgrade purposes. For this reason, the `sdk.Context` is also passed into each [route handler inside `msgServiceRouter`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/msg_service_router.go#L31-L32). For a stateless router that doesn't want to make use of this, you can just ignore the `ctx`. -The application's `msgServiceRouter` is initialized with all the routes using the application's [module manager](../../build/building-modules/01-module-manager.md#manager) (via the `RegisterServices` method), which itself is initialized with all the application's modules in the application's [constructor](../beginner/00-overview-app.md#constructor-function). +The application's `msgServiceRouter` is initialized with all the routes using the application's [module manager](../../build/building-modules/01-module-manager.md#manager) (via the `RegisterServices` method), which itself is initialized with all the application's modules in the application's [constructor](../beginner/00-app-anatomy.md#constructor-function). ### gRPC Query Router Similar to `sdk.Msg`s, [`queries`](../../build/building-modules/02-messages-and-queries.md#queries) need to be routed to the appropriate module's [`Query` service](../../build/building-modules/04-query-services.md). To do so, `BaseApp` holds a `grpcQueryRouter`, which maps modules' fully-qualified service methods (`string`, defined in their Protobuf `Query` gRPC) to their `QueryServer` implementation. The `grpcQueryRouter` is called during the initial stages of query processing, which can be either by directly sending a gRPC query to the gRPC endpoint, or via the [`Query` ABCI message](#query) on the CometBFT RPC endpoint. -Just like the `msgServiceRouter`, the `grpcQueryRouter` is initialized with all the query routes using the application's [module manager](../../build/building-modules/01-module-manager.md) (via the `RegisterServices` method), which itself is initialized with all the application's modules in the application's [constructor](../beginner/00-overview-app.md#app-constructor). +Just like the `msgServiceRouter`, the `grpcQueryRouter` is initialized with all the query routes using the application's [module manager](../../build/building-modules/01-module-manager.md) (via the `RegisterServices` method), which itself is initialized with all the application's modules in the application's [constructor](../beginner/00-app-anatomy.md#app-constructor). ## Main ABCI 2.0 Messages @@ -324,7 +324,7 @@ to do the following checks: with the transaction is superior to a minimum reference gas amount based on the raw transaction size, in order to avoid spam with transactions that provide 0 gas. -`CheckTx` does **not** process `sdk.Msg`s - they only need to be processed when the canonical state need to be updated, which happens during `FinalizeBlock`. +`CheckTx` does **not** process `sdk.Msg`s - they only need to be processed when the canonical state needs to be updated, which happens during `FinalizeBlock`. Steps 2. and 3. are performed by the [`AnteHandler`](../beginner/04-gas-fees.md#antehandler) in the [`RunTx()`](#runtx-antehandler-and-runmsgs) function, which `CheckTx()` calls with the `runTxModeCheck` mode. During each step of `CheckTx()`, a @@ -398,7 +398,7 @@ The `AnteHandler` is theoretically optional, but still a very important componen * Perform preliminary _stateful_ validity checks like ensuring signatures are valid or that the sender has enough funds to pay for fees. * Play a role in the incentivisation of stakeholders via the collection of transaction fees. -`BaseApp` holds an `anteHandler` as parameter that is initialized in the [application's constructor](../beginner/00-overview-app.md#application-constructor). The most widely used `anteHandler` is the [`auth` module](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/ante/ante.go). +`BaseApp` holds an `anteHandler` as parameter that is initialized in the [application's constructor](../beginner/00-app-anatomy.md#application-constructor). The most widely used `anteHandler` is the [`auth` module](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/ante/ante.go). Click [here](../beginner/04-gas-fees.md#antehandler) for more on the `anteHandler`. @@ -410,9 +410,10 @@ First, it retrieves the `sdk.Msg`'s fully-qualified type name, by checking the ` ### PostHandler -`PostHandler` is similar to `AnteHandler`, but it, as the name suggests, executes custom post tx processing logic after [`RunMsgs`](#runmsgs) is called. `PostHandler` receives the `Result` of the the `RunMsgs` in order to enable this customizable behavior. +`PostHandler` is similar to `AnteHandler`, but it, as the name suggests, executes custom post tx processing logic after [`RunMsgs`](#runmsgs) is called. `PostHandler` receives the `Result` of the `RunMsgs` in order to enable this customizable behavior. + +Like `AnteHandler`s, `PostHandler`s are theoretically optional. -Like `AnteHandler`s, `PostHandler`s are theoretically optional, one use case for `PostHandler`s is transaction tips (enabled by default in simapp). Other use cases like unused gas refund can also be enabled by `PostHandler`s. ```go reference @@ -431,7 +432,7 @@ The [`InitChain` ABCI message](https://github.com/cometbft/cometbft/blob/v0.37.x * [`checkState` and `finalizeBlockState`](#state-updates) via `setState`. * The [block gas meter](../beginner/04-gas-fees.md#block-gas-meter), with infinite gas to process genesis transactions. -Finally, the `InitChain(req abci.RequestInitChain)` method of `BaseApp` calls the [`initChainer()`](../beginner/00-overview-app.md#initchainer) of the application in order to initialize the main state of the application from the `genesis file` and, if defined, call the [`InitGenesis`](../../build/building-modules/08-genesis.md#initgenesis) function of each of the application's modules. +Finally, the `InitChain(req abci.RequestInitChain)` method of `BaseApp` calls the [`initChainer()`](../beginner/00-app-anatomy.md#initchainer) of the application in order to initialize the main state of the application from the `genesis file` and, if defined, call the [`InitGenesis`](../../build/building-modules/08-genesis.md#initgenesis) function of each of the application's modules. ### FinalizeBlock @@ -445,7 +446,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci.go#L623 #### PreBlock -* Run the application's [`preBlocker()`](../beginner/00-overview-app.md#preblocker), which mainly runs the [`PreBlocker()`](../../build/building-modules/17-preblock.md#preblock) method of each of the modules. +* Run the application's [`preBlocker()`](../beginner/00-app-anatomy.md#preblocker), which mainly runs the [`PreBlocker()`](../../build/building-modules/17-preblock.md#preblock) method of each of the modules. #### BeginBlock @@ -458,7 +459,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci.go#L623 This function also resets the [main gas meter](../beginner/04-gas-fees.md#main-gas-meter). * Initialize the [block gas meter](../beginner/04-gas-fees.md#block-gas-meter) with the `maxGas` limit. The `gas` consumed within the block cannot go above `maxGas`. This parameter is defined in the application's consensus parameters. -* Run the application's [`beginBlocker()`](../beginner/00-overview-app.md#beginblocker-and-endblocker), which mainly runs the [`BeginBlocker()`](../../build/building-modules/06-beginblock-endblock.md#beginblock) method of each of the modules. +* Run the application's [`beginBlocker()`](../beginner/00-app-anatomy.md#beginblocker-and-endblocker), which mainly runs the [`BeginBlocker()`](../../build/building-modules/06-beginblock-endblock.md#beginblock) method of each of the modules. * Set the [`VoteInfos`](https://github.com/cometbft/cometbft/blob/v0.37.x/spec/abci/abci++_methods.md#voteinfo) of the application, i.e. the list of validators whose _precommit_ for the previous block was included by the proposer of the current block. This information is carried into the [`Context`](./02-context.md) so that it can be used during transaction execution and EndBlock. #### Transaction Execution @@ -527,7 +528,7 @@ Each CometBFT `query` comes with a `path`, which is a `string` which denotes wha ### ExtendVote -`ExtendVote` allows an application to extend a pre-commit vote with arbitrary data. This process does NOT have be deterministic and the data returned can be unique to the validator process. +`ExtendVote` allows an application to extend a pre-commit vote with arbitrary data. This process does NOT have to be deterministic and the data returned can be unique to the validator process. In the Cosmos-SDK this is implemented as a NoOp: @@ -537,7 +538,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci_utils.go# ### VerifyVoteExtension -`VerifyVoteExtension` allows an application to verify that the data returned by `ExtendVote` is valid. This process does NOT have be deterministic and the data returned can be unique to the validator process. +`VerifyVoteExtension` allows an application to verify that the data returned by `ExtendVote` is valid. This process does NOT have to be deterministic and the data returned can be unique to the validator process. In the Cosmos-SDK this is implemented as a NoOp: diff --git a/versioned_docs/version-0.50/develop/advanced/01-transactions.md b/versioned_docs/version-0.50/learn/advanced/01-transactions.md similarity index 92% rename from versioned_docs/version-0.50/develop/advanced/01-transactions.md rename to versioned_docs/version-0.50/learn/advanced/01-transactions.md index a56aa3008..13d3727b6 100644 --- a/versioned_docs/version-0.50/develop/advanced/01-transactions.md +++ b/versioned_docs/version-0.50/learn/advanced/01-transactions.md @@ -10,7 +10,7 @@ sidebar_position: 1 :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK Application](../beginner/00-overview-app.md) +* [Anatomy of a Cosmos SDK Application](../beginner/00-app-anatomy.md) ::: @@ -91,7 +91,6 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/v1beta The use case is a multi-signer transaction, where one of the signers is appointed to gather all signatures, broadcast the signature and pay for fees, and the others only care about the transaction body. This generally allows for a better multi-signing UX. If Alice, Bob and Charlie are part of a 3-signer transaction, then Alice and Bob can both use `SIGN_MODE_DIRECT_AUX` to sign over the `TxBody` and their own signer info (no need an additional step to gather other signers' ones, like in `SIGN_MODE_DIRECT`), without specifying a fee in their SignDoc. Charlie can then gather both signatures from Alice and Bob, and create the final transaction by appending a fee. Note that the fee payer of the transaction (in our case Charlie) must sign over the fees, so must use `SIGN_MODE_DIRECT` or `SIGN_MODE_LEGACY_AMINO_JSON`. -A concrete use case is implemented in [transaction tips](./14-tips.md): the tipper may use `SIGN_MODE_DIRECT_AUX` to specify a tip in the transaction, without signing over the actual transaction fees. Then, the fee payer appends fees inside the tipper's desired `TxBody`, and as an exchange for paying the fees and broadcasting the transaction, receives the tipper's transaction tips as payment. #### `SIGN_MODE_TEXTUAL` @@ -99,7 +98,7 @@ A concrete use case is implemented in [transaction tips](./14-tips.md): the tipp #### Custom Sign modes -There is the the opportunity to add your own custom sign mode to the Cosmos-SDK. While we can not accept the implementation of the sign mode to the repository, we can accept a pull request to add the custom signmode to the SignMode enum located [here](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/signing/v1beta1/signing.proto#L17) +There is the opportunity to add your own custom sign mode to the Cosmos-SDK. While we can not accept the implementation of the sign mode to the repository, we can accept a pull request to add the custom signmode to the SignMode enum located [here](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/tx/signing/v1beta1/signing.proto#L17) ## Transaction Process @@ -146,7 +145,7 @@ As there are currently two sign modes for signing transactions, there are also t * [wrapper](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/tx/builder.go#L26-L43) for creating transactions for `SIGN_MODE_DIRECT`, * [StdTxBuilder](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/migrations/legacytx/stdtx_builder.go#L14-L17) for `SIGN_MODE_LEGACY_AMINO_JSON`. -However, the two implementation of `TxBuilder` should be hidden away from end-users, as they should prefer using the overarching `TxConfig` interface: +However, the two implementations of `TxBuilder` should be hidden away from end-users, as they should prefer using the overarching `TxConfig` interface: ```go reference https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/client/tx_config.go#L24-L34 diff --git a/versioned_docs/version-0.50/develop/advanced/02-context.md b/versioned_docs/version-0.50/learn/advanced/02-context.md similarity index 98% rename from versioned_docs/version-0.50/develop/advanced/02-context.md rename to versioned_docs/version-0.50/learn/advanced/02-context.md index d49d0bf04..0056ec10c 100644 --- a/versioned_docs/version-0.50/develop/advanced/02-context.md +++ b/versioned_docs/version-0.50/learn/advanced/02-context.md @@ -10,7 +10,7 @@ The `context` is a data structure intended to be passed from function to functio :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK Application](../beginner/00-overview-app.md) +* [Anatomy of a Cosmos SDK Application](../beginner/00-app-anatomy.md) * [Lifecycle of a Transaction](../beginner/01-tx-lifecycle.md) ::: @@ -62,7 +62,7 @@ explicitly pass a context `ctx` as the first argument of a process. ## Store branching -The `Context` contains a `MultiStore`, which allows for branchinig and caching functionality using `CacheMultiStore` +The `Context` contains a `MultiStore`, which allows for branching and caching functionality using `CacheMultiStore` (queries in `CacheMultiStore` are cached to avoid future round trips). Each `KVStore` is branched in a safe and isolated ephemeral storage. Processes are free to write changes to the `CacheMultiStore`. If a state-transition sequence is performed without issue, the store branch can diff --git a/versioned_docs/version-0.50/develop/advanced/03-node.md b/versioned_docs/version-0.50/learn/advanced/03-node.md similarity index 97% rename from versioned_docs/version-0.50/develop/advanced/03-node.md rename to versioned_docs/version-0.50/learn/advanced/03-node.md index 32ab24043..47b691b32 100644 --- a/versioned_docs/version-0.50/develop/advanced/03-node.md +++ b/versioned_docs/version-0.50/learn/advanced/03-node.md @@ -10,7 +10,7 @@ The main endpoint of a Cosmos SDK application is the daemon client, otherwise kn :::note Pre-requisite Readings -* [Anatomy of an SDK application](../beginner/00-overview-app.md) +* [Anatomy of an SDK application](../beginner/00-app-anatomy.md) ::: @@ -69,7 +69,7 @@ Note that an `appCreator` is a function that fulfills the `AppCreator` signature https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/server/types/app.go#L68 ``` -In practice, the [constructor of the application](../beginner/00-overview-app.md#constructor-function) is passed as the `appCreator`. +In practice, the [constructor of the application](../beginner/00-app-anatomy.md#constructor-function) is passed as the `appCreator`. ```go reference https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/simd/cmd/root_v2.go#L294-L308 diff --git a/versioned_docs/version-0.50/develop/advanced/04-store.md b/versioned_docs/version-0.50/learn/advanced/04-store.md similarity index 97% rename from versioned_docs/version-0.50/develop/advanced/04-store.md rename to versioned_docs/version-0.50/learn/advanced/04-store.md index a60ddd108..1419b54ae 100644 --- a/versioned_docs/version-0.50/develop/advanced/04-store.md +++ b/versioned_docs/version-0.50/learn/advanced/04-store.md @@ -10,7 +10,7 @@ A store is a data structure that holds the state of the application. :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK application](../beginner/00-overview-app.md) +* [Anatomy of a Cosmos SDK application](../beginner/00-app-anatomy.md) ::: @@ -138,7 +138,7 @@ A `KVStore` is a simple key-value store used to store and retrieve data. A `Comm Individual `KVStore`s are used by modules to manage a subset of the global state. `KVStores` can be accessed by objects that hold a specific key. This `key` should only be exposed to the [`keeper`](../../build/building-modules/06-keeper.md) of the module that defines the store. -`CommitKVStore`s are declared by proxy of their respective `key` and mounted on the application's [multistore](#multistore) in the [main application file](../beginner/00-overview-app.md#core-application-file). In the same file, the `key` is also passed to the module's `keeper` that is responsible for managing the store. +`CommitKVStore`s are declared by proxy of their respective `key` and mounted on the application's [multistore](#multistore) in the [main application file](../beginner/00-app-anatomy.md#core-application-file). In the same file, the `key` is also passed to the module's `keeper` that is responsible for managing the store. ```go reference https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/types/store.go#L229-L266 @@ -168,7 +168,7 @@ The documentation on the IAVL Tree is located [here](https://github.com/cosmos/i ### `DbAdapter` Store -`dbadapter.Store` is a adapter for `dbm.DB` making it fulfilling the `KVStore` interface. +`dbadapter.Store` is an adapter for `dbm.DB` making it fulfilling the `KVStore` interface. ```go reference https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/dbadapter/store.go#L13-L16 @@ -269,7 +269,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/store/prefix/store.go# When `Store.{Get, Set}()` is called, the store forwards the call to its parent, with the key prefixed with the `Store.prefix`. -When `Store.Iterator()` is called, it does not simply prefix the `Store.prefix`, since it does not work as intended. In that case, some of the elements are traversed even they are not starting with the prefix. +When `Store.Iterator()` is called, it does not simply prefix the `Store.prefix`, since it does not work as intended. In that case, some of the elements are traversed even if they are not starting with the prefix. ### `ListenKv` Store diff --git a/docs/develop/advanced/06-encoding.md b/versioned_docs/version-0.50/learn/advanced/05-encoding.md similarity index 77% rename from docs/develop/advanced/06-encoding.md rename to versioned_docs/version-0.50/learn/advanced/05-encoding.md index 9f854423f..0178884b2 100644 --- a/docs/develop/advanced/06-encoding.md +++ b/versioned_docs/version-0.50/learn/advanced/05-encoding.md @@ -8,11 +8,9 @@ sidebar_position: 1 While encoding in the Cosmos SDK used to be mainly handled by `go-amino` codec, the Cosmos SDK is moving towards using `gogoprotobuf` for both state and client-side encoding. ::: -:::note +:::note Pre-requisite Readings -### Pre-requisite Readings - -* [Anatomy of a Cosmos SDK application](../beginner/00-overview-app.md) +* [Anatomy of a Cosmos SDK application](../beginner/00-app-anatomy.md) ::: @@ -44,58 +42,14 @@ In the `codec` package, there exists two core interfaces, `BinaryCodec` and `JSO where the former encapsulates the current Amino interface except it operates on types implementing the latter instead of generic `interface{}` types. -In addition, there exists two implementations of `Codec`. The first being -`AminoCodec`, where both binary and JSON serialization is handled via Amino. The -second being `ProtoCodec`, where both binary and JSON serialization is handled -via Protobuf. - -This means that modules may use Amino or Protobuf encoding, but the types must +The `ProtoCodec`, where both binary and JSON serialization is handled +via Protobuf. This means that modules may use Protobuf encoding, but the types must implement `ProtoMarshaler`. If modules wish to avoid implementing this interface -for their types, they may use an Amino codec directly. - -### Amino - -Every module uses an Amino codec to serialize types and interfaces. This codec typically -has types and interfaces registered in that module's domain only (e.g. messages), -but there are exceptions like `x/gov`. Each module exposes a `RegisterLegacyAminoCodec` function -that allows a user to provide a codec and have all the types registered. An application -will call this method for each necessary module. - -Where there is no protobuf-based type definition for a module (see below), Amino -is used to encode and decode raw wire bytes to the concrete type or interface: - -```go -bz := keeper.cdc.MustMarshal(typeOrInterface) -keeper.cdc.MustUnmarshal(bz, &typeOrInterface) -``` - -Note, there are length-prefixed variants of the above functionality and this is -typically used for when the data needs to be streamed or grouped together -(e.g. `ResponseDeliverTx.Data`) - -#### Authz authorizations and Gov/Group proposals - -Since authz's `MsgExec` and `MsgGrant` message types, as well as gov's and group's `MsgSubmitProposal`, can contain different messages instances, it is important that developers -add the following code inside the `init` method of their module's `codec.go` file: - -```go -import ( - authzcodec "github.com/cosmos/cosmos-sdk/x/authz/codec" - govcodec "github.com/cosmos/cosmos-sdk/x/gov/codec" - groupcodec "github.com/cosmos/cosmos-sdk/x/group/codec" -) +for their types, this is autogenerated via [buf](https://buf.build/) -init() { - // Register all Amino interfaces and concrete types on the authz and gov Amino codec so that this can later be - // used to properly serialize MsgGrant, MsgExec and MsgSubmitProposal instances - RegisterLegacyAminoCodec(authzcodec.Amino) - RegisterLegacyAminoCodec(govcodec.Amino) - RegisterLegacyAminoCodec(groupcodec.Amino) -} -``` +If modules use [Collections](../../build/packages/02-collections.md) or [ORM](../../build/packages/03-orm.md), encoding and decoding are handled, marshal and unmarshal should not be handled manually unless for specific cases identified by the developer. -This will allow the `x/authz` module to properly serialize and de-serializes `MsgExec` instances using Amino, -which is required when signing this kind of messages using a Ledger. +```go reference ### Gogoproto @@ -117,7 +71,7 @@ Code generators can then match the `accepts_interface` and `implements_interface ### Transaction Encoding Another important use of Protobuf is the encoding and decoding of -[transactions](01-transactions.md). Transactions are defined by the application or +[transactions](./01-transactions.md). Transactions are defined by the application or the Cosmos SDK but are then passed to the underlying consensus engine to be relayed to other peers. Since the underlying consensus engine is agnostic to the application, the consensus engine accepts only transactions in the form of raw bytes. @@ -126,17 +80,17 @@ the consensus engine accepts only transactions in the form of raw bytes. * The `TxDecoder` object performs the decoding. ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/types/tx_msg.go#L76-L80 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/tx_msg.go#L91-L95 ``` A standard implementation of both these objects can be found in the [`auth/tx` module](../../build/modules/auth/2-tx.md): ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/auth/tx/decoder.go +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/tx/decoder.go ``` ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/auth/tx/encoder.go +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/tx/encoder.go ``` See [ADR-020](../../build/architecture/adr-020-protobuf-transaction-encoding.md) for details of how a transaction is encoded. @@ -157,7 +111,7 @@ message Profile { In this `Profile` example, we hardcoded `account` as a `BaseAccount`. However, there are several other types of [user accounts related to vesting](../../build/modules/auth/1-vesting.md), such as `BaseVestingAccount` or `ContinuousVestingAccount`. All of these accounts are different, but they all implement the `AccountI` interface. How would you create a `Profile` that allows all these types of accounts with an `account` field that accepts an `AccountI` interface? ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/auth/types/account.go#L307-L330 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/account.go#L15-L32 ``` In [ADR-019](../../build/architecture/adr-019-protobuf-state-encoding.md), it has been decided to use [`Any`](https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/any.proto)s to encode interfaces in protobuf. An `Any` contains an arbitrary serialized message as bytes, along with a URL that acts as a globally unique identifier for and resolves to that message's type. This strategy allows us to pack arbitrary Go types inside protobuf messages. Our new `Profile` then looks like: @@ -196,7 +150,7 @@ bz, err := cdc.Marshal(profile) jsonBz, err := cdc.MarshalJSON(profile) ``` -To summarize, to encode an interface, you must 1/ pack the interface into an `Any` and 2/ marshal the `Any`. For convenience, the Cosmos SDK provides a `MarshalInterface` method to bundle these two steps. Have a look at [a real-life example in the x/auth module](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/auth/keeper/keeper.go#L240-L243). +To summarize, to encode an interface, you must 1/ pack the interface into an `Any` and 2/ marshal the `Any`. For convenience, the Cosmos SDK provides a `MarshalInterface` method to bundle these two steps. Have a look at [a real-life example in the x/auth module](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/keeper/keeper.go#L240-L243). The reverse operation of retrieving the concrete Go type from inside an `Any`, called "unpacking", is done with the `GetCachedValue()` on `Any`. @@ -210,7 +164,7 @@ err := cdc.Unmarshal(profilebz, &myProfile) fmt.Printf("%T\n", myProfile.Account) // Prints "Any" fmt.Printf("%T\n", myProfile.Account.GetCachedValue()) // Prints "BaseAccount", "ContinuousVestingAccount" or whatever was initially packed in the Any. -// Get the address of the accountt. +// Get the address of the account. accAddr := myProfile.Account.GetCachedValue().(AccountI).GetAddress() ``` @@ -237,15 +191,15 @@ The above `Profile` example is a fictive example used for educational purposes. * the `cryptotypes.PubKey` interface for encoding different types of public keys, * the `sdk.Msg` interface for encoding different `Msg`s in a transaction, -* the `AccountI` interface for encodinig different types of accounts (similar to the above example) in the x/auth query responses, -* the `Evidencei` interface for encoding different types of evidences in the x/evidence module, +* the `AccountI` interface for encoding different types of accounts (similar to the above example) in the x/auth query responses, +* the `EvidenceI` interface for encoding different types of evidences in the x/evidence module, * the `AuthorizationI` interface for encoding different types of x/authz authorizations, -* the [`Validator`](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/staking/types/staking.pb.go#L340-L377) struct that contains information about a validator. +* the [`Validator`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/types/staking.pb.go#L340-L377) struct that contains information about a validator. A real-life example of encoding the pubkey as `Any` inside the Validator struct in x/staking is shown in the following example: ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/staking/types/validator.go#L41-L64 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/types/validator.go#L41-L64 ``` #### `Any`'s TypeURL @@ -321,7 +275,7 @@ The Cosmos SDK `codec.Codec` interface provides support methods `MarshalInterfac Module should register interfaces using `InterfaceRegistry` which provides a mechanism for registering interfaces: `RegisterInterface(protoName string, iface interface{}, impls ...proto.Message)` and implementations: `RegisterImplementations(iface interface{}, impls ...proto.Message)` that can be safely unpacked from Any, similarly to type registration with Amino: ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/codec/types/interface_registry.go#L24-L57 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/codec/types/interface_registry.go#L28-L75 ``` In addition, an `UnpackInterfaces` phase should be introduced to deserialization to unpack interfaces before they're needed. Protobuf types that contain a protobuf `Any` either directly or via one of their members should implement the `UnpackInterfacesMessage` interface: @@ -331,20 +285,3 @@ type UnpackInterfacesMessage interface { UnpackInterfaces(InterfaceUnpacker) error } ``` - -### Custom Stringer - -Using `option (gogoproto.goproto_stringer) = false;` in a proto message definition leads to unexpected behaviour, like returning wrong output or having missing fields in the output. -For that reason a proto Message's `String()` must not be customized, and the `goproto_stringer` option must be avoided. - -A correct YAML output can be obtained through ProtoJSON, using the `JSONToYAML` function: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/codec/yaml.go#L8-L20 -``` - -For example: - -```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/auth/types/account.go#L141-L151 -``` diff --git a/docs/develop/advanced/09-grpc_rest.md b/versioned_docs/version-0.50/learn/advanced/06-grpc_rest.md similarity index 84% rename from docs/develop/advanced/09-grpc_rest.md rename to versioned_docs/version-0.50/learn/advanced/06-grpc_rest.md index a1c648f67..f20a76750 100644 --- a/docs/develop/advanced/09-grpc_rest.md +++ b/versioned_docs/version-0.50/learn/advanced/06-grpc_rest.md @@ -20,14 +20,18 @@ Each node exposes the following endpoints for users to interact with a node, eac The node also exposes some other endpoints, such as the CometBFT P2P endpoint, or the [Prometheus endpoint](https://docs.cometbft.com/v0.37/core/metrics), which are not directly related to the Cosmos SDK. Please refer to the [CometBFT documentation](https://docs.cometbft.com/v0.37/core/configuration) for more information about these endpoints. ::: +:::note +All endpoints are defaulted to localhost and must be modified to be exposed to the public internet. +::: + ## gRPC Server -In the Cosmos SDK, Protobuf is the main [encoding](./06-encoding.md) library. This brings a wide range of Protobuf-based tools that can be plugged into the Cosmos SDK. One such tool is [gRPC](https://grpc.io), a modern open-source high performance RPC framework that has decent client support in several languages. +In the Cosmos SDK, Protobuf is the main [encoding](./encoding) library. This brings a wide range of Protobuf-based tools that can be plugged into the Cosmos SDK. One such tool is [gRPC](https://grpc.io), a modern open-source high performance RPC framework that has decent client support in several languages. Each module exposes a [Protobuf `Query` service](../../build/building-modules/02-messages-and-queries.md#queries) that defines state queries. The `Query` services and a transaction service used to broadcast transactions are hooked up to the gRPC server via the following function inside the application: ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/server/types/app.go#L46-L48 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/server/types/app.go#L46-L48 ``` Note: It is not possible to expose any [Protobuf `Msg` service](../../build/building-modules/02-messages-and-queries.md#messages) endpoints via gRPC. Transactions must be generated and signed using the CLI or programmatically before they can be broadcasted using gRPC. See [Generating, Signing, and Broadcasting Transactions](../../user/run-node/03-txs.md) for more information. @@ -62,7 +66,7 @@ If, for various reasons, you cannot use gRPC (for example, you are building a we [gRPC-gateway](https://grpc-ecosystem.github.io/grpc-gateway/) is a tool to expose gRPC endpoints as REST endpoints. For each gRPC endpoint defined in a Protobuf `Query` service, the Cosmos SDK offers a REST equivalent. For instance, querying a balance could be done via the `/cosmos.bank.v1beta1.QueryAllBalances` gRPC endpoint, or alternatively via the gRPC-gateway `"/cosmos/bank/v1beta1/balances/{address}"` REST endpoint: both will return the same result. For each RPC method defined in a Protobuf `Query` service, the corresponding REST endpoint is defined as an option: ```protobuf reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/query.proto#L23-L30 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/bank/v1beta1/query.proto#L23-L30 ``` For application developers, gRPC-gateway REST routes needs to be wired up to the REST server, this is done by calling the `RegisterGRPCGatewayRoutes` function on the ModuleManager. @@ -71,14 +75,14 @@ For application developers, gRPC-gateway REST routes needs to be wired up to the A [Swagger](https://swagger.io/) (or OpenAPIv2) specification file is exposed under the `/swagger` route on the API server. Swagger is an open specification describing the API endpoints a server serves, including description, input arguments, return types and much more about each endpoint. -Enabling the `/swagger` endpoint is configurable inside `~/.simapp/config/app.toml` via the `api.swagger` field, which is set to true by default. +Enabling the `/swagger` endpoint is configurable inside `~/.simapp/config/app.toml` via the `api.swagger` field, which is set to false by default. For application developers, you may want to generate your own Swagger definitions based on your custom modules. -The Cosmos SDK's [Swagger generation script](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/scripts/protoc-swagger-gen.sh) is a good place to start. +The Cosmos SDK's [Swagger generation script](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/scripts/protoc-swagger-gen.sh) is a good place to start. ## CometBFT RPC -Independently from the Cosmos SDK, CometBFT also exposes a RPC server. This RPC server can be configured by tuning parameters under the `rpc` table in the `~/.simapp/config/config.toml`, the default listening address is `tcp://localhost:26657`. An OpenAPI specification of all CometBFT RPC endpoints is available [here](https://docs.cometbft.com/master/rpc/). +Independently from the Cosmos SDK, CometBFT also exposes a RPC server. This RPC server can be configured by tuning parameters under the `rpc` table in the `~/.simapp/config/config.toml`, the default listening address is `tcp://localhost:26657`. An OpenAPI specification of all CometBFT RPC endpoints is available [here](https://docs.cometbft.com/main/rpc/). Some CometBFT RPC endpoints are directly related to the Cosmos SDK: @@ -86,10 +90,11 @@ Some CometBFT RPC endpoints are directly related to the Cosmos SDK: * any Protobuf fully-qualified service method, such as `/cosmos.bank.v1beta1.Query/AllBalances`. The `data` field should then include the method's request parameter(s) encoded as bytes using Protobuf. * `/app/simulate`: this will simulate a transaction, and return some information such as gas used. * `/app/version`: this will return the application's version. - * `/store/{path}`: this will query the store directly. + * `/store/{storeName}/key`: this will directly query the named store for data associated with the key represented in the `data` parameter. + * `/store/{storeName}/subspace`: this will directly query the named store for key/value pairs in which the key has the value of the `data` parameter as a prefix. * `/p2p/filter/addr/{port}`: this will return a filtered list of the node's P2P peers by address port. * `/p2p/filter/id/{id}`: this will return a filtered list of the node's P2P peers by ID. -* `/broadcast_tx_{aync,async,commit}`: these 3 endpoint will broadcast a transaction to other peers. CLI, gRPC and REST expose [a way to broadcast transations](01-transactions.md#broadcasting-the-transaction), but they all use these 3 CometBFT RPCs under the hood. +* `/broadcast_tx_{aync,async,commit}`: these 3 endpoints will broadcast a transaction to other peers. CLI, gRPC and REST expose [a way to broadcast transactions](./01-transactions.md#broadcasting-the-transaction), but they all use these 3 CometBFT RPCs under the hood. ## Comparison Table diff --git a/versioned_docs/version-0.50/develop/advanced/07-cli.md b/versioned_docs/version-0.50/learn/advanced/07-cli.md similarity index 93% rename from versioned_docs/version-0.50/develop/advanced/07-cli.md rename to versioned_docs/version-0.50/learn/advanced/07-cli.md index 140b13490..2762d83e6 100644 --- a/versioned_docs/version-0.50/develop/advanced/07-cli.md +++ b/versioned_docs/version-0.50/learn/advanced/07-cli.md @@ -5,7 +5,7 @@ sidebar_position: 1 # Command-Line Interface :::note Synopsis -This document describes how command-line interface (CLI) works on a high-level, for an [**application**](../beginner/00-overview-app.md). A separate document for implementing a CLI for a Cosmos SDK [**module**](../../build/building-modules/00-intro.md) can be found [here](../../build/building-modules/09-module-interfaces.md#cli). +This document describes how command-line interface (CLI) works on a high-level, for an [**application**](../beginner/00-app-anatomy.md). A separate document for implementing a CLI for a Cosmos SDK [**module**](../../build/building-modules/00-intro.md) can be found [here](../../build/building-modules/09-module-interfaces.md#cli). ::: ## Command-Line Interface @@ -153,7 +153,7 @@ Flags are added to commands directly (generally in the [module's CLI file](../.. ## Environment variables -Each flag is bound to it's respecteve named environment variable. Then name of the environment variable consist of two parts - capital case `basename` followed by flag name of the flag. `-` must be substituted with `_`. For example flag `--home` for application with basename `GAIA` is bound to `GAIA_HOME`. It allows reducing the amount of flags typed for routine operations. For example instead of: +Each flag is bound to its respective named environment variable. Then name of the environment variable consist of two parts - capital case `basename` followed by flag name of the flag. `-` must be substituted with `_`. For example flag `--home` for application with basename `GAIA` is bound to `GAIA_HOME`. It allows reducing the amount of flags typed for routine operations. For example instead of: ```shell gaia --home=./ --node= --chain-id="testchain-1" --keyring-backend=test tx ... --from= @@ -174,7 +174,7 @@ gaia tx ... --from= ## Configurations -It is vital that the root command of an application uses `PersistentPreRun()` cobra command property for executing the command, so all child commands have access to the server and client contexts. These contexts are set as their default values initially and maybe modified, scoped to the command, in their respective `PersistentPreRun()` functions. Note that the `client.Context` is typically pre-populated with "default" values that may be useful for all commands to inherit and override if necessary. +It is vital that the root command of an application uses `PersistentPreRun()` cobra command property for executing the command, so all child commands have access to the server and client contexts. These contexts are set as their default values initially and may be modified, scoped to the command, in their respective `PersistentPreRun()` functions. Note that the `client.Context` is typically pre-populated with "default" values that may be useful for all commands to inherit and override if necessary. Here is an example of an `PersistentPreRun()` function from `simapp`: @@ -187,7 +187,7 @@ The `SetCmdClientContextHandler` call reads persistent flags via `ReadPersistent The `InterceptConfigsPreRunHandler` call creates a viper literal, default `server.Context`, and a logger and sets that on the root command's `Context`. The `server.Context` will be modified and saved to disk. The internal `interceptConfigs` call reads or creates a CometBFT configuration based on the home path provided. In addition, `interceptConfigs` also reads and loads the application configuration, `app.toml`, and binds that to the `server.Context` viper literal. This is vital so the application can get access to not only the CLI flags, but also to the application configuration values provided by this file. :::tip -When willing to configure which logger is used, do not to use `InterceptConfigsPreRunHandler`, which sets the default SDK logger, but instead use `InterceptConfigsAndCreateContext` and set the server context and the logger manually: +When willing to configure which logger is used, do not use `InterceptConfigsPreRunHandler`, which sets the default SDK logger, but instead use `InterceptConfigsAndCreateContext` and set the server context and the logger manually: ```diff -return server.InterceptConfigsPreRunHandler(cmd, customAppTemplate, customAppConfig, customCMTConfig) diff --git a/versioned_docs/version-0.50/develop/advanced/08-events.md b/versioned_docs/version-0.50/learn/advanced/08-events.md similarity index 99% rename from versioned_docs/version-0.50/develop/advanced/08-events.md rename to versioned_docs/version-0.50/learn/advanced/08-events.md index cf2de31be..410e20ade 100644 --- a/versioned_docs/version-0.50/develop/advanced/08-events.md +++ b/versioned_docs/version-0.50/learn/advanced/08-events.md @@ -9,7 +9,7 @@ sidebar_position: 1 :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK application](../beginner/00-overview-app.md) +* [Anatomy of a Cosmos SDK application](../beginner/00-app-anatomy.md) * [CometBFT Documentation on Events](https://docs.cometbft.com/v0.37/spec/abci/abci++_basic_concepts#events) ::: diff --git a/docs/develop/advanced/11-telemetry.md b/versioned_docs/version-0.50/learn/advanced/09-telemetry.md similarity index 98% rename from docs/develop/advanced/11-telemetry.md rename to versioned_docs/version-0.50/learn/advanced/09-telemetry.md index 2be2dc680..c5916544f 100644 --- a/docs/develop/advanced/11-telemetry.md +++ b/versioned_docs/version-0.50/learn/advanced/09-telemetry.md @@ -18,8 +18,8 @@ To query active metrics (see retention note above) you have to enable API server ## Emitting metrics If telemetry is enabled via configuration, a single global metrics collector is registered via the -[go-metrics](https://github.com/armon/go-metrics) library. This allows emitting and collecting -metrics through simple [API](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/telemetry/wrapper.go). Example: +[go-metrics](https://github.com/hashicorp/go-metrics) library. This allows emitting and collecting +metrics through simple [API](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/telemetry/wrapper.go). Example: ```go func EndBlocker(ctx sdk.Context, k keeper.Keeper) { diff --git a/versioned_docs/version-0.50/develop/advanced/10-ocap.md b/versioned_docs/version-0.50/learn/advanced/10-ocap.md similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/10-ocap.md rename to versioned_docs/version-0.50/learn/advanced/10-ocap.md diff --git a/docs/develop/advanced/12-runtx_middleware.md b/versioned_docs/version-0.50/learn/advanced/11-runtx_middleware.md similarity index 94% rename from docs/develop/advanced/12-runtx_middleware.md rename to versioned_docs/version-0.50/learn/advanced/11-runtx_middleware.md index a62876566..f083a7788 100644 --- a/docs/develop/advanced/12-runtx_middleware.md +++ b/versioned_docs/version-0.50/learn/advanced/11-runtx_middleware.md @@ -8,12 +8,12 @@ sidebar_position: 1 Depending on the panic type different handler is used, for instance the default one prints an error log message. Recovery middleware is used to add custom panic recovery for Cosmos SDK application developers. -More context can found in the corresponding [ADR-022](../../build/architecture/adr-022-custom-panic-handling.md) and the implementation in [recovery.go](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/baseapp/recovery.go). +More context can found in the corresponding [ADR-022](../../build/architecture/adr-022-custom-panic-handling.md) and the implementation in [recovery.go](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/recovery.go). ## Interface ```go reference -https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/baseapp/recovery.go#L11-L14 +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/recovery.go#L14-L17 ``` `recoveryObj` is a return value for `recover()` function from the `buildin` Go package. diff --git a/docs/develop/advanced/13-simulation.md b/versioned_docs/version-0.50/learn/advanced/12-simulation.md similarity index 89% rename from docs/develop/advanced/13-simulation.md rename to versioned_docs/version-0.50/learn/advanced/12-simulation.md index 1ee27c861..dfbcddd0d 100644 --- a/docs/develop/advanced/13-simulation.md +++ b/versioned_docs/version-0.50/learn/advanced/12-simulation.md @@ -7,8 +7,8 @@ sidebar_position: 1 The Cosmos SDK offers a full fledged simulation framework to fuzz test every message defined by a module. -On the Cosmos SDK, this functionality is provided by [`SimApp`](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/app_v2.go), which is a -`Baseapp` application that is used for running the [`simulation`](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/simulation) module. +On the Cosmos SDK, this functionality is provided by [`SimApp`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app_v2.go), which is a +`Baseapp` application that is used for running the [`simulation`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/simulation) module. This module defines all the simulation logic as well as the operations for randomized parameters like accounts, balances etc. @@ -41,7 +41,7 @@ failure type: Each simulation must receive a set of inputs (_i.e_ flags) such as the number of blocks that the simulation is run, seed, block size, etc. -Check the full list of flags [here](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/simulation/client/cli/flags.go#L33-L57). +Check the full list of flags [here](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/simulation/client/cli/flags.go#L35-L59). ## Simulator Modes @@ -53,7 +53,7 @@ In addition to the various inputs and commands, the simulator runs in three mode This mode is helpful for running simulations on a known state such as a live network export where a new (mostly likely breaking) version of the application needs to be tested. 3. From a `params.json` file where the initial state is pseudo-randomly generated but the module and simulation parameters can be provided manually. This allows for a more controlled and deterministic simulation setup while allowing the state space to still be pseudo-randomly simulated. - The list of available parameters are listed [here](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/simulation/client/cli/flags.go#L59-L78). + The list of available parameters are listed [here](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/simulation/client/cli/flags.go#L59-L78). :::tip These modes are not mutually exclusive. So you can for example run a randomly @@ -63,7 +63,7 @@ generated genesis state (`1`) with manually generated simulation params (`3`). ## Usage This is a general example of how simulations are run. For more specific examples -check the Cosmos SDK [Makefile](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/Makefile#L282-L318). +check the Cosmos SDK [Makefile](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/Makefile#L282-L318). ```bash $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \ @@ -90,11 +90,11 @@ Here are some suggestions when encountering a simulation failure: * Run invariants on every operation with `-SimulateEveryOperation`. _Note_: this will slow down your simulation **a lot**. * Try adding logs to operations that are not logged. You will have to define a - [Logger](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/staking/keeper/keeper.go#L65-L68) on your `Keeper`. + [Logger](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/keeper/keeper.go#L65-L68) on your `Keeper`. ## Use simulation in your Cosmos SDK-based application -Learn how you can integrate the simulation into your Cosmos SDK-based application: +Learn how you can build the simulation into your Cosmos SDK-based application: * Application Simulation Manager * [Building modules: Simulator](../../build/building-modules/14-simulator.md) diff --git a/docs/develop/advanced/14-proto-docs.md b/versioned_docs/version-0.50/learn/advanced/13-proto-docs.md similarity index 100% rename from docs/develop/advanced/14-proto-docs.md rename to versioned_docs/version-0.50/learn/advanced/13-proto-docs.md diff --git a/docs/develop/advanced/16-upgrade.md b/versioned_docs/version-0.50/learn/advanced/15-upgrade.md similarity index 100% rename from docs/develop/advanced/16-upgrade.md rename to versioned_docs/version-0.50/learn/advanced/15-upgrade.md diff --git a/docs/develop/advanced/17-config.md b/versioned_docs/version-0.50/learn/advanced/16-config.md similarity index 100% rename from docs/develop/advanced/17-config.md rename to versioned_docs/version-0.50/learn/advanced/16-config.md diff --git a/versioned_docs/version-0.50/learn/advanced/17-autocli.md b/versioned_docs/version-0.50/learn/advanced/17-autocli.md new file mode 100644 index 000000000..516c7f495 --- /dev/null +++ b/versioned_docs/version-0.50/learn/advanced/17-autocli.md @@ -0,0 +1,196 @@ +--- +sidebar_position: 1 +--- + +# AutoCLI + +:::note Synopsis +This document details how to build CLI and REST interfaces for a module. Examples from various Cosmos SDK modules are included. +::: + +:::note Pre-requisite Readings + +* [CLI](https://docs.cosmos.network/main/core/cli) + +::: + +The `autocli` (also known as `client/v2`) package is a [Go library](https://pkg.go.dev/cosmossdk.io/client/v2/autocli) for generating CLI (command line interface) interfaces for Cosmos SDK-based applications. It provides a simple way to add CLI commands to your application by generating them automatically based on your gRPC service definitions. Autocli generates CLI commands and flags directly from your protobuf messages, including options, input parameters, and output parameters. This means that you can easily add a CLI interface to your application without having to manually create and manage commands. + +## Overview + +`autocli` generates CLI commands and flags for each method defined in your gRPC service. By default, it generates commands for each gRPC services. The commands are named based on the name of the service method. + +For example, given the following protobuf definition for a service: + +```protobuf +service MyService { + rpc MyMethod(MyRequest) returns (MyResponse) {} +} +``` + +For instance, `autocli` would generate a command named `my-method` for the `MyMethod` method. The command will have flags for each field in the `MyRequest` message. + +It is possible to customize the generation of transactions and queries by defining options for each service. + +## Application Wiring + +Here are the steps to use AutoCLI: + +1. Ensure your app's modules implements the `appmodule.AppModule` interface. +2. (optional) Configure how behave `autocli` command generation, by implementing the `func (am AppModule) AutoCLIOptions() *autocliv1.ModuleOptions` method on the module. +3. Use the `autocli.AppOptions` struct to specify the modules you defined. If you are using `depinject` / app v2, it can automatically create an instance of `autocli.AppOptions` based on your app's configuration. +4. Use the `EnhanceRootCommand()` method provided by `autocli` to add the CLI commands for the specified modules to your root command. + +:::tip +AutoCLI is additive only, meaning _enhancing_ the root command will only add subcommands that are not already registered. This means that you can use AutoCLI alongside other custom commands within your app. +::: + +Here's an example of how to use `autocli` in your app: + +``` go +// Define your app's modules +testModules := map[string]appmodule.AppModule{ + "testModule": &TestModule{}, +} + +// Define the autocli AppOptions +autoCliOpts := autocli.AppOptions{ + Modules: testModules, +} + +// Create the root command +rootCmd := &cobra.Command{ + Use: "app", +} + +if err := appOptions.EnhanceRootCommand(rootCmd); err != nil { + return err +} + +// Run the root command +if err := rootCmd.Execute(); err != nil { + return err +} +``` + +### Keyring + +`autocli` supports a keyring for key name resolving and signing transactions. Providing a keyring is optional, but if you want to use the `autocli` generated commands to sign transactions, you must provide a keyring. + +:::tip +This provides a better UX as it allows to resolve key names directly from the keyring in all transactions and commands. + +```sh + q bank balances alice + tx bank send alice bob 1000denom +``` + +::: + +The keyring to be provided to `client/v2` must match the `client/v2` keyring interface. The Cosmos SDK keyring and Hubl keyring both implement this interface. +The keyring should be provided in the `appOptions` struct as follows, and can be gotten from the client context: + +:::warning +When using AutoCLI the keyring will only be created once and before any command flag parsing. +::: + +```go +// Get the keyring from the client context +keyring := ctx.Keyring +// Set the keyring in the appOptions +appOptions.Keyring = keyring + +err := autoCliOpts.EnhanceRootCommand(rootCmd) +... +``` + +## Module Wiring & Customization + +The `AutoCLIOptions()` method on your module allows to specify custom commands, sub-commands or flags for each service, as it was a `cobra.Command` instance, within the `RpcCommandOptions` struct. Defining such options will customize the behavior of the `autocli` command generation, which by default generates a command for each method in your gRPC service. + +```go +*autocliv1.RpcCommandOptions{ + RpcMethod: "Params", // The name of the gRPC service + Use: "params", // Command usage that is displayed in the help + Short: "Query the parameters of the governance process", // Short description of the command + Long: "Query the parameters of the governance process. Specify specific param types (voting|tallying|deposit) to filter results.", // Long description of the command + PositionalArgs: []*autocliv1.PositionalArgDescriptor{ + {ProtoField: "params_type", Optional: true}, // Transform a flag into a positional argument + }, +} +``` + +### Specifying Subcommands + +By default, `autocli` generates a command for each method in your gRPC service. However, you can specify subcommands to group related commands together. To specify subcommands, use the `autocliv1.ServiceCommandDescriptor` struct. + +This example shows how to use the `autocliv1.ServiceCommandDescriptor` struct to group related commands together and specify subcommands in your gRPC service by defining an instance of `autocliv1.ModuleOptions` in your `autocli.go`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-beta.0/x/gov/autocli.go#L94-L97 +``` + +### Positional Arguments + +By default `autocli` generates a flag for each field in your protobuf message. However, you can choose to use positional arguments instead of flags for certain fields. + +To add positional arguments to a command, use the `autocliv1.PositionalArgDescriptor` struct, as seen in the example below. Specify the `ProtoField` parameter, which is the name of the protobuf field that should be used as the positional argument. In addition, if the parameter is a variable-length argument, you can specify the `Varargs` parameter as `true`. This can only be applied to the last positional parameter, and the `ProtoField` must be a repeated field. + +Here's an example of how to define a positional argument for the `Account` method of the `auth` service: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-beta.0/x/auth/autocli.go#L25-L30 +``` + +Then the command can be used as follows, instead of having to specify the `--address` flag: + +```bash + query auth account cosmos1abcd...xyz +``` + +### Customising Flag Names + +By default, `autocli` generates flag names based on the names of the fields in your protobuf message. However, you can customise the flag names by providing a `FlagOptions`. This parameter allows you to specify custom names for flags based on the names of the message fields. + +For example, if you have a message with the fields `test` and `test1`, you can use the following naming options to customise the flags: + +``` go +autocliv1.RpcCommandOptions{ + FlagOptions: map[string]*autocliv1.FlagOptions{ + "test": { Name: "custom_name", }, + "test1": { Name: "other_name", }, + }, +} +``` + +`FlagsOptions` is defined like sub commands in the `AutoCLIOptions()` method on your module. + +### Combining AutoCLI with Other Commands Within A Module + +AutoCLI can be used alongside other commands within a module. For example, the `gov` module uses AutoCLI to generate commands for the `query` subcommand, but also defines custom commands for the `proposer` subcommands. + +In order to enable this behavior, set in `AutoCLIOptions()` the `EnhanceCustomCommand` field to `true`, for the command type (queries and/or transactions) you want to enhance. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/fa4d87ef7e6d87aaccc94c337ffd2fe90fcb7a9d/x/gov/autocli.go#L98 +``` + +If not set to true, `AutoCLI` will not generate commands for the module if there are already commands registered for the module (when `GetTxCmd()` or `GetTxCmd()` are defined). + +### Use AutoCLI for non module commands + +It is possible to use `AutoCLI` for non module commands. The trick is still to implement the `appmodule.Module` interface and append it to the `appOptions.ModuleOptions` map. + +For example, here is how the SDK does it for `cometbft` gRPC commands: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/julien/autocli-comet/client/grpc/cmtservice/autocli.go#L52-L71 +``` + +## Summary + +`autocli` let you generate CLI to your Cosmos SDK-based applications without any cobra boilerplate. It allows you to easily generate CLI commands and flags from your protobuf messages, and provides many options for customising the behavior of your CLI application. + +To further enhance your CLI experience with Cosmos SDK-based blockchains, you can use `hubl`. `hubl` is a tool that allows you to query any Cosmos SDK-based blockchain using the new AutoCLI feature of the Cosmos SDK. With `hubl`, you can easily configure a new chain and query modules with just a few simple commands. + +For more information on `hubl`, including how to configure a new chain and query a module, see the [Hubl documentation](https://docs.cosmos.network/main/tooling/hubl). diff --git a/versioned_docs/version-0.50/develop/advanced/_category_.json b/versioned_docs/version-0.50/learn/advanced/_category_.json similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/_category_.json rename to versioned_docs/version-0.50/learn/advanced/_category_.json diff --git a/versioned_docs/version-0.50/develop/advanced/baseapp_state-begin_block.png b/versioned_docs/version-0.50/learn/advanced/baseapp_state-begin_block.png similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/baseapp_state-begin_block.png rename to versioned_docs/version-0.50/learn/advanced/baseapp_state-begin_block.png diff --git a/versioned_docs/version-0.50/develop/advanced/baseapp_state-checktx.png b/versioned_docs/version-0.50/learn/advanced/baseapp_state-checktx.png similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/baseapp_state-checktx.png rename to versioned_docs/version-0.50/learn/advanced/baseapp_state-checktx.png diff --git a/versioned_docs/version-0.50/develop/advanced/baseapp_state-commit.png b/versioned_docs/version-0.50/learn/advanced/baseapp_state-commit.png similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/baseapp_state-commit.png rename to versioned_docs/version-0.50/learn/advanced/baseapp_state-commit.png diff --git a/versioned_docs/version-0.50/develop/advanced/baseapp_state-deliver_tx.png b/versioned_docs/version-0.50/learn/advanced/baseapp_state-deliver_tx.png similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/baseapp_state-deliver_tx.png rename to versioned_docs/version-0.50/learn/advanced/baseapp_state-deliver_tx.png diff --git a/versioned_docs/version-0.50/develop/advanced/baseapp_state-initchain.png b/versioned_docs/version-0.50/learn/advanced/baseapp_state-initchain.png similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/baseapp_state-initchain.png rename to versioned_docs/version-0.50/learn/advanced/baseapp_state-initchain.png diff --git a/versioned_docs/version-0.50/develop/advanced/baseapp_state-prepareproposal.png b/versioned_docs/version-0.50/learn/advanced/baseapp_state-prepareproposal.png similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/baseapp_state-prepareproposal.png rename to versioned_docs/version-0.50/learn/advanced/baseapp_state-prepareproposal.png diff --git a/versioned_docs/version-0.50/develop/advanced/baseapp_state-processproposal.png b/versioned_docs/version-0.50/learn/advanced/baseapp_state-processproposal.png similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/baseapp_state-processproposal.png rename to versioned_docs/version-0.50/learn/advanced/baseapp_state-processproposal.png diff --git a/versioned_docs/version-0.50/develop/advanced/baseapp_state.png b/versioned_docs/version-0.50/learn/advanced/baseapp_state.png similarity index 100% rename from versioned_docs/version-0.50/develop/advanced/baseapp_state.png rename to versioned_docs/version-0.50/learn/advanced/baseapp_state.png diff --git a/versioned_docs/version-0.50/develop/beginner/00-overview-app.md b/versioned_docs/version-0.50/learn/beginner/00-app-anatomy.md similarity index 100% rename from versioned_docs/version-0.50/develop/beginner/00-overview-app.md rename to versioned_docs/version-0.50/learn/beginner/00-app-anatomy.md diff --git a/versioned_docs/version-0.50/develop/beginner/01-tx-lifecycle.md b/versioned_docs/version-0.50/learn/beginner/01-tx-lifecycle.md similarity index 99% rename from versioned_docs/version-0.50/develop/beginner/01-tx-lifecycle.md rename to versioned_docs/version-0.50/learn/beginner/01-tx-lifecycle.md index f8828e714..e3831ae3b 100644 --- a/versioned_docs/version-0.50/develop/beginner/01-tx-lifecycle.md +++ b/versioned_docs/version-0.50/learn/beginner/01-tx-lifecycle.md @@ -10,7 +10,7 @@ This document describes the lifecycle of a transaction from creation to committe :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK Application](./00-overview-app.md) +* [Anatomy of a Cosmos SDK Application](./00-app-anatomy.md) ::: ## Creation diff --git a/versioned_docs/version-0.50/develop/beginner/02-query-lifecycle.md b/versioned_docs/version-0.50/learn/beginner/02-query-lifecycle.md similarity index 100% rename from versioned_docs/version-0.50/develop/beginner/02-query-lifecycle.md rename to versioned_docs/version-0.50/learn/beginner/02-query-lifecycle.md diff --git a/versioned_docs/version-0.50/develop/beginner/03-accounts.md b/versioned_docs/version-0.50/learn/beginner/03-accounts.md similarity index 99% rename from versioned_docs/version-0.50/develop/beginner/03-accounts.md rename to versioned_docs/version-0.50/learn/beginner/03-accounts.md index 3e0bd45aa..7280108ac 100644 --- a/versioned_docs/version-0.50/develop/beginner/03-accounts.md +++ b/versioned_docs/version-0.50/learn/beginner/03-accounts.md @@ -11,7 +11,7 @@ This document describes the in-built account and public key system of the Cosmos :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK Application](./00-overview-app.md) +* [Anatomy of a Cosmos SDK Application](./00-app-anatomy.md) ::: diff --git a/versioned_docs/version-0.50/develop/beginner/04-gas-fees.md b/versioned_docs/version-0.50/learn/beginner/04-gas-fees.md similarity index 99% rename from versioned_docs/version-0.50/develop/beginner/04-gas-fees.md rename to versioned_docs/version-0.50/learn/beginner/04-gas-fees.md index b822d2bdf..2e04af462 100644 --- a/versioned_docs/version-0.50/develop/beginner/04-gas-fees.md +++ b/versioned_docs/version-0.50/learn/beginner/04-gas-fees.md @@ -10,7 +10,7 @@ This document describes the default strategies to handle gas and fees within a C :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK Application](./00-overview-app.md) +* [Anatomy of a Cosmos SDK Application](./00-app-anatomy.md) ::: diff --git a/versioned_docs/version-0.50/develop/beginner/_category_.json b/versioned_docs/version-0.50/learn/beginner/_category_.json similarity index 100% rename from versioned_docs/version-0.50/develop/beginner/_category_.json rename to versioned_docs/version-0.50/learn/beginner/_category_.json diff --git a/versioned_docs/version-0.50/develop/intro/00-what-is-sdk.md b/versioned_docs/version-0.50/learn/intro/00-overview.md similarity index 98% rename from versioned_docs/version-0.50/develop/intro/00-what-is-sdk.md rename to versioned_docs/version-0.50/learn/intro/00-overview.md index 72722b20c..616d33169 100644 --- a/versioned_docs/version-0.50/develop/intro/00-what-is-sdk.md +++ b/versioned_docs/version-0.50/learn/intro/00-overview.md @@ -6,7 +6,7 @@ sidebar_position: 1 The [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is an open-source framework for building multi-asset public Proof-of-Stake (PoS) blockchains, like the Cosmos Hub, as well as permissioned Proof-of-Authority (PoA) blockchains. Blockchains built with the Cosmos SDK are generally referred to as **application-specific blockchains**. -The goal of the Cosmos SDK is to allow developers to easily create custom blockchains from scratch that can natively interoperate with other blockchains. We envision the Cosmos SDK as the npm-like framework to build secure blockchain applications on top of [CometBFT](https://github.com/cometbft/cometbft). SDK-based blockchains are built out of composable [modules](../../build/building-modules/01-intro.md), most of which are open-source and readily available for any developers to use. Anyone can create a module for the Cosmos SDK, and integrating already-built modules is as simple as importing them into your blockchain application. What's more, the Cosmos SDK is a capabilities-based system that allows developers to better reason about the security of interactions between modules. For a deeper look at capabilities, jump to [Object-Capability Model](../advanced/10-ocap.md). +The goal of the Cosmos SDK is to allow developers to easily create custom blockchains from scratch that can natively interoperate with other blockchains. We envision the Cosmos SDK as the npm-like framework to build secure blockchain applications on top of [CometBFT](https://github.com/cometbft/cometbft). SDK-based blockchains are built out of composable [modules](../../build/building-modules/00-intro.md), most of which are open-source and readily available for any developers to use. Anyone can create a module for the Cosmos SDK, and integrating already-built modules is as simple as importing them into your blockchain application. What's more, the Cosmos SDK is a capabilities-based system that allows developers to better reason about the security of interactions between modules. For a deeper look at capabilities, jump to [Object-Capability Model](../advanced/10-ocap.md). ## What are Application-Specific Blockchains diff --git a/versioned_docs/version-0.50/develop/intro/01-why-app-specific.md b/versioned_docs/version-0.50/learn/intro/01-why-app-specific.md similarity index 100% rename from versioned_docs/version-0.50/develop/intro/01-why-app-specific.md rename to versioned_docs/version-0.50/learn/intro/01-why-app-specific.md diff --git a/versioned_docs/version-0.50/develop/intro/02-sdk-app-architecture.md b/versioned_docs/version-0.50/learn/intro/02-sdk-app-architecture.md similarity index 100% rename from versioned_docs/version-0.50/develop/intro/02-sdk-app-architecture.md rename to versioned_docs/version-0.50/learn/intro/02-sdk-app-architecture.md diff --git a/versioned_docs/version-0.50/develop/intro/03-sdk-design.md b/versioned_docs/version-0.50/learn/intro/03-sdk-design.md similarity index 99% rename from versioned_docs/version-0.50/develop/intro/03-sdk-design.md rename to versioned_docs/version-0.50/learn/intro/03-sdk-design.md index ae4a05611..78afb1298 100644 --- a/versioned_docs/version-0.50/develop/intro/03-sdk-design.md +++ b/versioned_docs/version-0.50/learn/intro/03-sdk-design.md @@ -15,7 +15,7 @@ Here is a simplified view of how transactions are handled by an application buil ## `baseapp` -`baseapp` is the boilerplate implementation of a Cosmos SDK application. It comes with an implementation of the ABCI to handle the connection with the underlying consensus engine. Typically, a Cosmos SDK application extends `baseapp` by embedding it in [`app.go`]../beginner/00-overview-app.md#core-application-file). +`baseapp` is the boilerplate implementation of a Cosmos SDK application. It comes with an implementation of the ABCI to handle the connection with the underlying consensus engine. Typically, a Cosmos SDK application extends `baseapp` by embedding it in [`app.go`]../beginner/00-app-anatomy.md#core-application-file). Here is an example of this from `simapp`, the Cosmos SDK demonstration app: diff --git a/versioned_docs/version-0.50/develop/intro/_category_.json b/versioned_docs/version-0.50/learn/intro/_category_.json similarity index 100% rename from versioned_docs/version-0.50/develop/intro/_category_.json rename to versioned_docs/version-0.50/learn/intro/_category_.json diff --git a/versioned_docs/version-0.50/user/run-node/00-keyring.md b/versioned_docs/version-0.50/user/run-node/00-keyring.md index a1fd99b72..0ed06d412 100644 --- a/versioned_docs/version-0.50/user/run-node/00-keyring.md +++ b/versioned_docs/version-0.50/user/run-node/00-keyring.md @@ -5,7 +5,7 @@ sidebar_position: 1 # Setting up the keyring :::note Synopsis -This document describes how to configure and use the keyring and its various backends for an [**application**](../../develop/beginner/00-overview-app.md). +This document describes how to configure and use the keyring and its various backends for an [**application**](../../learn/beginner/00-app-anatomy.md). ::: The keyring holds the private/public keypairs used to interact with a node. For instance, a validator key needs to be set up before running the blockchain node, so that blocks can be correctly signed. The private key can be stored in different locations, called "backends", such as a file or the operating system's own key storage. diff --git a/versioned_docs/version-0.50/user/run-node/01-run-node.md b/versioned_docs/version-0.50/user/run-node/01-run-node.md index 95c814e82..7bd1a9c79 100644 --- a/versioned_docs/version-0.50/user/run-node/01-run-node.md +++ b/versioned_docs/version-0.50/user/run-node/01-run-node.md @@ -10,7 +10,7 @@ Now that the application is ready and the keyring populated, it's time to see ho :::note Pre-requisite Readings -* [Anatomy of a Cosmos SDK Application](../../develop/beginner/00-overview-app.md) +* [Anatomy of a Cosmos SDK Application](../../learn/beginner/00-app-anatomy.md) * [Setting up the keyring](./00-keyring.md) ::: @@ -87,7 +87,7 @@ simd genesis add-genesis-account $MY_VALIDATOR_ADDRESS 100000000000stake Recall that `$MY_VALIDATOR_ADDRESS` is a variable that holds the address of the `my_validator` key in the [keyring](./00-keyring.md#adding-keys-to-the-keyring). Also note that the tokens in the Cosmos SDK have the `{amount}{denom}` format: `amount` is is a 18-digit-precision decimal number, and `denom` is the unique token identifier with its denomination key (e.g. `atom` or `uatom`). Here, we are granting `stake` tokens, as `stake` is the token identifier used for staking in [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/main/simapp). For your own chain with its own staking denom, that token identifier should be used instead. -Now that your account has some tokens, you need to add a validator to your chain. Validators are special full-nodes that participate in the consensus process (implemented in the [underlying consensus engine](../../develop/intro/02-sdk-app-architecture.md#cometbft)) in order to add new blocks to the chain. Any account can declare its intention to become a validator operator, but only those with sufficient delegation get to enter the active set (for example, only the top 125 validator candidates with the most delegation get to be validators in the Cosmos Hub). For this guide, you will add your local node (created via the `init` command above) as a validator of your chain. Validators can be declared before a chain is first started via a special transaction included in the genesis file called a `gentx`: +Now that your account has some tokens, you need to add a validator to your chain. Validators are special full-nodes that participate in the consensus process (implemented in the [underlying consensus engine](../../learn/intro/02-sdk-app-architecture.md#cometbft)) in order to add new blocks to the chain. Any account can declare its intention to become a validator operator, but only those with sufficient delegation get to enter the active set (for example, only the top 125 validator candidates with the most delegation get to be validators in the Cosmos Hub). For this guide, you will add your local node (created via the `init` command above) as a validator of your chain. Validators can be declared before a chain is first started via a special transaction included in the genesis file called a `gentx`: ```bash # Create a gentx. diff --git a/versioned_docs/version-0.50/user/run-node/02-interact-node.md b/versioned_docs/version-0.50/user/run-node/02-interact-node.md index 48193504a..a511aec41 100644 --- a/versioned_docs/version-0.50/user/run-node/02-interact-node.md +++ b/versioned_docs/version-0.50/user/run-node/02-interact-node.md @@ -10,7 +10,7 @@ There are multiple ways to interact with a node: using the CLI, using gRPC or us :::note Pre-requisite Readings -* [gRPC, REST and CometBFT Endpoints](../../develop/advanced/06-grpc_rest.md) +* [gRPC, REST and CometBFT Endpoints](../../learn/advanced/06-grpc_rest.md) * [Running a Node](./01-run-node.md) ::: @@ -54,7 +54,7 @@ You should see two delegations, the first one made from the `gentx`, and the sec ## Using gRPC -The Protobuf ecosystem developed tools for different use cases, including code-generation from `*.proto` files into various languages. These tools allow the building of clients easily. Often, the client connection (i.e. the transport) can be plugged and replaced very easily. Let's explore one of the most popular transport: [gRPC](../../develop/advanced/06-grpc_rest.md). +The Protobuf ecosystem developed tools for different use cases, including code-generation from `*.proto` files into various languages. These tools allow the building of clients easily. Often, the client connection (i.e. the transport) can be plugged and replaced very easily. Let's explore one of the most popular transport: [gRPC](../../learn/advanced/06-grpc_rest.md). Since the code generation library largely depends on your own tech stack, we will only present three alternatives: @@ -248,7 +248,7 @@ CosmJS documentation can be found at [https://cosmos.github.io/cosmjs](https://c ## Using the REST Endpoints -As described in the [gRPC guide](../../develop/advanced/06-grpc_rest.md), all gRPC services on the Cosmos SDK are made available for more convenient REST-based queries through gRPC-gateway. The format of the URL path is based on the Protobuf service method's full-qualified name, but may contain small customizations so that final URLs look more idiomatic. For example, the REST endpoint for the `cosmos.bank.v1beta1.Query/AllBalances` method is `GET /cosmos/bank/v1beta1/balances/{address}`. Request arguments are passed as query parameters. +As described in the [gRPC guide](../../learn/advanced/06-grpc_rest.md), all gRPC services on the Cosmos SDK are made available for more convenient REST-based queries through gRPC-gateway. The format of the URL path is based on the Protobuf service method's full-qualified name, but may contain small customizations so that final URLs look more idiomatic. For example, the REST endpoint for the `cosmos.bank.v1beta1.Query/AllBalances` method is `GET /cosmos/bank/v1beta1/balances/{address}`. Request arguments are passed as query parameters. Note that the REST endpoints are not enabled by default. To enable them, edit the `api` section of your `~/.simapp/config/app.toml` file: diff --git a/versioned_docs/version-0.50/user/run-node/03-txs.md b/versioned_docs/version-0.50/user/run-node/03-txs.md index faf1d8b65..106f02e8e 100644 --- a/versioned_docs/version-0.50/user/run-node/03-txs.md +++ b/versioned_docs/version-0.50/user/run-node/03-txs.md @@ -269,7 +269,7 @@ func sendTx() error { ### Broadcasting a Transaction -The preferred way to broadcast a transaction is to use gRPC, though using REST (via `gRPC-gateway`) or the CometBFT RPC is also posible. An overview of the differences between these methods is exposed [here](../../develop/advanced/06-grpc_rest.md). For this tutorial, we will only describe the gRPC method. +The preferred way to broadcast a transaction is to use gRPC, though using REST (via `gRPC-gateway`) or the CometBFT RPC is also posible. An overview of the differences between these methods is exposed [here](../../learn/advanced/06-grpc_rest.md). For this tutorial, we will only describe the gRPC method. ```go import ( diff --git a/versioned_docs/version-0.50/user/run-node/04-rosetta.md b/versioned_docs/version-0.50/user/run-node/04-rosetta.md new file mode 100644 index 000000000..936666fa8 --- /dev/null +++ b/versioned_docs/version-0.50/user/run-node/04-rosetta.md @@ -0,0 +1,144 @@ +# Rosetta + +The `rosetta` project implements Coinbase's [Rosetta API](https://www.rosetta-api.org). This document provides instructions on how to use the Rosetta API integration. For information about the motivation and design choices, refer to [ADR 035](https://docs.cosmos.network/main/architecture/adr-035-rosetta-api-support). + +## Installing Rosetta + +The Rosetta API server is a stand-alone server that connects to a node of a chain developed with Cosmos SDK. + +Rosetta can be added to any cosmos chain node. standalone or natively. + +### Standalone + +Rosetta can be executed as a standalone service, it connects to the node endpoints and expose the required endpoints. + +Install Rosetta standalone server with the following command: + +```bash +go install github.com/cosmos/rosetta +``` + +Alternatively, for building from source, simply run `make rosetta`. The binary will be located in the root folder. + +### Native - As a node command + +To enable Native Rosetta API support, it's required to add the `RosettaCommand` to your application's root command file (e.g. `simd/cmd/root.go`). + +Import the `rosettaCmd` package: + +```go +import "github.com/cosmos/rosetta/cmd" +``` + +Find the following line: + +```go +initRootCmd(rootCmd, encodingConfig) +``` + +After that line, add the following: + +```go +rootCmd.AddCommand( + rosettaCmd.RosettaCommand(encodingConfig.InterfaceRegistry, encodingConfig.Codec) +) +``` + +The `RosettaCommand` function builds the `rosetta` root command and is defined in the `rosettaCmd` package (`github.com/cosmos/rosetta/cmd`). + +Since we’ve updated the Cosmos SDK to work with the Rosetta API, updating the application's root command file is all you need to do. + +An implementation example can be found in `simapp` package. + +## Use Rosetta Command + +To run Rosetta in your application CLI, use the following command: + +> **Note:** if using the native approach, add your node name before any rosetta comand. + +```shell +rosetta --help +``` + +To test and run Rosetta API endpoints for applications that are running and exposed, use the following command: + +```shell +rosetta + --blockchain "your application name (ex: gaia)" + --network "your chain identifier (ex: testnet-1)" + --tendermint "tendermint endpoint (ex: localhost:26657)" + --grpc "gRPC endpoint (ex: localhost:9090)" + --addr "rosetta binding address (ex: :8080)" + --grpc-types-server (optional) "gRPC endpoint for message descriptor types" +``` + +## Plugins - Multi chain connections + +Rosetta will try to reflect the node types trough reflection over the node gRPC endpoints, there may be cases were this approach is not enough. It is possible to extend or implement the required types easily trough plugins. + +To use Rosetta over any chain, it is required to set up prefixes and registering zone specific interfaces through plugins. + +Each plugin is a minimalist implementation of `InitZone` and `RegisterInterfaces` which allow Rosetta to parse chain specific data. There is an example for cosmos-hub chain under `plugins/cosmos-hun/` folder +- **InitZone**: An empty method that is executed first and defines prefixes, parameters and other settings. +- **RegisterInterfaces**: This method receives an interface registry which is were the zone specific types and interfaces will be loaded + +In order to add a new plugin: +1. Create a folder over `plugins` folder with the name of the desired zone +2. Add a `main.go` file with the mentioned methods above. +3. Build the code binary through `go build -buildmode=plugin -o main.so main.go` + +The plugin folder is selected through the cli `--plugin` flag and loaded into the Rosetta server. + +## Extensions + +There are two ways in which you can customize and extend the implementation with your custom settings. + +### Message extension + +In order to make an `sdk.Msg` understandable by rosetta the only thing which is required is adding the methods to your messages that satisfy the `rosetta.Msg` interface. Examples on how to do so can be found in the staking types such as `MsgDelegate`, or in bank types such as `MsgSend`. + +### Client interface override + +In case more customization is required, it's possible to embed the Client type and override the methods which require customizations. + +Example: + +```go +package custom_client +import ( + +"context" +"github.com/coinbase/rosetta-sdk-go/types" +"github.com/cosmos/rosetta/lib" +) + +// CustomClient embeds the standard cosmos client +// which means that it implements the cosmos-rosetta-gateway Client +// interface while at the same time allowing to customize certain methods +type CustomClient struct { + *rosetta.Client +} + +func (c *CustomClient) ConstructionPayload(_ context.Context, request *types.ConstructionPayloadsRequest) (resp *types.ConstructionPayloadsResponse, err error) { + // provide custom signature bytes + panic("implement me") +} +``` + +NOTE: when using a customized client, the command cannot be used as the constructors required **may** differ, so it's required to create a new one. We intend to provide a way to init a customized client without writing extra code in the future. + +### Error extension + +Since rosetta requires to provide 'returned' errors to network options. In order to declare a new rosetta error, we use the `errors` package in cosmos-rosetta-gateway. + +Example: + +```go +package custom_errors +import crgerrs "github.com/cosmos/rosetta/lib/errors" + +var customErrRetriable = true +var CustomError = crgerrs.RegisterError(100, "custom message", customErrRetriable, "description") +``` + +Note: errors must be registered before cosmos-rosetta-gateway's `Server`.`Start` method is called. Otherwise the registration will be ignored. Errors with same code will be ignored too. diff --git a/versioned_docs/version-0.50/user/run-node/_category_.json b/versioned_docs/version-0.50/user/run-node/_category_.json index 375a7c423..65e64b945 100644 --- a/versioned_docs/version-0.50/user/run-node/_category_.json +++ b/versioned_docs/version-0.50/user/run-node/_category_.json @@ -1,5 +1,5 @@ { "label": "Running a Node, API and CLI", - "position": 5, + "position": 0, "link": null } \ No newline at end of file diff --git a/versioned_sidebars/version-0.47-sidebars.json b/versioned_sidebars/version-0.47-sidebars.json index 598a115b1..c6be3020a 100644 --- a/versioned_sidebars/version-0.47-sidebars.json +++ b/versioned_sidebars/version-0.47-sidebars.json @@ -1,5 +1,5 @@ { - "developSidebar": [ + "learnSidebar": [ { "type": "autogenerated", "dirName": "learn" diff --git a/versioned_sidebars/version-0.50-sidebars.json b/versioned_sidebars/version-0.50-sidebars.json index c50e64eaf..c6be3020a 100644 --- a/versioned_sidebars/version-0.50-sidebars.json +++ b/versioned_sidebars/version-0.50-sidebars.json @@ -1,8 +1,8 @@ { - "developSidebar": [ + "learnSidebar": [ { "type": "autogenerated", - "dirName": "develop" + "dirName": "learn" } ], "buildSidebar": [