Merge branch 'master' into am/10490-metadata

CHANGELOG.md

@@ -151,6 +151,7 @@ Ref: https://keepachangelog.com/en/1.0.0/
 
 ### Bug Fixes
 
+* (grpc) [\#10985](https://github.com/cosmos/cosmos-sdk/pull/10992) The `/cosmos/tx/v1beta1/txs/{hash}` endpoint returns a 404 when a tx does not exist.
 * (rosetta) [\#10340](https://github.com/cosmos/cosmos-sdk/pull/10340) Use `GenesisChunked(ctx)` instead `Genesis(ctx)` to get genesis block height
 * [#10180](https://github.com/cosmos/cosmos-sdk/issues/10180) Documentation: make references to Cosmos SDK consistent
 * [\#9651](https://github.com/cosmos/cosmos-sdk/pull/9651) Change inconsistent limit of `0` to `MaxUint64` on InfiniteGasMeter and add GasRemaining func to GasMeter.

docs/basics/tx-lifecycle.md

@@ -83,7 +83,18 @@ When `Tx` is received by the application from the underlying consensus engine (e
 
 ### ValidateBasic
 
-[`sdk.Msg`s](../core/transactions.md#messages) are extracted from `Tx`, and `ValidateBasic`, a method of the `sdk.Msg` interface implemented by the module developer, is run for each one. `ValidateBasic` should include basic **stateless** sanity checks. For example, if the message is to send coins from one address to another, `ValidateBasic` likely checks for nonempty addresses and a nonnegative coin amount, but does not require knowledge of state such as the account balance of an address.
+Messages ([`sdk.Msg`](../core/transactions.md#messages)) are extracted from transactions (`Tx`). The `ValidateBasic` method of the `sdk.Msg` interface implemented by the module developer is run for each transaction. 
+To discard obviously invalid messages, the BaseApp` type calls the `ValidateBasic` method very early in the processing of the message in the [`CheckTx`](../core/baseapp.md#checktx) and [`DeliverTx`](../core/baseapp.md#delivertx)) transactions.
+`ValidateBasic` can include only **stateless** checks (the checks that do not require access to the state). 
+
+#### Guideline
+
+Gas is not charged when `ValidateBasic` is executed so we recommend only performing all necessary stateless checks to enable middleware operations (for example, parsing the required signer accounts to validate a signature by a middleware) and stateless sanity checks not impacting performance of the CheckTx phase.
+Other validation operations must be performed when [handling a message](../building-modules/msg-services#Validation) in a module Msg Server.
+
+Example, if the message is to send coins from one address to another, `ValidateBasic` likely checks for non-empty addresses and a non-negative coin amount, but does not require knowledge of state such as the account balance of an address.
+
+See also [Msg Service Validation](../building-modules/msg-services.md#Validation).
 
 ### AnteHandler
 

docs/building-modules/messages-and-queries.md

@@ -55,7 +55,7 @@ It extends `proto.Message` and contains the following methods:
 
 - `Route() string`: Name of the route for this message. Typically all `message`s in a module have the same route, which is most often the module's name.
 - `Type() string`: Type of the message, used primarly in [events](../core/events.md). This should return a message-specific `string`, typically the denomination of the message itself.
-- `ValidateBasic() error`: This method is called by `BaseApp` very early in the processing of the `message` (in both [`CheckTx`](../core/baseapp.md#checktx) and [`DeliverTx`](../core/baseapp.md#delivertx)), in order to discard obviously invalid messages. `ValidateBasic` should only include *stateless* checks, i.e. checks that do not require access to the state. This usually consists in checking that the message's parameters are correctly formatted and valid (i.e. that the `amount` is strictly positive for a transfer).
+- [`ValidateBasic() error`](../basics/tx-lifecycle.md#ValidateBasic).
 - `GetSignBytes() []byte`: Return the canonical byte representation of the message. Used to generate a signature.
 - `GetSigners() []AccAddress`: Return the list of signers. The Cosmos SDK will make sure that each `message` contained in a transaction is signed by all the signers listed in the list returned by this method.
 

docs/building-modules/module-interfaces.md

@@ -35,7 +35,7 @@ In general, the getter function does the following:
     - **RunE:** Defines a function that can return an error. This is the function that is called when the command is executed. This function encapsulates all of the logic to create a new transaction.
         - The function typically starts by getting the `clientCtx`, which can be done with `client.GetClientTxContext(cmd)`. The `clientCtx` contains information relevant to transaction handling, including information about the user. In this example, the `clientCtx` is used to retrieve the address of the sender by calling `clientCtx.GetFromAddress()`.
         - If applicable, the command's arguments are parsed. In this example, the arguments `[to_address]` and `[amount]` are both parsed.
-        - A [message](./messages-and-queries.md) is created using the parsed arguments and information from the `clientCtx`. The constructor function of the message type is called directly. In this case, `types.NewMsgSend(fromAddr, toAddr, amount)`. Its good practice to call `msg.ValidateBasic()` after creating the message, which runs a sanity check on the provided arguments.
+        - A [message](./messages-and-queries.md) is created using the parsed arguments and information from the `clientCtx`. The constructor function of the message type is called directly. In this case, `types.NewMsgSend(fromAddr, toAddr, amount)`. Its good practice to call [`msg.ValidateBasic()`](../basics/tx-lifecycle.md#ValidateBasic) and other validation methods before broadcasting the message.
         - Depending on what the user wants, the transaction is either generated offline or signed and broadcasted to the preconfigured node using `tx.GenerateOrBroadcastTxCLI(clientCtx, flags, msg)`.
 - **Adds transaction flags:** All transaction commands must add a set of transaction [flags](#flags). The transaction flags are used to collect additional information from the user (e.g. the amount of fees the user is willing to pay). The transaction flags are added to the constructed command using `AddTxFlagsToCmd(cmd)`.
 - **Returns the command:** Finally, the transaction command is returned.

docs/building-modules/msg-services.md

@@ -29,20 +29,48 @@ When possible, the existing module's [`Keeper`](keeper.md) should implement `Msg
 
 +++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc1/x/bank/keeper/msg_server.go#L27-L28
 
-`sdk.Msg` processing usually follows these 2 steps:
+`sdk.Msg` processing usually follows these 3 steps:
 
-- First, they perform *stateful* checks to make sure the `message` is valid. At this stage, the `message`'s `ValidateBasic()` method has already been called, meaning *stateless* checks on the message (like making sure parameters are correctly formatted) have already been performed. Checks performed in the `msgServer` method can be more expensive and require access to the state. For example, a `msgServer` method for a `transfer` message might check that the sending account has enough funds to actually perform the transfer. To access the state, the `msgServer` method needs to call the [`keeper`'s](./keeper.md) getter functions.
-- Then, if the checks are successful, the `msgServer` method calls the [`keeper`'s](./keeper.md) setter functions to actually perform the state transition.
+### Validation 
 
-Before returning, `msgServer` methods generally emit one or more [events](../core/events.md) via the `EventManager` held in the `ctx`:
+Before a `msgServer` method is executed, the message's [`ValidateBasic()`](../basics/tx-lifecycle.md#ValidateBasic) method has already been called. Since `msg.ValidateBasic()`  performs only the most basic checks, this stage must perform all other validation (both *stateful* and *stateless*) to make sure the `message` is valid. Checks performed in the `msgServer` method can be more expensive and the signer is charged gas for these operations.
+For example, a `msgServer` method for a `transfer` message might check that the sending account has enough funds to actually perform the transfer. 
+
+It is recommended to implement all validation checks in a separate function that passes state values as arguments. This implementation simplifies testing. As expected, expensive validation functions charge additional gas. Example:
+
+```go
+ValidateMsgA(msg MsgA, now Time, gm GasMeter) error {
+	if now.Before(msg.Expire) {
+		return sdkerrrors.ErrInvalidRequest.Wrap("msg expired")
+	}
+	gm.ConsumeGas(1000, "signature verification")
+	return signatureVerificaton(msg.Prover, msg.Data)
+}
+```
+
+### State Transition
+
+After the validation is successful, the `msgServer` method uses the [`keeper`](./keeper.md) functions to access the state and perform a state transition.
+
+### Events 
+
+Before returning, `msgServer` methods generally emit one or more [events](../core/events.md) by using the `EventManager` held in the `ctx`. Use the new `EmitTypedEvent` function that uses protobuf-based event types:
+
+```
+ctx.EventManager().EmitTypedEvent(
+	&group.EventABC{Key1: Value1,  Key2, Value2})
+```
+
+or the older `EmitEvent` function: 
 
 ```go
 ctx.EventManager().EmitEvent(
-		sdk.NewEvent(
-			eventType,  // e.g. sdk.EventTypeMessage for a message, types.CustomEventType for a custom event defined in the module
-			sdk.NewAttribute(attributeKey, attributeValue),
-		),
-    )
+	sdk.NewEvent(
+		eventType,  // e.g. sdk.EventTypeMessage for a message, types.CustomEventType for a custom event defined in the module
+		sdk.NewAttribute(key1, value1),
+		sdk.NewAttribute(key2, value2),
+	),
+)
 ```
 
 These events are relayed back to the underlying consensus engine and can be used by service providers to implement services around the application. Click [here](../core/events.md) to learn more about events.

docs/core/baseapp.md

@@ -224,7 +224,9 @@ transaction is received by a full-node. The role of `CheckTx` is to guard the fu
 Unconfirmed transactions are relayed to peers only if they pass `CheckTx`.
 
 `CheckTx()` can perform both _stateful_ and _stateless_ checks, but developers should strive to
-make them lightweight. In the Cosmos SDK, after [decoding transactions](./encoding.md), `CheckTx()` is implemented
+make the checks **lightweight** because gas fees are not charged for the resources (CPU, data load...) used during the `CheckTx`. 
+
+In the Cosmos SDK, after [decoding transactions](./encoding.md), `CheckTx()` is implemented
 to do the following checks:
 
 1. Extract the `sdk.Msg`s from the transaction.
@@ -237,9 +239,8 @@ to do the following checks:
    as `sdk.Msg`s are not processed. Usually, the [`AnteHandler`](../basics/gas-fees.md#antehandler) will check that the `gas` provided
    with the transaction is superior to a minimum reference gas amount based on the raw transaction size,
    in order to avoid spam with transactions that provide 0 gas.
-4. Ensure that each `sdk.Msg`'s fully-qualified service method matches on of the routes inside the `msgServiceRouter`, but do **not** actually
-   process `sdk.Msg`s. `sdk.Msg`s only need to be processed when the canonical state need to be updated,
-   which happens during `DeliverTx`.
+
+`CheckTx` does **not** process `sdk.Msg`s -  they only need to be processed when the canonical state need to be updated, which happens during `DeliverTx`.
 
 Steps 2. and 3. are performed by the [`AnteHandler`](../basics/gas-fees.md#antehandler) in the [`RunTx()`](#runtx-antehandler-and-runmsgs)
 function, which `CheckTx()` calls with the `runTxModeCheck` mode. During each step of `CheckTx()`, a
@@ -269,7 +270,7 @@ The response contains:
 #### RecheckTx
 
 After `Commit`, `CheckTx` is run again on all transactions that remain in the node's local mempool
-after filtering those included in the block. To prevent the mempool from rechecking all transactions
+excluding the transactions that are included in the block. To prevent the mempool from rechecking all transactions
 every time a block is committed, the configuration option `mempool.recheck=false` can be set. As of
 Tendermint v0.32.1, an additional `Type` parameter is made available to the `CheckTx` function that
 indicates whether an incoming transaction is new (`CheckTxType_New`), or a recheck (`CheckTxType_Recheck`).

docs/core/transactions.md

@@ -25,7 +25,7 @@ Transaction objects are Cosmos SDK types that implement the `Tx` interface
 It contains the following methods:
 
 - **GetMsgs:** unwraps the transaction and returns a list of contained `sdk.Msg`s - one transaction may have one or multiple messages, which are defined by module developers.
-- **ValidateBasic:** includes lightweight, [_stateless_](../basics/tx-lifecycle.md#types-of-checks) checks used by ABCI messages [`CheckTx`](./baseapp.md#checktx) and [`DeliverTx`](./baseapp.md#delivertx) to make sure transactions are not invalid. For example, the [`auth`](https://github.com/cosmos/cosmos-sdk/tree/master/x/auth) module's `StdTx` `ValidateBasic` function checks that its transactions are signed by the correct number of signers and that the fees do not exceed what the user's maximum. Note that this function is to be distinct from the `ValidateBasic` functions for `sdk.Msg`s, which perform basic validity checks on messages only. For example, when [`runTx`](./baseapp.md#runtx) is checking a transaction created from the [`auth`](https://github.com/cosmos/cosmos-sdk/tree/master/x/auth/spec) module, it first runs `ValidateBasic` on each message, then runs the `auth` module AnteHandler which calls `ValidateBasic` for the transaction itself.
+- **ValidateBasic:** lightweight, [_stateless_](../basics/tx-lifecycle.md#types-of-checks) checks used by ABCI messages [`CheckTx`](./baseapp.md#checktx) and [`DeliverTx`](./baseapp.md#delivertx) to make sure transactions are not invalid. For example, the [`auth`](https://github.com/cosmos/cosmos-sdk/tree/master/x/auth) module's `StdTx` `ValidateBasic` function checks that its transactions are signed by the correct number of signers and that the fees do not exceed what the user's maximum. Note that this function is to be distinct from `sdk.Msg` [`ValidateBasic`](../basics/tx-lifecycle.md#ValidateBasic) methods, which perform basic validity checks on messages only. When [`runTx`](./baseapp.md#runtx) is checking a transaction created from the [`auth`](https://github.com/cosmos/cosmos-sdk/tree/master/x/auth/spec) module, it first runs `ValidateBasic` on each message, then runs the `auth` module AnteHandler which calls `ValidateBasic` for the transaction itself.
 
 As a developer, you should rarely manipulate `Tx` directly, as `Tx` is really an intermediate type used for transaction generation. Instead, developers should prefer the `TxBuilder` interface, which you can learn more about [below](#transaction-generation).
 

orm/encoding/ormkv/entry.go

@@ -4,10 +4,10 @@ import (
 	"fmt"
 	"strings"
 
-	"google.golang.org/protobuf/encoding/protojson"
+	"github.com/cosmos/cosmos-sdk/orm/internal/stablejson"
+
 	"google.golang.org/protobuf/proto"
 	"google.golang.org/protobuf/reflect/protoreflect"
-	"google.golang.org/protobuf/types/known/structpb"
 )
 
 // Entry defines a logical representation of a kv-store entry for ORM instances.
@@ -45,7 +45,12 @@ func (p *PrimaryKeyEntry) String() string {
 	if p.Value == nil {
 		return fmt.Sprintf("PK %s %s -> _", p.TableName, fmtValues(p.Key))
 	} else {
-		return fmt.Sprintf("PK %s %s -> %s", p.TableName, fmtValues(p.Key), p.Value)
+		valBz, err := stablejson.Marshal(p.Value)
+		valStr := string(valBz)
+		if err != nil {
+			valStr = fmt.Sprintf("ERR %v", err)
+		}
+		return fmt.Sprintf("PK %s %s -> %s", p.TableName, fmtValues(p.Key), valStr)
 	}
 }
 
@@ -56,19 +61,7 @@ func fmtValues(values []protoreflect.Value) string {
 
 	parts := make([]string, len(values))
 	for i, v := range values {
-		val, err := structpb.NewValue(v.Interface())
-		if err != nil {
-			parts[i] = "ERR"
-			continue
-		}
-
-		bz, err := protojson.Marshal(val)
-		if err != nil {
-			parts[i] = "ERR"
-			continue
-		}
-
-		parts[i] = string(bz)
+		parts[i] = fmt.Sprintf("%v", v.Interface())
 	}
 
 	return strings.Join(parts, "/")

orm/encoding/ormkv/entry_test.go

@@ -19,7 +19,7 @@ func TestPrimaryKeyEntry(t *testing.T) {
 		Key:       encodeutil.ValuesOf(uint32(1), "abc"),
 		Value:     &testpb.ExampleTable{I32: -1},
 	}
-	assert.Equal(t, `PK testpb.ExampleTable 1/"abc" -> i32:-1`, entry.String())
+	assert.Equal(t, `PK testpb.ExampleTable 1/abc -> {"i32":-1}`, entry.String())
 	assert.Equal(t, aFullName, entry.GetTableName())
 
 	// prefix key
@@ -28,7 +28,7 @@ func TestPrimaryKeyEntry(t *testing.T) {
 		Key:       encodeutil.ValuesOf(uint32(1), "abc"),
 		Value:     nil,
 	}
-	assert.Equal(t, `PK testpb.ExampleTable 1/"abc" -> _`, entry.String())
+	assert.Equal(t, `PK testpb.ExampleTable 1/abc -> _`, entry.String())
 	assert.Equal(t, aFullName, entry.GetTableName())
 }
 
@@ -40,7 +40,7 @@ func TestIndexKeyEntry(t *testing.T) {
 		IndexValues: encodeutil.ValuesOf(uint32(10), int32(-1), "abc"),
 		PrimaryKey:  encodeutil.ValuesOf("abc", int32(-1)),
 	}
-	assert.Equal(t, `IDX testpb.ExampleTable u32/i32/str : 10/-1/"abc" -> "abc"/-1`, entry.String())
+	assert.Equal(t, `IDX testpb.ExampleTable u32/i32/str : 10/-1/abc -> abc/-1`, entry.String())
 	assert.Equal(t, aFullName, entry.GetTableName())
 
 	entry = &ormkv.IndexKeyEntry{
@@ -50,7 +50,7 @@ func TestIndexKeyEntry(t *testing.T) {
 		IndexValues: encodeutil.ValuesOf(uint32(10)),
 		PrimaryKey:  encodeutil.ValuesOf("abc", int32(-1)),
 	}
-	assert.Equal(t, `UNIQ testpb.ExampleTable u32 : 10 -> "abc"/-1`, entry.String())
+	assert.Equal(t, `UNIQ testpb.ExampleTable u32 : 10 -> abc/-1`, entry.String())
 	assert.Equal(t, aFullName, entry.GetTableName())
 
 	// prefix key
@@ -70,6 +70,6 @@ func TestIndexKeyEntry(t *testing.T) {
 		IsUnique:    true,
 		IndexValues: encodeutil.ValuesOf("abc", int32(1)),
 	}
-	assert.Equal(t, `UNIQ testpb.ExampleTable str/i32 : "abc"/1 -> _`, entry.String())
+	assert.Equal(t, `UNIQ testpb.ExampleTable str/i32 : abc/1 -> _`, entry.String())
 	assert.Equal(t, aFullName, entry.GetTableName())
 }

orm/go.mod

@@ -4,8 +4,9 @@ go 1.17
 
 require (
 	github.com/cosmos/cosmos-proto v1.0.0-alpha6
-	github.com/cosmos/cosmos-sdk/api v0.1.0-alpha2
+	github.com/cosmos/cosmos-sdk/api v0.1.0-alpha3
 	github.com/cosmos/cosmos-sdk/errors v1.0.0-beta.2
+	github.com/stretchr/testify v1.7.0
 	github.com/tendermint/tm-db v0.6.6
 	google.golang.org/protobuf v1.27.1
 	gotest.tools/v3 v3.1.0
@@ -15,26 +16,33 @@ require (
 require (
 	github.com/DataDog/zstd v1.4.5 // indirect
 	github.com/cespare/xxhash v1.1.0 // indirect
+	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/dgraph-io/badger/v2 v2.2007.2 // indirect
 	github.com/dgraph-io/ristretto v0.0.3 // indirect
 	github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect
 	github.com/dustin/go-humanize v1.0.0 // indirect
 	github.com/facebookgo/ensure v0.0.0-20160127193407-b4ab57deab51 // indirect
 	github.com/facebookgo/stack v0.0.0-20160209184415-751773369052 // indirect
 	github.com/facebookgo/subset v0.0.0-20150612182917-8dac2c3c4870 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/golang/protobuf v1.5.2 // indirect
 	github.com/golang/snappy v0.0.3-0.20201103224600-674baa8c7fc3 // indirect
 	github.com/google/btree v1.0.0 // indirect
 	github.com/google/go-cmp v0.5.5 // indirect
 	github.com/jmhodges/levigo v1.0.0 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
 	github.com/syndtr/goleveldb v1.0.1-0.20200815110645-5c35d600f0ca // indirect
 	github.com/tecbot/gorocksdb v0.0.0-20191217155057-f0fad39f321c // indirect
 	go.etcd.io/bbolt v1.3.6 // indirect
 	golang.org/x/net v0.0.0-20210903162142-ad29c8ab022f // indirect
 	golang.org/x/sys v0.0.0-20210903071746-97244b99971b // indirect
+	golang.org/x/text v0.3.6 // indirect
 	golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
+	google.golang.org/genproto v0.0.0-20211223182754-3ac035c7e7cb // indirect
+	google.golang.org/grpc v1.43.0 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c // indirect
 )
 
 replace github.com/gogo/protobuf => github.com/regen-network/protobuf v1.3.3-alpha.regen.1