RFC-001: Epoch Staking

[PoisonPhang]

Epoch Staking

Epoch staking is the idea that validator set changes are buffered for some epoch before being applied at the end of the epoch. Instead of epoch staking, the x/staking module provided by the cosmos-sdk conducts staking updates at the end of each block. The benefits and drawbacks of epoch staking over per-block staking have been addressed in both ADR 039: Epoched Staking & cosmos/cosmos-sdk/discussions Epoched Staking. The reasoning for why the Union network would benefit from Epoch Staking is presented in the Union Whitepaper. The goal of this RFC is to determine the design and implementation of epoch staking that will be used in the Union network.

Requirements

• Allow for time based epochs

• Apply updates to the validator set at the end of an epoch

• Allow the duration of an epoch to be configurable

• Conduct staking updates at the end of each block

• Allow for validator set changes when too many validators are jailed

Design

Epoch Duration

To ensure a flexible design, the duration of an epoch should be a parameter for the staking module.

While a primitive solution may only allow for an epoch to be a number of blocks - a time-based epoch will be required for Union. In favor of flexibility, the unit used in defining the epoch parameter should also be configurable. For our use case, blocks and seconds should suffice.

This will result in the addition of two new parameters being added to the staking module: uint32 epoch_length and epoch_unit.

End of Epoch Detection

The end of an epoch is defined as when the current block/time is greater than or equal to the start_of_epoch + epoch_length.

Validator State & Stake Changes

To avoid a heavy block at the end of each epoch, we eagerly perform the stake and delegation changes at the end of each block but buffer the validator set state changes until the end of each epoch.

Two conditions trigger a validator set update:

1. The first block of an epoch. This will also determine the timestamp of the next epoch.

2. The number of jailed validators exceeding a defined threshold.

Existing Work

A closed but never merged pull request cosmos/cosmos-sdk#9043 into the cosmos-sdk contained most of @tac0turtle's implementation of epoched staking. The implementation provided by @tac0turtle depending on a now unavailable epoching module in the cosmos-sdk and was primarily comprised of changes to the staking module's keeper.

Epoching Module

The epoching module that originally existed in the cosmos-sdk is no longer present in v0.47 of the cosmos-sdk. The original epoching module was removed "in favour of a different design in the future" in cosmos/cosmos-sdk#13503. However, there has been no conformation of a new design from the cosmos devs.

The previous design is still available under the release/v0.46.x branch of the cosmos-sdk. As it stands, the available epoching module may not be production ready. The epoching module contains a list of pending to-do items as well as at lease one known issue. The to-dos and known issue are documented in the 03_to_improve.md. This document also mentions a pull request or issue "#46" and it's to-dos, but cosmos/cosmos-sdk#46 is seemingly unrelated to epoching.

The epoching module introduces a new epochkeeper, this keeper lets consumers create their own instance of the epoching keeper. Using an instance of the epoching keeper, you can queue messages as "actions" interact with actions via their actionID. You can also get information about a given epoch and progress the current epoch.

The epoching module does not execute queued actions itself, that is up to the consumer to do by using the interface provided by the keeper to judge when to conduct actions. This means that the epoching module is only responsible for storing state while consumers are expected to change state and act on the current state.

QA

The epoching module is missing any form of direct testing, instead - testing of the epoch functionality is conducted on the consumers of the epoching module.

Staking Keeper

In cosmos/cosmos-sdk#9043 several important changes were made to the staking module.

• The epoching keeper was added to the staking module's keeper.

  This provided the epoching interface to the staking module. They also introduced the EpochInterval parameter to determine the length of staking epochs.

• Wrapper functions for the staking module's keeper were created to interact with the epoching keeper.

  This exposed the epoching interface to the staking module via it's keeper.

• The epoch_exec.go file was created which contains the code needed to execute epoched staking messages.

  At the end of an epoch, all actions/messages in the epochs queue are executed here.

• The staking message server was altered to route messages throuhg epoching

  This enabled the epoching of several messages including:

  • MsgCreateValidator
  • MsgEditValidator
  • MsgDelegate
  • MsgBeginRedelegate
  • MsgUndelegate

QA

While tests were added to staking that were related to the epoching module, these tests only ensure that the epoching keeper is saving and loading data correctly.

Other Changes

• Slashing Module

  Delays both unjail and slash messages until end of epoch.

• Genesis Utilities

  Execute staking keepers ExecuteEpoch function to conduct genesis validator set changes.

Implementation

While the work from cosmos/cosmos-sdk#9043 will provide a good baseline for epoch staking, two facts of their implementation stand in direct contrast to the goals set out in the Union Whitepaper.

1. When delegations occur, funds are initially sent to an epoch module holding account until the end of epoch when they are appropriately delegated. This results in two transactions per delegation with the second taking place at the end of the epoch.

2. To conduct validator jailing, validator set changes are still checked every block. Instead of directly preventing the validator set changes, they instead prevent all state changes in delegation that would trigger a per-block validator set change until the end of the epoch.

This is, functionally, the opposite of what's proposed in the Union Whitepaper. The Union Whitepaper proposes that delegation transactions occur each block while the validator set changes are only conducted at the end of the epoch.

For our implementation of epoch staking, this means that we can mostly re-use the old epoching module provided by the cosmos-sdk while heavily redesigning the approach they attempted to take for implementing epoching in the staking module. The cosmos team will not start research into a new staking module until Q3 2023 according to their roadmap.

Epoching Module

The epoching module can be copied over from the v0.46.x branch of the cosmos-sdk. However, in an attempt to ensure quality without depending on other modules to test the epoching module - a test suite should be added to the epoching module.

This means the epoching module will be used to maintain the state of epochs and their action queues, execution of the actions will be the responsibility of the consuming modules.

Staking Module

To implement our changes to the staking module, we'll need to conditionally limit all validator set changes (besides jailing) until the end of an epoch or when too many validators are jailed.

To do this, we'll need three main changes to the staking module:

1. Track the number of jailed validators

2. Create a new function that will update the validator set in the case of newly jailed validators

3. Only conditionally run the full validator set update in the event of the number of jailed validators exceeding the threshold or the end of an epoch

Track the Number of Jailed Validators

Introduce a new value to the staking store to keep track of the jailed validators. This value can be incremented by the jailValidator function in val_state_change.go. It can also be decremented by the function unjailValidator in val_state_change.go.

Within staking's EndBlocker function, this value will be used to determine which type of validator set update to perform.

Create New Function for Minimum Validator Set Change

A new function will need to be introduced. This will be called by staking's EndBlocker whenever it's not calling BlockValidatorUpdates. The function BlockValidatorUpdates should be renamed to FullValidatorUpdates and the new function should be named MinimumValidatorUpdates.

This function will be responsible for only returning validator set changes for jailed/unjailed validators.

Update EndBlocker

Finally, EndBlocker will need to be updated to conditionally run validator set changes using the following logic:

flowchart TD
    A[EndBlocker]
    A --> C{End of Epoch?}
    C -->|Yes| D[FullValidatorUpdates]
    C -->|No| E{JailedValidators >= THRESHOLD}
    E -->|Yes| D[BlockValidatorUpdates]
    E -->|No| F[MinimumValidatorUpdates]

Loading

If a validator set update is forced by the THRESHOLD trigger, the epoch should be incremented and have the end of the new epoch re-calculated.

Add New Parameters

The staking module should also expose some new parameters for configuration:

• jailedThreshold - The number of jailed validators that will force a new validator set rotation.

• epochLength - The length of a normal epoch.

Questions

Some questions about the design and implementation of epoch staking in Union still need to be addressed.

1. If a validator is created mid-epoch, should the validator be created then or at the end of the epoch? If created then should users be able to delegate to them?

Naive Answer: If a validator is created mid-epoch, their delegation should be deposited the block it is confirmed and users should be able to delegate to them before the end of the current epoch.

2. Should we conduct a "dry run" of epoched messages to ensure they are valid before adding them epoch queue? Otherwise, users won't know if one of their queued messages failed until the end of the epoch.

3. Should validators be able to unjail themselves and become validators again before the end of an epoch

Resources

• ADR 039: Epoched Staking
• cosmos/cosmos-sdk/discussions Epoched Staking
• Union Whitepaper
• cosmos-sdk epoching module

[KaiserKarel]

1. If a validator is created mid-epoch, should the validator be created then or at the end of the epoch? If created then should users be able to delegate to them?

It should be created and delegatable immediately IMHO, that is the best validator experience overall.

2. Should we conduct a "dry run" of epoched messages to ensure they are valid before adding them epoch queue? Otherwise, users won't know if one of their queued messages failed until the end of the epoch.

We should verify that the validator exists, they have sufficient balance, etc.

3. Should validators be able to unjail themselves and become validators again before the end of an epoch?

Ideally yes. I'm unfamiliar with the current jail logic; does it entail taking validators from the inactive set to replace jailed validators?

[Raymond-kode]

Why not integrate paymaster to Union Testnet where each chain can also use their stable coin to pay for gas, like when on arbitrum chain, one can use USDC to pay for gas.