dApp staking v3 - part 7 (#1099)

* dApp staking v3 part 6

* Minor refactor of benchmarks

* Weights integration

* Fix

* remove orig file

* Minor refactoring, more benchmark code

* Extract on_init logic

* Some renaming

* More benchmarks

* Full benchmarks integration

* Testing primitives

* staking primitives

* dev fix

* Integration part1

* Integration part2

* Reward payout integration

* Replace lock functionality with freeze

* Cleanup TODOs

* More negative tests

* Frozen balance test

* Zero div

* Docs for inflation

* Rename is_active & add some more docs

* More docs

* pallet docs

* text

* scripts

* More tests

* Test, docs

* Review comment

* Runtime API

* Changes

* Change dep

* Comment

* Formatting

* Expired entry cleanup

* Cleanup logic test

* Remove tier labels

* fix

* Improve README

* Improved cleanup

* Review comments

* Docs

* Formatting

* Typo

Cargo.toml

@@ -284,6 +284,8 @@ pallet-inflation = { path = "./pallets/inflation", default-features = false }
 pallet-dynamic-evm-base-fee = { path = "./pallets/dynamic-evm-base-fee", default-features = false }
 pallet-unified-accounts = { path = "./pallets/unified-accounts", default-features = false }
 
+dapp-staking-v3-runtime-api = { path = "./pallets/dapp-staking-v3/rpc/runtime-api", default-features = false }
+
 astar-primitives = { path = "./primitives", default-features = false }
 astar-test-utils = { path = "./tests/utils", default-features = false }
 

pallets/dapp-staking-v3/README.md

@@ -27,7 +27,7 @@ Each period consists of two subperiods:
 Each period is denoted by a number, which increments each time a new period begins.
 Period beginning is marked by the `voting` subperiod, after which follows the `build&earn` period.
 
-Stakes are **only** valid throughout a period. When new period starts, all stakes are reset to **zero**. This helps prevent projects remaining staked due to intertia of stakers, and makes for a more dynamic staking system.
+Stakes are **only** valid throughout a period. When new period starts, all stakes are reset to **zero**. This helps prevent projects remaining staked due to intertia of stakers, and makes for a more dynamic staking system. Staker doesn't need to do anything for this to happen, it is automatic.
 
 Even though stakes are reset, locks (or freezes) of tokens remain.
 
@@ -39,7 +39,7 @@ Projects participating in dApp staking are expected to market themselves to (re)
 Stakers must assess whether the project they want to stake on brings value to the ecosystem, and then `vote` for it.
 Casting a vote, or staking, during the `Voting` subperiod makes the staker eligible for bonus rewards. so they are encouraged to participate.
 
-`Voting` subperiod length is expressed in _standard_ era lengths, even though the entire voting period is treated as a single _voting era_.
+`Voting` subperiod length is expressed in _standard_ era lengths, even though the entire voting subperiod is treated as a single _voting era_.
 E.g. if `voting` subperiod lasts for **5 eras**, and each era lasts for **100** blocks, total length of the `voting` subperiod will be **500** blocks.
 * Block 1, Era 1 starts, Period 1 starts, `Voting` subperiod starts
 * Block 501, Era 2 starts, Period 1 continues, `Build&Earn` subperiod starts
@@ -173,11 +173,70 @@ Rewards don't remain available forever, and if not claimed within some time peri
 
 Rewards are calculated using a simple formula: `staker_reward_pool * staker_staked_amount / total_staked_amount`.
 
-#### Claim Bonus Reward
+#### Claiming Bonus Reward
 
-If staker staked on a dApp during the voting period, and didn't reduce their staked amount below what was staked at the end of the voting period, this makes them eligible for the bonus reward.
+If staker staked on a dApp during the voting subperiod, and didn't reduce their staked amount below what was staked at the end of the voting subperiod, this makes them eligible for the bonus reward.
 
 Bonus rewards need to be claimed per contract, unlike staker rewards.
 
-Bonus reward is calculated using a simple formula: `bonus_reward_pool * staker_voting_period_stake / total_voting_period_stake`.
+Bonus reward is calculated using a simple formula: `bonus_reward_pool * staker_voting_subperiod_stake / total_voting_subperiod_stake`.
 
+#### Handling Expired Entries
+
+There is a limit to how much contracts can a staker stake on at once.
+Or to be more precise, for how many contract a database entry can exist at once.
+
+It's possible that stakers get themselves into a situation where some number of expired database entries associated to
+their account has accumulated. In that case, it's required to call a special extrinsic to cleanup these expired entries.
+
+### Developers
+
+Main thing for developers to do is develop a good product & attract stakers to stake on them.
+
+#### Claiming dApp Reward
+
+If at the end of an build&earn subperiod era dApp has high enough score to enter a tier, it gets rewards assigned to it.
+Rewards aren't paid out automatically but must be claimed instead, similar to staker & bonus rewards.
+
+When dApp reward is being claimed, both smart contract & claim era must be specified.
+
+dApp reward is calculated based on the tier in which ended. All dApps that end up in one tier will get the exact same reward.
+
+### Tier System
+
+At the end of each build&earn subperiod era, dApps are evaluated using a simple metric - total value staked on them.
+Based on this metric, they are sorted, and assigned to tiers.
+
+There is a limited number of tiers, and each tier has a limited capacity of slots.
+Each tier also has a _threshold_ which a dApp must satisfy in order to enter it.
+
+Better tiers bring bigger rewards, so dApps are encouraged to compete for higher tiers and attract staker's support.
+For each tier, the reward pool and capacity are fixed. Each dApp within a tier always gets the same amount of reward.
+Even if tier capacity hasn't been fully taken, rewards are paid out as if they were.
+
+For example, if tier 1 has capacity for 10 dApps, and reward pool is **500 ASTR**, it means that each dApp that ends up
+in this tier will earn **50 ASTR**. Even if only 3 dApps manage to enter this tier, they will still earn each **50 ASTR**.
+The rest, **350 ASTR** in this case, won't be minted (or will be _burned_ if the reader prefers such explanation).
+
+If there are more dApps eligible for a tier than there is capacity, the dApps with the higher score get the advantage.
+dApps which missed out get priority for entry into the next lower tier (if there still is any).
+
+In the case a dApp doesn't satisfy the entry threshold for any tier, even though there is still capacity, the dApp will simply
+be left out of tiers and won't earn **any** reward.
+
+In a special and unlikely case that two or more dApps have the exact same score and satisfy tier entry threshold, but there isn't enough
+leftover tier capacity to accomodate them all, this is considered _undefined_ behavior. Some of the dApps will manage to enter the tier, while
+others will be left out. There is no strict rule which defines this behavior - instead dApps are encouraged to ensure their tier entry by
+having a larger stake than the other dApp(s).
+
+### Reward Expiry
+
+Unclaimed rewards aren't kept indefinitely in storage. Eventually, they expire.
+Stakers & developers should make sure they claim those rewards before this happens.
+
+In case they don't, they will simply miss on the earnings.
+
+However, this should not be a problem given how the system is designed.
+There is no longer _stake&forger_ - users are expected to revisit dApp staking at least at the
+beginning of each new period to pick out old or new dApps on which to stake on.
+If they don't do that, they miss out on the bonus reward & won't earn staker rewards.

pallets/dapp-staking-v3/rpc/runtime-api/Cargo.toml

@@ -0,0 +1,25 @@
+[package]
+name = "dapp-staking-v3-runtime-api"
+version = "0.0.1-alpha"
+description = "dApp Staking v3 runtime API"
+authors.workspace = true
+edition.workspace = true
+homepage.workspace = true
+repository.workspace = true
+
+[package.metadata.docs.rs]
+targets = ["x86_64-unknown-linux-gnu"]
+
+[dependencies]
+sp-api = { workspace = true }
+
+astar-primitives = { workspace = true }
+pallet-dapp-staking-v3 = { workspace = true }
+
+[features]
+default = ["std"]
+std = [
+	"sp-api/std",
+	"pallet-dapp-staking-v3/std",
+	"astar-primitives/std",
+]

pallets/dapp-staking-v3/rpc/runtime-api/src/lib.rs

@@ -0,0 +1,40 @@
+// This file is part of Astar.
+
+// Copyright (C) 2019-2023 Stake Technologies Pte.Ltd.
+// SPDX-License-Identifier: GPL-3.0-or-later
+
+// Astar is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Astar is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Astar. If not, see <http://www.gnu.org/licenses/>.
+
+#![cfg_attr(not(feature = "std"), no_std)]
+
+use astar_primitives::BlockNumber;
+use pallet_dapp_staking_v3::EraNumber;
+
+sp_api::decl_runtime_apis! {
+
+    /// dApp Staking Api.
+    ///
+    /// Used to provide information otherwise not available via RPC.
+    pub trait DappStakingApi {
+
+        /// For how many standard era lengths does the voting subperiod last.
+        fn eras_per_voting_subperiod() -> EraNumber;
+
+        /// How many standard eras are there in the build&earn subperiod.
+        fn eras_per_build_and_earn_subperiod() -> EraNumber;
+
+        /// How many blocks are there per standard era.
+        fn blocks_per_era() -> BlockNumber;
+    }
+}

pallets/dapp-staking-v3/src/benchmarking/mod.rs

@@ -902,6 +902,44 @@ mod benchmarks {
         }
     }
 
+    #[benchmark]
+    fn on_idle_cleanup() {
+        // Prepare init config (protocol state, tier params & config, etc.)
+        initial_config::<T>();
+
+        // Advance enough periods to trigger the cleanup
+        let retention_period = T::RewardRetentionInPeriods::get();
+        advance_to_period::<T>(
+            ActiveProtocolState::<T>::get().period_number() + retention_period + 2,
+        );
+
+        let first_era_span_index = 0;
+        assert!(
+            EraRewards::<T>::contains_key(first_era_span_index),
+            "Sanity check - era reward span entry must exist."
+        );
+        let first_period = 1;
+        assert!(
+            PeriodEnd::<T>::contains_key(first_period),
+            "Sanity check - period end info must exist."
+        );
+        let block_number = System::<T>::block_number();
+
+        #[block]
+        {
+            DappStaking::<T>::on_idle(block_number, Weight::MAX);
+        }
+
+        assert!(
+            !EraRewards::<T>::contains_key(first_era_span_index),
+            "Entry should have been cleaned up."
+        );
+        assert!(
+            !PeriodEnd::<T>::contains_key(first_period),
+            "Period end info should have been cleaned up."
+        );
+    }
+
     impl_benchmark_test_suite!(
         Pallet,
         crate::benchmarking::tests::new_test_ext(),

pallets/dapp-staking-v3/src/lib.rs

@@ -64,8 +64,7 @@ mod test;
 mod benchmarking;
 
 mod types;
-use types::*;
-pub use types::{PriceProvider, TierThreshold};
+pub use types::*;
 
 pub mod weights;
 pub use weights::WeightInfo;
@@ -444,6 +443,10 @@ pub mod pallet {
     pub type DAppTiers<T: Config> =
         StorageMap<_, Twox64Concat, EraNumber, DAppTierRewardsFor<T>, OptionQuery>;
 
+    /// History cleanup marker - holds information about which DB entries should be cleaned up next, when applicable.
+    #[pallet::storage]
+    pub type HistoryCleanupMarker<T: Config> = StorageValue<_, CleanupMarker, ValueQuery>;
+
     #[pallet::genesis_config]
     #[derive(frame_support::DefaultNoBound)]
     pub struct GenesisConfig {
@@ -521,6 +524,10 @@ pub mod pallet {
         fn on_initialize(now: BlockNumber) -> Weight {
             Self::era_and_period_handler(now, TierAssignment::Real)
         }
+
+        fn on_idle(_block: BlockNumberFor<T>, remaining_weight: Weight) -> Weight {
+            Self::expired_entry_cleanup(&remaining_weight)
+        }
     }
 
     /// A reason for freezing funds.
@@ -580,7 +587,6 @@ pub mod pallet {
                     id: dapp_id,
                     state: DAppState::Registered,
                     reward_destination: None,
-                    tier_label: None,
                 },
             );
 
@@ -1131,9 +1137,8 @@ pub mod pallet {
             let earliest_staked_era = ledger
                 .earliest_staked_era()
                 .ok_or(Error::<T>::InternalClaimStakerError)?;
-            let era_rewards =
-                EraRewards::<T>::get(Self::era_reward_span_index(earliest_staked_era))
-                    .ok_or(Error::<T>::NoClaimableRewards)?;
+            let era_rewards = EraRewards::<T>::get(Self::era_reward_index(earliest_staked_era))
+                .ok_or(Error::<T>::NoClaimableRewards)?;
 
             // The last era for which we can theoretically claim rewards.
             // And indicator if we know the period's ending era.
@@ -1439,6 +1444,11 @@ pub mod pallet {
             .into())
         }
 
+        // TODO: this call should be removed prior to mainnet launch.
+        // It's super useful for testing purposes, but even though force is used in this pallet & works well,
+        // it won't apply to the inflation recalculation logic - which is wrong.
+        // Probably for this call to make sense, an outside logic should handle both inflation & dApp staking state changes.
+
         /// Used to force a change of era or subperiod.
         /// The effect isn't immediate but will happen on the next block.
         ///
@@ -1532,7 +1542,7 @@ pub mod pallet {
         }
 
         /// Calculates the `EraRewardSpan` index for the specified era.
-        pub(crate) fn era_reward_span_index(era: EraNumber) -> EraNumber {
+        pub(crate) fn era_reward_index(era: EraNumber) -> EraNumber {
             era.saturating_sub(era % T::EraRewardSpanLength::get())
         }
 
@@ -1829,7 +1839,7 @@ pub mod pallet {
 
             CurrentEraInfo::<T>::put(era_info);
 
-            let era_span_index = Self::era_reward_span_index(current_era);
+            let era_span_index = Self::era_reward_index(current_era);
             let mut span = EraRewards::<T>::get(&era_span_index).unwrap_or(EraRewardSpan::new());
             if let Err(_) = span.push(current_era, era_reward) {
                 // This must never happen but we log the error just in case.
@@ -1848,5 +1858,79 @@ pub mod pallet {
 
             consumed_weight
         }
+
+        /// Attempt to cleanup some expired entries, if enough remaining weight & applicable entries exist.
+        ///
+        /// Returns consumed weight.
+        fn expired_entry_cleanup(remaining_weight: &Weight) -> Weight {
+            // Need to be able to process full pass
+            if remaining_weight.any_lt(T::WeightInfo::on_idle_cleanup()) {
+                return Weight::zero();
+            }
+
+            // Get the cleanup marker
+            let mut cleanup_marker = HistoryCleanupMarker::<T>::get();
+
+            // Whitelisted storage, no need to account for the read.
+            let protocol_state = ActiveProtocolState::<T>::get();
+            let latest_expired_period = match protocol_state
+                .period_number()
+                .checked_sub(T::RewardRetentionInPeriods::get().saturating_add(1))
+            {
+                Some(latest_expired_period) => latest_expired_period,
+                None => {
+                    // Protocol hasn't advanced enough to have any expired entries.
+                    return T::WeightInfo::on_idle_cleanup();
+                }
+            };
+
+            // Get the oldest valid era - any era before it is safe to be cleaned up.
+            let oldest_valid_era = match PeriodEnd::<T>::get(latest_expired_period) {
+                Some(period_end_info) => period_end_info.final_era.saturating_add(1),
+                None => {
+                    // Can happen if it's period 0 or if the entry has already been cleaned up.
+                    return T::WeightInfo::on_idle_cleanup();
+                }
+            };
+
+            // Attempt to cleanup one expired `EraRewards` entry.
+            if let Some(era_reward) = EraRewards::<T>::get(cleanup_marker.era_reward_index) {
+                // If oldest valid era comes AFTER this span, it's safe to delete it.
+                if era_reward.last_era() < oldest_valid_era {
+                    EraRewards::<T>::remove(cleanup_marker.era_reward_index);
+                    cleanup_marker
+                        .era_reward_index
+                        .saturating_accrue(T::EraRewardSpanLength::get());
+                }
+            } else {
+                // Should never happen, but if it does, log an error and move on.
+                log::error!(
+                    target: LOG_TARGET,
+                    "Era rewards span for era {} is missing, but cleanup marker is set.",
+                    cleanup_marker.era_reward_index
+                );
+            }
+
+            // Attempt to cleanup one expired `DAppTiers` entry.
+            if cleanup_marker.dapp_tiers_index < oldest_valid_era {
+                DAppTiers::<T>::remove(cleanup_marker.dapp_tiers_index);
+                cleanup_marker.dapp_tiers_index.saturating_inc();
+            }
+
+            // One extra grace period before we cleanup period end info.
+            // This so we can always read the `final_era` of that period.
+            if let Some(period_end_cleanup) = latest_expired_period.checked_sub(1) {
+                PeriodEnd::<T>::remove(period_end_cleanup);
+            }
+
+            // Store the updated cleanup marker
+            HistoryCleanupMarker::<T>::put(cleanup_marker);
+
+            // We could try & cleanup more entries, but since it's not a critical operation and can happen whenever,
+            // we opt for the simpler solution where only 1 entry per block is cleaned up.
+            // It can be changed though.
+
+            T::WeightInfo::on_idle_cleanup()
+        }
     }
 }