change_feature_flags_for_testing/verification #12506

zjma · 2024-03-13T10:32:17Z

Description

Type of Change

Which Components or Systems Does This Change Impact?

How Has This Been Tested?

Key Areas to Review

Checklist

I have read and followed the CONTRIBUTING doc
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I identified and added all stakeholders and component owners affected by this change as reviewers
I tested both happy and unhappy path of the functionality
I have made corresponding changes to the documentation

trunk-io · 2024-03-13T10:32:21Z

⏱️ 25h 58m total CI duration on this PR

Job	Cumulative Duration	Recent Runs
rust-unit-coverage	12h	⬜ 🟥
windows-build	1h 48m	🟩 🟩 🟩 🟩 🟩 (+2 more)
rust-smoke-tests	1h 30m	🟥 ⬜ ⬜ 🟩 🟩
execution-performance / single-node-performance	1h 25m	🟥 🟥 🟩 🟩 🟩 (+1 more)
rust-unit-tests	1h 13m	🟥 🟥 ⬜ 🟥 🟩 (+1 more)
rust-move-unit-coverage	1h 8m	🟩 ⬜ ⬜ 🟩 🟩
rust-move-tests	1h 5m	🟥 🟥 ⬜ 🟥 🟥
rust-images / rust-all	57m	🟩 ⬜ ⬜ 🟩 🟩
forge-framework-upgrade-test / forge	56m	⬜ ⬜ 🟥
forge-e2e-test / forge	39m	⬜ 🟩 🟩
forge-compat-test / forge	39m	⬜ 🟩 🟩
rust-lints	36m	🟩 🟩 ⬜ 🟩 🟩 (+1 more)
check	21m	🟩 🟩 🟩 🟩 🟩 (+1 more)
run-tests-main-branch	21m	🟩 🟩 🟩 🟩 🟩
cli-e2e-tests / run-cli-tests	19m	🟩 🟩 🟩
rust-smoke-coverage	19m	⬜ 🟩
indexer-grpc-e2e-tests / test-indexer-grpc-docker-compose	11m	🟥 🟩 🟩
general-lints	10m	🟩 🟩 🟩 🟩 🟩 (+1 more)
check-dynamic-deps	10m	🟩 🟩 🟩 🟩 🟩 (+2 more)
semgrep/ci	3m	🟩 🟩 🟩 🟩 🟩 (+2 more)
node-api-compatibility-tests / node-api-compatibility-tests	3m	🟩 🟩 🟩
execution-performance / file_change_determinator	1m	🟩 🟩 🟩 🟩 🟩 (+1 more)
file_change_determinator	1m	🟩 🟩 🟩 🟩 🟩 (+1 more)
file_change_determinator	58s	🟩 🟩 🟩 🟩 🟩 (+1 more)
file_change_determinator	52s	🟩 🟩 🟩 🟩 🟩
permission-check	19s	🟩 🟩 🟩 🟩 🟩 (+2 more)
permission-check	19s	🟩 🟩 🟩 🟩 🟩 (+2 more)
permission-check	17s	🟩 🟩 🟩 🟩 🟩 (+2 more)
permission-check	16s	🟩 🟩 🟩 🟩 🟩 (+2 more)
determine-docker-build-metadata	15s	🟩 🟩 🟩 🟩 🟩
permission-check	14s	🟩 🟩 🟩 🟩 🟩 (+1 more)

🚨 1 job on the last run was significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
windows-build	12m	20m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

codecov · 2024-03-13T10:50:12Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 63.9%. Comparing base (920ad74) to head (9575044).

Additional details and impacted files

@@                             Coverage Diff                              @@
##           zjma/disable-config-functions-by-randomness   #12506   +/-   ##
============================================================================
  Coverage                                         63.9%    63.9%           
============================================================================
  Files                                              811      811           
  Lines                                           179999   179999           
============================================================================
  Hits                                            115093   115093           
  Misses                                           64906    64906

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2024-03-13T12:16:43Z

✅ Forge suite `compat` success on `aptos-node-v1.9.5` ==> `95750440092291be1083412a67db2965b1ac76b2`

Compatibility test results for aptos-node-v1.9.5 ==> 95750440092291be1083412a67db2965b1ac76b2 (PR)
1. Check liveness of validators at old version: aptos-node-v1.9.5
compatibility::simple-validator-upgrade::liveness-check : committed: 6712 txn/s, latency: 4936 ms, (p50: 4800 ms, p90: 8100 ms, p99: 10200 ms), latency samples: 234920
2. Upgrading first Validator to new version: 95750440092291be1083412a67db2965b1ac76b2
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 845 txn/s, latency: 32664 ms, (p50: 36000 ms, p90: 51600 ms, p99: 53000 ms), latency samples: 56620
3. Upgrading rest of first batch to new version: 95750440092291be1083412a67db2965b1ac76b2
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 37 txn/s, submitted: 606 txn/s, expired: 569 txn/s, latency: 43181 ms, (p50: 45700 ms, p90: 56500 ms, p99: 56600 ms), latency samples: 2485
4. upgrading second batch to new version: 95750440092291be1083412a67db2965b1ac76b2
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 2437 txn/s, latency: 12550 ms, (p50: 12900 ms, p90: 16100 ms, p99: 16800 ms), latency samples: 104820
5. check swarm health
Compatibility test for aptos-node-v1.9.5 ==> 95750440092291be1083412a67db2965b1ac76b2 passed
Test Ok

github-actions · 2024-03-13T12:18:26Z

✅ Forge suite `realistic_env_max_load` success on `95750440092291be1083412a67db2965b1ac76b2`

two traffics test: inner traffic : committed: 7827 txn/s, latency: 5012 ms, (p50: 4800 ms, p90: 6000 ms, p99: 10200 ms), latency samples: 3381440
two traffics test : committed: 100 txn/s, latency: 1852 ms, (p50: 1800 ms, p90: 2000 ms, p99: 7000 ms), latency samples: 1820
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.231, avg: 0.203", "QsPosToProposal: max: 0.284, avg: 0.255", "ConsensusProposalToOrdered: max: 0.466, avg: 0.410", "ConsensusOrderedToCommit: max: 0.303, avg: 0.286", "ConsensusProposalToCommit: max: 0.713, avg: 0.696"]
Max round gap was 1 [limit 4] at version 1577927. Max no progress secs was 4.777768 [limit 15] at version 1577927.
Test Ok

github-actions · 2024-03-13T12:28:51Z

❌ Forge suite `framework_upgrade` failure on `aptos-node-v1.9.5` ==> `95750440092291be1083412a67db2965b1ac76b2`

Compatibility test results for aptos-node-v1.9.5 ==> 95750440092291be1083412a67db2965b1ac76b2 (PR)
Upgrade the nodes to version: 95750440092291be1083412a67db2965b1ac76b2
Test Failed: API error: Unknown error Transaction committed on chain, but failed execution: Transaction Executed and Committed with Error EXECUTION_LIMIT_REACHED

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:565:25
   1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1963:27
   2: aptos_release_builder::validate::NetworkConfig::execute_proposal::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:364:9
   3: aptos_release_builder::validate::NetworkConfig::submit_and_execute_multi_step_proposal::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:124:64
   4: aptos_release_builder::validate::execute_release::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:432:22
   5: aptos_release_builder::validate::validate_config_and_generate_release::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:493:6
   6: aptos_release_builder::validate::validate_config::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:479:80
   7: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:63
   8: tokio::runtime::coop::with_budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:107:5
   9: tokio::runtime::coop::budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:73:5
  10: tokio::runtime::park::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:31
  11: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/blocking.rs:66:9
  12: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  13: tokio::runtime::context::runtime::enter_runtime
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16
  14: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  15: tokio::runtime::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/runtime.rs:350:50
  16: <aptos_testcases::framework_upgrade::FrameworkUpgrade as aptos_forge::interface::network::NetworkTest>::run
             at ./testsuite/testcases/src/framework_upgrade.rs:97:9
  17: aptos_forge::runner::Forge<F>::run::{{closure}}
             at ./testsuite/forge/src/runner.rs:598:42
  18: aptos_forge::runner::run_test
             at ./testsuite/forge/src/runner.rs:666:11
  19: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:598:30
  20: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
  21: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  22: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
  23: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
  24: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
  25: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
  26: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  27: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  28: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  29: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  30: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  31: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  32: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  33: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  34: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  35: __libc_start_main
  36: _start
Trailing Log Lines:
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  32: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  33: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  34: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  35: __libc_start_main
  36: _start


Swarm logs can be found here: See fgi output for more information.
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:292"},"thread_name":"main","hostname":"forge-framework-upgrade-pr-12506-1710331575-aptos-node-v1-9-5","timestamp":"2024-03-13T12:28:47.549919Z","message":"Deleting namespace forge-framework-upgrade-pr-12506: Some(NamespaceStatus { conditions: None, phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:400"},"thread_name":"main","hostname":"forge-framework-upgrade-pr-12506-1710331575-aptos-node-v1-9-5","timestamp":"2024-03-13T12:28:47.549944Z","message":"aptos-node resources for Forge removed in namespace: forge-framework-upgrade-pr-12506"}

failures:
    framework_upgrade::framework-upgrade
Failed to run tests:
Tests Failed

test result: FAILED. 0 passed; 1 failed; 0 filtered out

Error: Tests Failed

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:83:36
   1: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:618:13
   2: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
   3: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
   4: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
   5: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
   6: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
   7: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
   8: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
   9: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  10: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  11: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  12: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  13: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  14: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  15: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  16: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  17: __libc_start_main
  18: _start
Debugging output:
NAME                                   READY   STATUS      RESTARTS   AGE
aptos-node-0-validator-0               1/1     Running     0          19m
aptos-node-1-validator-0               1/1     Running     0          18m
aptos-node-2-validator-0               1/1     Running     0          17m
aptos-node-3-validator-0               1/1     Running     0          17m
genesis-aptos-genesis-eforge84-hq8tv   0/1     Completed   0          22m

release builder update for randomness framework (#12381) print jwk configs in release builder maybe keep the gas schedule update function fix tests using version_set_version fix doc test fixing test_register_and_update_validator fixing tests using consensus_config::set fixing storage_integration_test update spec also disable gas update function (#12431) * still have to disable gas function * update fix doc Co-authored-by: Greg Nazario <[email protected]> update update rename function update spec change_feature_flags_for_testing/verification (#12506) * change_feature_flags_for_testing/verification * update features function usage in genesis * update features function usage in harness * debug * update remaining reference in e2e-move-tests

) * disable X::set() release builder update for randomness framework (#12381) print jwk configs in release builder maybe keep the gas schedule update function fix tests using version_set_version fix doc test fixing test_register_and_update_validator fixing tests using consensus_config::set fixing storage_integration_test update spec also disable gas update function (#12431) * still have to disable gas function * update fix doc Co-authored-by: Greg Nazario <[email protected]> update update rename function update spec change_feature_flags_for_testing/verification (#12506) * change_feature_flags_for_testing/verification * update features function usage in genesis * update features function usage in harness * debug * update remaining reference in e2e-move-tests * a tmp release entry for randomness framework upgrade fix generate-proposals update release yaml for randomness framework (#12523) * debug * debug * debug * debug * debug * fix spec * update doc * update * update * support release entries for randomness config and jwk consensus config * update * lint

zjma marked this pull request as ready for review March 13, 2024 10:32

zjma requested review from alinush, junkil-park, mstraka100, davidiw, movekevin and wrwg as code owners March 13, 2024 10:32

zjma removed request for davidiw, alinush, junkil-park, wrwg, mstraka100 and movekevin March 13, 2024 10:32

zjma added CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR CICD:run-framework-upgrade-test labels Mar 13, 2024

This comment has been minimized.

Sign in to view

change_feature_flags_for_testing/verification

4582912

zjma force-pushed the disable-features-function branch from 4120f61 to 4582912 Compare March 13, 2024 10:58

zjma added 3 commits March 13, 2024 04:09

update features function usage in genesis

2713e61

update features function usage in harness

dee5f79

debug

19050ea

This comment has been minimized.

Sign in to view

update remaining reference in e2e-move-tests

9575044

This comment has been minimized.

Sign in to view

zjma merged commit e590cdd into zjma/disable-config-functions-by-randomness Mar 13, 2024
44 of 45 checks passed

zjma deleted the disable-features-function branch March 13, 2024 18:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

change_feature_flags_for_testing/verification #12506

change_feature_flags_for_testing/verification #12506

zjma commented Mar 13, 2024

trunk-io bot commented Mar 13, 2024 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

codecov bot commented Mar 13, 2024 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Mar 13, 2024

github-actions bot commented Mar 13, 2024

github-actions bot commented Mar 13, 2024

change_feature_flags_for_testing/verification #12506

change_feature_flags_for_testing/verification #12506

Conversation

zjma commented Mar 13, 2024

Description

Type of Change

Which Components or Systems Does This Change Impact?

How Has This Been Tested?

Key Areas to Review

Checklist

trunk-io bot commented Mar 13, 2024 • edited Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

codecov bot commented Mar 13, 2024 • edited Loading

Codecov Report

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Mar 13, 2024

✅ Forge suite compat success on aptos-node-v1.9.5 ==> 95750440092291be1083412a67db2965b1ac76b2

github-actions bot commented Mar 13, 2024

✅ Forge suite realistic_env_max_load success on 95750440092291be1083412a67db2965b1ac76b2

github-actions bot commented Mar 13, 2024

❌ Forge suite framework_upgrade failure on aptos-node-v1.9.5 ==> 95750440092291be1083412a67db2965b1ac76b2

trunk-io bot commented Mar 13, 2024 •

edited

Loading

codecov bot commented Mar 13, 2024 •

edited

Loading

✅ Forge suite `compat` success on `aptos-node-v1.9.5` ==> `95750440092291be1083412a67db2965b1ac76b2`

✅ Forge suite `realistic_env_max_load` success on `95750440092291be1083412a67db2965b1ac76b2`

❌ Forge suite `framework_upgrade` failure on `aptos-node-v1.9.5` ==> `95750440092291be1083412a67db2965b1ac76b2`