Determine the gas consumption of opcodes #1281
Assignees
Comments
piux2
added a commit
that referenced
this issue
Apr 25, 2024
<!-- please provide a detailed description of the changes made in this pull request. --> <details><summary>Contributors' checklist...</summary> - [x] Added new tests - [ ] Provided an example (e.g. screenshot) to aid review or the PR is self-explanatory - [ ] Updated the official documentation or not needed - [x] No breaking changes were made - [x] Added references to related issues and PRs - [ ] Provided any useful hints for running manual tests - [ ] Added new benchmarks to [generated graphs](https://gnoland.github.io/benchmarks), if any. More info [here](https://github.com/gnolang/gno/blob/master/.benchmarks/README.md). </details> Ref: #1070 #1067 #649 #1281 ## Summary The current gno.land node, optimized for development purposes, has a simplified verification process and gas meter implementation. To transition the gno.land node to a production-ready state, it is necessary to implement a comprehensive gas metering system that accurately accounts for VM gas consumption. This includes refining the gas fee structure to encompass all relevant costs, ensuring robust transaction validation, and calculating gas consumption based on actual computational load. This PR aims to address these limitations by introducing a complete gas meter and validation flow, laying the groundwork for further gas meter profiling and configuration. ## Problem Definition Current State and Limitations for Production: - **VM Gas Consumption Not Accounted in Gas Meter:** The current gas meter fails to calculate VM gas consumption, potentially allowing heavy contract loads without corresponding gas meter deductions. A refined system should measure and charge for VM gas usage accurately. - **Gas Fee Structure:** Presently, the gas fee structure only includes storage access, transaction size, and signature verification. VM gas fees are levied as a separate, flat fee, which might lead to confusion among users expecting the total fee to match the amount specified in the 'gas-fee' argument. For improved transparency and precision, the gas fee structure should integrate all these aspects. - **Transaction Validation:** The system currently validates basic information for VM msg_addpkg and msg_call. However, gas consumption cannot be determined before fully executing these messages against the VM. Consequently, VM transactions are placed in the mempool and propagated to other nodes, even if they may not meet the gas fee requirements to execute these transactions. This undermines the purpose of using gas fees to prevent VM spamming. ## Solution: ( Updated ) This is a high-level description of the implemented features: ~~Added an anteHandler in VM to monitor gas consumption~~ ~~Implemented chained VM anteHandler in auth.anteHandler~~ - Consume gas to verify account, signature and tx size in CheckTx - Consume VM gas in DeliverTx - Accumulated VM CPU cycles, memory allocation, store access, transaction size, and signature verification into a single gas meter. - Enabled local node checks of VM resource usage. The VM message is only aborted if it runs out of gas in basic CheckTx. However, the message is still propagated to other nodes if execution fails to prevent censorship - Introduced a structured format for logging gas consumption for profiling and metrics. - Introduced a gas factor linking gas to vm CPU cycles and memory allocation to balance between vm gas consumption with the rest. ## Trade-offs and Future Optimization: ( Updated ) ~~The current implementation processes messages against the VM to check gas consumption in abci.CheckTx() before inclusion in the mempool and propagation to other nodes.~~ ~~Messages lacking sufficient gas-wanted will be dropped, preventing abuse without adequate gas fees. However, the trade-off is that for each message with enough gas, the VM executes the transaction twice: once in CheckTx() and once in DeliverTx(). As these occur in separate execution contexts and are not in synchronized sequence, the performance impact is currently a secondary concern.~~ We moved the VM gas check from CheckTx to DeliverTx for the following reasons: - We only know the VM gas consumption after the messages have been processed. - Running VM execution for many CheckTx requests from the peers could overload the mempool that is executing CheckTx. - This could slow down the propagation of transactions across the entire network. By moving the VM gas check from CheckTx to DeliverTx, we are able to reduce the load on the mempool of a node and allow transactions to propagate through the network faster. In the future, we may use a predicted median value instead of the exact value from transaction execution for efficiency. ## What's Next: - Add a minimum gas price flag and configuration for node operation. - Provide a user-friendly fee input interface, offering 'gas-wanted' and 'gas price' as alternatives to the current 'gas-wanted' and 'gas-fee' inputs. - Tune the gas factor based on VM CPU and Memory Profiling. The current factor is 1:1 between gas and VM CPU cycles and memory allocation. --------- Co-authored-by: Thomas Bruyelle <[email protected]>
thehowl
added a commit
that referenced
this issue
Dec 18, 2024
<!-- please provide a detailed description of the changes made in this pull request. --> <details><summary>Contributors' checklist...</summary> - [ x] Added new tests, or not needed, or not feasible - [ x] Provided an example (e.g. screenshot) to aid review or the PR is self-explanatory - [ x] Updated the official documentation or not needed - [ x] No breaking changes were made, or a `BREAKING CHANGE: xxx` message was included in the description - [ x] Added references to related issues and PRs - [ x] Provided any useful hints for running manual tests - [ ] Added new benchmarks to [generated graphs](https://gnoland.github.io/benchmarks), if any. More info [here](https://github.com/gnolang/gno/blob/master/.benchmarks/README.md). </details> We build this tool mainly for the following issues #1826 #1828 #1281 #1973 We could also use it in the following cases. #1973 #2222 ### `gnobench` benchmarks the time consumed for each VM CPU OpCode and persistent access to the store, including marshalling and unmarshalling of realm objects. ## Design consideration ### Minimum Overhead and Footprint - Constant build flags enable benchmarking. - Encode operations and measurements in binary. - Dump to a local file in binary. - No logging, printout, or network access involved. ### Accuracy - Pause the timer for storage access while performing VM opcode benchmarking. - Measure each OpCode execution in nanoseconds. - Store access includes the duration for Amino marshalling and unmarshalling. It is built on top of @deelawn's design and framework with @jaekwon's input. #2073 ## Usage ### Simple mode The benchmark only involves the GnoVM and the persistent store. It benchmarks the bare minimum components, and the results are isolated from other components. We use standardize gno contract to perform the benchmarking. This mode is the best for benchmarking each major release and/or changes in GnoVM. make opcode make storage ### Production mode It benchmarks the node in the production environment with minimum overhead. We can only benchmark with standardize the contract but also capture the live usage in production environment. It gives us a complete picture of the node perform. 1. Build the production node with benchmarking flags: `go build -tags "benchmarkingstorage benchmarkingops" gno.land/cmd/gnoland` 2. Run the node in the production environment. It will dump benchmark data to a benchmark.bin file. 3. call the realm contracts at `gno.land/r/x/benchmark/opcodes` and `gno.land/r/x/benchmark/storage` 4. Stop the server after the benchmarking session is complete. 5. Run the following command to convert the binary dump: `gnobench -bin path_to_benchmark_bin` it converts the binary dump to results.csv and results_stats.csv. ## Results ( Examples ) The benchmarking results are stored in two files: 1. The raw results are saved in results.csv. | Operation | Elapsed Time | Disk IO Bytes | |-----------------|--------------|---------------| | OpEval | 40333 | 0 | | OpPopBlock | 208 | 0 | | OpHalt | 167 | 0 | | OpEval | 500 | 0 | | OpInterfaceType | 458 | 0 | | OpPopBlock | 166 | 0 | | OpHalt | 125 | 0 | | OpInterfaceType | 21125 | 0 | | OpEval | 541 | 0 | | OpEval | 209 | 0 | | OpInterfaceType | 334 | 0 | 2. The averages and standard deviations are summarized in results_stats.csv. | Operation | Avg Time | Avg Size | Time Std Dev | Count | |----------------|----------|----------|--------------|-------| | OpAdd | 101 | 0 | 45 | 300 | | OpAddAssign | 309 | 0 | 1620 | 100 | | OpArrayLit | 242 | 0 | 170 | 700 | | OpArrayType | 144 | 0 | 100 | 714 | | OpAssign | 136 | 0 | 95 | 2900 | | OpBand | 92 | 0 | 30 | 100 | | OpBandAssign | 127 | 0 | 62 | 100 | | OpBandn | 97 | 0 | 54 | 100 | | OpBandnAssign | 125 | 0 | 113 | 100 | | OpBinary1 | 128 | 0 | 767 | 502 | | OpBody | 127 | 0 | 145 | 13700 | --------- Co-authored-by: Morgan Bazalgette <[email protected]>
omarsy
pushed a commit
to omarsy/gno
that referenced
this issue
Dec 18, 2024
<!-- please provide a detailed description of the changes made in this pull request. --> <details><summary>Contributors' checklist...</summary> - [ x] Added new tests, or not needed, or not feasible - [ x] Provided an example (e.g. screenshot) to aid review or the PR is self-explanatory - [ x] Updated the official documentation or not needed - [ x] No breaking changes were made, or a `BREAKING CHANGE: xxx` message was included in the description - [ x] Added references to related issues and PRs - [ x] Provided any useful hints for running manual tests - [ ] Added new benchmarks to [generated graphs](https://gnoland.github.io/benchmarks), if any. More info [here](https://github.com/gnolang/gno/blob/master/.benchmarks/README.md). </details> We build this tool mainly for the following issues gnolang#1826 gnolang#1828 gnolang#1281 gnolang#1973 We could also use it in the following cases. gnolang#1973 gnolang#2222 ### `gnobench` benchmarks the time consumed for each VM CPU OpCode and persistent access to the store, including marshalling and unmarshalling of realm objects. ## Design consideration ### Minimum Overhead and Footprint - Constant build flags enable benchmarking. - Encode operations and measurements in binary. - Dump to a local file in binary. - No logging, printout, or network access involved. ### Accuracy - Pause the timer for storage access while performing VM opcode benchmarking. - Measure each OpCode execution in nanoseconds. - Store access includes the duration for Amino marshalling and unmarshalling. It is built on top of @deelawn's design and framework with @jaekwon's input. gnolang#2073 ## Usage ### Simple mode The benchmark only involves the GnoVM and the persistent store. It benchmarks the bare minimum components, and the results are isolated from other components. We use standardize gno contract to perform the benchmarking. This mode is the best for benchmarking each major release and/or changes in GnoVM. make opcode make storage ### Production mode It benchmarks the node in the production environment with minimum overhead. We can only benchmark with standardize the contract but also capture the live usage in production environment. It gives us a complete picture of the node perform. 1. Build the production node with benchmarking flags: `go build -tags "benchmarkingstorage benchmarkingops" gno.land/cmd/gnoland` 2. Run the node in the production environment. It will dump benchmark data to a benchmark.bin file. 3. call the realm contracts at `gno.land/r/x/benchmark/opcodes` and `gno.land/r/x/benchmark/storage` 4. Stop the server after the benchmarking session is complete. 5. Run the following command to convert the binary dump: `gnobench -bin path_to_benchmark_bin` it converts the binary dump to results.csv and results_stats.csv. ## Results ( Examples ) The benchmarking results are stored in two files: 1. The raw results are saved in results.csv. | Operation | Elapsed Time | Disk IO Bytes | |-----------------|--------------|---------------| | OpEval | 40333 | 0 | | OpPopBlock | 208 | 0 | | OpHalt | 167 | 0 | | OpEval | 500 | 0 | | OpInterfaceType | 458 | 0 | | OpPopBlock | 166 | 0 | | OpHalt | 125 | 0 | | OpInterfaceType | 21125 | 0 | | OpEval | 541 | 0 | | OpEval | 209 | 0 | | OpInterfaceType | 334 | 0 | 2. The averages and standard deviations are summarized in results_stats.csv. | Operation | Avg Time | Avg Size | Time Std Dev | Count | |----------------|----------|----------|--------------|-------| | OpAdd | 101 | 0 | 45 | 300 | | OpAddAssign | 309 | 0 | 1620 | 100 | | OpArrayLit | 242 | 0 | 170 | 700 | | OpArrayType | 144 | 0 | 100 | 714 | | OpAssign | 136 | 0 | 95 | 2900 | | OpBand | 92 | 0 | 30 | 100 | | OpBandAssign | 127 | 0 | 62 | 100 | | OpBandn | 97 | 0 | 54 | 100 | | OpBandnAssign | 125 | 0 | 113 | 100 | | OpBinary1 | 128 | 0 | 767 | 502 | | OpBody | 127 | 0 | 145 | 13700 | --------- Co-authored-by: Morgan Bazalgette <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
To determine the gas consumption of opcodes, we should collect metrics to establish the appropriate opcode gas configuration for the mainnet. Please refer to the following link for more information: github.com/gnolang/gno/blob/master/gnovm/pkg/gnolang/machine.go#L762-L881
cc @peter7891 @leohhhn @MichaelFrazzy @mvertes
Related with #1430
Related with #1106
The text was updated successfully, but these errors were encountered: