use storage multi_get #3306

nkysg · 2022-03-30T11:56:03Z

Pull request type

Please check the type of change your PR introduces:

What is the current behavior?

Issue Number: N/A

What is the new behavior?

use multi_get replace for loop

Other information

codecov · 2022-03-30T12:31:57Z

Codecov Report

Merging #3306 (3e43992) into master (7ab632b) will decrease coverage by 0.26%.
The diff coverage is 16.87%.

@@            Coverage Diff             @@
##           master    #3306      +/-   ##
==========================================
- Coverage   32.08%   31.82%   -0.25%     
==========================================
  Files         500      500              
  Lines       46282    46772     +490     
  Branches    20930    21268     +338     
==========================================
+ Hits        14843    14880      +37     
- Misses      17862    17912      +50     
- Partials    13577    13980     +403

Flag	Coverage Δ
unittests	`31.82% <16.87%> (-0.25%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
chain/api/src/message.rs	`0.00% <0.00%> (ø)`
chain/api/src/service.rs	`10.17% <0.00%> (+0.42%)`	⬆️
chain/service/src/chain_service.rs	`22.56% <0.00%> (-0.52%)`	⬇️
chain/src/chain.rs	`33.39% <0.00%> (-5.89%)`	⬇️
network-rpc/src/rpc.rs	`19.32% <0.00%> (+1.37%)`	⬆️
storage/src/block_info/mod.rs	`100.00% <ø> (ø)`
storage/src/transaction/mod.rs	`68.43% <0.00%> (-12.82%)`	⬇️
storage/src/tests/test_storage.rs	`15.32% <14.71%> (-0.14%)`	⬇️
storage/src/tests/test_batch.rs	`14.65% <15.67%> (+1.37%)`	⬆️
storage/src/lib.rs	`55.97% <17.65%> (-1.60%)`	⬇️
... and 93 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7ab632b...3e43992. Read the comment docs.

github-actions · 2022-03-30T12:44:39Z

Benchmark for `40a716b`

Click to view benchmark

Test	Base	PR	%
accumulator_append	595.5±11.93µs	596.2±14.85µs	+0.12%
block_apply/block_apply_10	267.2±1.49ms	267.3±0.49ms	+0.04%
block_apply/block_apply_1000	27.3±0.02s	27.4±0.03s	+0.37%
get_with_proof/db_store	36.6±3.66µs	35.8±0.27µs	-2.19%
get_with_proof/mem_store	31.0±0.35µs	31.2±0.38µs	+0.65%
put_and_commit/db_store/1	97.4±7.71µs	95.8±4.81µs	-1.64%
put_and_commit/db_store/10	852.5±41.56µs	857.7±37.15µs	+0.61%
put_and_commit/db_store/100	7.4±0.29ms	7.3±0.32ms	-1.35%
put_and_commit/db_store/5	440.2±24.42µs	439.1±23.76µs	-0.25%
put_and_commit/db_store/50	3.8±0.17ms	3.8±0.18ms	0.00%
put_and_commit/mem_store/1	60.8±5.64µs	61.0±5.96µs	+0.33%
put_and_commit/mem_store/10	567.9±47.29µs	569.3±47.34µs	+0.25%
put_and_commit/mem_store/100	5.6±0.83ms	5.6±0.84ms	0.00%
put_and_commit/mem_store/5	286.9±24.62µs	288.4±25.70µs	+0.52%
put_and_commit/mem_store/50	2.8±0.19ms	2.8±0.19ms	0.00%
query_block/query_block_in(10)_times(100)	3.8±0.11ms	3.9±0.10ms	+2.63%
query_block/query_block_in(10)_times(1000)	38.4±0.74ms	39.3±0.95ms	+2.34%
query_block/query_block_in(10)_times(10000)	384.1±6.09ms	390.3±9.74ms	+1.61%
query_block/query_block_in(1000)_times(100)	935.8±12.22µs	946.4±7.15µs	+1.13%
query_block/query_block_in(1000)_times(1000)	9.4±0.06ms	9.4±0.05ms	0.00%
query_block/query_block_in(1000)_times(10000)	94.0±0.74ms	94.3±0.76ms	+0.32%
storage_transaction	1066.1±289.42µs	1104.4±345.73µs	+3.59%
vm/transaction_execution/1	314.4±1.81ms	316.2±4.89ms	+0.57%
vm/transaction_execution/10	98.2±0.31ms	98.6±0.19ms	+0.41%
vm/transaction_execution/20	91.0±1.29ms	90.8±0.23ms	-0.22%
vm/transaction_execution/5	120.9±1.18ms	120.8±0.68ms	-0.08%
vm/transaction_execution/50	102.1±0.31ms	102.8±1.00ms	+0.69%

nkysg · 2022-03-30T13:02:09Z

list_block 需要修改下

nkysg · 2022-03-30T13:05:03Z

chain/service/src/chain_service.rs

@@ -298,6 +301,7 @@ impl ReadableChainService for ChainReaderServiceInner {
        self.storage.get_blocks(ids)
    }

+    // XXX FIXME different get_blocks


这里风格使用Vec，其他地方用的是Vec<Option>不统一

jolestar

LGTM

chain/src/chain.rs

github-actions · 2022-03-31T04:39:21Z

Benchmark for `cd79ea6`

Click to view benchmark

Test	Base	PR	%
accumulator_append	590.7±10.02µs	592.3±11.78µs	+0.27%
block_apply/block_apply_10	266.5±1.24ms	265.8±0.62ms	-0.26%
block_apply/block_apply_1000	27.5±0.02s	27.2±0.03s	-1.09%
get_with_proof/db_store	36.0±0.36µs	35.9±0.34µs	-0.28%
get_with_proof/mem_store	31.0±0.33µs	31.0±0.26µs	0.00%
put_and_commit/db_store/1	95.0±4.56µs	95.2±4.73µs	+0.21%
put_and_commit/db_store/10	853.3±39.12µs	844.7±39.15µs	-1.01%
put_and_commit/db_store/100	7.3±0.30ms	7.2±0.29ms	-1.37%
put_and_commit/db_store/5	441.3±20.77µs	432.4±23.40µs	-2.02%
put_and_commit/db_store/50	3.8±0.18ms	3.8±0.16ms	0.00%
put_and_commit/mem_store/1	61.7±5.45µs	60.9±5.46µs	-1.30%
put_and_commit/mem_store/10	577.9±48.13µs	569.5±46.13µs	-1.45%
put_and_commit/mem_store/100	5.6±0.80ms	5.6±0.80ms	0.00%
put_and_commit/mem_store/5	291.6±24.71µs	287.5±24.21µs	-1.41%
put_and_commit/mem_store/50	2.8±0.18ms	2.8±0.18ms	0.00%
query_block/query_block_in(10)_times(100)	3.9±0.23ms	3.9±0.09ms	0.00%
query_block/query_block_in(10)_times(1000)	39.2±0.98ms	37.9±1.35ms	-3.32%
query_block/query_block_in(10)_times(10000)	383.3±5.50ms	388.4±7.81ms	+1.33%
query_block/query_block_in(1000)_times(100)	953.2±13.25µs	949.2±6.98µs	-0.42%
query_block/query_block_in(1000)_times(1000)	9.4±0.09ms	9.6±0.08ms	+2.13%
query_block/query_block_in(1000)_times(10000)	94.4±0.79ms	95.7±0.88ms	+1.38%
storage_transaction	1037.1±311.61µs	1049.5±300.30µs	+1.20%
vm/transaction_execution/1	314.3±0.56ms	314.5±1.34ms	+0.06%
vm/transaction_execution/10	98.5±0.27ms	98.7±1.14ms	+0.20%
vm/transaction_execution/20	90.6±0.18ms	90.8±0.56ms	+0.22%
vm/transaction_execution/5	120.6±0.43ms	120.9±1.42ms	+0.25%
vm/transaction_execution/50	102.9±0.88ms	102.4±0.39ms	-0.49%

chain/src/chain.rs

github-actions · 2022-03-31T11:14:24Z

Benchmark for `c360298`

Click to view benchmark

Test	Base	PR	%
accumulator_append	587.2±11.27µs	592.4±12.71µs	+0.89%
block_apply/block_apply_10	269.0±0.78ms	267.5±0.70ms	-0.56%
block_apply/block_apply_1000	27.3±0.02s	27.2±0.02s	-0.37%
get_with_proof/db_store	35.9±0.35µs	35.8±0.28µs	-0.28%
get_with_proof/mem_store	31.1±0.47µs	31.2±0.48µs	+0.32%
put_and_commit/db_store/1	94.0±4.59µs	94.5±4.61µs	+0.53%
put_and_commit/db_store/10	851.6±42.84µs	850.3±37.83µs	-0.15%
put_and_commit/db_store/100	7.2±0.31ms	7.3±0.31ms	+1.39%
put_and_commit/db_store/5	430.0±20.36µs	434.2±20.75µs	+0.98%
put_and_commit/db_store/50	3.8±0.16ms	3.8±0.16ms	0.00%
put_and_commit/mem_store/1	60.7±5.40µs	61.3±5.51µs	+0.99%
put_and_commit/mem_store/10	569.2±45.28µs	569.1±44.93µs	-0.02%
put_and_commit/mem_store/100	5.6±0.80ms	5.6±0.80ms	0.00%
put_and_commit/mem_store/5	286.7±23.69µs	288.0±23.41µs	+0.45%
put_and_commit/mem_store/50	2.8±0.18ms	2.8±0.18ms	0.00%
query_block/query_block_in(10)_times(100)	3.9±0.20ms	4.0±0.09ms	+2.56%
query_block/query_block_in(10)_times(1000)	38.8±1.34ms	38.4±1.12ms	-1.03%
query_block/query_block_in(10)_times(10000)	384.3±8.19ms	388.0±5.96ms	+0.96%
query_block/query_block_in(1000)_times(100)	962.0±4.41µs	963.0±7.66µs	+0.10%
query_block/query_block_in(1000)_times(1000)	9.7±0.05ms	9.6±0.07ms	-1.03%
query_block/query_block_in(1000)_times(10000)	96.4±0.86ms	96.6±1.04ms	+0.21%
storage_transaction	1057.3±279.93µs	1058.0±271.88µs	+0.07%
vm/transaction_execution/1	314.3±1.56ms	314.1±0.45ms	-0.06%
vm/transaction_execution/10	98.4±0.38ms	98.7±0.73ms	+0.30%
vm/transaction_execution/20	90.8±0.33ms	90.7±0.50ms	-0.11%
vm/transaction_execution/5	121.6±3.88ms	120.5±0.16ms	-0.90%
vm/transaction_execution/50	102.3±0.54ms	102.8±1.17ms	+0.49%

github-actions · 2022-04-01T00:07:37Z

Benchmark for `6e28b2a`

Click to view benchmark

Test	Base	PR	%
accumulator_append	589.1±10.00µs	589.1±9.77µs	0.00%
block_apply/block_apply_10	266.0±0.66ms	265.9±0.91ms	-0.04%
block_apply/block_apply_1000	27.3±0.03s	27.3±0.02s	0.00%
get_with_proof/db_store	36.2±0.60µs	36.2±0.28µs	0.00%
get_with_proof/mem_store	31.4±0.27µs	31.1±0.29µs	-0.96%
put_and_commit/db_store/1	95.1±5.18µs	94.7±5.37µs	-0.42%
put_and_commit/db_store/10	849.2±40.77µs	845.6±41.59µs	-0.42%
put_and_commit/db_store/100	7.2±0.30ms	7.3±0.26ms	+1.39%
put_and_commit/db_store/5	432.2±22.41µs	432.9±21.48µs	+0.16%
put_and_commit/db_store/50	3.8±0.16ms	3.8±0.16ms	0.00%
put_and_commit/mem_store/1	60.6±5.34µs	60.9±5.38µs	+0.50%
put_and_commit/mem_store/10	570.8±48.20µs	570.8±48.34µs	0.00%
put_and_commit/mem_store/100	5.6±0.79ms	5.6±0.80ms	0.00%
put_and_commit/mem_store/5	288.4±24.82µs	288.5±25.38µs	+0.03%
put_and_commit/mem_store/50	2.8±0.18ms	2.8±0.19ms	0.00%
query_block/query_block_in(10)_times(100)	3.8±0.07ms	3.8±0.08ms	0.00%
query_block/query_block_in(10)_times(1000)	38.2±0.92ms	38.6±0.80ms	+1.05%
query_block/query_block_in(10)_times(10000)	385.4±6.22ms	384.4±5.23ms	-0.26%
query_block/query_block_in(1000)_times(100)	955.3±9.52µs	983.1±8.58µs	+2.91%
query_block/query_block_in(1000)_times(1000)	9.6±0.10ms	9.7±0.08ms	+1.04%
query_block/query_block_in(1000)_times(10000)	96.1±0.66ms	97.4±0.82ms	+1.35%
storage_transaction	1063.3±339.44µs	1046.0±294.99µs	-1.63%
vm/transaction_execution/1	313.8±1.22ms	315.5±3.26ms	+0.54%
vm/transaction_execution/10	98.5±0.83ms	98.8±1.09ms	+0.30%
vm/transaction_execution/20	90.7±0.38ms	90.8±0.82ms	+0.11%
vm/transaction_execution/5	120.3±0.36ms	121.1±1.10ms	+0.67%
vm/transaction_execution/50	102.0±0.15ms	102.2±0.25ms	+0.20%

nkysg added 9 commits March 28, 2022 18:51

add DBStorage multiple_get

862ebf5

add Result<Vec<Option<T>>> AsResultType

9ce5bdb

add multi_get implement and test code

a01660a

fix fmt

1855268

implement cache_storage multi_get

2351856

add StorageInstace multi_get

43672c1

add StorageInstance test

c50a763

export snapshot use multi-get

2008792

use multi_get replace foreach

32d6694

nkysg changed the title ~~Multiple get code~~ use storage multi_get Mar 30, 2022

nkysg commented Mar 30, 2022

View reviewed changes

jolestar requested review from sanlee42 and jolestar March 31, 2022 02:00

jolestar approved these changes Mar 31, 2022

View reviewed changes

chain/src/chain.rs Outdated Show resolved Hide resolved

nkysg added 3 commits March 31, 2022 11:36

get_blocks_by_number use multi_get

3c252eb

fix log error and use db.multi_get

e633ce6

fix typo

edcdde0

jolestar reviewed Mar 31, 2022

View reviewed changes

chain/src/chain.rs Outdated Show resolved Hide resolved

nkysg added 2 commits March 31, 2022 18:23

fix get_blocks_by_num

ae628be

fix fmt

b218044

add get_blocks_by_number test

3e43992

nkysg requested a review from jolestar March 31, 2022 23:32

jolestar approved these changes Apr 1, 2022

View reviewed changes

nkysg merged commit 168f640 into master Apr 1, 2022

nkysg deleted the multiple_get_code branch April 1, 2022 02:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use storage multi_get #3306

use storage multi_get #3306

nkysg commented Mar 30, 2022 •

edited

Loading

codecov bot commented Mar 30, 2022 •

edited

Loading

github-actions bot commented Mar 30, 2022

nkysg commented Mar 30, 2022

nkysg Mar 30, 2022

jolestar left a comment

github-actions bot commented Mar 31, 2022

github-actions bot commented Mar 31, 2022

github-actions bot commented Apr 1, 2022

use storage multi_get #3306

use storage multi_get #3306

Conversation

nkysg commented Mar 30, 2022 • edited Loading

Pull request type

What is the current behavior?

What is the new behavior?

Other information

codecov bot commented Mar 30, 2022 • edited Loading

Codecov Report

github-actions bot commented Mar 30, 2022

Benchmark for 40a716b

nkysg commented Mar 30, 2022

nkysg Mar 30, 2022

Choose a reason for hiding this comment

jolestar left a comment

Choose a reason for hiding this comment

github-actions bot commented Mar 31, 2022

Benchmark for cd79ea6

github-actions bot commented Mar 31, 2022

Benchmark for c360298

github-actions bot commented Apr 1, 2022

Benchmark for 6e28b2a

nkysg commented Mar 30, 2022 •

edited

Loading

codecov bot commented Mar 30, 2022 •

edited

Loading

Benchmark for `40a716b`

Benchmark for `cd79ea6`

Benchmark for `c360298`

Benchmark for `6e28b2a`