Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Optimize transaction signature recovery #6471

Merged
merged 23 commits into from
Dec 18, 2018
Merged

Optimize transaction signature recovery #6471

merged 23 commits into from
Dec 18, 2018

Conversation

heifner
Copy link
Contributor

@heifner heifner commented Dec 12, 2018

Change Description

  • Multi-thread transaction signature recovery for incoming transactions

Consensus Changes

None

API Changes

  • New producer_plugin option:
    -- --producer-threads arg (=2) Number of worker threads in producer thread pool

  • New chain_plugin option:
    -- --signature-cpu-billable-pct arg (=50) Percentage of actual signature recovery cpu to bill. Whole number percentages, e.g. 50 for 50%

Documentation Additions

See API Changes above

@heifner heifner changed the title [WIP] Optimize transaction signature recovery Optimize transaction signature recovery Dec 14, 2018
@heifner heifner requested a review from arhag December 14, 2018 21:51
@taokayan
Copy link
Contributor

taokayan commented Dec 17, 2018

Hi @heifner , I've tested & compared your change in this ticket vs my change(#6460) vs original (develop branch). I got almost the same performance(~3600 TPS) between your implementation and mine, while the original code shows a significant lower TPS number(~2200 TPS). Here's the details:

Testings steps:

1. start nodeos with txn_test_gen plugin, without history_plugin
2. run curl --data-binary '["eosio","5KQwrPbwdL6PhXujxW37FSSQZ1JiwsST4cqQzDeyXtP79zkvFD3"]' http://localhost:8888/v1/txn_test_gen/create_test_accounts
3. run curl --data-binary '["fooa", 2, 10]' http://localhost:8888/v1/txn_test_gen/start_generation
4. after 10 secs, run curl --data-binary '["fooa", 2, 10]' http://localhost:8888/v1/txn_test_gen/stop_generation

Your implementation(commit id 22ab63c):
test 1 (112.215us / transaction, or 3571 TPS):

info  2018-12-17T03:38:50.841 thread-0  txn_test_gen_plugin.cp:288    start_generation     ] Started transaction test plugin; performing 10 transactions every 2ms
info  2018-12-17T03:38:51.001 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000005a5d3ffadf... #90 @ 2018-12-17T03:38:51.000 signed by eosio [trxs: 540, lib: 89, confirmed: 0]
info  2018-12-17T03:38:51.512 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000005be221ca71... #91 @ 2018-12-17T03:38:51.500 signed by eosio [trxs: 1786, lib: 90, confirmed: 0]
info  2018-12-17T03:38:52.011 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000005ca3cb0b30... #92 @ 2018-12-17T03:38:52.000 signed by eosio [trxs: 1804, lib: 91, confirmed: 0]
info  2018-12-17T03:38:52.516 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000005d545b12b6... #93 @ 2018-12-17T03:38:52.500 signed by eosio [trxs: 1680, lib: 92, confirmed: 0]
info  2018-12-17T03:38:53.012 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000005e440186bf... #94 @ 2018-12-17T03:38:53.000 signed by eosio [trxs: 1806, lib: 93, confirmed: 0]
info  2018-12-17T03:38:53.512 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000005fd986f04e... #95 @ 2018-12-17T03:38:53.500 signed by eosio [trxs: 1815, lib: 94, confirmed: 0]
info  2018-12-17T03:38:54.013 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000060af321d1c... #96 @ 2018-12-17T03:38:54.000 signed by eosio [trxs: 1743, lib: 95, confirmed: 0]
info  2018-12-17T03:38:54.515 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 000000610e9832a4... #97 @ 2018-12-17T03:38:54.500 signed by eosio [trxs: 1815, lib: 96, confirmed: 0]
info  2018-12-17T03:38:55.014 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000062500143bb... #98 @ 2018-12-17T03:38:55.000 signed by eosio [trxs: 1810, lib: 97, confirmed: 0]
info  2018-12-17T03:38:55.514 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 000000637205ad34... #99 @ 2018-12-17T03:38:55.500 signed by eosio [trxs: 1737, lib: 98, confirmed: 0]
info  2018-12-17T03:38:56.010 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000064ebcec0cc... #100 @ 2018-12-17T03:38:56.000 signed by eosio [trxs: 1802, lib: 99, confirmed: 0]
info  2018-12-17T03:38:56.514 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 000000655c2dd588... #101 @ 2018-12-17T03:38:56.500 signed by eosio [trxs: 1815, lib: 100, confirmed: 0]
info  2018-12-17T03:38:57.013 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 000000666b94c8cf... #102 @ 2018-12-17T03:38:57.000 signed by eosio [trxs: 1694, lib: 101, confirmed: 0]
info  2018-12-17T03:38:57.513 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000067ca229b2f... #103 @ 2018-12-17T03:38:57.500 signed by eosio [trxs: 1818, lib: 102, confirmed: 0]
info  2018-12-17T03:38:58.014 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000068a99cb919... #104 @ 2018-12-17T03:38:58.000 signed by eosio [trxs: 1822, lib: 103, confirmed: 0]
info  2018-12-17T03:38:58.514 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000069228120ef... #105 @ 2018-12-17T03:38:58.500 signed by eosio [trxs: 1736, lib: 104, confirmed: 0]
info  2018-12-17T03:38:59.013 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000006a4125e5b8... #106 @ 2018-12-17T03:38:59.000 signed by eosio [trxs: 1813, lib: 105, confirmed: 0]
info  2018-12-17T03:38:59.508 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000006ba786e025... #107 @ 2018-12-17T03:38:59.500 signed by eosio [trxs: 1815, lib: 106, confirmed: 0]
info  2018-12-17T03:39:00.017 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000006c2c5d1dda... #108 @ 2018-12-17T03:39:00.000 signed by eosio [trxs: 1730, lib: 107, confirmed: 0]
info  2018-12-17T03:39:00.511 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000006dc45c0f01... #109 @ 2018-12-17T03:39:00.500 signed by eosio [trxs: 1810, lib: 108, confirmed: 0]
info  2018-12-17T03:39:00.869 thread-0  txn_test_gen_plugin.cp:370    stop_generation      ] Stopping transaction generation test
info  2018-12-17T03:39:00.869 thread-0  txn_test_gen_plugin.cp:373    stop_generation      ] 35734 transactions executed, 112.21539710080035945us / transaction

test 2 (111.379us / transaction, around 3591 TPS):

info  2018-12-17T03:45:36.978 thread-0  txn_test_gen_plugin.cp:288    start_generation     ] Started transaction test plugin; performing 10 transactions every 2ms
info  2018-12-17T03:45:37.000 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000386f53131f0... #902 @ 2018-12-17T03:45:37.000 signed by eosio [trxs: 50, lib: 901, confirmed: 0]
info  2018-12-17T03:45:37.516 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 000003879e9add0b... #903 @ 2018-12-17T03:45:37.500 signed by eosio [trxs: 1728, lib: 902, confirmed: 0]
info  2018-12-17T03:45:38.016 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 000003887e737f26... #904 @ 2018-12-17T03:45:38.000 signed by eosio [trxs: 1818, lib: 903, confirmed: 0]
info  2018-12-17T03:45:38.515 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000038920752f2c... #905 @ 2018-12-17T03:45:38.500 signed by eosio [trxs: 1814, lib: 904, confirmed: 0]
info  2018-12-17T03:45:39.014 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000038a0e337b9e... #906 @ 2018-12-17T03:45:39.000 signed by eosio [trxs: 1725, lib: 905, confirmed: 0]
info  2018-12-17T03:45:39.513 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000038ba50c7fe5... #907 @ 2018-12-17T03:45:39.500 signed by eosio [trxs: 1823, lib: 906, confirmed: 0]
info  2018-12-17T03:45:40.014 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000038c6faa76ca... #908 @ 2018-12-17T03:45:40.000 signed by eosio [trxs: 1828, lib: 907, confirmed: 0]
info  2018-12-17T03:45:40.513 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000038d4c345fe0... #909 @ 2018-12-17T03:45:40.500 signed by eosio [trxs: 1736, lib: 908, confirmed: 0]
info  2018-12-17T03:45:41.014 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000038e5653c9a3... #910 @ 2018-12-17T03:45:41.000 signed by eosio [trxs: 1827, lib: 909, confirmed: 0]
info  2018-12-17T03:45:41.512 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000038f833b6747... #911 @ 2018-12-17T03:45:41.500 signed by eosio [trxs: 1812, lib: 910, confirmed: 0]
info  2018-12-17T03:45:42.018 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 000003900f19be6b... #912 @ 2018-12-17T03:45:42.000 signed by eosio [trxs: 1763, lib: 911, confirmed: 0]
info  2018-12-17T03:45:42.512 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 000003912feb3a7b... #913 @ 2018-12-17T03:45:42.500 signed by eosio [trxs: 1829, lib: 912, confirmed: 0]
info  2018-12-17T03:45:43.014 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000392309be535... #914 @ 2018-12-17T03:45:43.000 signed by eosio [trxs: 1824, lib: 913, confirmed: 0]
info  2018-12-17T03:45:43.515 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000039317fde566... #915 @ 2018-12-17T03:45:43.500 signed by eosio [trxs: 1746, lib: 914, confirmed: 0]
info  2018-12-17T03:45:44.017 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 000003944fb4c026... #916 @ 2018-12-17T03:45:44.000 signed by eosio [trxs: 1813, lib: 915, confirmed: 0]
info  2018-12-17T03:45:44.515 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000395b3055776... #917 @ 2018-12-17T03:45:44.500 signed by eosio [trxs: 1833, lib: 916, confirmed: 0]
info  2018-12-17T03:45:45.018 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000396c690d506... #918 @ 2018-12-17T03:45:45.000 signed by eosio [trxs: 1754, lib: 917, confirmed: 0]
info  2018-12-17T03:45:45.511 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000039746ed0032... #919 @ 2018-12-17T03:45:45.500 signed by eosio [trxs: 1823, lib: 918, confirmed: 0]
info  2018-12-17T03:45:46.014 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000398c29eaf64... #920 @ 2018-12-17T03:45:46.000 signed by eosio [trxs: 1829, lib: 919, confirmed: 0]
info  2018-12-17T03:45:46.518 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 00000399f2b79f54... #921 @ 2018-12-17T03:45:46.500 signed by eosio [trxs: 1743, lib: 920, confirmed: 0]
info  2018-12-17T03:45:47.014 thread-0  producer_plugin.cpp:1509      produce_block        ] Produced block 0000039a1ca45436... #922 @ 2018-12-17T03:45:47.000 signed by eosio [trxs: 1831, lib: 921, confirmed: 0]
info  2018-12-17T03:45:47.020 thread-0  txn_test_gen_plugin.cp:370    stop_generation      ] Stopping transaction generation test
info  2018-12-17T03:45:47.020 thread-0  txn_test_gen_plugin.cp:373    stop_generation      ] 35959 transactions executed, 111.37937651213881907us / transaction

test results of my PR:
test 1 (111.732us / transaction, around 3579 TPS):

info  2018-12-17T04:00:51.285 thread-0  txn_test_gen_plugin.cp:288    start_generation     ] Started transaction test plugin; performing 10 transactions every 2ms
info  2018-12-17T04:00:51.503 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000000e39fc37db... #14 @ 2018-12-17T04:00:51.500 signed by eosio [trxs: 780, lib: 13, confirmed: 0]
info  2018-12-17T04:00:52.018 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000000f70aaa732... #15 @ 2018-12-17T04:00:52.000 signed by eosio [trxs: 1735, lib: 14, confirmed: 0]
info  2018-12-17T04:00:52.511 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000010f2ce23c4... #16 @ 2018-12-17T04:00:52.500 signed by eosio [trxs: 1815, lib: 15, confirmed: 0]
info  2018-12-17T04:00:53.012 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000011791d0796... #17 @ 2018-12-17T04:00:53.000 signed by eosio [trxs: 1830, lib: 16, confirmed: 0]
info  2018-12-17T04:00:53.508 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000012eff3fd8c... #18 @ 2018-12-17T04:00:53.500 signed by eosio [trxs: 1736, lib: 17, confirmed: 0]
info  2018-12-17T04:00:54.013 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000013fb297988... #19 @ 2018-12-17T04:00:54.000 signed by eosio [trxs: 1774, lib: 18, confirmed: 0]
info  2018-12-17T04:00:54.513 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000014786e4809... #20 @ 2018-12-17T04:00:54.500 signed by eosio [trxs: 1820, lib: 19, confirmed: 0]
info  2018-12-17T04:00:55.017 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000015369f8d44... #21 @ 2018-12-17T04:00:55.000 signed by eosio [trxs: 1731, lib: 20, confirmed: 0]
info  2018-12-17T04:00:55.514 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000016152c9085... #22 @ 2018-12-17T04:00:55.500 signed by eosio [trxs: 1809, lib: 21, confirmed: 0]
info  2018-12-17T04:00:56.014 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 000000177ebc4229... #23 @ 2018-12-17T04:00:56.000 signed by eosio [trxs: 1810, lib: 22, confirmed: 0]
info  2018-12-17T04:00:56.512 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000018b453df63... #24 @ 2018-12-17T04:00:56.500 signed by eosio [trxs: 1740, lib: 23, confirmed: 0]
info  2018-12-17T04:00:57.014 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000019e0ee17c3... #25 @ 2018-12-17T04:00:57.000 signed by eosio [trxs: 1820, lib: 24, confirmed: 0]
info  2018-12-17T04:00:57.513 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001a934f28f2... #26 @ 2018-12-17T04:00:57.500 signed by eosio [trxs: 1810, lib: 25, confirmed: 0]
info  2018-12-17T04:00:58.017 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001b8314771a... #27 @ 2018-12-17T04:00:58.000 signed by eosio [trxs: 1718, lib: 26, confirmed: 0]
info  2018-12-17T04:00:58.514 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001c8d6a118c... #28 @ 2018-12-17T04:00:58.500 signed by eosio [trxs: 1802, lib: 27, confirmed: 0]
info  2018-12-17T04:00:59.012 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001d4d8b24b1... #29 @ 2018-12-17T04:00:59.000 signed by eosio [trxs: 1770, lib: 28, confirmed: 0]
info  2018-12-17T04:00:59.510 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001e194b26be... #30 @ 2018-12-17T04:00:59.500 signed by eosio [trxs: 1726, lib: 29, confirmed: 0]
info  2018-12-17T04:01:00.013 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001fbbc6f5ac... #31 @ 2018-12-17T04:01:00.000 signed by eosio [trxs: 1794, lib: 30, confirmed: 0]
info  2018-12-17T04:01:00.512 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000020221bfdaf... #32 @ 2018-12-17T04:01:00.500 signed by eosio [trxs: 1796, lib: 31, confirmed: 0]
info  2018-12-17T04:01:01.017 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 000000219d2522a6... #33 @ 2018-12-17T04:01:01.000 signed by eosio [trxs: 1737, lib: 32, confirmed: 0]
info  2018-12-17T04:01:01.352 thread-0  txn_test_gen_plugin.cp:370    stop_generation      ] Stopping transaction generation test
info  2018-12-17T04:01:01.352 thread-0  txn_test_gen_plugin.cp:373    stop_generation      ] 35799 transactions executed, 111.73242269337131916us / transaction

test 2 (112.083us / transaction, around 3568 TPS):

info  2018-12-17T04:01:44.361 thread-0  txn_test_gen_plugin.cp:288    start_generation     ] Started transaction test plugin; performing 10 transactions every 2ms
info  2018-12-17T04:01:44.501 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000000ce8304f83... #12 @ 2018-12-17T04:01:44.500 signed by eosio [trxs: 470, lib: 11, confirmed: 0]
info  2018-12-17T04:01:45.013 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000000df26943d5... #13 @ 2018-12-17T04:01:45.000 signed by eosio [trxs: 1811, lib: 12, confirmed: 0]
info  2018-12-17T04:01:45.510 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000000ee9b48145... #14 @ 2018-12-17T04:01:45.500 signed by eosio [trxs: 1799, lib: 13, confirmed: 0]
info  2018-12-17T04:01:46.016 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000000fa34bcfe3... #15 @ 2018-12-17T04:01:46.000 signed by eosio [trxs: 1762, lib: 14, confirmed: 0]
info  2018-12-17T04:01:46.513 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000010f2e79c35... #16 @ 2018-12-17T04:01:46.500 signed by eosio [trxs: 1788, lib: 15, confirmed: 0]
info  2018-12-17T04:01:47.014 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001124a73443... #17 @ 2018-12-17T04:01:47.000 signed by eosio [trxs: 1816, lib: 16, confirmed: 0]
info  2018-12-17T04:01:47.511 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000012cfaa0781... #18 @ 2018-12-17T04:01:47.500 signed by eosio [trxs: 1724, lib: 17, confirmed: 0]
info  2018-12-17T04:01:48.025 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 000000139cfccbe9... #19 @ 2018-12-17T04:01:48.000 signed by eosio [trxs: 1760, lib: 18, confirmed: 0]
info  2018-12-17T04:01:48.513 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000014e841b8f5... #20 @ 2018-12-17T04:01:48.500 signed by eosio [trxs: 1750, lib: 19, confirmed: 0]
info  2018-12-17T04:01:49.015 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 000000154ef4552b... #21 @ 2018-12-17T04:01:49.000 signed by eosio [trxs: 1715, lib: 20, confirmed: 0]
info  2018-12-17T04:01:49.513 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000016ac452862... #22 @ 2018-12-17T04:01:49.500 signed by eosio [trxs: 1795, lib: 21, confirmed: 0]
info  2018-12-17T04:01:50.013 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000017c5e32266... #23 @ 2018-12-17T04:01:50.000 signed by eosio [trxs: 1800, lib: 22, confirmed: 0]
info  2018-12-17T04:01:50.518 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000018b971647a... #24 @ 2018-12-17T04:01:50.500 signed by eosio [trxs: 1733, lib: 23, confirmed: 0]
info  2018-12-17T04:01:51.014 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000019cee1a3f3... #25 @ 2018-12-17T04:01:51.000 signed by eosio [trxs: 1777, lib: 24, confirmed: 0]
info  2018-12-17T04:01:51.513 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001a6028d45e... #26 @ 2018-12-17T04:01:51.500 signed by eosio [trxs: 1810, lib: 25, confirmed: 0]
info  2018-12-17T04:01:52.017 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001be54680a2... #27 @ 2018-12-17T04:01:52.000 signed by eosio [trxs: 1736, lib: 26, confirmed: 0]
info  2018-12-17T04:01:52.513 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001c5d51923e... #28 @ 2018-12-17T04:01:52.500 signed by eosio [trxs: 1784, lib: 27, confirmed: 0]
info  2018-12-17T04:01:53.014 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001d830adcf4... #29 @ 2018-12-17T04:01:53.000 signed by eosio [trxs: 1800, lib: 28, confirmed: 0]
info  2018-12-17T04:01:53.510 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001e203316d6... #30 @ 2018-12-17T04:01:53.500 signed by eosio [trxs: 1737, lib: 29, confirmed: 0]
info  2018-12-17T04:01:54.014 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 0000001f9f5755b7... #31 @ 2018-12-17T04:01:54.000 signed by eosio [trxs: 1813, lib: 30, confirmed: 0]
info  2018-12-17T04:01:54.513 thread-0  producer_plugin.cpp:1503      produce_block        ] Produced block 00000020e8fdbfe3... #32 @ 2018-12-17T04:01:54.500 signed by eosio [trxs: 1781, lib: 31, confirmed: 0]
info  2018-12-17T04:01:54.856 thread-0  txn_test_gen_plugin.cp:370    stop_generation      ] Stopping transaction generation test
info  2018-12-17T04:01:54.856 thread-0  txn_test_gen_plugin.cp:373    stop_generation      ] 37173 transactions executed, 112.08344766362682776us / transaction

Original code (around 2200 TPS):

info  2018-12-17T07:28:57.780 thread-0  txn_test_gen_plugin.cp:269    start_generation     ] Started transaction test plugin; performing 10 transactions every 2ms
info  2018-12-17T07:28:58.003 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000000b7ca0d7aa... #11 @ 2018-12-17T07:28:58.000 signed by eosio [trxs: 649, lib: 10, confirmed: 0]
info  2018-12-17T07:28:58.511 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000000c6f6ec85f... #12 @ 2018-12-17T07:28:58.500 signed by eosio [trxs: 1069, lib: 11, confirmed: 0]
info  2018-12-17T07:28:59.013 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000000d48725b48... #13 @ 2018-12-17T07:28:59.000 signed by eosio [trxs: 1111, lib: 12, confirmed: 0]
info  2018-12-17T07:28:59.507 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000000ec8339ad1... #14 @ 2018-12-17T07:28:59.500 signed by eosio [trxs: 1089, lib: 13, confirmed: 0]
info  2018-12-17T07:29:00.014 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000000f5fc4754e... #15 @ 2018-12-17T07:29:00.000 signed by eosio [trxs: 1083, lib: 14, confirmed: 0]
info  2018-12-17T07:29:00.514 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 00000010ee778a01... #16 @ 2018-12-17T07:29:00.500 signed by eosio [trxs: 1102, lib: 15, confirmed: 0]
info  2018-12-17T07:29:01.014 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 00000011e700b953... #17 @ 2018-12-17T07:29:01.000 signed by eosio [trxs: 1108, lib: 16, confirmed: 0]
info  2018-12-17T07:29:01.513 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 00000012c91e2d4d... #18 @ 2018-12-17T07:29:01.500 signed by eosio [trxs: 1078, lib: 17, confirmed: 0]
info  2018-12-17T07:29:02.014 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000001320281d51... #19 @ 2018-12-17T07:29:02.000 signed by eosio [trxs: 1126, lib: 18, confirmed: 0]
info  2018-12-17T07:29:02.514 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 00000014a8a48277... #20 @ 2018-12-17T07:29:02.500 signed by eosio [trxs: 1123, lib: 19, confirmed: 0]
info  2018-12-17T07:29:03.013 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 00000015a1c2f604... #21 @ 2018-12-17T07:29:03.000 signed by eosio [trxs: 1089, lib: 20, confirmed: 0]
info  2018-12-17T07:29:03.515 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000001667ea1e6c... #22 @ 2018-12-17T07:29:03.500 signed by eosio [trxs: 1121, lib: 21, confirmed: 0]
info  2018-12-17T07:29:04.014 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 00000017603f1ad3... #23 @ 2018-12-17T07:29:04.000 signed by eosio [trxs: 1075, lib: 22, confirmed: 0]
info  2018-12-17T07:29:04.514 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 00000018c3c7be61... #24 @ 2018-12-17T07:29:04.500 signed by eosio [trxs: 1084, lib: 23, confirmed: 0]
info  2018-12-17T07:29:05.014 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 00000019e18ee478... #25 @ 2018-12-17T07:29:05.000 signed by eosio [trxs: 1113, lib: 24, confirmed: 0]
info  2018-12-17T07:29:05.509 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000001a17d7ab07... #26 @ 2018-12-17T07:29:05.500 signed by eosio [trxs: 1119, lib: 25, confirmed: 0]
info  2018-12-17T07:29:06.013 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000001b039fb2b5... #27 @ 2018-12-17T07:29:06.000 signed by eosio [trxs: 1077, lib: 26, confirmed: 0]
info  2018-12-17T07:29:06.514 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000001cb5c45a53... #28 @ 2018-12-17T07:29:06.500 signed by eosio [trxs: 1104, lib: 27, confirmed: 0]
info  2018-12-17T07:29:07.014 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000001d1591b9dc... #29 @ 2018-12-17T07:29:07.000 signed by eosio [trxs: 1107, lib: 28, confirmed: 0]
info  2018-12-17T07:29:07.514 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000001e9dba3bb4... #30 @ 2018-12-17T07:29:07.500 signed by eosio [trxs: 1080, lib: 29, confirmed: 0]
info  2018-12-17T07:29:08.013 thread-0  producer_plugin.cpp:1497      produce_block        ] Produced block 0000001fb3a88e49... #31 @ 2018-12-17T07:29:08.000 signed by eosio [trxs: 1068, lib: 30, confirmed: 0]
info  2018-12-17T07:29:08.179 thread-0  txn_test_gen_plugin.cp:353    stop_generation      ] Stopping transaction generation test

Also from my understanding, in on_incoming_transaction_async, producer_plugin.cpp, looks like if we configure 2 or more threads in the thread_pool, one thread would always sleep, waiting for another thread to complete signing_keys_future. Is it better to just use one thread in the thread_pool to complete signing_keys_future and then post process_incoming_transaction_async?

         transaction_metadata::create_signing_keys_future( trx, chain.get_thread_pool(), chain.get_chain_id() );
         boost::asio::post( chain.get_thread_pool(), [self = this, trx, persist_until_expired, next]() {
            if( trx->signing_keys_future.valid() )
               trx->signing_keys_future.wait();
            app().get_io_service().post( [self, trx, persist_until_expired, next]() {
               self->process_incoming_transaction_async( trx, persist_until_expired, next );
            });
         });

@heifner
Copy link
Contributor Author

heifner commented Dec 17, 2018

@taokayan Thanks for testing on your machine.

I was keeping it simple with one thread_pool since it is configurable and a user could just double the thread_pool size if this is an issue. But I can see adding another thread_pool to producer_plugin for these waits. I could also see using different thread_pools for apply_block sig recovery so that incoming transactions can't hold up apply_block.

libraries/chain/transaction.cpp Outdated Show resolved Hide resolved
libraries/chain/transaction.cpp Outdated Show resolved Hide resolved
@heifner heifner merged commit c9a21a4 into develop Dec 18, 2018
@heifner heifner deleted the producer-txns branch December 18, 2018 22:36
Vardhamana pushed a commit to boscore/bos that referenced this pull request Feb 28, 2019
* state history plugin

* bump version to 1.5.0-rc1

* Update README to reflect new package names

* Change from 1.5.0 to 1.5.0-rc1

* Fix requires in spec file to resolve dependency resolution issues with rpms

* Fix requires in spec file to resolve dependency resolution issues with rpms

* Remove unneeded req_trx vector

* Remove needless copy of signed_block. Only create pending_notify if needed.

* Remove unused blk_buffer

* separate out version suffix so that RPM packages produce an acceptable specfile

* address PR review feedback

* Make signed_block copy constructor private to avoid accidental copies.

* add release suffix to doxygen cmake module

* move signed_block instead of making copy

* Avoid copy constructor for push_block

* add release suffix to EOSIO_VERSION in tester cmake modules

* Add which comment

* Update tests for private signed_block copy constructor

* Avoid making copies of signed_block

* export the new bash vars

* Use rvalue for signed_block

* Update buildkite pipeline to use new queues

* Update buildkite pipeline to use new queues

* Special case msg_handler visit of signed_block to avoid hidden move from net_message

* pass along trigger_send

* Use move for rvalue

* Added Mongo History API link

* Added ZMQ Light History API

* Added Chintai ZMQ Watcher

* Added State History API

* Stop the unix_server from listening in shutdown stage as well.

* finish BOS basic functions

* bump version to rc2

* update rpm names to match the real files

* Fix deb package names in README

* Fix package naming scheme for debs

* Update LICENSE path

* add a test that reduces the size of the producer set to validate that it does not create protocol level issues

* promote rc2 to release

* add missing release to deb packages

* remove x86_64 arch dep specification on AMI2 build script

I can't find any reason we need to explcitly request the x86_64 versions of packages for AMI2. Removing this apparently extraneous specification makes the build script "just work" on new ARM8 instances

* Use 64-bit float printing of 128-bit floats on non x86_64

For platforms other then x86_64, have the 128-bit float console print API (printqf()) convert to a 64-bit float before printing. While this loses precision it’s enough to get the unit tests for printqf() working on ARM8, which actually was the only unit_test unit test that failed on that platform.

* prebump to 1.6.0 and add develop suffix

* Do not broadcast block if peer lib is larger than block number

* Provide more efficient sha256_less

* Simply by removing unused large_msg_notify

* Remove unused node_transaction_state.packed_txn

* Store serialized transaction as shared_ptr to minimize copies

* Not possible for serialized_txn to be empty

* Remove find_plugin overhead

* Minimize shared_ptr copies

* Use sha256_less for all sets/maps with ids

* Add explicit constructors

* Add accept_transaction that takes a packed_transaction_ptr to avoid copy

* Pass id and packed_transaction by const& since never moved

* Remove copy assignment operator. Remove used std::move.

* Minimize packed_transaction copies. Store packed_transaction_ptr in transaction_metadata.

* Cache chain_plugin lookup

* Update tester for transaction_metadata packed_transaction_ptr

* Remove packed_tansaction to net_message copy

* Remove unused constructor

* Explicitly disable copy/move construction

* Remove inflight update of node_transaction_state as it is not needed

* Remove connections for unused signals that only logged

* Fix sync check for lib

* add a test that verifies proposing an empty producer schedule does not break things

* Allow disablement of net_plugin's listen socket

Disable net_plugin's listen socket when p2p-listen-endpoint is empty. This is useful for security and for an easier way of running multiple nodeos without it conflicting when you don't actually need to service incoming connections.

* More descriptive memo to distinguish from other transfers

* Revert "Fix sync check for lib"

This reverts commit 52a6f19.

* Update the bios-boot-tutorial.py script and readme.md:
1. args.contracts_dir doesn't end in a forward slash, therefor when it is concatenated with a subpath, the subpath has to start with a forward slash; this pattern is present in other similar variables as well
2. system contract has to be initialized before it can be used (e.g. creating staked acconts)
3. commands list updated to include the new addition stepInitSystemContract
4. readme.md updated as well to reflect the current status of the script and requirements

* Release/1.0.x (#11)

* boscore basic improvement (#2)

* kafka_plugin code

* Automatic installation librdkafka/cppkafka

* Feature/ci

* Feature/48 kafka plugin

* add CMakeModules/FindCppkafka.cmake

* Production of block in time zone sequence

* P2p self discovery

* P2p self discovery

* add notify_plugin

* add api "get_block_detail"

* add free res limit and blklst code

* update free res limit and blklst code

* update res code

* update unittest code

* revert submodule version

* code typo

* update  blklist code

* update sync name list db object error code

* update  code

* update  index code

* Feature/5 ramdom

* Revert "Merge branch 'feature/5-ramdom' into 'develop'"

This reverts merge request !8

* adjust for setup BOSCore

* change description

* adjust the kafka plugin dependency be more special

* use boscore repository to improve security

* change version tag

* finish for docker/builder

* pass to build docker and update readme

* add actionseed, global action sequence (#5)

* delete renamed old file

* BOSCore v1.0.1-1.4.3

* restructure the version schema

* fix __gmpn_set_str error when build bos.contract

* prepare for the v1.0.1

* add README files

* update info

* readme for kafka & add time for action (#5)

* 重启 节点,黑名单 失效,fixes #7 (#8)

* restart sync list db

* recovery system account bos to eosio

* recovery system account bos to eosio

* recovery system account bos to eosio

* Fix/#3 notify plugin (#10)

* Add debug info

* comment log

* rm log for notify_plugin

* README.md refactored
- correcting spelling errors, and caps
- better bash commands
- simpler command line to launch the script
- improved prerequisites
- include steps to follow after prerequisites

* Correct typos

* Correct link format

* Add eosio.cdt steps

* merge v1.0.2 (#13)

* boscore basic improvement (#2)

* kafka_plugin code

* Automatic installation librdkafka/cppkafka

* Feature/ci

* Feature/48 kafka plugin

* add CMakeModules/FindCppkafka.cmake

* Production of block in time zone sequence

* P2p self discovery

* P2p self discovery

* add notify_plugin

* add api "get_block_detail"

* add free res limit and blklst code

* update free res limit and blklst code

* update res code

* update unittest code

* revert submodule version

* code typo

* update  blklist code

* update sync name list db object error code

* update  code

* update  index code

* Feature/5 ramdom

* Revert "Merge branch 'feature/5-ramdom' into 'develop'"

This reverts merge request !8

* adjust for setup BOSCore

* change description

* adjust the kafka plugin dependency be more special

* use boscore repository to improve security

* change version tag

* finish for docker/builder

* pass to build docker and update readme

* add actionseed, global action sequence (#5)

* delete renamed old file

* BOSCore v1.0.1-1.4.3

* restructure the version schema

* fix __gmpn_set_str error when build bos.contract

* prepare for the v1.0.1

* add README files

* update info

* prepare for v1.0.2

* Better name for eosio contracts directory

* dummy checkin to trigger another build

* Added scrit to support recursive search for files with FC_REFLECT or FC_REFLECT_ENUM to verify order and completeness.  Supports comments with @swap to support different order from definition and @ignore to ignore a specific field for reflection.  Still need to support FC_REFLECT_DERIVED. GH EOSIO#3127

* Added @ignore comment for field that is left out of reflect definition. GH EOSIO#3127

* Fixed pull request comments.

* bump version to 1.5.1

* Consolidated Security Fixes for 1.5.1

- Only allow authorizations that are satisfiable by `eosio.code` for self-addressed deferred transactions
- Only allow authorizations that are satisfiable by `eosio.code` OR on the parent action for self-addressed inline actions sent from direct actions
- Only allow authorizations that are satisfiable by `eosio.code` for self-addressed inline actions sent from recipient handlers

Co-authored-by: arhag <[email protected]>
Co-authored-by: Bart Wyatt <[email protected]>

* Peer review changes. Fix move.

* Spelling correction

* Spelling correction #2

* Remove unused max-implicit-request config

* Switch interface from packed_transaction_ptr to transaction_metadata_ptr

* Thread pool does not need to be optional

* Add transaction_metadata create_signing_keys_future method

* Start transaction signature earily in thread pool

* Refactor packed_transaction for better encapsulation

* Add transaction_metadata create_signing_keys_future method

* Start transaction signature earily in thread pool

* Update txn_test_gen_plugin to overlap transaction submit @taokayan

* merge 1.0.3 to develop (#16)

* boscore basic improvement (#2)

* kafka_plugin code

* Automatic installation librdkafka/cppkafka

* Feature/ci

* Feature/48 kafka plugin

* add CMakeModules/FindCppkafka.cmake

* Production of block in time zone sequence

* P2p self discovery

* P2p self discovery

* add notify_plugin

* add api "get_block_detail"

* add free res limit and blklst code

* update free res limit and blklst code

* update res code

* update unittest code

* revert submodule version

* code typo

* update  blklist code

* update sync name list db object error code

* update  code

* update  index code

* Feature/5 ramdom

* Revert "Merge branch 'feature/5-ramdom' into 'develop'"

This reverts merge request !8

* adjust for setup BOSCore

* change description

* adjust the kafka plugin dependency be more special

* use boscore repository to improve security

* change version tag

* finish for docker/builder

* pass to build docker and update readme

* add actionseed, global action sequence (#5)

* delete renamed old file

* BOSCore v1.0.1-1.4.3

* restructure the version schema

* fix __gmpn_set_str error when build bos.contract

* prepare for the v1.0.1

* add README files

* update info

* prepare for v1.0.2

* merge v1.0.2 (#12)

* boscore basic improvement (#2)

* kafka_plugin code

* Automatic installation librdkafka/cppkafka

* Feature/ci

* Feature/48 kafka plugin

* add CMakeModules/FindCppkafka.cmake

* Production of block in time zone sequence

* P2p self discovery

* P2p self discovery

* add notify_plugin

* add api "get_block_detail"

* add free res limit and blklst code

* update free res limit and blklst code

* update res code

* update unittest code

* revert submodule version

* code typo

* update  blklist code

* update sync name list db object error code

* update  code

* update  index code

* Feature/5 ramdom

* Revert "Merge branch 'feature/5-ramdom' into 'develop'"

This reverts merge request !8

* adjust for setup BOSCore

* change description

* adjust the kafka plugin dependency be more special

* use boscore repository to improve security

* change version tag

* finish for docker/builder

* pass to build docker and update readme

* add actionseed, global action sequence (#5)

* delete renamed old file

* BOSCore v1.0.1-1.4.3

* restructure the version schema

* fix __gmpn_set_str error when build bos.contract

* prepare for the v1.0.1

* finish BOS basic functions

* add README files

* update info

* Release/1.0.x (#11)

* boscore basic improvement (#2)

* kafka_plugin code

* Automatic installation librdkafka/cppkafka

* Feature/ci

* Feature/48 kafka plugin

* add CMakeModules/FindCppkafka.cmake

* Production of block in time zone sequence

* P2p self discovery

* P2p self discovery

* add notify_plugin

* add api "get_block_detail"

* add free res limit and blklst code

* update free res limit and blklst code

* update res code

* update unittest code

* revert submodule version

* code typo

* update  blklist code

* update sync name list db object error code

* update  code

* update  index code

* Feature/5 ramdom

* Revert "Merge branch 'feature/5-ramdom' into 'develop'"

This reverts merge request !8

* adjust for setup BOSCore

* change description

* adjust the kafka plugin dependency be more special

* use boscore repository to improve security

* change version tag

* finish for docker/builder

* pass to build docker and update readme

* add actionseed, global action sequence (#5)

* delete renamed old file

* BOSCore v1.0.1-1.4.3

* restructure the version schema

* fix __gmpn_set_str error when build bos.contract

* prepare for the v1.0.1

* add README files

* update info

* readme for kafka & add time for action (#5)

* 重启 节点,黑名单 失效,fixes #7 (#8)

* restart sync list db

* recovery system account bos to eosio

* recovery system account bos to eosio

* recovery system account bos to eosio

* Fix/#3 notify plugin (#10)

* Add debug info

* comment log

* rm log for notify_plugin

* prepare for v1.0.2

* patch the EOSIO 1.5.1 security bug fixes

* prepare for v1.0.3

* adjust the slogon

* Remove redundant signing_keys check

* state history plugin: permission_object bug EOSIO#6495

* Add deadline to key recovery

* Modify producer_plugin to have its own thead_pool instead of using chain-threads

* Move thread_pool join/stop to plugin shutdown so that they are joined before application quit

* Fix signature future deadline from starting too early

* Fix overflow of deadline and deadline check

* initial setup of billing CPU for signatures recovered earlier

* Return 400 on get_block with uonexist block number.
For issue 6374

* has_contract() : Determine whether to deploy the contract (#18)

* fix issue 5488

* Fix comments and function names (#19)

* Add action_trace.block_num index. Remove action_trace.trx_id index.

* Fix return codes of build scripts so that buildkite can fail properly

* Make recovery cache non-thread local and guard by mutex

* Calculate cpu usage of signature recovery

* Add signature-cpu-billable-pct option to chain_plugin

* Add missing include of mutex

* Assert signature-cpu-billable-pct is 0-100

* Fix capture of cpu_usage. move flat_set<public_key_type> into attribute

* clear recovered_pub_keys to preserve previous behaviour

* use `assign` instead of `resize`+`memcpy` to update `shared_blob` data

Co-Authored-By: Kayan <[email protected]>

* Add move into tuple creation

* bump version to 1.6.0-rc1

* add explicitly defaulted move semantics

* Store unpacked_trx as signed_transaction

* get_uncached_id no longer needed for thread safety

* get_raw_transaction no longer needed for thread safety

* Remove cached signed_transaction since now cached in packed_transaction

* Test should honor existing compression

* Cleanup of packed_transaction interface. Fixes for comments on PR EOSIO#6471

* Update to fc with unpack verify

* fc renamed reflector_verify to reflector_init

* bump version to 1.5.2

* Update to latest fc with updated verify_init

* Consolidated Security Fixes for 1.5.3

- Add missing implementation of dtor, copy ctor, etc for blob_types of fc::variant

Co-Authored-By: Matt Witherspoon <[email protected]>

* bump version to 1.5.3

* bump version to 1.6.0-rc2

* Consolidated Security Fixes for 1.6.0-rc2

- Add missing implementation of dtor, copy ctor, etc for blob_types of fc::variant

Co-Authored-By: Matt Witherspoon <[email protected]>

* update developer portal link

update the getting started link to the latest version

* fix tag version typo of cppkafka (#24)

* get_contract_code (#26)

* fixed #20Using greylist-account while starting nodeos for the first time throws exception (#23)

* restart sync list db

* recovery system account bos to eosio

* catch exception plugin initialize sync list before initialize database

* fixed  'cleos system bidname info'  cmd exec  return       could not parse uint64_t (#30)

* recovery system account bos to eosio

* catch exception plugin initialize sync list before initialize database

* fixed bidnameinfo could not parse uint64_t

* prepare for 2.0.1

* adjust the location paramater

* apply 1.5.3 patch

* modify the location type

* fix cmake error

* Use https url for wabt submodule

A user agent that doesn't respect hsts could potentially be coerced into downloading malicious sources for wabt via a mitm attack. Prevent this by using a https upstream like the other submodules already do

* Add libtinfo5 dependency to deb package. Resolves EOSIO#6590

Also clean up unused variables and force control directory permissions
to comply with Debian specifications regardless of umask.

* fix aragument name error

* Bump version to 1.6.0

* Consolidated Security Fixes for 1.6.0

- Force compilation support for c++ exceptions for the wabt submodule
- Allow limiting the time a node will spend processing scheduled transactions

Co-Authored-By: Matt Witherspoon <[email protected]>
Co-Authored-By: Bart Wyatt <[email protected]>

* Consolidated Security Fixes for 1.6.1

- net_plugin security fixes
- Additional checktime calls to limit cpu usage
- Limit memory usage in producer_plugin

Co-Authored-By: Kevin Heifner <[email protected]>
Co-Authored-By: Matt Witherspoon <[email protected]>
Co-authored-by: Kayan <[email protected]>

* Bump version to 1.6.1

* Fix boost 1.67 brew install

The latest homebrew code balks at something in the old 1.67 package file. Fix the package file and store it locally in our repo for now.

We try and pin the boost version because boost upgrades invalidate nodeos data files.

* Set proper directory for baked in macOS LLVM_DIR

Some users (including myself) were seeing llvm@4 unpacked to 4.0.1_1 instead of 4.0.1. Stuff unpacked to the Cellar directory appears to be a kind of implementation detail — /usr/local/opt is the proper place to reference here.

* Add bk step to gather brew files for automatic update

* Update version to 1.6.2

* add noninteractive option for build scripts

* Limit assert message to 1024 chars

* Don't unlink what we install via brew

unlinking eveything we install makes no sense -- it means things like cmake aren't in the path any longer like the script expects. So don't do that any more. Unfortuately this old script requires that gettext be force linked. So implement that behavior explictly for now

* Make sure python-devel is installed for amazon linux builds

* fc version 1.6.x

* merge missed code

* merge missed code

* merge missed code

* prepare v2.0.2
Vardhamana pushed a commit to boscore/bos that referenced this pull request Feb 28, 2019
* Remove unnecessary conversion of contractPath to empty path

* enhance cleos multisig review to show proposed transaction ID and optionally show requested approvals EOSIO#6272

* add cleos multisig invalidate command to call eosio.msig::invalidate

* state history plugin

* fix GMP & secp256 linker order for EosioTesterBuild.cmake.in

This is the same fix from pr EOSIO#6268 but for EosioTesterBuild.cmake.in too

* state history plugin

* bump version to 1.5.0-rc1

* Update README to reflect new package names

* Change from 1.5.0 to 1.5.0-rc1

* Fix requires in spec file to resolve dependency resolution issues with rpms

* Fix requires in spec file to resolve dependency resolution issues with rpms

* Remove unneeded req_trx vector

* Remove needless copy of signed_block. Only create pending_notify if needed.

* Remove unused blk_buffer

* separate out version suffix so that RPM packages produce an acceptable specfile

* address PR review feedback

* Make signed_block copy constructor private to avoid accidental copies.

* add release suffix to doxygen cmake module

* move signed_block instead of making copy

* Avoid copy constructor for push_block

* add release suffix to EOSIO_VERSION in tester cmake modules

* Add which comment

* Update tests for private signed_block copy constructor

* Avoid making copies of signed_block

* export the new bash vars

* Use rvalue for signed_block

* Update buildkite pipeline to use new queues

* Update buildkite pipeline to use new queues

* Special case msg_handler visit of signed_block to avoid hidden move from net_message

* pass along trigger_send

* Use move for rvalue

* Added Mongo History API link

* Added ZMQ Light History API

* Added Chintai ZMQ Watcher

* Added State History API

* Stop the unix_server from listening in shutdown stage as well.

* bump version to rc2

* update rpm names to match the real files

* Fix deb package names in README

* Fix package naming scheme for debs

* Update LICENSE path

* add a test that reduces the size of the producer set to validate that it does not create protocol level issues

* promote rc2 to release

* add missing release to deb packages

* remove x86_64 arch dep specification on AMI2 build script

I can't find any reason we need to explcitly request the x86_64 versions of packages for AMI2. Removing this apparently extraneous specification makes the build script "just work" on new ARM8 instances

* Use 64-bit float printing of 128-bit floats on non x86_64

For platforms other then x86_64, have the 128-bit float console print API (printqf()) convert to a 64-bit float before printing. While this loses precision it’s enough to get the unit tests for printqf() working on ARM8, which actually was the only unit_test unit test that failed on that platform.

* prebump to 1.6.0 and add develop suffix

* Do not broadcast block if peer lib is larger than block number

* Provide more efficient sha256_less

* Simply by removing unused large_msg_notify

* Remove unused node_transaction_state.packed_txn

* Store serialized transaction as shared_ptr to minimize copies

* Not possible for serialized_txn to be empty

* Remove find_plugin overhead

* Minimize shared_ptr copies

* Use sha256_less for all sets/maps with ids

* Add explicit constructors

* Add accept_transaction that takes a packed_transaction_ptr to avoid copy

* Pass id and packed_transaction by const& since never moved

* Remove copy assignment operator. Remove used std::move.

* Minimize packed_transaction copies. Store packed_transaction_ptr in transaction_metadata.

* Cache chain_plugin lookup

* Update tester for transaction_metadata packed_transaction_ptr

* Remove packed_tansaction to net_message copy

* Remove unused constructor

* Explicitly disable copy/move construction

* Remove inflight update of node_transaction_state as it is not needed

* Remove connections for unused signals that only logged

* Fix sync check for lib

* add a test that verifies proposing an empty producer schedule does not break things

* Allow disablement of net_plugin's listen socket

Disable net_plugin's listen socket when p2p-listen-endpoint is empty. This is useful for security and for an easier way of running multiple nodeos without it conflicting when you don't actually need to service incoming connections.

* More descriptive memo to distinguish from other transfers

* Revert "Fix sync check for lib"

This reverts commit 52a6f19.

* Update the bios-boot-tutorial.py script and readme.md:
1. args.contracts_dir doesn't end in a forward slash, therefor when it is concatenated with a subpath, the subpath has to start with a forward slash; this pattern is present in other similar variables as well
2. system contract has to be initialized before it can be used (e.g. creating staked acconts)
3. commands list updated to include the new addition stepInitSystemContract
4. readme.md updated as well to reflect the current status of the script and requirements

* README.md refactored
- correcting spelling errors, and caps
- better bash commands
- simpler command line to launch the script
- improved prerequisites
- include steps to follow after prerequisites

* Correct typos

* Correct link format

* Add eosio.cdt steps

* merge v1.0.2 (#12)

* boscore basic improvement (#2)

* kafka_plugin code

* Automatic installation librdkafka/cppkafka

* Feature/ci

* Feature/48 kafka plugin

* add CMakeModules/FindCppkafka.cmake

* Production of block in time zone sequence

* P2p self discovery

* P2p self discovery

* add notify_plugin

* add api "get_block_detail"

* add free res limit and blklst code

* update free res limit and blklst code

* update res code

* update unittest code

* revert submodule version

* code typo

* update  blklist code

* update sync name list db object error code

* update  code

* update  index code

* Feature/5 ramdom

* Revert "Merge branch 'feature/5-ramdom' into 'develop'"

This reverts merge request !8

* adjust for setup BOSCore

* change description

* adjust the kafka plugin dependency be more special

* use boscore repository to improve security

* change version tag

* finish for docker/builder

* pass to build docker and update readme

* add actionseed, global action sequence (#5)

* delete renamed old file

* BOSCore v1.0.1-1.4.3

* restructure the version schema

* fix __gmpn_set_str error when build bos.contract

* prepare for the v1.0.1

* finish BOS basic functions

* add README files

* update info

* Release/1.0.x (#11)

* boscore basic improvement (#2)

* kafka_plugin code

* Automatic installation librdkafka/cppkafka

* Feature/ci

* Feature/48 kafka plugin

* add CMakeModules/FindCppkafka.cmake

* Production of block in time zone sequence

* P2p self discovery

* P2p self discovery

* add notify_plugin

* add api "get_block_detail"

* add free res limit and blklst code

* update free res limit and blklst code

* update res code

* update unittest code

* revert submodule version

* code typo

* update  blklist code

* update sync name list db object error code

* update  code

* update  index code

* Feature/5 ramdom

* Revert "Merge branch 'feature/5-ramdom' into 'develop'"

This reverts merge request !8

* adjust for setup BOSCore

* change description

* adjust the kafka plugin dependency be more special

* use boscore repository to improve security

* change version tag

* finish for docker/builder

* pass to build docker and update readme

* add actionseed, global action sequence (#5)

* delete renamed old file

* BOSCore v1.0.1-1.4.3

* restructure the version schema

* fix __gmpn_set_str error when build bos.contract

* prepare for the v1.0.1

* add README files

* update info

* readme for kafka & add time for action (#5)

* 重启 节点,黑名单 失效,fixes #7 (#8)

* restart sync list db

* recovery system account bos to eosio

* recovery system account bos to eosio

* recovery system account bos to eosio

* Fix/#3 notify plugin (#10)

* Add debug info

* comment log

* rm log for notify_plugin

* prepare for v1.0.2

* Better name for eosio contracts directory

* dummy checkin to trigger another build

* Added scrit to support recursive search for files with FC_REFLECT or FC_REFLECT_ENUM to verify order and completeness.  Supports comments with @swap to support different order from definition and @ignore to ignore a specific field for reflection.  Still need to support FC_REFLECT_DERIVED. GH EOSIO#3127

* Added @ignore comment for field that is left out of reflect definition. GH EOSIO#3127

* Fixed pull request comments.

* bump version to 1.5.1

* Consolidated Security Fixes for 1.5.1

- Only allow authorizations that are satisfiable by `eosio.code` for self-addressed deferred transactions
- Only allow authorizations that are satisfiable by `eosio.code` OR on the parent action for self-addressed inline actions sent from direct actions
- Only allow authorizations that are satisfiable by `eosio.code` for self-addressed inline actions sent from recipient handlers

Co-authored-by: arhag <[email protected]>
Co-authored-by: Bart Wyatt <[email protected]>

* Peer review changes. Fix move.

* Spelling correction

* Spelling correction #2

* Remove unused max-implicit-request config

* Switch interface from packed_transaction_ptr to transaction_metadata_ptr

* Thread pool does not need to be optional

* Add transaction_metadata create_signing_keys_future method

* Start transaction signature earily in thread pool

* Refactor packed_transaction for better encapsulation

* Add transaction_metadata create_signing_keys_future method

* Start transaction signature earily in thread pool

* Update txn_test_gen_plugin to overlap transaction submit @taokayan

* release v1.0.3 to master (#15)

* boscore basic improvement (#2)

* kafka_plugin code

* Automatic installation librdkafka/cppkafka

* Feature/ci

* Feature/48 kafka plugin

* add CMakeModules/FindCppkafka.cmake

* Production of block in time zone sequence

* P2p self discovery

* P2p self discovery

* add notify_plugin

* add api "get_block_detail"

* add free res limit and blklst code

* update free res limit and blklst code

* update res code

* update unittest code

* revert submodule version

* code typo

* update  blklist code

* update sync name list db object error code

* update  code

* update  index code

* Feature/5 ramdom

* Revert "Merge branch 'feature/5-ramdom' into 'develop'"

This reverts merge request !8

* adjust for setup BOSCore

* change description

* adjust the kafka plugin dependency be more special

* use boscore repository to improve security

* change version tag

* finish for docker/builder

* pass to build docker and update readme

* add actionseed, global action sequence (#5)

* delete renamed old file

* BOSCore v1.0.1-1.4.3

* restructure the version schema

* fix __gmpn_set_str error when build bos.contract

* prepare for the v1.0.1

* finish BOS basic functions

* add README files

* update info

* Release/1.0.x (#11)

* boscore basic improvement (#2)

* kafka_plugin code

* Automatic installation librdkafka/cppkafka

* Feature/ci

* Feature/48 kafka plugin

* add CMakeModules/FindCppkafka.cmake

* Production of block in time zone sequence

* P2p self discovery

* P2p self discovery

* add notify_plugin

* add api "get_block_detail"

* add free res limit and blklst code

* update free res limit and blklst code

* update res code

* update unittest code

* revert submodule version

* code typo

* update  blklist code

* update sync name list db object error code

* update  code

* update  index code

* Feature/5 ramdom

* Revert "Merge branch 'feature/5-ramdom' into 'develop'"

This reverts merge request !8

* adjust for setup BOSCore

* change description

* adjust the kafka plugin dependency be more special

* use boscore repository to improve security

* change version tag

* finish for docker/builder

* pass to build docker and update readme

* add actionseed, global action sequence (#5)

* delete renamed old file

* BOSCore v1.0.1-1.4.3

* restructure the version schema

* fix __gmpn_set_str error when build bos.contract

* prepare for the v1.0.1

* add README files

* update info

* readme for kafka & add time for action (#5)

* 重启 节点,黑名单 失效,fixes #7 (#8)

* restart sync list db

* recovery system account bos to eosio

* recovery system account bos to eosio

* recovery system account bos to eosio

* Fix/#3 notify plugin (#10)

* Add debug info

* comment log

* rm log for notify_plugin

* prepare for v1.0.2

* merge v1.0.2 (#13)

* boscore basic improvement (#2)

* kafka_plugin code

* Automatic installation librdkafka/cppkafka

* Feature/ci

* Feature/48 kafka plugin

* add CMakeModules/FindCppkafka.cmake

* Production of block in time zone sequence

* P2p self discovery

* P2p self discovery

* add notify_plugin

* add api "get_block_detail"

* add free res limit and blklst code

* update free res limit and blklst code

* update res code

* update unittest code

* revert submodule version

* code typo

* update  blklist code

* update sync name list db object error code

* update  code

* update  index code

* Feature/5 ramdom

* Revert "Merge branch 'feature/5-ramdom' into 'develop'"

This reverts merge request !8

* adjust for setup BOSCore

* change description

* adjust the kafka plugin dependency be more special

* use boscore repository to improve security

* change version tag

* finish for docker/builder

* pass to build docker and update readme

* add actionseed, global action sequence (#5)

* delete renamed old file

* BOSCore v1.0.1-1.4.3

* restructure the version schema

* fix __gmpn_set_str error when build bos.contract

* prepare for the v1.0.1

* add README files

* update info

* prepare for v1.0.2

* patch the EOSIO 1.5.1 security bug fixes

* prepare for v1.0.3

* adjust the slogon

* Remove redundant signing_keys check

* state history plugin: permission_object bug EOSIO#6495

* Add deadline to key recovery

* Modify producer_plugin to have its own thead_pool instead of using chain-threads

* Move thread_pool join/stop to plugin shutdown so that they are joined before application quit

* Fix signature future deadline from starting too early

* Fix overflow of deadline and deadline check

* initial setup of billing CPU for signatures recovered earlier

* Return 400 on get_block with uonexist block number.
For issue 6374

* fix issue 5488

* Add action_trace.block_num index. Remove action_trace.trx_id index.

* Fix return codes of build scripts so that buildkite can fail properly

* Make recovery cache non-thread local and guard by mutex

* Calculate cpu usage of signature recovery

* Add signature-cpu-billable-pct option to chain_plugin

* Add missing include of mutex

* Assert signature-cpu-billable-pct is 0-100

* Fix capture of cpu_usage. move flat_set<public_key_type> into attribute

* clear recovered_pub_keys to preserve previous behaviour

* use `assign` instead of `resize`+`memcpy` to update `shared_blob` data

Co-Authored-By: Kayan <[email protected]>

* Add move into tuple creation

* bump version to 1.6.0-rc1

* add explicitly defaulted move semantics

* Store unpacked_trx as signed_transaction

* get_uncached_id no longer needed for thread safety

* get_raw_transaction no longer needed for thread safety

* Remove cached signed_transaction since now cached in packed_transaction

* Test should honor existing compression

* Cleanup of packed_transaction interface. Fixes for comments on PR EOSIO#6471

* Update to fc with unpack verify

* fc renamed reflector_verify to reflector_init

* bump version to 1.5.2

* Update to latest fc with updated verify_init

* Consolidated Security Fixes for 1.5.3

- Add missing implementation of dtor, copy ctor, etc for blob_types of fc::variant

Co-Authored-By: Matt Witherspoon <[email protected]>

* bump version to 1.5.3

* bump version to 1.6.0-rc2

* Consolidated Security Fixes for 1.6.0-rc2

- Add missing implementation of dtor, copy ctor, etc for blob_types of fc::variant

Co-Authored-By: Matt Witherspoon <[email protected]>

* update developer portal link

update the getting started link to the latest version

* prepare for 2.0.1

* adjust the location paramater

* apply 1.5.3 patch

* modify the location type

* fix cmake error

* Use https url for wabt submodule

A user agent that doesn't respect hsts could potentially be coerced into downloading malicious sources for wabt via a mitm attack. Prevent this by using a https upstream like the other submodules already do

* Add libtinfo5 dependency to deb package. Resolves EOSIO#6590

Also clean up unused variables and force control directory permissions
to comply with Debian specifications regardless of umask.

* fix aragument name error

* merge v2.0.1 version (#36)

* Bump version to 1.6.0

* Consolidated Security Fixes for 1.6.0

- Force compilation support for c++ exceptions for the wabt submodule
- Allow limiting the time a node will spend processing scheduled transactions

Co-Authored-By: Matt Witherspoon <[email protected]>
Co-Authored-By: Bart Wyatt <[email protected]>

* Consolidated Security Fixes for 1.6.1

- net_plugin security fixes
- Additional checktime calls to limit cpu usage
- Limit memory usage in producer_plugin

Co-Authored-By: Kevin Heifner <[email protected]>
Co-Authored-By: Matt Witherspoon <[email protected]>
Co-authored-by: Kayan <[email protected]>

* Bump version to 1.6.1

* Fix boost 1.67 brew install

The latest homebrew code balks at something in the old 1.67 package file. Fix the package file and store it locally in our repo for now.

We try and pin the boost version because boost upgrades invalidate nodeos data files.

* Set proper directory for baked in macOS LLVM_DIR

Some users (including myself) were seeing llvm@4 unpacked to 4.0.1_1 instead of 4.0.1. Stuff unpacked to the Cellar directory appears to be a kind of implementation detail — /usr/local/opt is the proper place to reference here.

* Add bk step to gather brew files for automatic update

* Update version to 1.6.2

* add noninteractive option for build scripts

* Limit assert message to 1024 chars

* Don't unlink what we install via brew

unlinking eveything we install makes no sense -- it means things like cmake aren't in the path any longer like the script expects. So don't do that any more. Unfortuately this old script requires that gettext be force linked. So implement that behavior explictly for now

* Make sure python-devel is installed for amazon linux builds

* fc version 1.6.x

* merge missed code

* merge missed code

* merge missed code

* prepare v2.0.2
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants