Add fuzzing seeds to recently contributed fuzzing harnesses

[practicalswift]

Add fuzzing seeds to recently contributed fuzzing harnesses.

Seeds for:

• tests: Add fuzzing harnesses for functions parsing scripts, numbers, JSON and HD keypaths (bip32) bitcoin/bitcoin#17050 – "tests: Add fuzzing harnesses for functions parsing scripts, numbers, JSON and HD keypaths (bip32)"
• tests: Add deserialization fuzzing harnesses bitcoin/bitcoin#17051 – "tests: Add deserialization fuzzing harnesses"
• tests: Add fuzzing harness for CheckBlock(...) and other CBlock related functions bitcoin/bitcoin#17071 – "tests: Add fuzzing harness for CheckBlock(...) and other CBlock related functions"
• tests: Add fuzzing harness for various CTx{In,Out} related functions bitcoin/bitcoin#17093 – "tests: Add fuzzing harness for various CTx{In,Out} related functions"
• tests: Add fuzzing harness for various functions consuming only integrals bitcoin/bitcoin#17109 – "tests: Add fuzzing harness for various functions consuming only integrals"
• tests: Add fuzzing harness for various PSBT related functions bitcoin/bitcoin#17136 – "tests: Add fuzzing harness for various PSBT related functions"
• tests: Add fuzzing harnesses for various Base{32,58,64} and hex related functions bitcoin/bitcoin#17229 – "tests: Add fuzzing harnesses for various Base{32,58,64} and hex related functions"

[maflcko]

Thanks, I will merge them with libFuzzer -merge=1

[maflcko]

I found that only some script flags seeds increased coverage. See 3c3b16c

[practicalswift]

@MarcoFalke That's expected since the script flags seeds I submitted were not bootstrapped with the seeds in the repo :)

Anything I need to do to get the seed corpora for the other fuzzing harnesses in? :)

[maflcko]

All others didn't increase coverage for me. Feel free to double check

[practicalswift]

@MarcoFalke Please note that script flags was the only fuzzer in master that got added any seeds. The rest of the seeds are to fuzzers not yet merged (see linked PR:s).

Take fuzz_seed_corpus/addr_info_deserialize/166c9b7e8a5d54837ddf5f752051c692115219ab in this PR as an example. That is a seed for addr_info_deserialize and AFAICT there are no seeds for that fuzzer in this repo, right? :)

Note that addr_info_deserialize is waiting to be merged in bitcoin/bitcoin#17051.

Sames goes for all the added seeds in this PR (with the exception of the script flag seeds).

[maflcko]

Ah, thx

[practicalswift]

Can we move forward with adding the additional seeds? :)

[maflcko]

I'd like to add them only after the fuzzers have been merged. Otherwise I can't calculate coverage information.

[practicalswift]

Is there anything I can do to get the fuzzers merged? :)

[maflcko]

review on them would help

[practicalswift]

@MarcoFalke Can we move forward with some of the remaining fuzzing harnesses? :)

[maflcko]

Thanks merged in 5c4e9f9

[maflcko]

Let me know if I forgot some files which increase coverage, otherwise feel free to close

[maflcko]

Before:

After:

[practicalswift]

@MarcoFalke I'll investigate! To make sure we measure exactly the same way: what commands/options did you use to generate the coverage reports?

[maflcko]

It looks like this:

Run prefilled_transaction_deserialize with args ...
... ['./src/test/fuzz/prefilled_transaction_deserialize', '-merge=1', '../btc_qa_assets/fuzz_seed_corpus/prefilled_transaction_deserialize', '../btc_qa_assets/fuzz_seed_corpus_other/prefilled_transaction_deserialize']
Output: INFO: Seed: 4151550544
INFO: Loaded 1 modules   (14253 inline 8-bit counters): 14253 [0x55a62cdea258, 0x55a62cdeda05), 
INFO: Loaded 1 PC tables (14253 PCs): 14253 [0x55a62cdeda08,0x55a62ce254d8), 
MERGE-OUTER: 417 files, 207 in the initial corpus
MERGE-OUTER: attempt 1
INFO: Seed: 4201440395
INFO: Loaded 1 modules   (14253 inline 8-bit counters): 14253 [0x5615a461a258, 0x5615a461da05), 
INFO: Loaded 1 PC tables (14253 PCs): 14253 [0x5615a461da08,0x5615a46554d8), 
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
MERGE-INNER: using the control file '/tmp/libFuzzerTemp.14696.txt'
MERGE-INNER: 417 total files; 0 processed earlier; will process 417 files now
#1	pulse  lim: 4 exec/s: 0 rss: 33Mb
#2	pulse  lim: 4 exec/s: 0 rss: 33Mb
#4	pulse  lim: 4 exec/s: 0 rss: 33Mb
#8	pulse  lim: 4 exec/s: 0 rss: 33Mb
#16	pulse  lim: 4 exec/s: 0 rss: 34Mb
#32	pulse  lim: 4 exec/s: 0 rss: 37Mb
#64	pulse  lim: 4 exec/s: 0 rss: 44Mb
#128	pulse  lim: 4 exec/s: 0 rss: 51Mb
#256	pulse  lim: 4 exec/s: 0 rss: 59Mb
MERGE-OUTER: succesfull in 1 attempt(s)
MERGE-OUTER: the control file has 114441 bytes
MERGE-OUTER: consumed 0Mb (34Mb rss) to parse the control file
MERGE-OUTER: 0 new files with 0 new features added

[maflcko]

So that is using libfuzzer. And libfuzzer is using "features" to determine if a seed adds coverage. You can get the "feature" coverage by just running libfuzzer. One of the lines will say:

#25	INITED cov: 714 ft: 1087 corp: 16/645b lim: 4 exec/s: 0 rss: 35Mb

[practicalswift]

@MarcoFalke I'm well aware of libFuzzer reports: I was thinking about what series of commands you ran to collect data for and to generate the lcov report. Trying to reproduce your results :)

[practicalswift]

Did you use test/fuzz/test_runner.py?

[maflcko]

Oh 😅

https://github.com/MarcoFalke/btc_cov/blob/travis/.travis.yml

[maflcko]

Hosted here: https://marcofalke.github.io/btc_cov/fuzz.coverage/index.html

[practicalswift]

Excellent! Exactly what I needed. Thanks!