Don't know how to fix wasmtime's coverage build

[alexcrichton]

When we first added wasmtime to oss-fuzz we quckly got a bug opened about the coverage build failing. Checking out the build log the end of it has failures that look like:

Step #5: du: cannot access '/workspace/out/coverage/dumps/compile.*.profraw': No such file or directory
Step #5: du: cannot access '/workspace/out/coverage/dumps/instantiate.*.profraw': No such file or directory
Step #5: Error occured while running api_calls:
Step #5: INFO: Seed: 4260620746
Step #5: INFO: Loaded 1 modules   (617915 inline 8-bit counters): 617915 [0x55afc02fd368, 0x55afc0394123), 
Step #5: INFO: Loaded 1 PC tables (617915 PCs): 617915 [0x55afc0394128,0x55afc0d01cd8), 
Step #5: MERGE-OUTER: 5557 files, 0 in the initial corpus, 0 processed earlier
Step #5: MERGE-OUTER: attempt 1
Step #5: ==46== libFuzzer: run interrupted; exiting
Step #5: du: cannot access '/workspace/out/coverage/dumps/api_calls.*.profraw': No such file or directory
Step #5: Error occured while running differential:
Step #5: INFO: Seed: 4260836248
Step #5: INFO: Loaded 1 modules   (617824 inline 8-bit counters): 617824 [0x563ef0cb2468, 0x563ef0d491c8), 
Step #5: INFO: Loaded 1 PC tables (617824 PCs): 617824 [0x563ef0d491c8,0x563ef16b67c8), 
Step #5: MERGE-OUTER: 2588 files, 0 in the initial corpus, 0 processed earlier
Step #5: MERGE-OUTER: attempt 1
Step #5: ==90== libFuzzer: run interrupted; exiting
Step #5: du: cannot access '/workspace/out/coverage/dumps/differential.*.profraw': No such file or directory
Step #5: Error occured while running instantiate_translated:
Step #5: INFO: Seed: 4261298422
Step #5: INFO: Loaded 1 modules   (617786 inline 8-bit counters): 617786 [0x557d38f2e428, 0x557d38fc5162), 
Step #5: INFO: Loaded 1 PC tables (617786 PCs): 617786 [0x557d38fc5168,0x557d39932508), 
Step #5: MERGE-OUTER: 6338 files, 0 in the initial corpus, 0 processed earlier
Step #5: MERGE-OUTER: attempt 1
Step #5: ==178== libFuzzer: run interrupted; exiting
Step #5: du: cannot access '/workspace/out/coverage/dumps/instantiate_translated.*.profraw': No such file or directory
Step #5: error: /workspace/out/coverage/dumps/*.profdata: No such file or directory

Notably it looks like no *.profraw or *.profdata entries are getting emitted by the fuzzers. This is almost for sure due to how wasmtime is a Rust project and the Rust integration with oss-fuzz isn't complete in some way.

I'm curious though if we could get some help in deciphering this a bit? I believe these files are supposed to be generated by instrumentation in LLVM, for example I've seen them get generated for PGO builds of Rust at some point. I'm not entirely sure how the fuzzers are supposed to generate them though. Is there an example of what this looks like in C/C++ to see what we need to mirror into Rust? Or is it known what LLVM intrinsic libraries are necessary to get linked in (or passes to run?) that we need to configure rustc to execute? Additionally I wasn't entirely sure how we would know to change our build to a "coverage mode" because at least reading the build logs I didn't see env vars or such indicating that we should be building the fuzzers a different way.

Another question may just be if this is that relevant to fix. Are we perhaps losing out on the quality of fuzzing because we don't have this coverage information?

In any case any help in debugging this would be much appreciated! I also wasn't sure how to reproduce and poke around locally, so if there's a one-liner for that I can dig more in locally myself too.

[Dor1s]

Thanks for reporting this and sorry for the trouble. Coverage should be running for C/C++ projects only, as of now, hm...

This coverage build is used only for generating HTML reports, so the fuzzing quality or any other OSS-Fuzz bits aren't affected by this.

We use https://clang.llvm.org/docs/SourceBasedCodeCoverage.html if you want to learn more, and for languages other than C/C++ we just skip the coverage job for now, until we reach some critical mass of fuzzers/users of a given language :)

[Dor1s]

Ah, I see what's going on. The coverage build failure got opened before we started to skip coverage job for non C/C++ projects. After that change was made, no new bugs are being opened for such projects, but the one for wasmtime wasn't autoclosed. Same for mtail (written in Go).

I've just WontFix'ed both.

If you know how to generate code coverage reports for Rust, we can discuss it in #3469, but I should say it's not likely that we'll add a support for it soon. We'd love to get more Rust projects signed up first :)

[alexcrichton]

Oh heh that all makes sense, thanks for the info and quick resolution!