API Access for testcases

[mhlakhani]

It would be great if we could build an API into clusterfuzz that provides access to testcases, and then expose this API for oss-fuzz issues. This would make it easier for organizations to centrally manage findings.

For my use case, the API should provide the following (perhaps as separate API endpoints)?

• The list of all open testcases
• For each testcase, the following
  • Tracking metadata (project name, fuzzer name, date it started firing, the monorail issue #, whether it is still open or if it is considered fixed)
  • Bug data (stack trace, crashing input, crash type, sanitizer, etc) -- the more information the better, but we could probably get away with just the crashing input to start with

It would be fine if we can just hit this API in a cron job every N hours.

As discussed, having such an API would also make it easier to integrate other bug trackers (google/clusterfuzz#389) as they could simply build on top of this API.

(I filed this here because the goal for us is to get access to oss-fuzz issues, please let me know if it is better to move this to the clusterfuzz issue tracker instead).

[badass-aoz]

ditto that. Having this set of apis can make integration a lot more useful.

[oliverchang]

The work that would go into supporting this would go into ClusterFuzz, so we should move it there.

However before we do that, can I understand a bit more what the intended use case is here? How would this data be used by you?

[oliverchang]

I ask because we have a project in the works to expose OSS-Fuzz findings in a more friendly way to the open source community, e.g. something like an API that be queried to see if there are any known vulnerabilities at a given tag/revision. I'm not sure if that will replace your need for this though.

[mhlakhani]

@oliverchang:

I'm happy to add more context (it's quite worthwhile, since not everyone was in the call we had earlier today). This was prompted by the recent onboarding of Hermes onto oss-fuzz, and our experience tracking those fixes internally.

My understanding of the current oss-fuzz model is that permissions are tied to a list of individuals who are listed in the project.yaml configuration file. With vendor-ccs we can get access to know that an issue was found, but that does not provide more detail on the issue itself.

The current problem we are facing is that we (a centralized security team) would like to know about all open oss-fuzz issues for projects owned at the company; and help engineering teams track the fuzzing findings in a way that is more inline with their usual workflow.

Right now, when we get an email about a new issue, we have to find the person who has access to the bug (this actually involved some guesswork, the emails did not mention which project had an issue) and they have to basically export the data manually into our internal bug tracker, otherwise we have to add every interested person to every project's config, which does not scale. For now, I've just had myself added to most/all of the projects and have imported these by hand.

With a solution that lets us programmatically access issues, we could write a script to pull these issues in directly and freely loop in engineers/security folk to help fix or triage the issues.

The work that would go into supporting this would go into ClusterFuzz, so we should move it there.

Should I just copy paste this issue and file it on the clusterfuzz github?

[evverx]

Looks like a duplicate of #3027

[evverx]

e.g. something like an API that be queried to see if there are any known vulnerabilities at a given tag/revision

@oliverchang I've never seen anyone ask for this (here at least) (probably because it's not useful for developers or, more generally, for people triaging and fixing bugs). I wonder if you're planning to integrate OSS-Fuzz into vulnerability scanners. If so, why would it be more important than the API allowing people to move issues from OSS-Fuzz (Monorail) to bug trackers they're used to.

[inferno-chromium]

e.g. something like an API that be queried to see if there are any known vulnerabilities at a given tag/revision

@oliverchang I've never seen anyone ask for this (here at least) (probably because it's not useful for developers or, more generally, for people triaging and fixing bugs). I wonder if you're planning to integrate OSS-Fuzz into vulnerability scanners. If so, why would it be more important than the API allowing people to move issues from OSS-Fuzz (Monorail) to bug trackers they're used to.

@evverx - this is a different usecase. Basically consumers of open source software e.g. Facebook, Chrome, Android, Mozilla, etc need to know the state of that open source library at a particular revision, before they roll in their products. For other things, these folks had relied on CVEs but as we know CVEs usually never get allocated in open source libraries (other than high profile ones e.g. openssl).

[evverx]

Basically consumers of open source software e.g. Facebook, Chrome, Android, Mozilla, etc need to know the state of that open source library at a particular revision, before they roll in their products

It seems to be based on the assumption that they consume open source libraries directly. If packages of some kind are installed instead, those OSS-Fuzz queries wouldn't help much because bug fixes tend to be backported making raw tags and revisions irrelevant.

[inferno-chromium]

Hasnain, we have a solution for this usecase now, can you please check this - https://security.googleblog.com/2021/02/launching-osv-better-vulnerability.html. Would love to have you onboard.

[evverx]

@inferno-chromium I wonder what is considered a vulnerability there? I skimmed over https://osv.dev/docs/#tag/faq but couldn't find anything.

[inferno-chromium]

vulnerability = security bug = OSS-Fuzz bugs with Type=Bug-Security label - https://bugs.chromium.org/p/oss-fuzz/issues/list?q=Type%3DBug-Security&can=1

[mhlakhani]

Hasnain, we have a solution for this usecase now, can you please check this - https://security.googleblog.com/2021/02/launching-osv-better-vulnerability.html. Would love to have you onboard.

Thanks! I've flagged to the team that works on ingesting external vulnerability feeds and they're taking a look

[inferno-chromium]

Hasnain, we have a solution for this usecase now, can you please check this - https://security.googleblog.com/2021/02/launching-osv-better-vulnerability.html. Would love to have you onboard.

Thanks! I've flagged to the team that works on ingesting external vulnerability feeds and they're taking a look

Great, thanks!

[evverx]

@inferno-chromium I took a look at the systemd "vulnerabilities" and 4 of them point to the commit where I added a fuzz target: systemd/systemd@e27aac1 (which makes sense because it's unlikely that the script looking for bugs can get past the point where the fuzz targets were introduced). I'm not sure why OSV thinks that "no versions" were affected though and hid all those bug reports by default. Given that (in my experience) a lot of bugs are found and fixed right after fuzz targets are added it seems OSV should at least show bug reports with "no versions affected" and probably the documentation should mention that limitation.

[evverx]

On a somewhat related note, I think the requirement to fix fuzz targets to make bad_build_check pass isn't compatible with OSV in the sense that bugs that are fixed that way never pop up on Monorail. For example, a bug in systemd (to which CVE-2018-16866 was assigned) was found when the fuzz target was being integrated into systemd but it was fixed before OSS-Fuzz had a chance to find and report it. Though if it had been found by OSS-Fuzz it would have been hidden by default as far as I can tell.