Review Security Vulnerability/Communication response policy for CNCF incubated/graduated projects

[ultrasaurus]

Description: as discussed in Feb 16 TOC meeting (TBD toc issue), take a look at incubated/graduated projects and check to what degree projects have documented policies for users to report security vulnerabilities and how known vulnerabilities are communicated

Impact: support TOC in evaluating proposed criteria to validate requirement which is already part of CII badge requirement

Scope: take a look at CII badge requirements. Per Feb 17 SIG-Security meeting, projects are already required to document a link, maybe this could be simply automated or perhaps few enough CNCF projects to manually create list of links and visually review them. Then report back on CNCF TOC issue

TO DO

• TAG Representative: @ultrasaurus
• Project leader: Alex Floyd Marshall (@apmarshall)
• first draft targeted mid-Q3
• presentation to TOC: late Q3 or Q4? (assess schedule after first draft)

[joshuagl]

#183 is a related issue proposing we create a template security reporting policy for projects to use

[apmarshall]

To clarify the project here, two pieces:

1. Identify which CNCF incubated/graduated projects have documented vulnerability reporting procedures.
2. Create a list of links to those documents for the ones that exist.

Procedures we are looking for from projects:

1. How to report discovered vulnerabilities to the project.
2. How project leaders communicate vulnerabilities to their users.

Is that an accurate summary?

[apmarshall]

Here is the collected data: CNCF Security Procedures

From what I have seen, the following projects have no defined security policy:

• CloudEvents
• fluentd*
• OpenTracing
• KubeEdge
• SPIFFE (but maybe this is superseded by SPIRE)

The following projects have a defined procedure for reporting vulnerabilities, but no defined procedure for fixing and communicating about such vulnerabilities:

• NATS
• Buildpacks
• Prometheus*
• Thanos
• gRPC
• Envoy*
• CNI
• Jaeger*
• Dragonfly
• TUF*
• Falco
• Notary
• SPIRE

*Graduated Projects

Happy to convert this data from spreadsheet form to either markdown we can keep in this repo or some sort of "dashboard" we can publish somewhere.

[philpennock]

Please clarify "defined" and where this needs to be defined to meet the requirements?

NATS: we use https://advisories.nats.io/ as a repository of known security advisories. At our discretion, we also email oss-security, where "discretion" means "don't spam them with trivialities" but in practice so far all issues have been mailed there.

The procedure for fixing them is to triage the report, fix it if it's a real issue, and include the fix in the next release and publish the advisory then; for most such issues, we refrain from pushing the fix to the repo until just before the release. For certain classes of vulnerability which are DoS and only affect some circumstances, we might fix on git only and not rush to a new release.

We can pretty this up with words (including going into more details about the git-only circumstances) in a specific format in a specific place, I just need to know where to do so.

[caniszczyk]

ah interest @philpennock, has NATS considered just using the built in github security advisory mechanism now? https://github.com/containerd/containerd/security/advisories?state=published

[philpennock]

Not really, but we could do so in addition. We think that the canonical results should be easy for any search engine to find and so put them on a hostname under our control, with clean content which renders without javascript etc. Security researchers are cautious like that.

I will add to my ToDo list to populate the advisories sections and update our process docs to do so.

[caniszczyk]

@philpennock I only ask as the github one you can programmatically check for easily via their API (you can also request CVEs via their system which as an added bonus)

anyways, thank you!

[apmarshall]

@philpennock There are a few ways that projects "define" and make available their disclosure policies. One way is to put a SECURITY file in either the project home directory or the .github directory (if you use that) and describe the policy there. Another way, which is particularly useful if you have multiple repositories you want the same policy to apply to, is to create a community or policy repo to collect all the common, policy stuff (security, contribution policies, governance, etc) and then link to the policy in that repo through the Security tab in GitHub in all the repos that it applies to. You can see examples of both in the spreadsheet of data I linked above. Hope that's helpful to you and your team!

[philpennock]

Re NATS https://github.com/nats-io:

@caniszczyk I've backfilled GitHub with GHSAs for all existing advisories in the NATS project and cross-linked appropriately.

@apmarshall We'll discuss structure of layout of .github vs extra-repo in our upcoming planning meeting. Thanks. I should have remembered that flow existed, my head was juggling other stuff yesterday. Duh. Well, this is why we document things instead of relying upon memory. 😄

[RichiH]

With my Prometheus hat on: Revamping our reporting and disclosure policies etc is on the backlog and should be discussed in one of the next few dev summits.

While I can't make hard promises on speed without wider discussion within -team, we would most certainly be interested in following, potentially shaping, a security BCP which can be used as a default across CNCF or even LF.

[ultrasaurus]

@apmarshall sorry to be late responding -- would be awesome if you could convert into some kind of dashboard or markdown. In my dreams would be auto-updated programmatically, but wouldn't be a big deal to update manually when projects move from incubation to graduation or annually or something. Still up for that? If so, wanna propose something specific or go ahead and prototype something?

[apmarshall]

Sure! I am away for the next couple of months with limited internet access, but I would be glad to build something when I return. If projects use the GitHub "security policy" feature to point to their docs, should be fairly easy to use the GitHub API to keep track of that automatically.

[ultrasaurus]

Excellent! thanks @apmarshall and API-driven approach would be awesome, as long as there's someway for people to link in their stuff (maybe via some standard text file or README link) if their stuff lives elsewhere. We don't want to require the exact mechanism, but reasonable IMHO to require README reference!

With your taking on the issue, I suggest we formally stick this on the roadmap, and maybe aim for Q3? Perhaps first draft end of of Q2 or early Q3 and then time for community feedback and iteration before formalizing and presenting to TOC?

[apmarshall]

I'm back in July, so maybe let's say first draft targeted mid-Q3, aiming for presentation late Q3 or early Q4?

I believe (and I need to look at this in more detail, I've just surmised this by seeing how people seem to be using it) that the newish GitHub security policy feature for repos basically let's you populate a field with a link to your security policy, wherever that policy may live. Then I think we should be able to grab the contents of that field (the link) via the API to programmatically update the dashboard should a repo change the location of their policy. So I don't think it determines for anyone exactly where/how they store their policy, it just gives us an easy mechanism for discovery provided they've populated the field with the appropriate link. Again, that's how I think it works and I need to actually read the docs and confirm that. But assuming I've surmised correctly based on what I've seen repos doing, it should give us a fairly straightforward implementation for those repos using this feature. Will have to do a bit more legwork for those that don't, but I'll figure that out when I dig into this.

[ultrasaurus]

that makes sense to me, @philpennock do you think that would work for NATS

Separately, we'll want some kind of human review of the contents of the link, periodically and when projects seek to graduate. @apmarshall you mentioned a spreadsheet -- if you have that somewhere with access in your travels, could you post a link to it so maybe there are some folks who would want to take a look in parallel and we can muse on the process for the human review (which is typically part of the security review, but maybe we want to factor out, so it could be de-coupled so we don't need full security review, just to do this part)

[apmarshall]

Sure. I'll put the link here with permissions for commenting and then I'll share with you for editing as well.

https://docs.google.com/spreadsheets/d/1EuI5SOTJSYAClgvz0QGPXi8FC6EhsH9h1fzfs8kDPfM/edit?usp=sharing

[philpennock]

If the question to me is around relying on GitHub to act as an index pointing to security policy docs elsewhere, then: seems reasonable for us. NATS should have a vulnerability disclosure program sometime "soon", it's the second item on my (current) priority list.

[RichiH]

Irregular check-in from Prometheus: Is there a guesstimated time? We would like to update our security handling, but it's not urgent either.

I see it's in "Planned and Scheduled" state, but couldn't determine if there's a time guesstimate beyond "by EOY 2022".

[anvega]

Currently "Vulnerability report process" is captured under the "Reporting" section of the "Silver" level criteria in CII badges. This may be an issue for the OpenSSF to move from "Silver" to "Best Practices: Passing" level.

A "Best Practices: Passing" badge is expected for CNCF projects moving from incubation to graduation.

[anvega]

Hey @apmarshall , is https://docs.google.com/spreadsheets/d/1EuI5SOTJSYAClgvz0QGPXi8FC6EhsH9h1fzfs8kDPfM/edit?usp=sharing reasonably up to date? How much work is involved in keeping it current? Interest in visualizing it publicly to have it as something we could host somewhere?

[anvega]

Closing due to inactivity as discussed during today's weekly call. The spreadsheet link above documents most incubated and graduated projects at that point in time and shows only a few missing a vulnerability disclosure process. With the recent announcement of GH advisory database might be worthwhile considering how to leverage it as support for other ecosystems is expanded.