[SecAssess WG] Provide some consistency across reviews

[lumjjb]

This issue was created from results of the Security Assessment Improvement Working Group (#167 (comment)).

Provide additional consistency across security assessment reviews

Premise

• Auditing imperative systems is more of an art than a science
• Threat modeling is important for projects, we should figure out how to help them be consistent with it
• Different projects have a variety in the level of detail for different sections and emphasis on different aspects

Ideas

• Make adjustments to assessment doc structure
• Give recommendations of word length for sections
• Map assessment findings to MIRTE ATTACK or similar
• Use an existing assessment framework + template - can be tweaked for CNCF but start with that so there is a consistent process and vocabulary
• Provide qualitative ranking for projects i.e. scoring between 1 - 10
• Provide ideas/guidelines for how one should perform a review

Logistics

• Contributors (For multiple contributors, 1 lead to coordinate)
• Placeholder_2
• SIG-Representative

[JustinCappos]

I don't really know how to be more consistent with threat modeling unless we have a dedicated team. From teaching this at NYU, the abilities and issues found widely vary by the person unless you are really experienced doing this. The difficulty though is that really doing a proper threat model requires knowing the system and the operational environment well...

The past idea was that the project provided a sketch of the operational model and system and the threat model evolved by talking with the assessors. I'm not sure if there is a better way.

[JustinCappos]

Provide qualitative ranking for projects i.e. scoring between 1 - 10

As someone who gives rankings like this (grades) all the time, I do not think you want to open this can of worms. Projects will lobby hard to increase a number which will be semi arbitrary. I think explaining the pros and cons is a lot easier to justify and harder to argue against.

[stale]

This issue has been automatically marked as inactive because it has not had recent activity.

[stale]

This issue has been automatically marked as inactive because it has not had recent activity.

[anvega]

Closing the issue as assessments are as consistent as they can get. The assessment book publication already synthesizes the current "pensum" which strives for conformance with a baseline.