Skip to content

Commit

Permalink
Merge branch 'main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
laurie-tyz authored Oct 12, 2020
2 parents e41873c + 70b8cd6 commit fc63d08
Show file tree
Hide file tree
Showing 4 changed files with 59 additions and 18 deletions.
39 changes: 39 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# How to contribute

Thanks for your help on improving our stakeholder-specific vulnerability categorization work. To account for different stakeholder perspectives, we benefit from a diverse group of contributors.

## Where to contribute

This repository contains both a written document with the English-langauge spec, and some code for automating application of SSVC. Contributions to these two parts of the project look different. We are focusing on getting the English right first, so we know what code to write.
Right now we don't have any plans for translations, but if you have interest in that let us know.

# Contributing to the document

The English text lives in the `doc` [subfolder](https://github.com/CERTCC/SSVC/tree/main/doc).
We welcome any issues from anyone in the community, so we can discuss them and improve SSVC. If you have a suggestion, please create an issue.
In general, please create an issue before making a pull request to submit a change, except in the case of fixing a small typo, etc.
Please check that your suggestion does not overlap with existing [issues](https://github.com/CERTCC/SSVC/issues) (including [closed ones](https://github.com/CERTCC/SSVC/issues?q=is%3Aissue+is%3Aclosed+))

In the `doc` folder, please see the `style-guide`, `crossref-how-to`, and `reference-how-to` for how to keep any suggestions or commits aligned with our style consistently.

# Contributing code

The tools for working with SSVC live in the `src` [subfolder](https://github.com/CERTCC/SSVC/tree/main/src).

We have limited tooling at the moment. The expectation is that these will mostly be flexible helper-type scripts and plug-ins. Therefore, interoperability is important.
Where the code implements or directly references some aspect of the English document, please make that linkage explicit. We use config files stored in `data` to to prevent code in `src` from having fragile dependencies on the English doc.
We would like to minimize manual change management, but at the very least we need to document where changes in the document need to result in changes to code.
Information likely to change based on changes to the English should go in config files to be stored in the `data` [subfolder](https://github.com/CERTCC/SSVC/tree/main/data). Code in the `src` folder should (as robustly as plausible) be reading that data in.

The process is similar to that for the doc, though the language is different. Please create issues before making pull requests.
Pull requests on code should be clear about what they've changed and what you've done. Thanks in advance!

# Licenses

- The license for all code in the repository is [here](https://github.com/CERTCC/SSVC/blob/main/LICENSE)
- The license for all English writing in the repository is [here](https://github.com/CERTCC/SSVC/blob/main/doc/version_1/900_license.md)

# Questions

If you have any questions, a message to j--- should work, or tweet @zmanion or @\_\_adh\_\_.

31 changes: 16 additions & 15 deletions doc/version_1/040_treesForVulMgmt.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,23 +123,24 @@ When evaluating *technical impact*, recall the scope definition above. Total con
### Utility (Supplier, Deployer)
> The Usefulness of the Exploit to the Adversary
Heuristically, we base *utility* on a combination of value density of vulnerable components and virulence of potential exploitation. This framing makes it easier to analytically derive these categories from a description of the vulnerability and the affected component. Virulence (slow or rapid) and value density (diffuse or concentrated) are defined in Sections 4.4.3.1 and 4.4.3.2. Deployers currently use this feature only as a suggested constraint on the values for *Mission Impact*.
Heuristically, we base *utility* on a combination of value density of vulnerable components and automatability of potential exploitation. This framing makes it easier to analytically derive these categories from a description of the vulnerability and the affected component. Automatability (slow or rapid) and value density (diffuse or concentrated) are defined in Sections 4.4.3.1 and 4.4.3.2. Deployers currently use this feature only as a suggested constraint on the values for *Mission Impact*.
main

Roughly, *utility* is a combination of two things: (1) the value of each exploitation event and (2) the ease and speed with which the adversary can cause exploitation events. We define *utility* as laborious, efficient, or super effective, as described in Table 6.

| | Table 6: Utility Decision Values |
| --------------- | ------------------------------------------------------------------------------ |
| Laborious | Slow virulence and diffuse value |
| Efficient | {Rapid virulence and diffuse value} OR {Slow virulence and concentrated value} |
| Super Effective | Rapid virulence and concentrated value |
| Laborious | Slow automatability and diffuse value |
| Efficient | {Rapid automatability and diffuse value} OR {Slow automatability and concentrated value} |
| Super Effective | Rapid automatability and concentrated value |

#### Virulence
#### Automatability

*Virulence* is described as slow or rapid:
*Automatability* is described as slow or rapid:

- **Slow**. Steps 1-4 of the kill chain [@hutchins2011intelligence] cannot be reliably
automated for this vulnerability for some reason. These steps are
reconnaissance, weaponization, delivery, and exploitation. Example
- **Slow**. Attackers cannot reliably automate steps 1-4 of the kill chain
[@hutchins2011intelligence] for this vulnerability for some reason. These
steps are reconnaissance, weaponization, delivery, and exploitation. Example
reasons for why a step may not be reliably automatable include (1)
the vulnerable component is not searchable or enumerable on the
network, (2) weaponization may require human direction for each
Expand All @@ -148,9 +149,9 @@ Roughly, *utility* is a combination of two things: (1) the value of each exploit
frustrated by adequate exploit-prevention techniques enabled by
default; ASLR is an example of an exploit-prevention tool.

- **Rapid**. Steps 1-4 of the of the kill chain can be reliably
automated. If the vulnerability allows remote code execution or
command injection, the default response should be rapid.
- **Rapid**. Attackers can reliably automate steps 1-4 of the of the kill
chain. If the vulnerability allows remote code execution or command
injection, the default response should be rapid.

#### Value Density

Expand Down Expand Up @@ -181,17 +182,17 @@ Roughly, *utility* is a combination of two things: (1) the value of each exploit

The output for the *Utility* decision point is visualized in Table 7.

Table 7: Utility to the Adversary, as a Combination of Virulence and Value Density
Table 7: Utility to the Adversary, as a Combination of Automatability and Value Density

| *Virulence* | *Value Density* | *Utility* |
| *Automatability* | *Value Density* | *Utility* |
| ----------- | --------------- | --: |
| **slow** | **diffuse** | laborious |
| **slow** | **concentrated** | efficient |
| **rapid** | **diffuse** | efficient |
| **rapid** | **concentrated** | super effective |


Alternative heuristics for proxying adversary utility are plausible. One such example is the value the vulnerability would have were it sold on the open market. Some firms, such as [Zerodium](https://zerodium.com/program.html), make such pricing structures public. The valuable exploits track the virulence and value density heuristics for the most part. Within a single system—whether it is Apache, Windows, iOS or WhatsApp—more automated kill chain steps successfully leads to higher exploit value. Remote code execution with sandbox escape and without user interaction are the most valuable exploits, and those features describe automation of the relevant kill chain steps. How equivalently virulent exploits for different systems are priced relative to each other is more idiosyncratic. Price does not only track value density of the system, but presumably also the existing supply of exploits and the installation distribution among the targets of Zerodium’s customers. Currently, we simplify the analysis and ignore these factors. However, future work should look for and prevent large mismatches between the outputs of the *utility* decision point and the exploit markets.
Alternative heuristics for proxying adversary utility are plausible. One such example is the value the vulnerability would have were it sold on the open market. Some firms, such as [Zerodium](https://zerodium.com/program.html), make such pricing structures public. The valuable exploits track the automatability and value density heuristics for the most part. Within a single system—whether it is Apache, Windows, iOS or WhatsApp—more automated kill chain steps successfully leads to higher exploit value. Remote code execution with sandbox escape and without user interaction are the most valuable exploits, and those features describe automation of the relevant kill chain steps. How equivalently virulent exploits for different systems are priced relative to each other is more idiosyncratic. Price does not only track value density of the system, but presumably also the existing supply of exploits and the installation distribution among the targets of Zerodium’s customers. Currently, we simplify the analysis and ignore these factors. However, future work should look for and prevent large mismatches between the outputs of the *utility* decision point and the exploit markets.

### Safety Impact (Supplier, Deployer)
> Safety Impacts of Affected System Compromise
Expand Down
5 changes: 3 additions & 2 deletions doc/version_1/050_evaluationDraftTrees.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ For all decision points, the presumed goal is for κ to be close or equal to 1.

This solution identifies another difficulty of Fleiss’ kappa, namely that it does not preserve any order; none and catastrophic are considered the same level of disagreement as none and minor. Table 13 displays a sense of the range of disagreement to complement this weakness. This value is the largest distance between rater selections on a single vulnerability out of the maximum possible distance. So, for safety impact, the most two raters disagreed was by two steps (none to major, minor to hazardous, or major to catastrophic) out of the four possible steps (none to catastrophic). The only values of κ that are reliably comparable are those with the same number of options (that is, the same maximum distance). In other cases, closer to 1 is better, but how close is close enough to be considered “good” changes. In all but one case, if raters differed by two steps then there were raters who selected the central option between them. The exception was mission impact for CVE-201814781; it is unclear whether this discrepancy should be localized to a poor test scenario description, or to SSVC’s mission impact definition. Given it is an isolated occurrence, we expect the scenario description at least partly.

Nonetheless, κ provides some way to measure improvement on this a conceptual engineering task. The pilot evaluation can be repeated, with more diverse groups of stakeholders after the descriptions have been refined by stakeholder input, to measure fit to this goal. For a standard to be reliably applied across different analyst backgrounds, skill sets, and cultures, a set of decision point descriptions should ideally achieve κ of 1 for each item in multiple studies with diverse participants. Such a high level of agreement would be difficult to achieve, but it would ensure that when two analysts assign a priority with the system that they get the same answer. Such agreement is not the norm with CVSS currently [@allodi2018effect].
Nonetheless, κ provides some way to measure improvement on this a conceptual engineering task. The pilot evaluation can be repeated, with more diverse groups of stakeholders after the descriptions have been refined by stakeholder input, to measure fit to this goal. For a standard to be reliably applied across different analyst backgrounds, skill sets, and cultures, a set of decision point descriptions should ideally achieve κ of 1 for each item in multiple studies with diverse participants. Such a high level of agreement would be difficult to achieve, but it would ensure that when two analysts assign a priority with the system that they get the same answer. Such agreement is not the norm with CVSS currently [@allodi2018effect].

We have not discussed a convenient compressed expression of a set of SSVC decision points. Table 14 uses initialisms similar to a CVSS vector string; Exploitation (E), Technical impact (T), Utility (U), Safety Impact (S), Exposure (X), and Mission impact (M). S:M is minor, S:J is major; M:F is MEF failure, M:M is mission failure. However, since we created Utility in response to the System Value metric’s shortcomings, the pilot results do not include systematic consensus on Utility values.

Expand Down Expand Up @@ -210,4 +210,5 @@ Some of these points left marks on other decision points. The decision point “

Three of the above decision points left no trace on the current system. “State of legal or regulatory obligations,” “cost of developing a patch,” and “patch distribution readiness” were dropped as either being too vaguely defined, too high level, or otherwise not within the scope of decisions by the defined stakeholders. The remaining decision point, “adversary’s strategic benefit of exploiting the vulnerability,” transmuted to a different sense of system value. In an attempt to be more concrete and not speculate about adversary motives, we considered a different sense of value: supplier portfolio value.

The only decision point that we have removed following the pilot is supplier portfolio value. This notion of value was essentially an over-correction to the flaws identified in the “adversary’s strategic benefit of exploiting the vulnerability” decision point. “Supplier portfolio value” was defined as “the value of the affected component as a part of the supplier’s product portfolio. Value is some combination of importance of a given piece of software, number of deployed instances of the software, and how many people rely on each. The supplier may also include lifecycle stage (early development, stable release, decommissioning, etc.) as an aspect of value.” It had two possible values: low and high. As Table 13 demonstrates, there was relatively little agreement among the six analysts about how to evaluate this decision point. We replaced this sense of portfolio value with *Utility*, which combines *Value Density* and *Virulence*.
The only decision point that we have removed following the pilot is developer portfolio value. This notion of value was essentially an over-correction to the flaws identified in the “adversary’s strategic benefit of exploiting the vulnerability” decision point. “Supplier portfolio value” was defined as “the value of the affected component as a part of the developer’s product portfolio. Value is some combination of importance of a given piece of software, number of deployed instances of the software, and how many people rely on each. The developer may also include lifecycle stage (early development, stable release, decommissioning, etc.) as an aspect of value.” It had two possible values: low and high. As Table 13 demonstrates, there was relatively little agreement among the six analysts about how to evaluate this decision point. We replaced this sense of portfolio value with *Utility*, which combines *Value Density* and *Automatability*.

2 changes: 1 addition & 1 deletion ssvc-calc/ssvc.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/* SSVC code for graph building */
const _version = 2.9
const _version = 2.91
var showFullTree = false
var diagonal,tree,svg,duration,root
var treeData = []
Expand Down

0 comments on commit fc63d08

Please sign in to comment.