Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update SARIF format #534

Merged
merged 38 commits into from
Sep 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
909ba19
sarif output attempt 2
another-rex Aug 29, 2023
e1cc2bc
Remove unused lines
another-rex Aug 29, 2023
a68755d
Edit workflow for testing
another-rex Aug 29, 2023
f911b44
Update workflow again for testing
another-rex Aug 29, 2023
44e0b33
Add ID to short description
another-rex Aug 29, 2023
585403a
Attempt at fancy template
another-rex Aug 29, 2023
2a84401
Update template
another-rex Aug 29, 2023
1a4707f
Quote the details
another-rex Aug 29, 2023
c296d9b
Replace header with just bold
another-rex Aug 29, 2023
1b4f365
Merge remote-tracking branch 'upstream/main' into sarif-updates
another-rex Sep 11, 2023
313f80e
Update to new format
another-rex Sep 12, 2023
fc042cc
Fix sarif comparison and add tests
another-rex Sep 12, 2023
697c916
Refactor and fix tests
another-rex Sep 13, 2023
295fbce
Remove commented out code
another-rex Sep 13, 2023
398be8b
Refactor to make it clearer
another-rex Sep 14, 2023
99a93bc
Add tests
another-rex Sep 14, 2023
66e00f8
Undo reusable pr workflow change
another-rex Sep 14, 2023
094b1fd
Fix lints
another-rex Sep 14, 2023
0ce3212
Minor fixes
another-rex Sep 14, 2023
2a7d9ef
Fix tests and "also known as" output
another-rex Sep 14, 2023
b82d1e8
Add github annotation tests
another-rex Sep 14, 2023
5f7342d
Address PR comments
another-rex Sep 15, 2023
45366cc
Update .goreleaser.yml
another-rex Sep 15, 2023
3cc256a
Change test func name
another-rex Sep 15, 2023
fd433a6
Merge branch 'main' into sarif-updates
another-rex Sep 15, 2023
8b79763
Fix lint and tests
another-rex Sep 15, 2023
85ad73f
fix string interpolation
another-rex Sep 15, 2023
b40ca28
Update sarif dependency and place deprecatedIds into the correct place
another-rex Sep 18, 2023
34e977d
Disable lint
another-rex Sep 18, 2023
e1eeb32
Have reporter use internal version
another-rex Sep 18, 2023
c9e2a2e
Merge branch 'main' into sarif-updates
another-rex Sep 19, 2023
bccf282
Fix tests
another-rex Sep 19, 2023
8b54451
Merge branch 'main' into sarif-updates
another-rex Sep 19, 2023
cf2a65e
Fix behavior with aliases, add additional tests
another-rex Sep 19, 2023
08672c1
Remove unused lint
another-rex Sep 19, 2023
9254a31
Fix test paths
another-rex Sep 19, 2023
4b15862
Add code cov ignore for testutility
another-rex Sep 19, 2023
941f031
Deterministic sorting
another-rex Sep 19, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ builds:
- -trimpath
ldflags:
# prettier-ignore
- '-s -w -X main.version={{.Version}} -X main.commit={{.Commit}} -X main.date={{.CommitDate}}'
- '-s -w -X github.com/google/osv-scanner/internal/version.OSVVersion={{.Version}} -X main.commit={{.Commit}} -X main.date={{.CommitDate}}'
goos:
# Further testing before supporting freebsd
# - freebsd
Expand Down
8 changes: 4 additions & 4 deletions cmd/osv-reporter/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"strings"

"github.com/google/osv-scanner/internal/ci"
"github.com/google/osv-scanner/internal/version"
"github.com/google/osv-scanner/pkg/models"
"github.com/google/osv-scanner/pkg/osvscanner"
"github.com/google/osv-scanner/pkg/reporter"
Expand All @@ -17,9 +18,8 @@ import (

var (
// Update this variable when doing a release
version = "1.4.0"
commit = "n/a"
date = "n/a"
commit = "n/a"
date = "n/a"
)

// splitLastArg splits the last argument by new lines and appends the split
Expand Down Expand Up @@ -47,7 +47,7 @@ func run(args []string, stdout, stderr io.Writer) int {

app := &cli.App{
Name: "osv-scanner-action-reporter",
Version: version,
Version: version.OSVVersion,
Usage: "(Experimental) generates github action output",
Description: "(Experimental) Used specifically to generate github action output ",
Suggest: true,
Expand Down
11 changes: 5 additions & 6 deletions cmd/osv-scanner/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"os"
"strings"

"github.com/google/osv-scanner/internal/version"
"github.com/google/osv-scanner/pkg/osv"
"github.com/google/osv-scanner/pkg/osvscanner"
"github.com/google/osv-scanner/pkg/reporter"
Expand All @@ -17,10 +18,8 @@ import (
)

var (
// Update this variable when doing a release
version = "1.4.0"
commit = "n/a"
date = "n/a"
commit = "n/a"
date = "n/a"
)

func run(args []string, stdout, stderr io.Writer) int {
Expand All @@ -32,11 +31,11 @@ func run(args []string, stdout, stderr io.Writer) int {
r.PrintText(fmt.Sprintf("osv-scanner version: %s\ncommit: %s\nbuilt at: %s\n", ctx.App.Version, commit, date))
}

osv.RequestUserAgent = "osv-scanner/" + version
osv.RequestUserAgent = "osv-scanner/" + version.OSVVersion

app := &cli.App{
Name: "osv-scanner",
Version: version,
Version: version.OSVVersion,
Usage: "scans various mediums for dependencies and matches it against the OSV database",
Suggest: true,
Writer: stdout,
Expand Down
182 changes: 106 additions & 76 deletions cmd/osv-scanner/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"testing"

"github.com/go-git/go-git/v5"
"github.com/google/osv-scanner/internal/version"
)

func createTestDir(t *testing.T) (string, func()) {
Expand Down Expand Up @@ -134,7 +135,7 @@ func TestRun(t *testing.T) {
osv-scanner version: %s
commit: n/a
built at: n/a
`, version),
`, version.OSVVersion),
wantStderr: "",
},
// one specific supported lockfile
Expand Down Expand Up @@ -314,31 +315,25 @@ func TestRun(t *testing.T) {
name: "Empty sarif output",
args: []string{"", "--format", "sarif", "./fixtures/locks-many/composer.lock"},
wantExitCode: 0,
wantStdout: `
{
"version": "2.1.0",
"$schema": "https://json.schemastore.org/sarif-2.1.0.json",
"runs": [
{
"tool": {
"driver": {
"informationUri": "https://github.com/google/osv-scanner",
"name": "osv-scanner",
"rules": [
{
"id": "vulnerable-packages",
"shortDescription": {
"text": "This manifest file contains one or more vulnerable packages."
}
}
]
}
},
"results": []
}
]
}
`,
wantStdout: fmt.Sprintf(`
{
"version": "2.1.0",
"$schema": "https://json.schemastore.org/sarif-2.1.0.json",
"runs": [
{
"tool": {
"driver": {
"informationUri": "https://github.com/google/osv-scanner",
"name": "osv-scanner",
"rules": [],
"version": "%s"
}
},
"results": []
}
]
}
`, version.OSVVersion),
wantStderr: `
Scanning dir ./fixtures/locks-many/composer.lock
Scanned %%/fixtures/locks-many/composer.lock file and found 1 package
Expand All @@ -348,60 +343,95 @@ func TestRun(t *testing.T) {
name: "Sarif with vulns",
args: []string{"", "--format", "sarif", "--config", "./fixtures/osv-scanner-empty-config.toml", "./fixtures/locks-many/package-lock.json"},
wantExitCode: 1,
wantStdout: `
{
"version": "2.1.0",
"$schema": "https://json.schemastore.org/sarif-2.1.0.json",
"runs": [
{
"tool": {
"driver": {
"informationUri": "https://github.com/google/osv-scanner",
"name": "osv-scanner",
"rules": [
{
"id": "vulnerable-packages",
"shortDescription": {
"text": "This manifest file contains one or more vulnerable packages."
}
}
]
}
},
"artifacts": [
{
"location": {
"uri": "fixtures/locks-many/package-lock.json"
},
"length": -1
}
],
"results": [
{
"ruleId": "vulnerable-packages",
"ruleIndex": 0,
"level": "warning",
"message": {
"text": "+-----------+-------------------------------------+------+-----------------+---------------+\n| PACKAGE \u0026nbsp; | VULNERABILITY ID \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;| CVSS | CURRENT VERSION | FIXED VERSION |\n+-----------+-------------------------------------+------+-----------------+---------------+\n| ansi-html | https://osv.dev/GHSA-whgm-jr23-g3j9 | 7.5 \u0026nbsp;| 0.0.1 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; | 0.0.8 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; |\n+-----------+-------------------------------------+------+-----------------+---------------+"
},
"locations": [
{
"physicalLocation": {
"artifactLocation": {
"uri": "fixtures/locks-many/package-lock.json"
}
}
}
]
}
]
}
]
}
wantStdout: fmt.Sprintf(`
{
"version": "2.1.0",
"$schema": "https://json.schemastore.org/sarif-2.1.0.json",
"runs": [
{
"tool": {
"driver": {
"informationUri": "https://github.com/google/osv-scanner",
"name": "osv-scanner",
"rules": [
{
"id": "CVE-2021-23424",
"shortDescription": {
"text": "Uncontrolled Resource Consumption in ansi-html"
},
"fullDescription": {
"text": "This affects all versions of package ansi-html. If an attacker provides a malicious string, it will get stuck processing the input for an extremely long time.",
"markdown": "This affects all versions of package ansi-html. If an attacker provides a malicious string, it will get stuck processing the input for an extremely long time."
},
"deprecatedIds": [
"CVE-2021-23424",
"GHSA-whgm-jr23-g3j9"
],
"help": {
"text": "\n**Your dependency is vulnerable to [CVE-2021-23424](https://osv.dev/vulnerability/CVE-2021-23424)** \n.\n\n\n\u003e ## [GHSA-whgm-jr23-g3j9](https://osv.dev/vulnerability/GHSA-whgm-jr23-g3j9)\n\u003e \n\u003e This affects all versions of package ansi-html. If an attacker provides a malicious string, it will get stuck processing the input for an extremely long time.\n\u003e \n\n\n---\n\n### Affected Packages\n| Source | Package Name | Package Version |\n| --- | --- | --- |\n| lockfile:/%%%%/fixtures/locks-many/package-lock.json | ansi-html | 0.0.1 |\n\n",
"markdown": "\n**Your dependency is vulnerable to [CVE-2021-23424](https://osv.dev/vulnerability/CVE-2021-23424)** \n.\n\n\n\u003e ## [GHSA-whgm-jr23-g3j9](https://osv.dev/vulnerability/GHSA-whgm-jr23-g3j9)\n\u003e \n\u003e This affects all versions of package ansi-html. If an attacker provides a malicious string, it will get stuck processing the input for an extremely long time.\n\u003e \n\n\n---\n\n### Affected Packages\n| Source | Package Name | Package Version |\n| --- | --- | --- |\n| lockfile:/%%%%/fixtures/locks-many/package-lock.json | ansi-html | 0.0.1 |\n\n"
}
}
],
"version": "%s"
}
},
"artifacts": [
{
"location": {
"uri": "file://%%%%/fixtures/locks-many/package-lock.json"
},
"length": -1
}
],
"results": [
{
"ruleId": "CVE-2021-23424",
"ruleIndex": 0,
"level": "warning",
"message": {
"text": "Package '[email protected]' is vulnerable to 'CVE-2021-23424' (also known as 'GHSA-whgm-jr23-g3j9')."
},
"locations": [
{
"physicalLocation": {
"artifactLocation": {
"uri": "file://%%%%/fixtures/locks-many/package-lock.json"
}
}
}
]
}
]
}
]
}
`, version.OSVVersion),
wantStderr: `
Scanning dir ./fixtures/locks-many/package-lock.json
Scanned %%/fixtures/locks-many/package-lock.json file and found 1 package
`,
},
// output format: gh-annotations
{
name: "Empty gh-annotations output",
args: []string{"", "--format", "gh-annotations", "./fixtures/locks-many/composer.lock"},
wantExitCode: 0,
wantStdout: ``,
wantStderr: `
Scanning dir ./fixtures/locks-many/composer.lock
Scanned %%/fixtures/locks-many/composer.lock file and found 1 package
`,
},
{
name: "gh-annotations with vulns",
args: []string{"", "--format", "gh-annotations", "--config", "./fixtures/osv-scanner-empty-config.toml", "./fixtures/locks-many/package-lock.json"},
wantExitCode: 1,
wantStdout: ``,
wantStderr: `
Scanning dir ./fixtures/locks-many/package-lock.json
Scanned %%/fixtures/locks-many/package-lock.json file and found 1 package
::error file=fixtures/locks-many/package-lock.json::fixtures/locks-many/package-lock.json%0A+-----------+-------------------------------------+------+-----------------+---------------+%0A| PACKAGE | VULNERABILITY ID | CVSS | CURRENT VERSION | FIXED VERSION |%0A+-----------+-------------------------------------+------+-----------------+---------------+%0A| ansi-html | https://osv.dev/GHSA-whgm-jr23-g3j9 | 7.5 | 0.0.1 | 0.0.8 |%0A+-----------+-------------------------------------+------+-----------------+---------------+
`,
},
// output format: markdown table
Expand Down
2 changes: 2 additions & 0 deletions codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
ignore:
- internal/testutility
4 changes: 2 additions & 2 deletions internal/ci/fixtures/vulns/test-vuln-diff-a-a-1.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"results": [
{
"source": {
"path": "/home/rexpan/Documents/Projects/different-dir/go.mod",
"path": "/path/to/different-dir/go.mod",
"type": "lockfile"
},
"packages": [
Expand Down Expand Up @@ -82,4 +82,4 @@
]
}
]
}
}
4 changes: 2 additions & 2 deletions internal/ci/fixtures/vulns/test-vuln-diff-a-b.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"results": [
{
"source": {
"path": "/home/rexpan/Documents/Projects/scorecard-check-osv-e2e/go.mod",
"path": "/path/to/scorecard-check-osv-e2e/go.mod",
"type": "lockfile"
},
"packages": [
Expand Down Expand Up @@ -119,4 +119,4 @@
]
}
]
}
}
6 changes: 3 additions & 3 deletions internal/ci/fixtures/vulns/test-vuln-diff-c-b.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"results": [
{
"source": {
"path": "/home/rexpan/Documents/Projects/scorecard-check-osv-e2e/go.mod",
"path": "/path/to/scorecard-check-osv-e2e/go.mod",
"type": "lockfile"
},
"packages": [
Expand Down Expand Up @@ -120,7 +120,7 @@
},
{
"source": {
"path": "/home/rexpan/Documents/Projects/scorecard-check-osv-e2e/sub-rust-project/Cargo.lock",
"path": "/path/to/scorecard-check-osv-e2e/sub-rust-project/Cargo.lock",
"type": "lockfile"
},
"packages": [
Expand Down Expand Up @@ -326,4 +326,4 @@
]
}
]
}
}
4 changes: 2 additions & 2 deletions internal/ci/fixtures/vulns/test-vuln-results-a-1.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"results": [
{
"source": {
"path": "/home/rexpan/Documents/Projects/different-dir/go.mod",
"path": "/path/to/different-dir/go.mod",
"type": "lockfile"
},
"packages": [
Expand Down Expand Up @@ -83,7 +83,7 @@
},
{
"source": {
"path": "/home/rexpan/Documents/Projects/scorecard-check-osv-e2e/sub-rust-project/Cargo.lock",
"path": "/path/to/scorecard-check-osv-e2e/sub-rust-project/Cargo.lock",
"type": "lockfile"
},
"packages": [
Expand Down
4 changes: 2 additions & 2 deletions internal/ci/fixtures/vulns/test-vuln-results-a.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"results": [
{
"source": {
"path": "/home/rexpan/Documents/Projects/scorecard-check-osv-e2e/go.mod",
"path": "/path/to/scorecard-check-osv-e2e/go.mod",
"type": "lockfile"
},
"packages": [
Expand Down Expand Up @@ -83,7 +83,7 @@
},
{
"source": {
"path": "/home/rexpan/Documents/Projects/scorecard-check-osv-e2e/sub-rust-project/Cargo.lock",
"path": "/path/to/scorecard-check-osv-e2e/sub-rust-project/Cargo.lock",
"type": "lockfile"
},
"packages": [
Expand Down
Loading