Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github action #420

Closed
wants to merge 56 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
21d1fe3
Markdown report output
another-rex May 22, 2023
0a10852
Add report output option
another-rex May 22, 2023
068ce6f
Add format helper function
another-rex May 25, 2023
4d4ed89
Merge remote-tracking branch 'upstream/main' into markdown-output
another-rex May 25, 2023
170ea18
Move reporter to centralize format
another-rex May 26, 2023
f41037c
Add sarif reporter
another-rex Jun 21, 2023
2f24c82
Merge branch 'main' into markdown-output
another-rex Jun 21, 2023
a6478f4
Test github action
another-rex Jun 21, 2023
d750a85
Readd return err with comment
another-rex Jun 21, 2023
d5429c4
Fix lints
another-rex Jun 21, 2023
49376ad
reorder
another-rex Jun 21, 2023
a503b0d
Fix args
another-rex Jun 21, 2023
ebdd3b5
update-image
another-rex Jun 22, 2023
f7e7525
print to stderr
another-rex Jun 22, 2023
27630d3
Fix sarif output
another-rex Jun 22, 2023
e0c3fad
Add recursive, disable osv-scanner ignore
another-rex Jun 22, 2023
fd815a0
Always upload
another-rex Jun 22, 2023
af3c71a
sarif report message with text
another-rex Jun 22, 2023
f578ebc
Fix markdown/text hybrid output
another-rex Jun 22, 2023
03b97ef
Re-enable filtering
another-rex Jun 22, 2023
ba5beff
Test working dir
another-rex Jun 23, 2023
21388d3
Another fix
another-rex Jun 23, 2023
d03e948
Merge remote-tracking branch 'upstream/main' into markdown-output
another-rex Jun 23, 2023
cdd7118
more action
another-rex Jun 23, 2023
1f921d3
testing with custom dockerfile
another-rex Jun 23, 2023
82d0c36
Fancy bash script
another-rex Jun 23, 2023
3450395
use actual image
another-rex Jun 23, 2023
bb27580
Fix yaml syntax error
another-rex Jun 23, 2023
56cbd27
Minor fix
another-rex Jun 23, 2023
b540dc6
Merge branch 'main' into markdown-output
another-rex Jun 23, 2023
45f3e52
Add test, refactor to make it more testable
another-rex Jun 26, 2023
d417ec9
Address PR comments
another-rex Jun 26, 2023
533876e
Fix action
another-rex Jun 26, 2023
0d55a3a
Print exit code
another-rex Jun 26, 2023
7902166
Add osv-scanner ignore file
another-rex Jun 26, 2023
821dfeb
Fix tests again
another-rex Jun 26, 2023
4e9b4a6
Account for FixedVersions having multiple packages
another-rex Jul 4, 2023
fc36f5e
Merge branch 'main' into markdown-output
another-rex Jul 4, 2023
cd0a96c
Add tests for the new grouping code
another-rex Jul 4, 2023
88356df
Remove comma in severity in table output
another-rex Jul 4, 2023
6eec552
Fix linter issues
another-rex Jul 4, 2023
a8e179a
Test reusable workflow and detecting changed files
another-rex Jul 4, 2023
0e64498
Fix yaml syntax
another-rex Jul 4, 2023
7a2df07
Fix indentation again
another-rex Jul 4, 2023
91444a5
Fix usage of reusable workflow
another-rex Jul 4, 2023
fae34e7
Attempt is made
another-rex Jul 4, 2023
8d994c8
Maybe it's a permission error
another-rex Jul 4, 2023
8ac0340
Quote the path
another-rex Jul 4, 2023
3ee828a
Fix yaml indentation... again.
another-rex Jul 4, 2023
9ab4a4f
Fix more indentation
another-rex Jul 4, 2023
c713ec8
Fix more errors
another-rex Jul 4, 2023
e2a427c
Add scanner run
another-rex Jul 5, 2023
c089132
echo last arg
another-rex Jul 5, 2023
8b7afec
exit_code_redirect split by space as well
another-rex Jul 5, 2023
6bd7ac3
Echo out split
another-rex Jul 5, 2023
aa90aa1
Finally fixed splitting
another-rex Jul 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions .github/workflows/osv-scanner-pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

name: OSV-Scanner PR scanning

# env:
# # Generator
# BUILDER_BINARY: slsa-generator-generic-linux-amd64 # Name of the binary in the release assets.
# BUILDER_DIR: internal/builders/generic # Source directory if we compile the builder.

# defaults:
# run:
# shell: bash

on:
workflow_call:

jobs:
scan-pr:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
persist-credentials: false
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files@v37

# To compare changes between the current commit and the last pushed remote commit set `since_last_remote_commit: true`. e.g
# with:
# since_last_remote_commit: true

- name: List all changed files
run: |
echo ${{ steps.changed-files.outputs.all_changed_files }}
echo ------------
for file in ${{ steps.changed-files.outputs.all_changed_files }}; do
echo "$file was changed"
done
- name: "Run scanner"
uses: ./ # Uses ./action.yaml
with:
results-format: sarif
results-file: results.sarif
to-scan: ${{ steps.changed-files.outputs.all_changed_files }}

75 changes: 75 additions & 0 deletions .github/workflows/osv-scanner.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

name: osv-scanner

on:
push:
branches: [ main ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ main ]
merge_group:
branches: [ main ]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make this a reusable workflow to let users easily use this? https://github.blog/2022-02-10-using-reusable-workflows-github-actions/

Disclaimer: I haven't looked too deeply into this. There's also composite actions which is something different.


# Declare default permissions as read only.
permissions: read-all

jobs:
scan-pr-attempt:
uses: "./.github/workflows/osv-scanner-pr.yml"
# scan:
# name: OSV-Scanner scan
# runs-on: ubuntu-latest
# permissions:
# # Needed to upload the results to code-scanning dashboard.
# security-events: write
# # Needed to publish results and get a badge (see publish_results below).
# id-token: write
# # Uncomment the permissions below if installing in a private repository.
# # contents: read
# # actions: read

# steps:
# - name: "Checkout code"
# uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
# with:
# persist-credentials: false

# - name: "Run scanner"
# uses: ./ # Uses ./action.yaml
# with:
# results-format: sarif
# results-file: results.sarif
# to-scan: |-
# ./
# ./cmd/osv-scanner/fixtures/locks-many/


# # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
# # format to the repository Actions tab.
# - name: "Upload artifact"
# if: '!cancelled()'
# uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
# with:
# name: SARIF file
# path: results.sarif
# retention-days: 5

# # Upload the results to GitHub's code scanning dashboard.
# - name: "Upload to code-scanning"
# if: '!cancelled()'
# uses: github/codeql-action/upload-sarif@6c089f53dd51dc3fc7e599c3cb5356453a52ca9e # v2.20.0
# with:
# sarif_file: results.sarif
42 changes: 42 additions & 0 deletions action.dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

FROM golang:alpine@sha256:fd9d9d7194ec40a9a6ae89fcaef3e47c47de7746dd5848ab5343695dbbd09f8c

RUN mkdir /src
WORKDIR /src

COPY ./go.mod /src/go.mod
COPY ./go.sum /src/go.sum
RUN go mod download

COPY ./ /src/
RUN go build -o osv-scanner ./cmd/osv-scanner/

FROM alpine:3.17@sha256:124c7d2707904eea7431fffe91522a01e5a861a624ee31d03372cc1d138a3126
RUN apk --no-cache add \
ca-certificates \
git \
bash

# Allow git to run on mounted directories
RUN git config --global --add safe.directory '*'

WORKDIR /root/
COPY --from=0 /src/osv-scanner ./
COPY ./exit_code_redirect.sh ./

ENV PATH="${PATH}:/root"

ENTRYPOINT ["bash", "/root/exit_code_redirect.sh"]
25 changes: 16 additions & 9 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,26 @@ name: 'osv-scanner'
description: 'Scans your directory against the OSV database (Experimental)'
inputs:
to-scan:
description: 'Directory to scan'
required: true
default: '/github/workspace'
version:
description: 'osv-scanner version to use'
description: 'Directories to scan'
required: false
default: "./"
results-file:
description: 'OUTPUT: Path to file to store results'
required: true
arch:
description: 'osv-scanner architecture'
results-format:
description: 'Output result format'
required: false
default: 'sarif'
recursive-search:
description: 'Recursively scan though subdirectories'
required: false
default: 'amd64'
default: false
runs:
using: 'docker'
image: 'ghcr.io/google/osv-scanner:${{ inputs.version }}-${{ inputs.arch }}'
image: 'action.dockerfile'
args:
- '--skip-git'
- '--output=${{ inputs.results-file }}'
- '--format=${{ inputs.results-format }}'
- '--recursive=${{ inputs.recursive-search}}'
- ${{ inputs.to-scan }}
4 changes: 4 additions & 0 deletions cmd/osv-scanner/fixtures/locks-many/osv-scanner.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[[IgnoredVulns]]
id = "GHSA-whgm-jr23-g3j9"
# ignore_until = 2022-11-09
reason = "Test manifest file"
9 changes: 9 additions & 0 deletions cmd/osv-scanner/fixtures/locks-many/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions cmd/osv-scanner/fixtures/osv-scanner-empty-config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# An empty config file to override the ignore config
51 changes: 34 additions & 17 deletions cmd/osv-scanner/main.go
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
package main

import (
"bytes"
"errors"
"fmt"
"io"
"os"
"strings"

"github.com/google/osv-scanner/pkg/osv"
"github.com/google/osv-scanner/pkg/osvscanner"
"github.com/google/osv-scanner/pkg/reporter"
"golang.org/x/exp/slices"

"github.com/urfave/cli/v2"
)
Expand All @@ -25,7 +28,7 @@ func run(args []string, stdout, stderr io.Writer) int {

cli.VersionPrinter = func(ctx *cli.Context) {
// Use the app Writer and ErrWriter since they will be the writers to keep parallel tests consistent
r = reporter.NewTableReporter(ctx.App.Writer, ctx.App.ErrWriter, false)
r = reporter.NewTableReporter(ctx.App.Writer, ctx.App.ErrWriter, false, 0)
r.PrintText(fmt.Sprintf("osv-scanner version: %s\ncommit: %s\nbuilt at: %s\n", ctx.App.Version, commit, date))
}

Expand Down Expand Up @@ -68,21 +71,22 @@ func run(args []string, stdout, stderr io.Writer) int {
Usage: "sets the output format",
Value: "table",
Action: func(context *cli.Context, s string) error {
switch s {
case
"table",
"json", //nolint:goconst
"markdown":
if slices.Contains(reporter.Format(), s) {
return nil
}

return fmt.Errorf("unsupported output format \"%s\" - must be one of: \"table\", \"json\", \"markdown\"", s)
return fmt.Errorf("unsupported output format \"%s\" - must be one of: %s", s, strings.Join(reporter.Format(), ", "))
},
},
&cli.BoolFlag{
Name: "json",
Usage: "sets output to json (deprecated, use --format json instead)",
},
&cli.StringFlag{
Name: "output",
Usage: "saves the result to the given file path",
TakesFile: true,
},
&cli.BoolFlag{
Name: "skip-git",
Usage: "skip scanning git repositories",
Expand Down Expand Up @@ -113,15 +117,15 @@ func run(args []string, stdout, stderr io.Writer) int {
format = "json"
}

switch format {
case "json":
r = reporter.NewJSONReporter(stdout, stderr)
case "table":
r = reporter.NewTableReporter(stdout, stderr, false)
case "markdown":
r = reporter.NewTableReporter(stdout, stderr, true)
default:
return fmt.Errorf("%v is not a valid format", format)
outputPath := context.String("output")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason to create an in-memory buffer instead of directly writing to the file ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this way it only writes it to file when osv-scanner succeeds, if it's writing directly to file it'll always create a file, but upon an error (e.g. the file it's pointed to scan doesn't exit, the flags passed in are wrong...etc) it'll exit early and nothing will be written to it.

Though now thinking about it, it might be surprising behavior, maybe it should just create an empty file in those cases.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, creating an empty file seems like reasonable behaviour. Would we still need this buffer in that case?

We could also alternatively just clear the file on an error to avoid this indirection with the buffer in that case?

outputBuffer := &bytes.Buffer{}
if outputPath != "" {
stdout = outputBuffer
}

var err error
if r, err = reporter.GetReporter(format, stdout, stderr, outputPath != ""); err != nil {
return err
}

vulnResult, err := osvscanner.DoScan(osvscanner.ScannerActions{
Expand All @@ -147,13 +151,26 @@ func run(args []string, stdout, stderr io.Writer) int {
return fmt.Errorf("failed to write output: %w", errPrint)
}

if outputPath != "" {
file, err := os.Create(outputPath)
if err != nil {
return fmt.Errorf("failed to create output file: %w", err)
}

_, err = file.Write(outputBuffer.Bytes())
if err != nil {
return fmt.Errorf("failed to write to output file: %w", err)
}
}

// Could be nil, VulnerabilitiesFoundErr, or OnlyUncalledVulnerabilitiesFoundErr
return err
},
}

if err := app.Run(args); err != nil {
if r == nil {
r = reporter.NewTableReporter(stdout, stderr, false)
r = reporter.NewTableReporter(stdout, stderr, false, 0)
}
if errors.Is(err, osvscanner.VulnerabilitiesFoundErr) {
return 1
Expand Down
Loading