Start to scan all distroless images with Trivy

[mattmoor]

With 0.3.1 Trivy is able to recognize that we're an Alpine image:

2022-04-13T23:40:25.809Z	INFO	Need to update DB
2022-04-13T23:40:25.809Z	INFO	Downloading DB...
2022-04-13T23:40:27.895Z	INFO	Detected OS: alpine
2022-04-13T23:40:27.895Z	INFO	Detecting Alpine vulnerabilities...
2022-04-13T23:40:27.896Z	INFO	Number of language-specific files: 0

This change will make it so that all of the distroless releases will scan the image (you can control whether this is fatal with trivy-exit-code: '0'.

Signed-off-by: Matt Moore [email protected]

[mattmoor]

@kaniini I YOLO'd all of them here, but we can tune this based on what we reasonable expect to be fixed quickly in the feeds.

[mattmoor]

Actually, if I drop the os-release stanza I'd added I get:

2022-04-13T23:48:27.323Z	INFO	Need to update DB
2022-04-13T23:48:27.323Z	INFO	Downloading DB...
2022-04-13T23:48:29.299Z	INFO	Detected OS: alpine
2022-04-13T23:48:29.299Z	WARN	This OS version is not on the EOL list: alpine 3.16
2022-04-13T23:48:29.299Z	INFO	Detecting Alpine vulnerabilities...
2022-04-13T23:48:29.299Z	INFO	Number of language-specific files: 0
2022-04-13T23:48:29.299Z	WARN	This OS version is no longer supported by the distribution: alpine 3.16
2022-04-13T23:48:29.299Z	WARN	The vulnerability detection may be insufficient because security updates are not provided

I believe this is where @knqyf263 work will help? 🤞

[mattmoor]

I think the way I want this to evolve is as follows:

1. Start to have the Trivy stuff emit SARIF format findings,
2. Start to upload the SARIF files to github codeql,
3. Start to attest with --type vuln using the SARIF result,
4. (once Ville's fixes land) Start to use cosign verify-attestation --type vuln with a Cue policy to determine the failure criteria we want.

[mattmoor]

Ok, I added 1/2 here, and TODOs for the other bits.

[mattmoor]

It looks like uploading CodeQL results need contents: write permission, so we should update those before landing this.

[mattmoor]

Hmm, even with that is fails 🤔

[mattmoor]

I'll back out the codeql stuff for now...

[knqyf263]

I believe this is where @knqyf263 work will help? 🤞

Yes, we're trying to capture Alpine version from /etc/apk/repositories when /etc/os-version and /etc/alpine-version don't exist. As you can see in the following PR, it works in my local. We'll merge it and release a new version today if we don't see any issues.
aquasecurity/trivy#1987

[mattmoor]

Alright, I have a basic form of this that will start publishing vulnerability attestations as well, I'm going to pull that in here as a second commit.

[mattmoor]

@knqyf263 interesting, so @kaniini added support for emitting /etc/alpine-release as a workaround, but I think it's getting 3.16 not edge, so I suspect our workaround has unfortunately collided with your fix, and our current release may actually be incompatible 😅

@kaniini should we change things to emit edge here instead? I believe it's based on os-release.id so perhaps once the new trivy drop lands, I can try specifying edge there, but we should sort out the right way to do this longer term.

Thank you both for the quick fixes here 🙏

[knqyf263]

As I talked with @kaniini, I believe edge should be detected.
aquasecurity/trivy#1975 (comment)

Otherwise, we can not know which stream should be used since v3.16 doesn't exist yet in secdb.
https://secdb.alpinelinux.org/

[mattmoor]

@knqyf263 Ok, I think the latest ghcr.io/distroless/git image was rebuilt with her latest changes this morning, so if those are detected properly then I think we are golden 🤩

[mattmoor]

Here are some of the relevant files:

$ tar xf foo.tar -O etc/alpine-release
3.16

$ tar xf foo.tar -O etc/os-release
ID=alpine
NAME="apko-generated image"
PRETTY_NAME="apko-generated image"
VERSION_ID=3.16
HOME_URL="https://github.com/chainguard-dev/apko"

$ tar xf foo.tar -O etc/apk/repositories
https://dl-cdn.alpinelinux.org/alpine/edge/main
https://dl-cdn.alpinelinux.org/alpine/edge/community

# no /etc/secfixes.d yet

I defer to you two on the best course forward here, I just wanted to call out the (potential) inconsistency early, so we could sort it out if it were a problem.

[knqyf263]

We can't detect it now😄 As @mattmoor pointed out, we are inconsistent now.

ghcr.io/distroless/git (alpine 3.16)
====================================
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)

Currently /etc/alpine-release is prioritized than /etc/apk/repositories. If @kaniini thinks /etc/alpine-release should be 3.16, I guess there are two approaches on my end.

1. Prefer /etc/apk/repositories than /etc/os-release and /etc/apk-release
2. Assume v3.16 is edge and use the edge stream in secdb.

Or, you can put edge to /etc/os-release and /etc/apk-release instead of v3.16. Which is the best?

[knqyf263]

After thinking about it for a while, I realized that it is better to use the version of the repository for vulnerability detection, regardless of the OS version. This is because packages are installed according to the repository settings and the secdb stream should be determined from where a package is installed. Please correct me if I'm wrong.

It might be problematic if user installs some packages from 3.14 and others from 3.15, for example. But I didn't find repository info in /lib/apk/db/installed for each package.

[kaniini]

@kaniini should we change things to emit edge here instead? I believe it's based on os-release.id so perhaps once the new trivy drop lands, I can try specifying edge there, but we should sort out the right way to do this longer term.

No, /etc/alpine-release is derived from the alpine-base package version. 3.16 is right for edge at the moment.

[kaniini]

After thinking about it for a while, I realized that it is better to use the version of the repository for vulnerability detection, regardless of the OS version. This is because packages are installed according to the repository settings and the secdb stream should be determined from where a package is installed. Please correct me if I'm wrong.

In the long term, the idea is that scanners would consume /etc/secfixes.d files to determine the feeds to use for scanning an image. I plan to write a reference implementation of this soon, but I am busy with the Alpine 3.16 release at the moment.

It might be problematic if user installs some packages from 3.14 and others from 3.15, for example. But I didn't find repository info in /lib/apk/db/installed for each package.

Mixing Alpine versions results in an unsupported configuration. I think Trivy should warn about this.

[knqyf263]

Just FYI, v0.26.0 is out and /etc/os-release and /etc/alpine-release are no longer needed as a version is taken from /etc/apk/repositories. You can drop them off if you want.
aquasecurity/trivy#2001

Also, once @kaniini writes a reference about /etc/secfixes.d, we will follow it. Meanwhile, we will look into some examples to understand the concept.

[mattmoor]

@knqyf263 when do you think this will land in the trivy action?

[knqyf263]

It would be in a few days if we don't see any issue.

[mattmoor]

ok cool, I will circle back next week. I want to make sure this is WAI once that lands.

[knqyf263]

Done
https://github.com/aquasecurity/trivy-action/releases/tag/0.2.5

[mattmoor]

Seems to work, thanks!

[tuananh-finx]

@mattmoor can we disable trivy ? i'm using private repo and it's unable to fetch the image.