Create SBOMs for images pulled during package creation

[mikhailswift]

Description

Creates SBOMs for all images pulled during zarf package create. Stores them in a sboms directory within the package.

Related Issue

Fixes #22

Type of change

• New feature (non-breaking change which adds functionality)

Checklist before merging

• Tests have been added/updated as necessary (add the needs-tests label)
• Documentation has been updated as necessary (add the needs-docs label)
• An ADR has been written as necessary (add the needs-adr label) [ 1 2 3 ]
• (Optional) Changes have been linted locally with golangci-lint. (NOTE: We haven't turned on lint checks in the pipeline yet so linting may be hard if it shows a lot of lint errors in places that weren't touched by changes. Thus, linting is optional right now.)

[mikhailswift]

Relevant pr: in-toto/witness#167

[jeff-mccoy]

This is going to need some more work before we can merge it. We will need to have an ADR on the libs we pulled in (it adds multiple deps and increases the binary size by > 10%) as well as a test to verify it's being created. Also not sure what's going on with the go.mod, but it's enormous compared to master. Finally, we'll need some doc entry to explain what users should see. ADR won't be needed after the first SBOM PR and we can help with that if needed.

[jeff-mccoy]

As already stated, will need ADR/Docs/Tests and items discussed below. I also noticed SBOM generation can take quite a while, any comments on performance or possible future improvements for that?

[mikhailswift]

As already stated, will need ADR/Docs/Tests and items discussed below. I also noticed SBOM generation can take quite a while, any comments on performance or possible future improvements for that?

I've been working on the adr/docs/test, will get something in hsortly.

I can do some tracing to see what part of the process is taking time and identify areas for improvement.

[jeff-mccoy]

That would be super helpful, thanks!

[mikhailswift]

Ran some quick tests today profiling sbom generation on package-example-single-big-bang-package

Here's a flame graph of the sbom code on a run with a cold image cache

The total run time of sbom.CatalogImages was 11.19 seconds.

And here's a flame graph for the same package but with a hot cache

The total run time of sbom.CatalogImages was 4.57 seconds.

From the flame graphs it appears that it's rebuilding the image tar from the layer cache instead of just using the cache directly as I hoped it would. This explains why sbom generation is much faster on subsequent runs for the same image since the tar is already built.

Simplest fix will be to try to get the cataloger to use the layer cache directories directly instead of re-building the image's tar file.

[mikhailswift]

I spent the last bit chasing down performance improvements. I'll get the ADR and documentation in asap this week while I continue working tests and performance issues.

[jeff-mccoy]

This still needs docs/tests, any updates on your end @mikhailswift?

[jeff-mccoy]

BB Core SBOM, it runs--it's also > 200 MBs 😱🤣

[mikhailswift]

Overall this PR seems reasonable after messing with it for a couple of days, but I have concerns with adding SBOM and slowing down the package create stage without some visible utility to the user. So I propose we finish and merge this ASAP but not cut a release until we add something to view/use the SBOM. Here's something I've been playing around with the past couple of days that consumes our SBOM JSON data: https://www.loom.com/share/d85b02b13df442d7a0bae012307db3ee. Thoughts @mikhailswift?

Sound good to me, will get the docs & tests in asap this week.

I was looking at similar, either generating a static html page or starting a web server from zarf that allowed navigation of the sbom. Look like you got a good start on it.

I'm still investigating performance concerns regarding the first time building SBOMs for an image. It seems like syft is maintaining it's own cache and not using the layer cache that zarf creates on pull which is causing the slow downs. I think I may have a way forward on a solution.

[mikhailswift]

BB Core SBOM, it runs--it's also > 200 MBs screamrofl

wew. something i was considering at the grocery store today is that we could potentially generate the sboms on demand instead of packaging them if the size becomes an issue. main problem there is that currently syft doesn't support multi-image tar files, and of course it may not be an amazing user experience for the consumer of the sbom.

[jeff-mccoy]

TIL some ppl daydream about SBOM while getting groceries...

[mikhailswift]

TIL some ppl daydream about SBOM while getting groceries...

Send help please

[mikhailswift]

Okay, I'm doing some last rounds of testing but I believe I've got the performance of generating the SBOMs improved significantly:

Both of the below timings were measured with .zarf-image-cache erased for a cold run of the single-big-bang-package exampoe

Current code:

make package-example-single-big-bang-package 25.37s user 4.31s system 38% cpu 1:16.58 total

Not a precise measurement but watching the timer on the SBOM step seemed to take ~30 seconds

Changed:

make package-example-single-big-bang-package 23.31s user 3.43s system 57% cpu 46.313 total

Same imprecise measurement as above but the SBOM step seemed to take ~10 seconds.

Image pull on both examples took about 30 seconds. Of note is the increased CPU percentage: 38% of the elapsed time was spent on the CPU while 57% was spent on the CPU in the updated code -- indicating less time spent on IO

Looking at cpuprofile comparisons again,

Current code:

In this case image.Read is taking up ~30% of the cpu time in the CatalogImages function call, and there are some sneaky http and tls calls in there that lead me to believe it could actually be pulling layers over again.

Changed code:

In this case image.Read is taking ~22% of the cpu time in CatalogImages, but no tls calls happen.

Will have this + docs & tests pushed today

[mikhailswift]

added some basic docs and updates to the adr.

worked a bit on presentation: 1d5d02f

That is very much a hacky work in progress, though

[jeff-mccoy]

@Madeline-UX
Latest update with the SBOM Viewer thangz