SBOM improvements

[MikeMcQuaid]

• write a schema when installing formulae (if not already present)
• cache the schema on disk rather than downloading it every time
• make more methods/attributes private
• allow validation to be optional, only enable for Homebrew developers at installation time
• use the tab for more, correct information
• ensure that dependencies/bottles are written correctly
• use new SBOM 3 schema URL
• improve test coverage

[SMillerDev]

I'm not sure how useful the SBOM writing on install is since it's supposed to be a record of the build.

[MikeMcQuaid]

I'm not sure how useful the SBOM writing on install is since it's supposed to be a record of the build.

@SMillerDev My thinking is: it still records the build but this avoids the need to backfill this information into all bottles and also gets it when building from source.

[SMillerDev]

One thing I noticed (that might be worth checking here) is that the build time is always a 0 unix timestamp due to our reproducible builds. If you have any ideas for that, let me know. Otherwise maybe we add a TODO: figure out how to record the actual build time

[Bo98]

Depending on json_schemer on formula installs is a little messy since it depends on a native gem so we may want to visit whether we can have an alternative gem that doesn't do that.

[MikeMcQuaid]

Depending on json_schemer on formula installs

@Bo98 It's not doing that unless you've set HOMEBREW_DEVELOPER. I could make this even stricter and only do so on HOMEBREW_ENFORCE_SBOM.

[Bo98]

Yes, HOMEBREW_ENFORCE_SBOM would be nice as HOMEBREW_DEVELOPER is fairly liberally set in many CIs that don't install gems.

[MikeMcQuaid]

Yes, HOMEBREW_ENFORCE_SBOM would be nice as HOMEBREW_DEVELOPER is fairly liberally set in many CIs that don't install gems.

@Bo98 In this case it doesn't matter because the require? call will just fail. Would you rather the warning message is not printed in that case? What harm is it causing? What is the case/path you're concerned about causing issues with?

On reflection: HOMEBREW_DEVELOPER is definitely the right setting here. It already does stuff like e.g. raises in some places instead of catching the exception, raises deprecation failures rather than warnings, etc.

I'm all for trying to avoid breaking based on actual usage rather than documentation but HOMEBREW_DEVELOPER is something we always have and always will use as the front-line for stricter handing like this.

[MikeMcQuaid]

One thing I noticed (that might be worth checking here) is that the build time is always a 0 unix timestamp due to our reproducible builds. If you have any ideas for that, let me know. Otherwise maybe we add a TODO: figure out how to record the actual build time

• We could set this properly for the "there's no SBOM already" case
• We could put this in the bottle manifest (if we don't already, have a feeling we do) and then use that information to then populate the SBOM from the bottle manifest at pour time without harming reproducibility

Happy to add this on with a ✅ now and I'll add it in tomorrow.

[Bo98]

Would you rather the warning message is not printed in that case? What harm is it causing? What is the case/path you're concerned about causing issues with?

This + #17255 will lead to warning noise in some of our CI.

Example: https://github.com/Homebrew/brew/blob/master/.github/workflows/actionlint.yml

[Bo98]

Haven't combed through the code but overall idea looks great. Thanks for this ❤️

[MikeMcQuaid]

This + #17255 will lead to warning noise in some of our CI.

I've adjusted this to avoid printing any warnings when the gem is missing and instead lean in on the HOMEBREW_ENFORCE_SBOM when we want to actually error. Similarly, I've made brew install --build-bottle not attempt to validate the SBOM and instead do it at brew bottle time.

• We could put this in the bottle manifest (if we don't already, have a feeling we do) and then use that information to then populate the SBOM from the bottle manifest at pour time without harming reproducibility

Did this (but with Tab instead of bottle manifest)

[SMillerDev]

The relocating of functions is making the diff a bit hard to read, but generally this looks good

[SMillerDev]

One thing I see now in testing that I didn't consider before, we might need to URL encode the full name since it can contain @ too. @woodruffw you know the purl spec better. Is pkg:brew/homebrew/core/[email protected]@3.12.3 allowed?

[MikeMcQuaid]

@SMillerDev merged this to get the fixes in but: I'll open up a follow-up PR to address this once @woodruffw confirms 👍🏻