Fixes to SBOM libraries

[puerco]

What type of PR is this?

/kind feature
/kind design
/kind bug

What this PR does / why we need it:

This PR compiles several fixes and features that need to be addressed to the sbom libraries after the first SBOM test after alpha.3. Namely, this PR has the following changes:

• Adds to bom the capability to configure the sbom components from a YAML file *more below)
• Allows more options to be passed to the SPDX document builder
• File analysis is now done in parallel speeding the kubernetes bom generation significally
• When generating a SPDX package from a directory, file paths will now be relative to the dir root
• Golang packages that have local replacements will be honored saving a considerable amount of downloads
• Fixed a bug where we would erase the local golang package install
• Fixed a bug where license data would be saved in the download cache directory, resulting in the license classifier having a lower accuracy
• Golang packages will now include all license text in the SBOM as well as the SPDX license identifier
• New function license.ReadTopLicense() will scan and return only the most significant license in a directory, potentially avoiding thousands of operations in the classifier code.

SBOM artifacts YAML definition file

Defining a bill of materials that has multiple packages, from different sources (files, images, directories, etc), can be cumbersome to do from the command line. This PR adds initial support to define a BOM in a declarative way from a yaml file. In this iteration, bom has a new flag -c --config which points to the yaml file. In future versions and use cases (for example when run from CI or GitHub actions), we could default bom to look for a .sbom.yaml file in a repository.

A sample of a yaml file looks like this:

---
namespace: http://www.example.com/
license: Apache-2.0
name: bom-test
creator:
    person: Kubernetes Release Managers ([email protected])
artifacts:
    - type: directory
      source: .
      license: Apache-2.0
      gomodules: true
    - type: file
      source: ./SECURITY.md
    - type: image
      source: k8s.gcr.io/kube-apiserver:v1.22.0-alpha.2
    - type: docker-archive
      source: tmp/sample-images/kube-apiserver.tar

To run bom with that file:

bom -c sbom.yaml

This configuration would render a SBOM with one loose file (SECURITY.md) and three packages: a directory, an image and a docker archive tarball.

Which issue(s) this PR fixes:

Part of #2085
Part of #1837

Special notes for your reviewer:

The format of the config file will be evolving as we start using it and finding new needs

Requires #2085 to merge and a rebase (only the last three commits are relevant)

Does this PR introduce a user-facing change?

* Allows more options to be passed to the SPDX document builder
* File analysis is now done in parallel speeding the kubernetes bom generation significally
* When generating a SPDX package from a directory, file paths will now be relative to the dir root
* Golang packages that have local replacements will be honored saving a considerable amount of downloads
* Fixed a bug where we would erase the local golang package install
* Fixed a bug where license data would be saved in the download cache directory, resulting in the license classifier having a lower accuracy
* Golang packages will now include all license text in the SBOM as well as the SPDX license identifier
* New function `license.ReadTopLicense()` will scan and return only the most significant license in a directory, potentially avoiding thousands of operations in the classifier code.

/milestone v1.22

[puerco]

Since this PR was on hold, I've pushed the next set of the SBOM fixes and features that surfaced after v1.22.0-alpha.3 in individual commits, each one explains in its message what it does. This PR is now ready for review.

I have tested these changes in a stage run here:
https://console.cloud.google.com/cloud-build/builds;region=global/ae1c6300-95f7-4ee9-bbc8-72d5a063a71a?folder=&organizationId=&project=kubernetes-release-test

[puerco]

/hold cancel

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: puerco, saschagrunert

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [puerco,saschagrunert]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment