Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable importing schema v6 grype archives #441

Merged
merged 3 commits into from
Dec 4, 2024
Merged

Enable importing schema v6 grype archives #441

merged 3 commits into from
Dec 4, 2024

Conversation

wagoodman
Copy link
Contributor

@wagoodman wagoodman commented Dec 4, 2024

This allows for processing of v1-5 and v6 archives. V6 archives are distributed in tar.zstd archives and have no metadata.json to read from. What is needed is the ability to get a unique ID for the DB being used, so calculating the checksum of the archive was done.

Sample run:

❯ yardstick -vvv result capture -r default
2024-12-04 17:09:36,663 [INFO] capturing data result_set=default
2024-12-04 17:09:36,663 [DEBUG] loading result set 'default' location='.yardstick/result/sets/default.json'
2024-12-04 17:09:36,664 [INFO] capturing data for request 1 of 1
2024-12-04 17:09:36,664 [DEBUG] capturing data image=docker.io/anchore/test_images:grype-quality-python-d89207b@sha256:b2b58a55c0b03c1626d2aaae2add9832208b02124dda7b7b41811e14f0fb272c tool=grype@main+import-db=/Users/wagoodman/Downloads/vulnerability-db_v6.0.0_2024-12-03T01:31:03Z_1733340937.tar.zst profile=None
2024-12-04 17:09:36,664 [DEBUG] capturing via run config image=docker.io/anchore/test_images@sha256:b2b58a55c0b03c1626d2aaae2add9832208b02124dda7b7b41811e14f0fb272c tool=grype@main+import-db=/Users/wagoodman/Downloads/vulnerability-db_v6.0.0_2024-12-03T01:31:03Z_1733340937.tar.zst
2024-12-04 17:09:37,811 [DEBUG] parsed import-db='/Users/wagoodman/Downloads/vulnerability-db_v6.0.0_2024-12-03T01:31:03Z_1733340937.tar.zst' from version='main+import-db=/Users/wagoodman/Downloads/vulnerability-db_v6.0.0_2024-12-03T01:31:03Z_1733340937.tar.zst' new version='main'
2024-12-04 17:09:37,811 [DEBUG] installing grype version='main' from git
2024-12-04 17:09:37,811 [DEBUG] cloning the grype git repo: 'github.com/anchore/grype'
2024-12-04 17:09:38,821 [DEBUG] found grype git description='v0.85.0-22-g37245b0'
2024-12-04 17:09:38,821 [DEBUG] installing grype to '/Users/wagoodman/code/yardstick/.yardstick/tools/grype/main/git_install/v0.85.0-22-g37245b0'
2024-12-04 17:09:38,821 [DEBUG] installing grype via build to '/Users/wagoodman/code/yardstick/.yardstick/tools/grype/main/git_install/v0.85.0-22-g37245b0'
2024-12-04 17:09:38,821 [DEBUG] running 'go build -ldflags "-w -s -extldflags \'-static\' -X github.com/anchore/grype/internal/version.version=v0.85.0-22-g37245b0" -o /Users/wagoodman/code/yardstick/.yardstick/tools/grype/main/git_install/v0.85.0-22-g37245b0 ./cmd/grype'
2024-12-04 17:09:44,342 [INFO] importing given (custom) db from '/Users/wagoodman/Downloads/vulnerability-db_v6.0.0_2024-12-03T01:31:03Z_1733340937.tar.zst'
Vulnerability database imported
2024-12-04 17:09:48,324 [DEBUG] running grype with input=docker.io/anchore/test_images:grype-quality-python-d89207b@sha256:b2b58a55c0b03c1626d2aaae2add9832208b02124dda7b7b41811e14f0fb272c
... (fails since we don't have matchers working yet)

With configs:

# .yardstick.yaml
result-sets:
  default:
    description: "..."
    matrix:
      images:
        - docker.io/anchore/test_images:grype-quality-python-d89207b@sha256:b2b58a55c0b03c1626d2aaae2add9832208b02124dda7b7b41811e14f0fb272c

      tools:
        - name: grype
          version: main+import-db=/Users/wagoodman/Downloads/vulnerability-db_v6.0.0_2024-12-03T01:31:03Z_1733340937.tar.zst



# .grype.yaml
db:
  validate-age: false

exp:
    dbv6: true

While working on this the precommit tool was telling me to update, so I did:

git push origin zstd-import                     
[WARNING] hook id `check-added-large-files` uses deprecated stage names (push) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.
[WARNING] hook id `check-docstring-first` uses deprecated stage names (push) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.
[WARNING] hook id `check-yaml` uses deprecated stage names (push) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.
[WARNING] hook id `check-json` uses deprecated stage names (push) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.
[WARNING] hook id `end-of-file-fixer` uses deprecated stage names (push) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.
[WARNING] hook id `trailing-whitespace` uses deprecated stage names (push) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.
[WARNING] hook id `check-merge-conflict` uses deprecated stage names (push) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.
[WARNING] hook id `check-ast` uses deprecated stage names (push) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.
[WARNING] hook id `check-case-conflict` uses deprecated stage names (push) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.
[WARNING] top-level `default_stages` uses deprecated stage names (push) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.

Related to anchore/grype-db#367

@wagoodman wagoodman self-assigned this Dec 4, 2024
@wagoodman wagoodman requested a review from a team December 4, 2024 22:14
@wagoodman wagoodman added the enhancement New feature or request label Dec 4, 2024
@wagoodman wagoodman merged commit a992d49 into main Dec 4, 2024
3 checks passed
@wagoodman wagoodman deleted the zstd-import branch December 4, 2024 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants