Introduce new format pattern + port json processing #550
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Benchmark Test ResultsBenchmark results from the latest changes vs base branch |
5 tasks
wagoodman
commented
Oct 18, 2021
syft/format/format.go
Outdated
| return f.decoder(reader) | ||
| } | ||
|
|
||
| func (f Format) Detect(b []byte) bool { |
There was a problem hiding this comment.
From @luhring : could this be Validate ?
There was a problem hiding this comment.
(and if so, probably adjust the signature to be consistent with our new Validator concept (e.g. return an error))
wagoodman
commented
Oct 18, 2021
syft/format/format.go
Outdated
| } | ||
|
|
||
| if err := f.validator(bytes.NewReader(b)); err != nil { | ||
| return false |
There was a problem hiding this comment.
From @spiffcs : maybe log debug?
There was a problem hiding this comment.
logging here probably won't be helpful since we're expecting validation errors (which could get noisy)
spiffcs
approved these changes
Oct 18, 2021
| ) | ||
|
|
||
| // Encode takes all SBOM elements and a format option and encodes an SBOM document. | ||
| // TODO: encapsulate input data into common sbom document object |
There was a problem hiding this comment.
Want me to keep track of the TODO still found in this PR and break them into separate smaller issues to solve?
luhring
approved these changes
Oct 18, 2021
|
|
||
| import "io" | ||
|
|
||
| type Validator func(reader io.Reader) error |
There was a problem hiding this comment.
nit: For future implementers, it could be nice to add a doc comment to explain what's intended to be implemented in a Validator. E.g., // A Validator assess the input to determine blah blah blah ...
There was a problem hiding this comment.
agreed -- let me follow up with some useful doc strings
| return err | ||
| } | ||
| s.Target = payload | ||
| default: |
There was a problem hiding this comment.
nit: blank line above this line for consistency within the switch
| return fmt.Errorf("unable to decode: %w", err) | ||
| } | ||
|
|
||
| // note: we accept al schema versions |
internal/formats/formats.go
Outdated
|
|
||
| func Identify(by []byte) (*format.Format, error) { | ||
| for _, f := range All() { | ||
| if f.Detect(by) { |
There was a problem hiding this comment.
(Potentially changing this to create a new reader, based on a previous comment)
internal/formats/formats_test.go
Outdated
| ) | ||
|
|
||
| func TestIdentify(t *testing.T) { | ||
|
|
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
wagoodman
force-pushed
the
new-format-pattern
branch
from
d11e487 to
139c2b8
Compare
This was referenced Oct 21, 2021
Merged
GijsCalis
pushed a commit
to GijsCalis/syft
that referenced
this pull request
Feb 19, 2024
* add new format pattern Signed-off-by: Alex Goodman <[email protected]> * add syftjson format Signed-off-by: Alex Goodman <[email protected]> * add internal formats helper Signed-off-by: Alex Goodman <[email protected]> * add SBOM encode/decode to lib API Signed-off-by: Alex Goodman <[email protected]> * remove json presenter + update presenter tests to use common utils Signed-off-by: Alex Goodman <[email protected]> * remove presenter format enum type + add formats shim in presenter helper Signed-off-by: Alex Goodman <[email protected]> * add MustCPE helper for tests Signed-off-by: Alex Goodman <[email protected]> * update usage of format enum Signed-off-by: Alex Goodman <[email protected]> * add test fixtures for encode/decode tests Signed-off-by: Alex Goodman <[email protected]> * fix integration test Signed-off-by: Alex Goodman <[email protected]> * migrate format detection to use reader Signed-off-by: Alex Goodman <[email protected]> * address review comments Signed-off-by: Alex Goodman <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Partially addresses anchore/grype#395
Today we have the
presenterabstraction to write out internal SBOM results to a particular format (e.g. JSON, table, SPDX, CycloneDX). All model details about the format itself is contained under the presenter objects themselves. This abstraction was never meant to encapsulate handling intake (parsing) of these formats. As we add more formats it would be ideal to be able to parse these formats for use in other tools (such as syft). For this reason this PR adds a newformatabstraction for specifying encoding/decoding details for a particular SBOM format (and the presenter can use format encoders to deal with presentation concerns).The first commit of this PR adds the new format abstractions:
type Encoder ...: a function signature for encoding a given set of SBOM objects into bytes that are written to a writer.type Decoder ...: a function signature for decoding an SBOM from a reader and returning SBOM objects.type Validator ...: a function signature for observing the bytes of an SBOM document via a reader and returns any errors if the given document is not of a specific format.type Format ...: ties together the above bits of functionality into a single object tied to a specificformat.Option(with helper functions)This new pattern allows for:
This PR ports encoding from the syft presenters and decoding from the grype pkg.Provider implementation for the
syftjsonformat.Additionally the encode/decode behavior has been exposed in the syft top-level API , bypassing the presenter abstraction for the typical user (where deferred encoding is not necessary).