Add producer API to write specs #233

elezar · 2024-10-15T19:15:01Z

This change proposes a "producer" API for generating and writing specs.

The more general cdi package imports this package when writing specs.

pkg/cdi/producer/producer.go

pkg/cdi/producer/options.go

klihub · 2024-10-16T08:40:50Z

pkg/cdi/producer/api.go

+)
+
+// a specValidator is used to validate a CDI spec.
+type specValidator interface {


nit: Shouldn't we export this interface, too (even if it is so trivial) ?

The package-provided Default and Disabled implementations are exported (and are supposed to be referenced directly as such, and not for instance indirectly by a name, etc.). I am guessing that we want to allow folks to plug in their own validation logic. Since (assuming those folks use the stock c-d-i package also for eventual interpretation/injection) custom validators should always be stricter than the default one (to ensure injection by the same package wouldn't fail). That suggests that most custom validators will be a composition of the default validator + some extra custom restrictions.

Now, it is already possible to do composition using the default validator, but if my aforementioned assumptions are true, it feels strange to not expose the interface itself.

My thinking was that one could plug in the schema validators here as well. I had some local changes that would expose those as interfaces instead of as raw function pointers.

I'm happy to expose the interface as public though to make the intent clearer and allow users to compose these as required.

Having the ability to plug in additional schema-based validation here feels like a good idea.

elezar · 2024-10-22T05:38:05Z

@klihub I simplified this PR a bit to start with. If you're ok with the general direction, we can get this in and then move some of the other functionalty (annotations, device name generation) to this package as well.

I would also like to add functionality to allow the minimum verison to be determined at save, but would rather do that as a follow up.

bart0sh · 2024-10-23T10:20:46Z

pkg/cdi/producer/api.go

+// A SpecValidator is used to validate a CDI spec.
+type SpecValidator interface {
+	Validate(*cdi.Spec) error
+}


Would it make sense to move this to the specs-go package? Validation is generally usable on both producer and consumer side, I guess.

I don't want to add unnecessary things to the specs-go package. I think we made the call that code to determine the minimum version was close enough to the spec to implement there, but I think an operation on the spec such as validation is not.

Note that due to the nature of go interfaces, this type need not be exported at all -- although it may be useful if it is.

Sorry for not being clear with this. I didn't mean moving only SpecValidator interface. I was talking about moving the whole validator package there. The idea was that it doesn't logically belong to producer. Having it here looks a bit confusing to me.

There is at least one reason not to move it to the spec module though. If it brings a lot of additional external dependencies to the spec package, we should probably avoid doing this.

@klihub @elezar WDYT?

I would rather suggest that we move the version specifics out of specs-go implement them as part of the "producer" package. Ideally we would implement this as a submodule as well so that we can properly separate the dependencies.

In the end we will have specs-go defining the actual data structures and helpers around this -- including validation. Implemented in other packages / submodules.

Sorry for not being clear with this. I didn't mean moving only SpecValidator interface. I was talking about moving the whole validator package there. The idea was that it doesn't logically belong to producer. Having it here looks a bit confusing to me.

There is at least one reason not to move it to the spec module though. If it brings a lot of additional external dependencies to the spec package, we should probably avoid doing this.

@klihub @elezar WDYT?

Yes, I had the same gut feeling. As long as the producer and the consumer were under a single package, having the validator as a subpackage thereof felt like a logical choice. Now if we split them, it does not feel so natural to have it under either... so maybe a pkg/validator. This can be adjusted with subsequent PRs, too.

I was actually wondering whether we shouldn't make this split at a module level so as to better manage dependencies. Meaning that we'd have:

tags.cncf.io/container-device-interface/api/v1/consumer tags.cncf.io/container-device-interface/api/v1/producer

as top-level go submodules.

For the producer specifically, we would start out with:

tags.cncf.io/container-device-interface/api/v1/producer/cdi

with the types in this package.

We could keep the existing pkg/cdi package to provide backward compatibility.

A client that produces specs would then be something like:

package main import ( "tags.cncf.io/container-device-interface/api/v1/producer/cdi" ) func main() { raw := generateRawCDISpec() saver, _ := cdi.New(raw, cdi.WithPermissions(0644)) err := saver.Save("/var/run/cdi/example.json) }

A client that produces specs would then be something like:

package main import ( "tags.cncf.io/container-device-interface/api/v1/producer/cdi" ) func main() { raw := generateRawCDISpec() saver, _ := cdi.New(raw, cdi.WithPermissions(0644)) err := saver.Save("/var/run/cdi/example.json) }

As abstractions go, it feels a bit artificial to have a 1-to-1 association between a producer and a Spec, IOW to essentially make SpecProducer a simple Spec wrapper. I would expect a producer to produce one or more Spec files, typically with the same file format and permissions (at least by default). So maybe I'd allow passing those as options to the constructor, and rather pass the raw spec to be written to a Save or some similar function of the producer, but not to its constructor itself.

That's a fair point. One question in this regard, let's say that the producer is configured to set the minimum version of the spec on save is it OK to modify the spec in-place?

For the producer specifically, we would start out with: tags.cncf.io/container-device-interface/api/v1/producer/cdi

It'd look a bit inconsistent that producer and consumer APIs have a bit different paths. I'd propose to use
tags.cncf.io/container-device-interface/api/v1/producer and tags.cncf.io/container-device-interface/api/v1/consumer as you've proposed above.

klihub · 2024-10-25T05:45:35Z

pkg/cdi/producer/producer.go

+		pWithFormat, err := cloneWithOptions(p, WithSpecFormat(format))
+		if err != nil {
+			return "", err
+		}
+		return pWithFormat.Save(filename)


nit: again, this it not a biggie, but it feels a bit weird, having to 'clone' the producer just to change the format... and clone is only used for this.

An initial implementation that I had was to have a separate type to do the writing / saving and to construct this when Save is called. Let me look at it a bit again and see if I can come up with a better abstraction.

klihub · 2024-10-25T05:47:30Z

pkg/cdi/producer/options.go

+}
+
+// WithSpecFormat sets the output format of a CDI specification.
+func WithSpecFormat(format SpecFormat) Option {


nit: Perhaps call it WithDefaultSpecFormat()... now it is possible to configure a writer supposedly for a format, which is then overridden by the filename extension if it does not match... so semantically this is a default format.

Note that the type also exposes a WriteTo(io.Writer) function which is where this spec format is used. As you note, the Save(string) function does override the format to be consistent with the filename.

Happy to change the method name though.

Signed-off-by: Evan Lezar <[email protected]>

This change adds a SpecProducer that can be used by clients that are only concerned with outputing specs. A spec producer is configured on construction to allow for the default output format, file permissions, and spec validation to be specified. Signed-off-by: Evan Lezar <[email protected]>

bart0sh · 2024-11-27T17:14:36Z

@elezar @klihub I thought that we agreed to introduce validator package as a separate PR, no?

klihub · 2024-11-27T17:43:29Z

@elezar @klihub I thought that we agreed to introduce validator package as a separate PR, no?

That was also my understanding.

elezar requested a review from klihub October 15, 2024 19:15

elezar force-pushed the producer-api branch 6 times, most recently from 39d1b55 to 1282f64 Compare October 15, 2024 23:05

klihub reviewed Oct 16, 2024

View reviewed changes

pkg/cdi/producer/producer.go Outdated Show resolved Hide resolved

klihub reviewed Oct 16, 2024

View reviewed changes

pkg/cdi/producer/producer.go Outdated Show resolved Hide resolved

klihub reviewed Oct 16, 2024

View reviewed changes

pkg/cdi/producer/options.go Outdated Show resolved Hide resolved

klihub reviewed Oct 16, 2024

View reviewed changes

elezar force-pushed the producer-api branch 4 times, most recently from 3205f41 to 7ef814c Compare October 22, 2024 05:36

elezar force-pushed the producer-api branch from 7ef814c to 1e3a085 Compare October 22, 2024 05:39

elezar marked this pull request as ready for review October 22, 2024 05:39

elezar requested review from kad, klueska, bart0sh, marquiz and zvonkok October 22, 2024 05:39

elezar self-assigned this Oct 22, 2024

elezar force-pushed the producer-api branch from 1e3a085 to 609fad4 Compare October 22, 2024 05:41

elezar requested a review from klihub October 22, 2024 09:37

bart0sh reviewed Oct 23, 2024

View reviewed changes

klihub reviewed Oct 25, 2024

View reviewed changes

bart0sh mentioned this pull request Nov 7, 2024

Drive CDI specification to v1.0.0 #206

Open

18 tasks

elezar force-pushed the producer-api branch 5 times, most recently from a34dc06 to a66a415 Compare November 25, 2024 16:05

elezar added 2 commits November 26, 2024 21:54

Add api/validator submodule

5dc8945

Signed-off-by: Evan Lezar <[email protected]>

elezar force-pushed the producer-api branch from a66a415 to d18f1c1 Compare November 26, 2024 20:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add producer API to write specs #233

Add producer API to write specs #233

elezar commented Oct 15, 2024 •

edited

Loading

klihub Oct 16, 2024

elezar Oct 22, 2024

klihub Oct 25, 2024

elezar commented Oct 22, 2024

bart0sh Oct 23, 2024 •

edited

Loading

elezar Oct 24, 2024

bart0sh Oct 24, 2024

elezar Oct 24, 2024

klihub Oct 25, 2024

elezar Oct 26, 2024

klihub Oct 28, 2024 •

edited

Loading

elezar Oct 28, 2024

bart0sh Nov 27, 2024

klihub Oct 25, 2024

elezar Oct 26, 2024

klihub Oct 25, 2024

elezar Oct 26, 2024

bart0sh commented Nov 27, 2024

klihub commented Nov 27, 2024

Add producer API to write specs #233

Are you sure you want to change the base?

Add producer API to write specs #233

Conversation

elezar commented Oct 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elezar commented Oct 22, 2024

bart0sh Oct 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

klihub Oct 28, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bart0sh commented Nov 27, 2024

klihub commented Nov 27, 2024

elezar commented Oct 15, 2024 •

edited

Loading

bart0sh Oct 23, 2024 •

edited

Loading

klihub Oct 28, 2024 •

edited

Loading