Is it possible to do highly user-centric JSON Schema validation?

[andrewpollock]

The purist in me would like to do OSV record correctness checks as JSON Schema validation as much as possible, however the main problem I'm seeing with this approach is you're completely at the mercy of the validation tooling used for the user experience.

I'm exploring https://pkg.go.dev/github.com/santhosh-tekuri/jsonschema/v5, which seems pretty decent, but even that spits out:

$ ~/go/bin/jv ~/gosst/osv/osv-schema/validation/schema.json ~/gosst/osv/osv-validation/lastaffected_and_fixed-CVE-2023-41045.json 
[I#] [S#] doesn't validate with https://raw.githubusercontent.com/ossf/osv-schema/main/validation/schema.json#
  [I#/affected/0/ranges/0] [S#/properties/affected/items/properties/ranges/items/allOf/1] allOf failed
    [I#/affected/0/ranges/0] [S#/properties/affected/items/properties/ranges/items/allOf/1/then] if-then failed
      [I#/affected/0/ranges/0] [S#/properties/affected/items/properties/ranges/items/allOf/1/then/not] not failed

In the spirit of 1. Focus on the user and all else will follow I can't in good conscience subject OSV's record provider users to this sort of user experience.

I had thought that maybe judicious use of title and/or description in sub-schemas would help, and maybe it does (I'm finding the documentation on validation about as difficult to understand as the validation output) but I think it also requires the validation tooling to do the right thing with the annotations, and it's not looking like relying on that to be the case universally is a winning strategy for usability.

[gregsdennis]

Personally, I think that's some of the better output I've seen. Each error gives you:

• the location within the instance
• the location within the schema

Most tools won't give you these data points, and they're pretty important.

Granted, it doesn't say why it failed... It could use some error messages.

---

That aside, there are a few things to conisder.

First, the spec-defined output formats are recommended, not required. They're still somewhat in development. In fact, for the next version they're changing them and we're removing them and giving them their own spec. Check out https://json-everything.net/json-schema to see what the next generation output should look like. Also, bear in mind that this output was originally intended to be read by machines and potentially translated into something more human-friendly by a consuming application.

(I think that last sentence is really the answer to the subject question.)

Second, collecting annotations is also a recommendation, not a requirement. Sadly, few implementations actually support it.

[andrewpollock]

Second, collecting annotations is also a recommendation, not a requirement. Sadly, few implementations actually support it.

I think this is key here and confirms my overall concern. I can't really jam all possible the record validation into the schema and then hope that any random validation implementation does the right, useful or user-helpful things when validating records against it, now or in the future, by the sounds of it. I'm going to have to create and rely on my own linter-style validator to do a comprehensive job.

[benjagm]

Hi everyone! Do you know about https://github.com/atlassian/better-ajv-errors?

This ajv complementary tool does a pretty good job with error reporting.

[andrewpollock]

Hi everyone! Do you know about https://github.com/atlassian/better-ajv-errors?

Thanks for the pointer, I did not. At this point in time I was looking for CLI solutions, this doesn't appear to have an out-of-the-box one?