Integrate with the JSON Schema CLI for formatting and linting on CI

[jviotti]

Hey there! This is a PR to mainly start a conversation. I'm a member of JSON Schema organization involved in maintaining various JSON Schema related tooling (like https://alterschema.sourcemeta.com) and documentation (like https://www.learnjsonschema.com), plus co-author of the recently published O'Reilly book touching on JSON Schema (https://www.oreilly.com/library/view/unifying-business-data/9781098144999/).

Based on our experience working with organizations maintaining repositories of schemas (like you!), we've been working on an open-source CLI tool (https://github.com/Intelligence-AI/jsonschema) that aims to simplify development and CI/CD when dealing with schemas. This CLI supports a growing amount of commands, but I'm proposing two for this repository to start with:

• Formatting: making sure your schemas following a consistent style (i.e. indentation) plus a consistent ordering of keywords that was designed to make schemas easier to read

• Linting: we are collecting common rules to highlight anti-patterns, common pitfalls, etc when using JSON Schema. There are not that many implemented yet (thought we are rapidly adding them!), but the linter already caught a little anti-pattern in Reporter: setting enum alongside type

I would love to hear your feedback, feature requests, etc to help you make better use of JSON Schema with this CLI! We also have other things on the way, like a proper JSON Schema test runner that could eventually simplify your AJV scripts!

[jviotti]

Most of the changes are the fmt command making the schemas consistent. Any feedback on the formatter would be hugely appreciated so that you like how it looks like too!

[jviotti]

BTW, I also added this repo to my https://github.com/sourcemeta/awesome-jsonschema

[lpsinger]

This is neat! We currently use prettier for formatting and ajv for linting. It's important that we can install our primary formatting and linting tools using npm because our pre-commit hooks depend upon that. Is packaging for npm on your roadmap?

[jviotti]

@lpsinger Glad you like it!

Is packaging for npm on your roadmap?

Yeah, I can certainly make that happen! I'll let you know when its up :)

[jviotti]

@lpsinger

and ajv for linting

Also on this: the JSON Schema team discourages the use of AJV, as despite being popular, it is not spec compliant and its use can lead to interoperability issues with other JSON Schema consumers. The JSON Schema team maintains Bowtie (https://bowtie.report), which is a real-time dashboard for spec compliance for a wide range of implementations.

There, you can see that AJV is actually one of the worst scorers for JSON Schema 2020-12 with 24% of official tests failing: https://bowtie.report/#/implementations/js-ajv

There are other good JavaScript ones you can use (like https://github.com/hyperjump-io/json-schema, by another TSC member) and our CLI will support the test runner on 2020-12 soon!

[jviotti]

@lpsinger Done! I believe this should work fine now:

npm install --global @intelligence-ai/jsonschema

Do you want me to add it to the dev dependencies and tweak the GitHub Actions workflow to use it from there?

[jviotti]

@lpsinger Friendly ping! Let me know if the NPM integration is working on your end and if you have any other requirement for making the tool useful to you. We are rapidly adding more linting rules and other niceties.

[lpsinger]

sourcemeta/jsonschema#76

[jviotti]

Done! @lpsinger v0.5.0 should work on macOS Intel. Let me know if you need anything else :)

[lpsinger]

I tried out your tool locally on one of our files, but it could not find one of the referenced schemas:

$ npx @intelligence-ai/jsonschema validate gcn/notice
s/glowbug/Alert.schema.json gcn/notices/glowbug/Alert.example.json
Could not resolve schema: https://gcn.nasa.gov/schema/main/gcn/notices/core/Reporter.schema.json

In Alert.schema.json, we refer to Reporter.schema.json using a relative path:

{
    "$ref": "../core/Reporter.schema.json",
    "description": "mission=GLOWBUG"
}

and we like that Ajv resolves that file using its relative path on the filesystem, because we want to validate against the local copy rather than actually fetching it from the Web on the main branch.

However, I get the same error even if I add the -h flag:

$ npx @intelligence-ai/jsonschema validate gcn/notices/glowbug/Alert.schema.json gcn/notices/glowbug/Alert.example.json -h
Could not resolve schema: https://gcn.nasa.gov/schema/main/gcn/notices/core/Reporter.schema.json

[jviotti]

Hi @lpsinger , which version of the tool are you using? I'm getting a different output with v0.5.1 (the latest at the time of this writing). Maybe you kept an old one?

In any case, while the CLI supports Draft 0 to 2020-12 in most of the commands, it does not support 2020-12 on the validate command yet (we are working on it!) That's why I suggested the adoption of at least fmt and lint to start with for this repo!

With v0.5.1, you should be getting a nicer error that 2020-12 vocabularies are not supported yet. i.e.:

$ jsonschema validate gcn/notices/glowbug/Alert.schema.json
Error: Cannot compile unsupported vocabulary: https://json-schema.org/draft/2020-12/vocab/applicator

and we like that Ajv resolves that file using its relative path on the filesystem, because we want to validate against the local > copy rather than actually fetching it from the Web on the main branch.

With this CLI, you do it with the --resolve option. i.e. you pass a directory to --resolve to pre-register schemas inside of it. Some implementations like AJV attempt to do file resolution, which is not strictly compliant. In theory, implementations are supposed to purely do URI resolution.

[jviotti]

One cool thing you could try on 2020-12 right now is bundling (inline remote references into a schema). For example, as you pointed out, gcn/notices/burstcube/alert.schema.json contains references to other schemas in core. You can get a variant of it that inlines all the relevant core schemas and print it to standard output as follows (make sure you are on v0.5.1 though!):

jsonschema bundle gcn/notices/burstcube/alert.schema.json --resolve gcn/notices/core --extension .schema.json

In comparison to some other tools that attempt to do this, the CLI does it following the standardised bundling procedure: https://json-schema.org/blog/posts/bundling-json-schema-compound-documents

We are working super hard to get coverage for every dialect, including 2020-12, on validate, so hopefully that lands very soon. The reason this is complex is that we are building a schema compiler for doing high-performance streaming validation (doing micro-second validation on uni-kernels), so a lot of complexity on supporting every dialect compared to more traditional JSON Schema evaluators. We'll get there though!

[jviotti]

As a tiny update, we definitely didn't forget about this. We are working hard to get 2020-12 working, and we are pretty close! Expect an update to this PR hopefully soon.