feat(core): support JSON ruleset validation

[P0lip]

A small subset of changes from #2026
The only change shere are:

• Added JSON Schema definitions that are suitable for JSON/YAML rulesets - going to be useful internally, particularly in the case of the ruleset-migrator package. Due to that, assertValidRuleset accepts a second parameter, namely format.
• Moved JSON Schema definitions describing rulesets from src/meta to src/ruleset/meta
• Setup package exports so it's easier to import the validator

The remaining changes such as the addition of AggregateError, changes around typings, some bug fixes, etc. aren't a part of this PR.

Checklist

• Tests added / updated
• Docs added / updated

Does this PR introduce a breaking change?

• Yes
• No

[P0lip]

this is not a new code, it was copied over from packages/core/src/ruleset/validation.ts

[P0lip]

copied over from packages/core/src/ruleset/validation.ts, the only change is the addition of format in assertValidRuleset

[P0lip]

Largely copied from packages/core/src/ruleset/validation.ts with a few subtle adjustments to account for 2 different rulesets formats.

[Nezteb]

@P0lip @mmiask This PR is also somewhat old; do we have a priority on it?

[mallachari]

Looks good!
If it's still relevant.

[stoplight-spectral-bot]

🎉 This PR is included in version @stoplight/spectral-core-v1.12.0 🎉

The release is available on npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

[tylerlong]

I believe this PR breaks the lib.

Now it complaints "node_modules/@stoplight/spectral-core/dist/ruleset/validation/assertions.js:10
throw new Error('Provided ruleset is not an object');"

Ruleset never changed. And the previous version of @stoplight/spectral-core works well

@P0lip @Nezteb

[P0lip]

@tylerlong I'm sorry to hear that! Would you mind sharing your ruleset with us?

[tylerlong]

Here you go:

extends: ["spectral:oas", "spectral:asyncapi"]
rules:
  ensure-paths-kebab-case:
    description: Paths must be in kebab-case.
    message: '{{description}} (lower case and separated with hyphens)'
    formats: [oas3]
    type: style
    severity: error
    given: $.paths[*]~
    then:
      function: pattern
      functionOptions:
        match: "^(\/|[a-z0-9-.]+|{[a-zA-Z0-9]+})+$"

[P0lip]

Hm, I don't seem to be able to reproduce the issue using the latest.
Would you mind sharing the JS piece of code that you have? Or, if that's possible, a reproducible sample?

[tylerlong]

Do you know how to run a vscode extension: https://github.com/ringcentral/vscode-openapi-linter
In this project, if you upgrade the lib to latest version you will reproduce the issue.

If you don't want to run it, you may read the code here:
https://github.com/ringcentral/vscode-openapi-linter/blob/main/server/src/server.ts

[tylerlong]

Let me find some time to create a simple demo project.

[tylerlong]

Here is the simple project to reproduce it https://github.com/tylerlong/spectral-demo

How to run the project?

yarn install
yarn test

After upgrading to "@stoplight/spectral-core": "^1.12.0", the first issue is:

/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:820
    return new TSError(diagnosticText, diagnosticCodes);
           ^
TSError: ⨯ Unable to compile TypeScript:
src/index.ts:45:23 - error TS2345: Argument of type 'Ruleset' is not assignable to parameter of type 'RulesetDefinition | Ruleset'.
  Type 'import("/Users/tyler.liu/src/ts/spectral-demo/node_modules/@stoplight/spectral-ruleset-bundler/node_modules/@stoplight/spectral-core/dist/ruleset/ruleset").Ruleset' is not assignable to type 'import("/Users/tyler.liu/src/ts/spectral-demo/node_modules/@stoplight/spectral-core/dist/ruleset/ruleset").Ruleset'.
    Property '#private' in type 'Ruleset' refers to a different member that cannot be accessed from within type 'Ruleset'.

45   spectral.setRuleset(customRules);
                         ~~~~~~~~~~~

    at createTSError (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:820:12)
    at reportTSError (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:824:19)
    at getOutput (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1014:36)
    at Object.compile (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1322:43)
    at Module.m._compile (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1454:30)
    at Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Object.require.extensions.<computed> [as .ts] (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1458:12)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) {
  diagnosticCodes: [ 2345 ]
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I changed line 45 from spectral.setRuleset(customRules); to spectral.setRuleset(customRules as any); to workaround the compiling issue.

Then there is this issue:

yarn test
yarn run v1.22.18
$ ts-node src/index.ts
/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:820
    return new TSError(diagnosticText, diagnosticCodes);
           ^
TSError: ⨯ Unable to compile TypeScript:
src/index.ts:3:9 - error TS2305: Module '"@stoplight/spectral-ruleset-bundler"' has no exported member 'bundleAndLoadRuleset'.

3 import {bundleAndLoadRuleset} from '@stoplight/spectral-ruleset-bundler';
          ~~~~~~~~~~~~~~~~~~~~

    at createTSError (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:820:12)
    at reportTSError (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:824:19)
    at getOutput (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1014:36)
    at Object.compile (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1322:43)
    at Module.m._compile (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1454:30)
    at Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Object.require.extensions.<computed> [as .ts] (/Users/tyler.liu/src/ts/spectral-demo/node_modules/ts-node/src/index.ts:1458:12)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) {
  diagnosticCodes: [ 2305 ]
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Root cause is TypeScript: microsoft/TypeScript#33079

I workaround it by deleting "exports" section from package.json of @stoplight/spectral-ruleset-bundler and import {xxx} from @stoplight/spectral-ruleset-bundler/dist/loader/node.

Then the error is:

yarn run v1.22.18
$ ts-node src/index.ts
Error: Provided ruleset is not an object
    at assertValidRuleset (/Users/tyler.liu/src/ts/spectral-demo/node_modules/@stoplight/spectral-core/src/ruleset/validation/assertions.ts:11:11)
    at new Ruleset (/Users/tyler.liu/src/ts/spectral-demo/node_modules/@stoplight/spectral-core/src/ruleset/ruleset.ts:63:25)
    at Spectral.setRuleset (/Users/tyler.liu/src/ts/spectral-demo/node_modules/@stoplight/spectral-core/src/spectral.ts:101:59)
    at main (/Users/tyler.liu/src/ts/spectral-demo/src/index.ts:45:12)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Which I believe is a new bug cause by this PR.

Thanks for your time.

@P0lip

[P0lip]

Hey @tylerlong.
Thank you so much for getting back to me with a very thorough example.

Speaking of the first issue, you can run npx yarn-deduplicate --scopes @stoplight && yarn to get rid of it.
as any shouldn't needed once the commands above get executed.

Furthermore, it's likely the error you're experiencing will be resolved altogether, since I'm able to run the code just fine

yarn run v1.22.17
$ ts-node src/index.ts
[
  {
    "code": "oas3-server-trailing-slash",
    "message": "Server URL must not have trailing slash.",
    "path": [
      "servers",
      "0",
      "url"
    ],
    "severity": 1,
    "source": "spec.yml",
    "range": {
      "start": {
        "line": 13,
        "character": 9
      },
      "end": {
        "line": 13,
        "character": 42
      }
    }
  },
  {
    "code": "ensure-paths-kebab-case",
    "message": "Paths must be in kebab-case. (lower case and separated with hyphens)",
    "path": [
      "paths",
      "/rest_api"
    ],
    "severity": 0,
    "source": "spec.yml",
    "range": {
      "start": {
        "line": 17,
        "character": 12
      },
      "end": {
        "line": 36,
        "character": 26
      }
    }
  }
]
Done in 2.74s.

LMK if that did the trick.

[tylerlong]

I think I have found the root cause. It can only be reproduced in some scenarios.

spectral/packages/core/src/spectral.ts

Line 101 in 3baba29

this.ruleset = ruleset instanceof Ruleset ? ruleset : new Ruleset(ruleset);

ruleset instanceof Ruleset is problematic.

Especially when you need to compile the code to js first (or even more complicated, bundle the code using some kind of tools like webpack).

Especially when you rule set is loaded by @stoplight/spectral-ruleset-bundler/dist/loader/node which has different class definition for the RuleSet.

Yes npx yarn-deduplicate --scopes @stoplight && yarn does fix the issue.