Improve logging output from Nimma when a custom function errors

[SensibleWood]

User story.

As a user of Spectral
When I write a custom function that is erroneous
I want to see more verbose logging than "Error running Nimma"
That helps me diagnose the problem with my code

Is your feature request related to a problem?

I've across to v6 of Spectral and gone about rewriting my custom functions in the revised style. In doing so I (naturally) wrote some poor code that was incorrect.

However, Spectral told me virtually nothing about my mistake other than telling me "Error running Nimma". After poking around in the Nimma package I found that the errors are correctly collected, but not echoed out to the console in any useful way. I added console.error(__filename, this.errors) in a couple of places and Bob's your uncle and Fanny is your mother's sister I had the logging I need.

Describe the solution you'd like

I'm sure these logs are available to be echoed out so some means to see them when they happen would be very useful

Additional context

Given this clearly broken function:

import { createRulesetFunction } from '@stoplight/spectral-core';

export default createRulesetFunction({
  input: null,
  options: {
    type: 'object',
    additionalProperties: false,
    properties: {
      value: true,
    },
    required: ['value'],
  },
}, (input, options) => {
  const { value } = options;

  const results = input.filter((server) => server.url.brokenFunction);

  if (results.length !== 0) {
    return [
      {
        message: `This is just for debug`,
      },
    ];
  }
});

And this here ruleset:

extends: spectral:oas
functions: [empty]
rules:
  operation-operationId: off
  oas3-api-servers: off
  info-contact: off
  test-empty:
    message: This is broken
    severity: error
    given: "$.servers"
    then:
      function: empty
      functionOptions:
        value: "abc"

I get this result even with the --verbose flag on:

➜  spectral-ruleset git:(develop) ✗ ./node_modules/.bin/spectral lint --ruleset rulesets/debug.yaml test/broken-build.yaml.txt --verbose
Found 51 rules (37 enabled)
Linting /Users/chris/Documents/git/github/spectral-ruleset/test/broken-build.yaml.txt
Error running Nimma
➜  spectral-ruleset git:(develop) ✗

[P0lip]

I agree!
I've actually have some work left on it -> https://github.com/stoplightio/spectral/tree/feat/cli/rich-errors
I'll try to open a PR soon-ish

[padamstx]

We're also seeing reports of this error with the IBM openapi-validator which uses spectral as a dependency.

[padamstx]

@P0lip I work on a team in IBM (with @dpopp07) that supports the IBM openapi-validator that is now based on Spectral.
When we encounter one of these "Error running Nimma" situations, I assume it's because there's some sort of error in a custom rule function. Is there anything we can do from a debugging standpoint to get more information about the error when it occurs? i.e. any advice on perhaps adding some debug logging code to a certain spectral function, etc.?
Thanks!

Edit: After digging into the Nimma code a little bit I was able to see where the "Error running Nimma" errors are originating from. This led me to simply display more of the exception info that we receive in the AggregateError exceptions that are propagated from Spectral.
This solution might be adequate for projects that wrap the Spectral core, but probably not for folks that use the spectral CLI directly.

[stoplight-bot]

🎉 This issue has been resolved in version @stoplight/spectral-cli-v6.3.0 🎉

The release is available on npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

[stoplight-bot]

🎉 This issue has been resolved in version @stoplight/spectral-core-v1.11.0 🎉

The release is available on npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

[stoplight-bot]

🎉 This issue has been resolved in version @stoplight/spectral-functions-v1.6.0 🎉

The release is available on npm package (@latest dist-tag)

Your semantic-release bot 📦🚀