Support outputting results in JUnit XML format

[yehonatanz]

Support outputting check/lint/format results in a machine-readable format, to allow running biome in CI environments (read: Jenkins) and clearly see failures in the UI.
JUnit XML is pretty common and the de-facto standard of many tools (+ it's supported by all major test runners and eslint).

I thought of something along the lines of:

{
  // biome.json
  // ...
  "outputs": [
    {
      "format": "junit",
      "path": "./biome.junit.xml"
    },
    {
      "format": "console"
    }
  ]
}

which I think will provide good flexibility and extensibility for future formats.

Supporting such a feature will make the decision to adopt biome way easier for us.
Does that make sense?

[ematipico]

It does make sense. Just to understand, you expect from the CLI the capability to emit multiple formats at the time? Or it would just be one?

[yehonatanz]

I was thinking multiple formats at the same time.
The reason is mostly to enable outputting machine-readable file for CI and human-friendly styled console output within the same config.
I usually use eslint-output plugin for that and love it.

Honestly that's the only multiple-format use case I can think of, so anything supporting it would be fine. For example: always outputting to console and optionally outputting a file would also be great for us. I'd just thought why not already go with the more extensible approach from the start 🤷.

[blutorange]

Another use case would be integration with other tools such as the Maven / Gradle spotless plugin. I already added support for the Biome formatter. The plugin might soon support linters and it would be great if we could also integrate the Biome linter. Any kind of structured, machine-readable output would be sufficient for that use case, be it XML or JSON etc.

[Conaclos]

Could you share specs of the structured formats you talked about?

[blutorange]

I don't have any particular structure in mind, as long as it's something like JSON or XML that contains the data and we don't have to parse the plain text output. For reference, the planned linter support for the spotless plugin currently expects a list of lint messages, where each lint message consists of the start line number, the end line number, a code for the lint rule (e.g. no-non-null-assertion) and a message with an English description.

[ucwillg]

For a use case on our team, we'd like to have the ability to output a full list of all of the rules that were violated, along with the files the violations occurred in, as structured JSON.

A separate program, probably written in python, would consume this output and implement a linter on top of biome. This linter would maintain, in a flat file committed to our repo, a list of "known" rule violations that are ignored. Each file would have a "budget" of violations per rule that cannot be exceeded. New files do not have a violation budget, generally. Periodically we will shrink the violation budgets as we fix violations. We have used this strategy with other static analysis techniques successfully in the past.

We'd eventually like to be able to use the system i described above to suppress/filter the original biome output, but that could be accomplished in various ways and doesn't need to be in scope for a v1. Something like:

[{ rule: ..., file: ..., line: ..., message: ... }, ...]

would be sufficient for us to get started - we don't need enough info to reconstruct the original cli output.

[martinal]

Just adding to the choir: I came here looking for a way to get diagnostics output in a structured format that's easy to parse, e.g. like ruff has the option --outputFormat=json.

The use case is using biome in a pipeline where the consumer is not a human but code doing something with the diagnostics.

Any structured output format/schema that is easy to parse and can stay stable over time is fine IMO. JSON is convenient, but consumers will always have different needs anyway.

[ematipico]

Our APIs allow us now to easily create new reporters.

If anyone is interested in new reports and knows their formats, you could send a PR