[CT-3543] [Feature] Global config to apply pretty-print formatting to JSON artifacts

[dbeatty10]

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

For human consumers, it's nice to apply pretty-printing to dbt artifacts like semantic_manifest.json, manifest.json, catalog.json, run_results.json, sources.json, etc. so that they are more readable for debugging or development purposes.

So we should enable a global config that allows this to be configured:

• config: pretty_print_json
• environment variable: DBT_PRETTY_PRINT_JSON
• CLI: --pretty-print-json / --no-pretty-print-json

For consideration: whether or not it would default to true. Selfishly, I wouldn't mind if it defaulted to pretty-printing 😎. But there could be storage/performance implications to consider.

Other use cases

Both dbt list and dbt show subcommands allow specifying --output json.

Ideally, any commands that allow JSON-formatted output would also respect the pretty_print_json flag.

Describe alternatives you've considered

I just use the Shift + Option + F shortcut in vscode currently.

In the past, I've jq commands like this:

cat target/manifest.json | jq . > target/manifest.pp.json

Who will this benefit?

This will benefit anyone who does debugging of dbt JSON artifacts or does development that consumes these artifacts.

e.g.

This is not easy to read:

{"metadata": {"dbt_schema_version": "https://schemas.getdbt.com/dbt/catalog/v1.json", "dbt_version": "1.8.0a1", "generated_at": "2024-01-10T22:52:35.009683Z", "invocation_id": "5fa63ffe-8c3d-4107-b9ca-751dbfd3c8e5", "env": {}}}

But this is:

{
  "metadata": {
    "dbt_schema_version": "https://schemas.getdbt.com/dbt/catalog/v1.json",
    "dbt_version": "1.8.0a1",
    "generated_at": "2024-01-10T22:52:35.009683Z",
    "invocation_id": "5fa63ffe-8c3d-4107-b9ca-751dbfd3c8e5",
    "env": {}
  }
}

Are you interested in contributing this feature?

No response

Anything else?

No response

[graciegoheen]

• I don't think we should default this to on, due to the storage considerations
• I like the idea of this being all three - global config, CLI flag, and env var.

[jx2lee]

@dbeatty10 I want to solve this, and it'll be solved to fix dbt_common.events._JsonLogger L195
(then printed pretty json style)

Is there a process for contributing to both repos (dbt_common, dbt-core)? This needs to be modified in both repos.

[graciegoheen]

@jx2lee You can open up a PR in both repos, and we'll have an engineer review both in tandem