planner: add EXPLAIN FORMAT=JSON

[ghost]

What problem does this PR solve?

Issue Number: Fixes #20378

What is changed and how it works?

What's Changed:

This implements EXPLAIN FORMAT=JSON.

Because the operatorInfo is built by concatenating strings at a much lower level (planner/core/explain.go:37 - dataAccessor interface), it is just returned as a string (not ideal). I have forked #20520 to convert the delimiter to a ;.

Here is an example:

*************************** 1. row ***************************
EXPLAIN: {
  "id": "HashJoin_17",
  "estRows": 12487.5,
  "task": "",
  "accessObject": "",
  "operatorInfo": "inner join, equal:[eq(test.t2.t1_id, test.t1.id)]",
  "children": [
    {
      "id": "TableReader_20(Build)",
      "estRows": 9990,
      "task": "",
      "accessObject": "",
      "operatorInfo": "data:Selection_19",
      "children": [
        {
          "id": "Selection_19",
          "estRows": 9990,
          "task": "",
          "accessObject": "",
          "operatorInfo": "not(isnull(test.t2.t1_id))",
          "children": [
            {
              "id": "TableFullScan_18",
              "estRows": 10000,
              "task": "",
              "accessObject": "table:t2",
              "operatorInfo": "keep order:false, stats:pseudo",
              "children": null
            }
          ]
        }
      ]
    },
    {
      "id": "TableReader_22(Probe)",
      "estRows": 10000,
      "task": "",
      "accessObject": "",
      "operatorInfo": "data:TableFullScan_21",
      "children": [
        {
          "id": "TableFullScan_21",
          "estRows": 10000,
          "task": "",
          "accessObject": "table:t1",
          "operatorInfo": "keep order:false, stats:pseudo",
          "children": null
        }
      ]
    }
  ]
}

1 row in set (0.00 sec)

Related changes

• Parser PR: ast: add explain format=json parser#1055

Check List

Tests

• Unit tests

Side effects

• None

Release note

• The EXPLAIN statement now supports the optional FORMAT=JSON syntax.

[ghost]

@breeswish @zz-jason may I ask your opinion on these two questions in the description?

[breezewish]

(1) The golang JSON encoder will by default escape HTML characters. I think this creates ugly output in operatorInfo

Agree. Golang JSON encoder is escaped for HTML purpose and we don't need it.

(2) The operatorInfo is built by concatenating strings at a much lower level (planner/core/explain.go:37 - dataAccessor interface)

We should change it (in future PRs). It is also the foundation of enabling more data to be visualized in TiDB Dashboard, for example, an interactive and nice plan view from slow query and statement page. My own preference is to use Protobuf, whose structure is JSON serializable and also allow TiDB Dashboard to parse binary directly. @crazycs520 PTAL, we have talked about this previously.

[zz-jason]

(1) The golang JSON encoder will by default escape HTML characters. I think this creates ugly output in operatorInfo, and suggest we disable it.

Agree.

The best solution I have is to change concatenation between info items to use the semicolon character ;, but this would be an incompatible change that also affects other explain formats.

@qw4990 do you have any idea?

[qw4990]

The best solution I have is to change concatenation between info items to use the semicolon character ; , but this would be an incompatible change that also affects other explain formats.

I agree with it. Actually, I also meet this problem in another project where I want to parse and compare plans.

[ghost]

This is now ready to review. The tests are just failing waiting on the parser PR to merge.

[ghost]

/run-all-tests

[qw4990]

LGTM

[qw4990]

Please add more cases to cover more operators.

[ghost]

@qw4990 I was worried about test stability as statistics and operator auto-naming can change. What I arrived at, was to use a regular expression. PTAL again, thx :-)

[ti-srebot]

@qw4990, @winoros, PTAL.

[ti-srebot]

@qw4990, @winoros, PTAL.

[ti-srebot]

@qw4990, @winoros, PTAL.

[ghost]

We can revisit this when #20520 is complete.