log: add tidb_log_desensitization global variable to control whether do desensitization when log query

[crazycs520]

What problem does this PR solve?

ref: #18566

Add tidb_log_desensitization global variable to control whether do desensitization when log query.

Here are some examples:

Before

[2020/07/15 13:39:17.601 +08:00] [INFO] [session.go:2247] [GENERAL_LOG] [conn=9] [user=root@127.0.0.1] [schemaVersion=36] [txnStartTS=0] [forUpdateTS=0] [isReadConsistency=false] [current_db=test] [txn_mode=PESSIMISTIC] [sql="insert into t values (2,2),(3,3)"]
[2020/07/15 13:39:17.602 +08:00] [ERROR] [conn.go:737] ["command dispatched failed"] [conn=9] [connInfo="id:9, addr:127.0.0.1:54956 status:10, collation:utf8_general_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="insert into t values (2,2),(3,3)"] [txn_mode=PESSIMISTIC] [err="[kv:1062]Duplicate entry '2' for key 'idx1'"]

Now

after set @@global.tidb_log_desensitization=1;;

[2020/07/15 13:38:36.353 +08:00] [INFO] [session.go:2247] [GENERAL_LOG] [conn=6] [user=root@127.0.0.1] [schemaVersion=36] [txnStartTS=0] [forUpdateTS=0] [isReadConsistency=false] [current_db=test] [txn_mode=PESSIMISTIC] [sql="insert into t values ( ... ) , ( ... )"]
[2020/07/15 13:38:36.354 +08:00] [ERROR] [conn.go:737] ["command dispatched failed"] [conn=6] [connInfo="id:6, addr:127.0.0.1:54947 status:10, collation:utf8_general_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="insert into t values ( ... ) , ( ... )"] [txn_mode=PESSIMISTIC] [err="[kv:1062]Duplicate entry '2' for key 'idx1'"]

What is changed and how it works?

Related changes

• Need to cherry-pick to the release branch

Check List

Tests

• Manual test (add detailed scripts or steps below)

set @@global.tidb_log_desensitization=1;
set @@tidb_general_log=1;
-- execute some sql, and then check the tidb log output.

Side effects

• Performance regression
  • Consumes more CPU
  • Consumes more MEM
• Breaking backward compatibility

Release note

• add tidb_log_desensitization global variable to control whether do desensitization when log query.

[crazycs520]

/rebuild

[codecov]

Codecov Report

Merging #18578 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #18578   +/-   ##
===========================================
  Coverage   79.2934%   79.2934%           
===========================================
  Files           540        540           
  Lines        145601     145601           
===========================================
  Hits         115452     115452           
  Misses        20858      20858           
  Partials       9291       9291           

[crazycs520]

/run-all-tests

[tiancaiamao]

LGTM

[crazycs520]

/run-all-tests

[crazycs520]

/run-all-tests

[bb7133]

LGTM

[ti-srebot]

@bb7133,Thanks for your review. However, LGTM is restricted to Reviewers or higher roles.See the corresponding SIG page for more information. Related SIGs: execution(slack).

[winoros]

LGTM

[SunRunAway]

/merge

[ti-srebot]

/run-all-tests

[ti-srebot]

cherry pick to release-4.0 in PR #18581

[lysu]

closes #18034