forked from facebook/mysql-5.6
-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[bypass] MySQL bypass change #2 - SELECT parsing [RocksDB part]
Summary: This is change #2 that ports the SELECT statement parsing from feature-myrocks-bypass branch, including: 1. select_parser class that parses SELECT statements and validates the scenarios are supported, and error with proper message if not supported 2. You can control bypass policy with rocksdb_select_bypass_policy variable = 0 (ALWAYS OFF) / 1 (ALWAYS ON) / 2 (OPT IN) / 3 (OPT OUT). 3. A few metrics for number of SELECT bypass statements executed/rejected (due to not supported) /failed (something wrong in execution) At this point this only dumps the SELECT statement parsed results without doing anything useful. The next change will be porting the query plan execution and point/range query execution part. Please ignore the unused attributes for now - they'll be removed in next change once the execution part is ported. Reference Patch: facebook@e79b20a facebook@8072e3d --- Porting Notes: There are a lot of small changes in this area but nothing too bad: * Some bypass scenarios no longer fail or fail with different error message because of the new parse tree -> AST conversion process. For example, `SELECT *` now works correctly * st_select_lex_t -> SELECT_LEX, and many members has changed name, such as where -> where_cond() * protocol usage has changed to start_row, store, end_row in 8.0, which is much better. * thd->query() is now a LEX_STRING * SELECT option check (such as SELECT_DISTINCT) now uses active_options() and only check for SELECT_DISTINCT as there seems to be default value being set most likely due to side effect of having more parse tree processing being done. I checked the flags and it looks like only SELECT_DISTINCT are interesting and needs to be blocked. * SELECT INTO OUTFILE check is changed to use results->needs_file_priviledge * True/False const are now represented as Item_func_true/Item_func_false. For now I'm over-generalizing to functions that are const, and expect val_int/etc would do the right thing Reviewed By: luqun Differential Revision: D22808305 ----------------------------------------------------------------------------------------------- Introduce base select parser class Summary: To make it easier to add a new bypass rpc parser into existing bypass implementation (nosql_access.cc), and make minimum change to bypass engine while adding a bypass rpc parser, I introduced a new base class, base_select_parser. It includes all essential members and methods for parsing bypass sq/rpc queries. Existing bypass sql parser (select_parser) and a new bypass rpc parser (not included in this diff) inherits the base function. Bypass RPC reference diff: D30493654 Reviewed By: yizhang82 Differential Revision: D31870628
- Loading branch information
Showing
18 changed files
with
2,454 additions
and
635 deletions.
There are no files selected for viewing
608 changes: 320 additions & 288 deletions
608
mysql-test/suite/rocksdb/r/bypass_select_basic.result
Large diffs are not rendered by default.
Oops, something went wrong.
608 changes: 320 additions & 288 deletions
608
mysql-test/suite/rocksdb/r/bypass_select_basic_bloom.result
Large diffs are not rendered by default.
Oops, something went wrong.
275 changes: 275 additions & 0 deletions
275
mysql-test/suite/rocksdb/r/bypass_select_unsupported.result
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,275 @@ | ||
| # restart | ||
| SELECT @@rocksdb_select_bypass_policy; | ||
| @@rocksdb_select_bypass_policy | ||
| 0 | ||
| SELECT @@rocksdb_select_bypass_policy into @save_rocksdb_select_bypass_policy; | ||
| set global rocksdb_select_bypass_policy=2; | ||
| SELECT @@rocksdb_select_bypass_policy; | ||
| @@rocksdb_select_bypass_policy | ||
| 2 | ||
| create table t1 (pk INT PRIMARY KEY, a INT, b INT, c INT, KEY a (a, b, c)) | ||
| ENGINE=ROCKSDB; | ||
| create table t2 (pk INT PRIMARY KEY, a INT, b INT, c INT, KEY a (a, b, c)) | ||
| ENGINE=ROCKSDB; | ||
| create table t3 (pk INT PRIMARY KEY, a INT, b INT, c VARCHAR(15) CHARACTER SET latin1, KEY a (a, b)) | ||
| ENGINE=ROCKSDB; | ||
| SELECT @@rocksdb_select_bypass_fail_unsupported; | ||
| @@rocksdb_select_bypass_fail_unsupported | ||
| 1 | ||
| SELECT @@rocksdb_select_bypass_fail_unsupported into @save_fail_unsupported; | ||
| set global rocksdb_select_bypass_fail_unsupported=true; | ||
| SELECT @@rocksdb_select_bypass_fail_unsupported; | ||
| @@rocksdb_select_bypass_fail_unsupported | ||
| 1 | ||
| SELECT /*+ bypass */ pk from t1 WHERE pk=1 having pk=1; | ||
| ERROR 42000: SELECT statement pattern not supported: HAVING not supported | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 0 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 1 | ||
| SELECT /*+ bypass */ pk from t1 WHERE pk=1 group by pk; | ||
| ERROR 42000: SELECT statement pattern not supported: GROUP BY not supported | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 0 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 2 | ||
| SELECT /*+ bypass */ * from t1 WHERE pk=1; | ||
| Dump | ||
| Index=0, Order=ASC, Fields={ (Name=pk, Index=0), (Name=a, Index=1), (Name=b, Index=2), (Name=c, Index=3) }, Cond={ (pk == 1) } | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 1 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 2 | ||
| SELECT /*+ bypass */ 1 from t1 WHERE pk=1; | ||
| ERROR 42000: SELECT statement pattern not supported: SELECT expressions can only be field | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 1 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 3 | ||
| SELECT /*+ bypass */ 1+2 from t1 WHERE pk=1; | ||
| ERROR 42000: SELECT statement pattern not supported: SELECT expressions can only be field | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 1 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 4 | ||
| SELECT /*+ bypass */ COUNT(*) from t1 WHERE pk=1; | ||
| ERROR 42000: SELECT statement pattern not supported: SELECT expressions can only be field | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 1 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 5 | ||
| SELECT /*+ bypass */ COUNT(pk) from t1 WHERE pk=1; | ||
| ERROR 42000: SELECT statement pattern not supported: SELECT expressions can only be field | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 1 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 6 | ||
| SELECT /*+ bypass */ pk+1 from t1 WHERE pk=1; | ||
| ERROR 42000: SELECT statement pattern not supported: SELECT expressions can only be field | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 1 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 7 | ||
| SELECT /*+ bypass */ pk from t1 USE INDEX (PRIMARY) WHERE pk=1; | ||
| ERROR 42000: SELECT statement pattern not supported: Index hint must be FORCE INDEX | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 1 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 8 | ||
| SELECT /*+ bypass */ pk from t1; | ||
| ERROR 42000: SELECT statement pattern not supported: Unsupported WHERE: should be expr [(AND expr)*] where expr only contains >, >=, <, <=, =, IN | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 1 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 9 | ||
| SELECT /*+ bypass */ pk from t1 WHERE pk=1 or pk=2; | ||
| ERROR 42000: SELECT statement pattern not supported: Unsupported WHERE: should be expr [(AND expr)*] where expr only contains >, >=, <, <=, =, IN | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 1 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 10 | ||
| SELECT /*+ bypass */ pk from t1 WHERE pk<=>1; | ||
| ERROR 42000: SELECT statement pattern not supported: Unsupported WHERE - needs to be >, >=, <, <=, =, IN | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 1 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 11 | ||
| SELECT /*+ bypass */ pk from t1 WHERE pk=(1,2,3); | ||
| ERROR 21000: Operand should contain 1 column(s) | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 1 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 11 | ||
| SELECT /*+ bypass */ pk from t1 WHERE pk=DATE '2019-03-25'; | ||
| Dump | ||
| Index=0, Order=ASC, Fields={ (Name=pk, Index=0) }, Cond={ (pk == 2019-03-25) } | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 2 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 11 | ||
| SELECT /*+ bypass */ pk from t1 WHERE pk=TIME '18:01:00'; | ||
| Dump | ||
| Index=0, Order=ASC, Fields={ (Name=pk, Index=0) }, Cond={ (pk == 18:01:00) } | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 3 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 11 | ||
| SELECT /*+ bypass */ pk from t1 WHERE pk=TIMESTAMP '2019-03-25 18:01:00'; | ||
| Dump | ||
| Index=0, Order=ASC, Fields={ (Name=pk, Index=0) }, Cond={ (pk == 2019-03-25 18:01:00) } | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 4 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 11 | ||
| SELECT /*+ bypass */ pk from t1 WHERE pk>ALL (SELECT pk FROM t1); | ||
| ERROR 42000: SELECT statement pattern not supported: Unsupported WHERE - needs to be >, >=, <, <=, =, IN | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 4 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 12 | ||
| SELECT /*+ bypass */ DISTINCT a from t1; | ||
| ERROR 42000: SELECT statement pattern not supported: SELECT options not supported (such as SELECT DISTINCT) | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 4 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 13 | ||
| SELECT /*+ bypass */ ALL a, b from t1; | ||
| ERROR 42000: SELECT statement pattern not supported: Unsupported WHERE: should be expr [(AND expr)*] where expr only contains >, >=, <, <=, =, IN | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 4 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 14 | ||
| SELECT /*+ bypass */ t1.pk, t2.pk from t1, t2; | ||
| pk pk | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 4 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 14 | ||
| SELECT /*+ bypass */ t1.pk FROM t1 LEFT JOIN t2 using (pk) | ||
| WHERE t1.pk=1; | ||
| pk | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 4 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 14 | ||
| SELECT /*+ bypass */ pk, a from (SELECT pk, a FROM t1) AS t1_temp; | ||
| pk a | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 4 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 14 | ||
| SELECT /*+ bypass */ pk FROM t1 WHERE pk=3 UNION DISTINCT | ||
| SELECT pk FROM t2 WHERE pk=3; | ||
| pk | ||
| SELECT /*+ bypass */ pk FROM t1 WHERE pk=3 UNION ALL | ||
| SELECT pk FROM t2 WHERE pk=3; | ||
| pk | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 4 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 14 | ||
| SELECT /*+ bypass */ a, b, c FROM t1 WHERE a=1 ORDER BY a; | ||
| ERROR 42000: SELECT statement pattern not supported: ORDER BY field doesn't belong to the index | ||
| SELECT /*+ bypass */ a, b, c FROM t1 FORCE INDEX(`a`) WHERE a=1 ORDER BY b, a; | ||
| ERROR 42000: SELECT statement pattern not supported: ORDER BY is not in index order | ||
| SELECT /*+ bypass */ a, b, c FROM t1 FORCE INDEX(`a`) WHERE a=1 | ||
| ORDER BY a, b, c, c, c, a, b, c, d; | ||
| ERROR 42S22: Unknown column 'd' in 'order clause' | ||
| SELECT /*+ bypass */ a, b, c FROM t1 FORCE INDEX(`a`) WHERE a=1 ORDER BY c, a; | ||
| ERROR 42000: SELECT statement pattern not supported: ORDER BY is not in index order | ||
| SELECT /*+ bypass */ a, b, c FROM t1 FORCE INDEX(`a`) WHERE a=1 ORDER BY a, a; | ||
| ERROR 42000: SELECT statement pattern not supported: ORDER BY is not in index order | ||
| SELECT /*+ bypass */ a, b, c FROM t1 FORCE INDEX(`a`) WHERE a=1 | ||
| ORDER BY a, a, a, a, a, a, a, a; | ||
| ERROR 42000: SELECT statement pattern not supported: ORDER BY is not in index order | ||
| SELECT /*+ bypass */ a, b, c FROM t1 FORCE INDEX(`a`) WHERE a=1 ORDER BY b, b; | ||
| ERROR 42000: SELECT statement pattern not supported: ORDER BY is not in index order | ||
| SELECT /*+ bypass */ a, b, c FROM t3 FORCE INDEX(`a`) WHERE a=1 ORDER BY b, b; | ||
| ERROR 42000: SELECT statement pattern not supported: only utf8_bin, latin1_bin is supported for varchar field | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 4 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 21 | ||
| SELECT /*+ bypass */ a, b, c FROM t1 FORCE INDEX(`abc`) WHERE a=1 ORDER BY b, a; | ||
| ERROR 42000: Key 'abc' doesn't exist in table 't1' | ||
| SELECT /*+ bypass */ a from t1 WHERE a=1 INTO OUTFILE 'datadir/select.out'; | ||
| ERROR 42000: SELECT statement pattern not supported: SELECT INTO/DUMP not supported | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 4 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 22 | ||
| SELECT /*+ bypass */ a from t1 WHERE a=1 INTO DUMPFILE 'datadir/select.dump'; | ||
| ERROR 42000: SELECT statement pattern not supported: SELECT INTO/DUMP not supported | ||
| SHOW STATUS LIKE 'rocksdb_select_bypass%'; | ||
| Variable_name Value | ||
| rocksdb_select_bypass_executed 4 | ||
| rocksdb_select_bypass_failed 0 | ||
| rocksdb_select_bypass_rejected 23 | ||
| SELECT /*+ bypass */ a, b, c FROM t3 WHERE pk=1 FOR UPDATE; | ||
| ERROR 42000: SELECT statement pattern not supported: Only SELECT with default READ lock is supported | ||
| SELECT /*+ bypass */ a, b, c FROM t3 WHERE pk=1 FOR UPDATE SKIP LOCKED; | ||
| ERROR 42000: This version of MySQL doesn't yet support 'SKIP LOCKED | NOWAIT is not yet supported in RocksDB storage engine' | ||
| SELECT /*+ bypass */ a, b, c FROM t3 WHERE pk=1 FOR UPDATE NOWAIT; | ||
| ERROR 42000: This version of MySQL doesn't yet support 'SKIP LOCKED | NOWAIT is not yet supported in RocksDB storage engine' | ||
| SELECT /*+ bypass */ a, b, c FROM t3 WHERE pk=1 LOCK IN SHARE MODE; | ||
| ERROR 42000: SELECT statement pattern not supported: Only SELECT with default READ lock is supported | ||
| SELECT /*+ bypass */ HIGH_PRIORITY a, b, c FROM t3 WHERE pk=1; | ||
| ERROR 42000: SELECT statement pattern not supported: Only SELECT with default READ lock is supported | ||
| SELECT /*+ bypass */ a,a,a,a,a, a,a,a,a,a, a,a,a,a,a, a,a FROM t3 WHERE pk=1; | ||
| ERROR 42000: SELECT statement pattern not supported: Too many SELECT expressions | ||
| SELECT /*+ bypass */ a FROM t3 WHERE a=1 AND a=1 AND a=1 AND a=1 AND a=1 AND | ||
| a=1 AND a=1 AND a=1 AND a=1 AND a=1 AND | ||
| a=1 AND a=1 AND a=1 AND a=1 AND a=1 AND | ||
| a=1 AND a=1; | ||
| ERROR 42000: SELECT statement pattern not supported: Too many WHERE expressions | ||
| SELECT @@rocksdb_select_bypass_policy; | ||
| @@rocksdb_select_bypass_policy | ||
| 2 | ||
| set global rocksdb_select_bypass_policy=@save_rocksdb_select_bypass_policy; | ||
| SELECT @@rocksdb_select_bypass_policy; | ||
| @@rocksdb_select_bypass_policy | ||
| 0 | ||
| SELECT @@rocksdb_select_bypass_fail_unsupported; | ||
| @@rocksdb_select_bypass_fail_unsupported | ||
| 1 | ||
| set global rocksdb_select_bypass_fail_unsupported=false; | ||
| SELECT @@rocksdb_select_bypass_fail_unsupported; | ||
| @@rocksdb_select_bypass_fail_unsupported | ||
| 0 | ||
| SELECT /*+ bypass */ * from t1; | ||
| pk a b c | ||
| SELECT @@rocksdb_select_bypass_fail_unsupported; | ||
| @@rocksdb_select_bypass_fail_unsupported | ||
| 0 | ||
| set global rocksdb_select_bypass_fail_unsupported=@save_fail_unsupported; | ||
| SELECT @@rocksdb_select_bypass_fail_unsupported; | ||
| @@rocksdb_select_bypass_fail_unsupported | ||
| 1 | ||
| drop table t1; | ||
| drop table t2; | ||
| drop table t3; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.