Skip to content

Commit

Permalink
Merge #26164
Browse files Browse the repository at this point in the history
26164: importccl: Support `IMPORT MYSQLDUMP file` r=dt a=dt

This adds a new `IMPORT` syntax:
```
IMPORT MYSQLDUMP (s3://path/to/some/file.sql) 
```
as well as variant to select a single table:
```
IMPORT TABLE simple FROM MYSQLDUMP (s3://path/to/some/file.sql) 
```

This parses the dump file to read the table schema(s), converting their the mysql-dialect `CREATE TABLE` statements to cockroach's corresponding table definitions.

When a tables named, only the named table is imported -- if the dump contains other tables, they are ignored.

The supported conversions of the various MySQL column types to corresponding Cockroach column types is somewhat documented by Cockroach `CREATE` statement in the `everything`  test case of `TestMysqldumpSchemaReader`, which is tested to correspond to the example in the `everything.sql` testdata file, though these mappings remain subject to change as we test the behaviors of the corresponding types.

This is a big step towards our mysqldump support, though we will be continuing to actively work on this for the near future, so this is a rough, initial version that we can continue to polish. Future work should allow reading create table definitions _during_ sampling, rather than prior to starting the DistSQL job, to eliminate the duplicate read cost. 

Co-authored-by: David Taylor <[email protected]>
  • Loading branch information
craig[bot] and dt committed Jun 12, 2018
2 parents 857b3c9 + 7fa5485 commit 90f48d5
Show file tree
Hide file tree
Showing 35 changed files with 2,157 additions and 1,319 deletions.
4 changes: 3 additions & 1 deletion docs/generated/sql/bnf/import_table.bnf
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import_stmt ::=
'IMPORT' 'TABLE' table_name 'CREATE' 'USING' string_or_placeholder import_data_format 'DATA' '(' string_or_placeholder_list ')' opt_with_options
'IMPORT' import_bundle_format '(' string_or_placeholder ')'
| 'IMPORT' 'TABLE' table_name 'FROM' import_bundle_format '(' string_or_placeholder ')'
| 'IMPORT' 'TABLE' table_name 'CREATE' 'USING' string_or_placeholder import_data_format 'DATA' '(' string_or_placeholder_list ')' opt_with_options
| 'IMPORT' 'TABLE' table_name '(' table_elem_list ')' import_data_format 'DATA' '(' string_or_placeholder_list ')' opt_with_options
11 changes: 8 additions & 3 deletions docs/generated/sql/bnf/stmt_block.bnf
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,9 @@ insert_stmt ::=
| opt_with_clause 'INSERT' 'INTO' insert_target insert_rest on_conflict returning_clause

import_stmt ::=
'IMPORT' 'TABLE' table_name 'CREATE' 'USING' string_or_placeholder import_data_format 'DATA' '(' string_or_placeholder_list ')' opt_with_options
'IMPORT' import_bundle_format '(' string_or_placeholder ')'
| 'IMPORT' 'TABLE' table_name 'FROM' import_bundle_format '(' string_or_placeholder ')'
| 'IMPORT' 'TABLE' table_name 'CREATE' 'USING' string_or_placeholder import_data_format 'DATA' '(' string_or_placeholder_list ')' opt_with_options
| 'IMPORT' 'TABLE' table_name '(' table_elem_list ')' import_data_format 'DATA' '(' string_or_placeholder_list ')' opt_with_options

pause_stmt ::=
Expand Down Expand Up @@ -327,9 +329,9 @@ explain_option_list ::=
( explain_option_name ) ( ( ',' explain_option_name ) )*

import_data_format ::=
'CSV'
import_bundle_format
| 'CSV'
| 'MYSQLOUTFILE'
| 'MYSQLDUMP'
| 'PGCOPY'

privileges ::=
Expand All @@ -355,6 +357,9 @@ on_conflict ::=
'ON' 'CONFLICT' opt_conf_expr 'DO' 'UPDATE' 'SET' set_clause_list where_clause
| 'ON' 'CONFLICT' opt_conf_expr 'DO' 'NOTHING'

import_bundle_format ::=
'MYSQLDUMP'

string_or_placeholder_list ::=
( string_or_placeholder ) ( ( ',' string_or_placeholder ) )*

Expand Down
Loading

0 comments on commit 90f48d5

Please sign in to comment.