diff --git a/docs/content/connectors/oracle-cdc.md b/docs/content/connectors/oracle-cdc.md index b4c9379cdbd..aaa4dc37104 100644 --- a/docs/content/connectors/oracle-cdc.md +++ b/docs/content/connectors/oracle-cdc.md @@ -24,7 +24,7 @@ In order to setup the Oracle CDC connector, the following table provides depende Download [flink-sql-connector-oracle-cdc-2.4-SNAPSHOT.jar](https://repo1.maven.org/maven2/com/ververica/flink-sql-connector-oracle-cdc/2.4-SNAPSHOT/flink-sql-connector-oracle-cdc-2.4-SNAPSHOT.jar) and put it under `/lib/`. -**Note:** flink-sql-connector-oracle-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-oracle-cdc-2.2.1.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-oracle-cdc), the released version will be available in the Maven central warehouse. +**Note:** flink-sql-connector-oracle-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-oracle-cdc-2.3.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-oracle-cdc), the released version will be available in the Maven central warehouse. Setup Oracle ---------------- @@ -98,6 +98,7 @@ You have to enable log archiving for Oracle database and define an Oracle user w GRANT LOGMINING TO flinkuser; GRANT CREATE TABLE TO flinkuser; + -- need not to execute if set scan.incremental.snapshot.enabled=true(default) GRANT LOCK ANY TABLE TO flinkuser; GRANT ALTER ANY TABLE TO flinkuser; GRANT CREATE SEQUENCE TO flinkuser; @@ -169,6 +170,7 @@ Overall, the steps for configuring CDB database is quite similar to non-CDB data GRANT SELECT ANY TRANSACTION TO flinkuser CONTAINER=ALL; GRANT LOGMINING TO flinkuser CONTAINER=ALL; GRANT CREATE TABLE TO flinkuser CONTAINER=ALL; + -- need not to execute if set scan.incremental.snapshot.enabled=true(default) GRANT LOCK ANY TABLE TO flinkuser CONTAINER=ALL; GRANT CREATE SEQUENCE TO flinkuser CONTAINER=ALL; @@ -217,6 +219,9 @@ Flink SQL> SELECT * FROM products; **Note:** When working with the CDB + PDB model, you are expected to add an extra option `'debezium.database.pdb.name' = 'xxx'` in Flink DDL to specific the name of the PDB to connect to. +**Note:** +While the connector might work with a variety of Oracle versions and editions, only Oracle 9i, 10g, 11g and 12c have been tested. + Connector Options ----------------
@@ -302,38 +307,46 @@ Connector Options Optional startup mode for Oracle CDC consumer, valid enumerations are "initial" and "latest-offset". Please see Startup Reading Position section for more detailed information. - + - chunk-meta.group.size - optional - 1000 - Integer - The group size of chunk meta, if the meta size exceeds the group size, the meta will be divided into multiple groups. + scan.incremental.snapshot.enabled + optional + true + Boolean + Incremental snapshot is a new mechanism to read snapshot of a table. Compared to the old snapshot mechanism, + the incremental snapshot has many advantages, including: + (1) source can be parallel during snapshot reading, + (2) source can perform checkpoints in the chunk granularity during snapshot reading, + (3) source doesn't need to acquire ROW SHARE MODE lock before snapshot reading. + - connect.timeout + scan.incremental.snapshot.chunk.size optional - 30s - Duration - The maximum time that the connector should wait after trying to connect to the Oracle database server before timing out. - + 8096 + Integer + The chunk size (number of rows) of table snapshot, captured tables are split into multiple chunks when read the snapshot of table. + + + scan.snapshot.fetch.size + optional + 1024 + Integer + The maximum fetch size for per poll when read table snapshot. + - chunk-key.even-distribution.factor.lower-bound + connect.max-retries optional - 0.05d - Double - The lower bound of chunk key distribution factor. The distribution factor is used to determine whether the table is evenly distribution or not. - The table chunks would use evenly calculation optimization when the data distribution is even, and the query for splitting would happen when it is uneven. - The distribution factor could be calculated by (MAX(id) - MIN(id) + 1) / rowCount. - - - chunk-key.even-distribution.factor.upper-bound + 3 + Integer + The max retry times that the connector should retry to build Oracle database server connection. + + + connection.pool.size optional - 1000.0d - Double - The upper bound of chunk key distribution factor. The distribution factor is used to determine whether the table is evenly distribution or not. - The table chunks would use evenly calculation optimization when the data distribution is even, and the query for splitting would happen when it is uneven. - The distribution factor could be calculated by (MAX(id) - MIN(id) + 1) / rowCount. + 20 + Integer + The connection pool size. debezium.* @@ -418,11 +431,13 @@ CREATE TABLE products ( 'password' = 'flinkpw', 'database-name' = 'XE', 'schema-name' = 'inventory', - 'table-name' = 'products' + 'table-name' = 'products', + 'debezium.log.mining.strategy' = 'online_catalog', + 'debezium.log.mining.continuous.mine' = 'true' ); ``` -**Note** : The Oracle dialect is case-sensitive, it converts field name to uppercase if the field name is not quoted, Flink SQL doesn't convert the field name. Thus for physical columns from oracle database, we should use its converted field name in Oracle when define an `oracle-cdc` table in Flink SQL. +**Note** : The Oracle dialect is case-sensitive, it converts field name to uppercase if the field name is not quoted, Flink SQL doesn't convert the field name. Thus for physical columns from oracle database, we should use its converted field name in Oracle when define an `oracle-cdc` table in Flink SQL. Features -------- @@ -487,7 +502,7 @@ Data Type Mapping - + @@ -569,7 +584,8 @@ Data Type Mapping VARCHAR2(n)
CLOB
NCLOB
- XMLType + XMLType
+ SYS.XMLTYPE
Oracle typeOracle type Flink SQL type
STRING