Coordinator
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| taskTwoActive | BOOLEAN | false | Specifies whether the project is an active-active disaster recovery one. This parameter is for full migration only. | OMS V3.3.1 and later |
| taskIdentity | STRING | 1 | The project ID. | OMS V3.3.1 and later |
| checkDstTableEmpty | BOOLEAN | true | Specifies whether to check for empty tables in the destination. | OMS V3.3.1 and later |
| precheckSinkSchema | BOOLEAN | true | Specifies whether to query the destination for a precheck. If the destination is not queried, no precheck is performed on the destination. | OMS V3.3.1 and later |
| selectEmptyTableParallel | LONG | 16 | The number of concurrent queries for empty tables in a precheck for full migration (DATAFLOW-SOURCE). | OMS V4.2.0 and later |
| timezone | STRING | +08:00 | The time zone of the extract-transform-load (ETL) operation. | OMS V3.3.1 and later |
| systemDefaultTimezone | STRING | null | The time zone for reading the time. By default, the current time zone is read. | OMS V3.3.1 and later |
| sourceType | STRING | null | The type of the source data source. | OMS V3.3.1 and later |
| sinkType | STRING | null | The type of the destination data source. | OMS V3.3.1 and later |
| consistencyMigration | BOOLEAN | false | Specifies whether to lock full migration snapshots when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| allowRecordTypes | STRING | null | The one or more types of messages allowed for a project. Separate multiple types with a comma (,). Valid values: ROW, DELETE, UPDATE, INSERT, HEARTBEAT, REPLACE, and DDL. | OMS V3.3.1 and later |
| tableFilterCacheSize | LONG | 512 | The LRU cache size for CreateTableIfNotExistDDLFilter. | OMS V3.3.1 and later |
| throttleMemoryBound | STRING | null | The maximum memory size allowed, in bytes. | OMS V3.3.1 and later |
| throttleRecordBound | STRING | null | The maximum number of records allowed within the framework. | OMS V3.3.1 and later |
| throttleIOPS | STRING | null | The maximum I/O operations per second (IOPS) allowed, in bytes. | OMS V3.3.1 and later |
| throttleRps | STRING | null | The maximum requests per second (RPS) allowed. | OMS V3.3.1 and later |
| enablePreprocessConfig | BOOLEAN | true | Specifies whether to allow internal automatic calculation of queue length based on memory. | OMS V4.0.2 and later |
| connectorJvmParam | STRING | null | The JVM parameters for starting the connector. | OMS V3.3.1 and later |
| bridgeQueueSize | LONG | 256 | The capacity of the bridge cache queue. The unit of the value is RecordBatch. | OMS V3.3.1 and later |
| shuffleBucketSize | LONG | 128 | The number of buckets. This parameter is used together with ShuffleRecordDispatcher to control the concurrency. |
OMS V3.3.1 and later |
| batchAllInOneTable | BOOLEAN | false | Specifies whether messages in one batch belong to the same table. Default value: true. |
OMS V3.3.1 and later |
| shuffleFlushIntervalMs | LONG | The system provides a recommended value. | The interval at which messages are collected into one batch, in milliseconds. | OMS V3.3.1 and later |
| shuffleMinBatchSize | LONG | 20 | The minimum batch size. This parameter is used together with ShuffleRecordDispatcher. ShuffleRecordDispatcher is a message distributor that synchronizes messages to message queues (Kafka, RocketMQ, and DataHub). This mechanism improves the messaging concurrency. |
OMS V3.3.1 and later |
| shuffleDdlToAllRoute | BOOLEAN | false | Specifies whether to synchronize DDL statements to all queues when shuffleRecordDispatcher is specified. |
OMS V4.0.1 and later |
| shuffleMaxBatchSize | LONG | 64 | The maximum batch size. This parameter is used together with ShuffleRecordDispatcher. |
OMS V3.3.1 and later |
| shuffleBucketMaxInFlightBatchSize | LONG | 2147483647 | The maximum number of messages without an acknowledgment allowed in a bucket. This parameter is used together with ShuffleRecordDispatcher. |
OMS V3.3.1 and later |
| printTraceMsg | BOOLEAN | The system provides a recommended value. | Specifies whether to print msg logs. By default, msg logs are printed for incremental synchronization and are not printed for full migration. | OMS V3.3.1 and later |
| loggerLevel | STRING | INFO | The log printing level. Valid values: DEBUG, INFO, WARN, and ERROR. | OMS V3.3.1 and later |
| destSubId | STRING | null | The subID of the destination. This parameter is a JDBCWriter-compatible parameter. | OMS V3.3.1 and later |
| daemonIp | STRING | null | The IP address of the connector. This parameter is a JDBCWriter-compatible parameter. | OMS V3.3.1 and later |
| destSubTopic | STRING | null | The subtopic of the destination. This parameter is a JDBCWriter compatible parameter. | OMS V3.3.1 and later |
| checkpointUseTaskName | BOOLEAN | false | Specifies whether the checkpoint file is in the light-connector format. | OMS V3.3.1 and later |
| daemonPort | STRING | null | The daemon port, which is usually 9000. This parameter is a JDBCWriter-compatible parameter. | OMS V3.3.1 and later |
| enableMetricReportTask | BOOLEAN | true | Specifies whether to generate checkpoints. In OMS V4.0.0 and later, if you specify enableOmsConnectorV2Report, you can set this parameter to false. |
OMS V3.3.1 and later |
| ListenPort | LONG | 0 | The HTTP listening port. | OMS V3.3.1 and later |
| httpBacklog | LONG | 16 | The length of the TCP connection queue. | OMS V3.3.1 and later |
| taskName | STRING | null | The task name. | OMS V3.3.1 and later |
| caseStrategy | STRING | FOLLOW-SOURCE | The case-sensitivity strategy. Valid values: DEFAULT (to use the case based on the query result), FOLLOW_SOURCE (to follow the case in the source database), UPPER (to use the upper case), and LOWER (to use the lower case). | OMS V3.3.1 and later |
| isCaseSensitive | BOOLEAN | false | Specifies whether to enable case sensitivity. | OMS V3.3.1 and later |
| dbTableCaseStrategy | STRING | FOLLOW-SOURCE | The case-sensitivity strategy for database tables. Valid values: DEFAULT (to use the case based on the query result), FOLLOW_SOURCE (to follow the case in the source database), UPPER (to use the upper case), and LOWER (to use the lower case). | OMS V4.2.2 and later |
| columnCaseStrategy | STRING | FOLLOW-SOURCE | The case-sensitivity strategy for columns. Valid values: DEFAULT (to use the case based on the query result), FOLLOW_SOURCE (to follow the case in the source database), UPPER (to use the upper case), and LOWER (to use the lower case). | OMS V4.2.2 and later |
| noneedRetryException | STRING | null | The exception that a precheck does not retry for full migration. | OMS V3.3.1 and later |
| runIgnoreError | BOOLEAN | false | Specifies whether to exit when table synchronization fails due to an exception. | OMS V3.3.1 and later |
| maxRecordCapacity | LONG | 16384 | The size of the dispatcher queue. | OMS V3.3.1 and later |
| dispatcherClassName | STRING | The system provides a recommended value based on the sinkType and hotKeyMerge parameters. |
The implementation type of the dispatcher. This parameter is usually specified internally by the system. The default value is MetricDispatcher for full migration and is DefaultTransactionScheduler for incremental synchronization. The default value is ShuffleRecordDispatcher for a message queue destination. Hotspot merge and writing to partitions influence the parameter value. |
OMS V3.3.1 and later |
| enableActiveReportTask | BOOLEAN | true | Specifies whether to report the drc.active.xxx file. In OMS V4.0.0 and later, if you specify enableOmsConnectorV2Report, you can set this parameter to false. |
OMS V3.3.1 and later |
| reportIntervalSec | LONG | 5 | The interval for generating checkpoints, heartbeat packets, or diagnosis files. | OMS V3.3.1 and later |
| enableDmlDdlTypeStatistics | BOOLEAN | false | Specifies whether to enable DDL/DML statistics collection. | OMS V3.3.1 and later |
| enableOmsConnectorV2Report | BOOLEAN | false | Specifies whether to provide the new checkpoint format after a component merge, which changes the connector file name to oms.connector.checkpoint. To ensure the compatibility with connectors of earlier versions, use the default value false. You can change the value to true after most components are upgraded. |
OMS V4.0.1 and later |
| reportMetricsIntervalSec | LONG | 10 | The interval for generating metrics logs. | OMS V3.3.1 and later |
| castConfig | STRING | null | The serialization settings. Byte data is written in the Base64 format, timestamp data is written in the sec.mill form, and datetime data is written in the Base64 format with the year, month, date, hour, minute, and second values. Details: {"bytesType": "BASE64: converts byte data to a Base64 string","timestampType": "SEC_NANO: a timestamp string in the format of sec.mill, with a maximum of 9 digits for mill. SEC: a second-level timestamp, which is of the long type; MILLI_STR: a millisecond-level timestamp, which is of the long type; MILLI_STR: a millisecond-level timestamp string; DATETIME_NORMAL: a normal timestamp with the year, month, day, hour, minute, and second values","dateTimeType": "ISO: a timestamp with the year, month, day, hour, minute, and second values started with the letter T; NORMAL: a normal timestamp with the year, month, day, hour, minute, and second values; TIMESTAMP_MILL: a millisecond timestamp of the Long type","bitType": "NUMBER: converts bit data to the Long type; BASE64: converts bit data to the Base64 type","appendZoneId": "specifies whether to add the zone ID. Default value: true"} |
OMS V3.3.1 and later |
Condition
The format of the Condition module is as follows:
{
"briefWhiteCondition": [], // The brief allowlist. If an allowlist exists, the filtering rules are created for the store based on this parameter.
"whiteCondition": [], // The allowlist configuration.
"blackCondition": [] // The blocklist configuration.
}
The following example shows the Condition module for data synchronization from source_db.source_table to sink_db.sink_table:
{
"whiteCondition": [
{
"name": "source_db",
"map": "sink_db",
"all": false,
"subs": [
{
"name": "source_table",
"map": "sink_table"
}
]
}
],
"blackCondition": [
]
}
The Condition module is a multi-layer nested parameter structure. The first layer is the database layer, the second layer is the table layer, and the third layer is the column layer.
| Parameter | Description |
|---|---|
| name | The name of the object at the current layer in the source. For example, the first layer is a database name and the second layer is a table name. |
| map | The name of the object written to the destination. If the destination is a database, the object type in the destination is the same as that in the source. If the destination is a message queue, such as Kafka or RocketMQ, where only topic namespaces exist, map at the table level indicates the name of a table. For a destination type with the project or topic two-layer mapping, such as DataHub, map corresponds to a database and a table. Note that columns map indicates a column renaming mechanism. |
| all | Specifies whether to synchronize lower-level objects. For example, if all is set to true at the database level, all tables in the source database need to be synchronized. Otherwise, only tables defined by subs are synchronized. |
The preceding parameters can satisfy common migration requirements. OceanBase Migration Service (OMS) also provides wildcard and regular expression (regex) matching capabilities and attrs capabilities.
Wildcard and regex matching capabilities
{
"whiteCondition": [
{
"name": "source_*",
"map": "sink_db",
"func": "fn",
"all": false,
"subs": [
{
"name": "source_*",
"map": "sink_table",
"func": "fn"
}
]
}
],
"blackCondition": [
{
"name": "source_db_[0-10]",
"map": "sink_db",
"func": "reg",
"all": false,
"subs": [
{
"name": "source_table_[0-10]",
"map": "sink_table",
"func": "reg"
}
]
}
]
}
The func parameter specifies the matching function, which is equals by default. reg indicates regex match. fn indicates fnmatch.
The preceding example specifies to synchronize all tables starting with source in all databases starting with source to sink_db.sink_table, excluding the source_table_0...source_table_10 table in the source_db_0...source_db_10 database.
attrs capabilities
Filter DML statements
{ "whiteCondition": [ { "name": "source_db", "map": "sink_db", "all": false, "attr": { "dml": "i,u,d" }, "subs": [ { "name": "source_table", "map": "sink_table" } ] } ], "blackCondition": [] }In
attr,dml:i,u,dindicates that the project only synchronizes messages ofINSERT,DELETE, andUPDATEchanges. Note thatINSERTchanges do not involve full migration messages.Default
attrproperties are inherited. For example, if the table-levelattrproperty does not exist, OMS tries to obtain the parent-levelattrproperty.Filter DDL statements
{ "whiteCondition": [ { "name": "source_db", "map": "sink_db", "all": false, "subs": [ { "name": "source_table", "map": "sink_table", "attr": { "ddl": "ct,dt" } } ] } ], "blackCondition": [] }In
attr,ddl: ct,dtindicates that onlyCREATE TABLEandDROP TABLEDDL statements are accepted.Data conversion using SQL syntax
{ "whiteCondition": [ { "name": "source_db", "map": "sink_db", "all": false, "subs": [ { "name": "source_table", "map": "sink_table", "attr": { "op": "select substr(c1, 1, 2) as c3 from record where c2 = x" } } ] } ], "blackCondition": [] }In
attr,opuses SQL syntax to indicate the column computing and filtering capabilities of records. In the preceding example,select substr(c1, 1, 2) as c3 from record where c2 = xindicates that the following operations need to be performed onsource_db.source_table:Retrieve the data that matches
c2 = x.Truncate the
c1column to 2 bits, rename the new column asc3, and pass it down. Note that thec1column still exists.
Some
attrsproperties in message queue scenarios{ "whiteCondition": [ { "name": "source_db", "all": false, "subs": [ { "name": "source_table", "map": "topic_1", "attr": { "shard_fields": "c1,c2", "ddl_map": "topic_2" } } ] } ], "blackCondition": [] }shard_fieldsindicates that thec1andc2columns are used as the sharding key columns ofsource_table. This parameter specifies the partition, shard, or queue to which the data record is delivered.ddl_mapspecifies to deliver the data change message totopic_2when a DDL change is performed onsource_table.
Parameters of a destination MySQL tenant of OceanBase Database
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| enablePartitionBucket | BOOLEAN | false | Specifies whether to enable the feature of writing data to partitions. | OMS V3.3.1 and later |
| sysInstance | STRING | null | The database service connection information of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| maxRetryTimes | LONG | 1000 | The maximum number of retries allowed in the case of a destination writing failure. | OMS V3.3.1 and later |
| useSSL | BOOLEAN | false | Specifies whether to use SSL. | OMS V3.3.1 and later |
| retryIntervalTime | LONG | 1000 | The retry interval, in milliseconds. | OMS V3.3.1 and later |
| transactionTableReserveDays | LONG | 7 | The retention period for transaction tables, in days. | OMS V3.3.1 and later |
| shardFuncClassName | STRING | com.oceanbase.oms.connector.jdbc.sink.JdbcSinkRouteComputeFunc |
The implementation of the partition function. This parameter is used together with ShuffleDispatcher. This parameter is not specified by default. |
OMS V3.3.1 and later |
| workerNum | LONG | 16 | The number of threads for data writing. | OMS V3.3.1 and later |
| ignoreRedunantColumnsReplicate | BOOLEAN | false | Specifies whether to ignore redundant columns in the source to be written. If the parameter is set to true, redundant columns in the source are ignored. If the parameter is set to true, data cannot be written when the source has redundant columns. |
OMS V3.3.1 and later |
| correctMode | LIST | null | The description of the compatible format for writing incompatible data to the destination. Currently, this parameter only supports the Oracle mode of OceanBase Database. Valid values are the following ones. Separate different values with a comma (,).
|
OMS V4.2.2 and later |
| delayThresholdToCompareAllColumns | LONG | 10 | The incremental delay threshold for switching to primary key matching when isCompareAllColumns is set to true. |
OMS V3.3.1 and later |
| jdbcUrl | STRING | null | The connection string of the destination. | OMS V3.3.1 and later |
| timezone | STRING | +08:00 | The time zone. | OMS V3.3.1 and later |
| useTxId | BOOLEAN | false | Specifies whether to use a transaction ID. | OMS V3.3.1 and later |
| canGeometryBatched | BOOLEAN | The system provides a recommended value based on the sinkTypeparameter. |
Specifies whether the geometry format supports batch writing. | OMS V3.3.1 and later |
| binlogOffsetDb | STRING | null | The database for storing binlog offsets when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| binlogOffsetTable | STRING | binlog_offset | The table for storing binlog offsets when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| isCompareAllColumns | BOOLEAN | false | Specifies whether to enable full column matching, that is, whether to add all columns to the WHERE condition for incremental UPDATE and DELETE operations. | OMS V3.3.1 and later |
| password | STRING | null | The JDBC password. | OMS V3.3.1 and later |
| sysPassword | STRING | null | The password of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| disableForeignKeyCheck | BOOLEAN | true | Specifies whether to disable the foreign key check. | OMS V3.3.1 and later |
| enableTransactionWriter | BOOLEAN | true | Specifies whether to enable transaction writing capabilities. | OMS V3.3.1 and later |
| isUkDependSource | BOOLEAN | false | An Oracle or PostgreSQL source may have incomplete incremental logs. If the source table does not have a primary key, all source columns can be selected by using the index in the destination. The value of this parameter is automatically calculated internally. | OMS V3.3.1 and later |
| clientQueryPartitionWithInnerTable | BOOLEAN | true | Specifies whether to use internal tables to query partition metadata. | OMS V4.2.1 and later |
| txnDb | STRING | * | The database in which the transaction table that prevents cyclical replication and use is located. | OMS V3.3.1 and later |
| dependTargetSchema | BOOLEAN | true | Specifies whether to write data to the destination based on its schema. The default value is true. If the parameter is set to false, the destination schema is not queried. In some cases where the destination is an OceanBase database, you may need to set this parameter to false. |
OMS V3.3.1 and later |
| enableNoUniqueConstraintTableReplicate | BOOLEAN | false | Specifies whether to migrate tables without a primary key during data migration from a MySQL database, to a PyMySQL or PostgreSQL database, and then to an Oracle tenant of OceanBase Database, or from an Oracle tenant of OceanBase Database to a PyMySQL or PostgreSQL database, and then to a MySQL database. | null |
| sysUsername | STRING | null | The username of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| sinkType | STRING | null | The type of the destination data source. | OMS V3.3.1 and later |
| localRegionNo | STRING | null | The number of the region to which data is written. | OMS V3.3.1 and later |
| dryRun | BOOLEAN | false | Specifies whether to enable dry-run without executing any SQL statements. | OMS V3.3.1 and later |
| username | STRING | null | The JDBC username. | OMS V3.3.1 and later |
| filterHiddenPK | BOOLEAN | false | This parameter takes effect only when enableNoUniqueConstraintTableReplicate is set to true and the source and destination are of a certain combination. This parameter specifies whether to ignore the hidden primary keys of tables without primary keys in OceanBase Database. The hidden columns will be filtered out during data writing to the destination. |
OMS V3.3.1 and later |
| sqlMode | STRING | null | The SQL mode. | OMS V3.3.1 and later |
| charset | STRING | The system provides a recommended value based on the jdbcUrl, username, password, and sinkType parameters. |
The character set. | OMS V3.3.1 and later |
| writeMode | STRING | batch | The write mode of the destination. Batch writing and non-batch writing are supported. | OMS V3.3.1 and later |
| isPreLoadAllTableSchema | BOOLEAN | false | Specifies whether to preload all table schemas in the destination to the memory for a query when dependTargetSchema is set to true. |
OMS V4.2.2 and later |
| Booster class | STRING | null | The startup class, which is com.oceanbase.oms.connector.jdbc.sink.obmysql.OBMySQLJDBCSinkBooster. |
OMS V3.3.1 and later |
| isMysqlReplaceMode | BOOLEAN | false | Specifies whether to use the replace mode to write data to the destination MySQL database or the MySQL tenant of OceanBase Database. | OMS V3.3.1 and later |
| canLobBatched | BOOLEAN | The system provides a recommended value based on the sinkTypeparameter. |
Specifies whether LOB data can be rewritten in batch. This feature does not support Oracle or the Oracle mode of OceanBase Database, because null values of the LOB type are written by using the empty_clob and empty_blob functions. |
OMS V3.3.1 and later |
| slowSqlThreshold | LONG | 15 | The threshold of a slow SQL query, which controls the printing and execution of SQL statements. | OMS V3.3.1 and later |
| printSql | BOOLEAN | false | Specifies whether to print executed SQL statements. | OMS V3.3.1 and later |
| tenant | STRING | null | The tenant. | OMS V3.3.1 and later |
| obFreeMemoryMin | LONG | 20 | The threshold of the available memory of OceanBase Database for suspending data writing. If the value is greater than or equal to 0, OMS checks the available memory of OceanBase Database in the destination against the threshold, and suspends data writing if the available memory is less than the threshold until it exceeds the specified value in the next detection cycle. | OMS V3.3.1 and later |
| highWatermarkQueryIntervalSec | LONG | 1 | The interval, in seconds, at which the program queries the CPU idle rate or available memory in OceanBase Database if the value of obFreeCpuMin is larger than 0 or the value of obFreeMemoryMin is larger than 0. |
OMS V3.3.1 and later |
| obFreeCpuMin | LONG | 0 | The threshold of the idle CPU rate of OceanBase Database for suspending data writing. If the value is greater than 0, OMS checks the idle CPU rate of OceanBase Database in the destination against the threshold, and suspends data writing if the idle CPU rate is less than the threshold until it exceeds the specified value in the next detection cycle. Note that CPU statistics may be inaccurate. Therefore, we recommend that you do not specify this parameter. | OMS V3.3.1 and later |
| retryErrorMessage | STRING | null | The message for the error for which a retry is performed. | OMS V3.3.1 and later |
| skipErrorMessage | STRING | null | The message for the error to skip. | OMS V3.3.1 and later |
| skipErrorCode | STRING | null | The code for the error to skip. | OMS V3.3.1 and later |
| retryErrorCode | STRING | null | The code for the error for which a retry is performed. | OMS V3.3.1 and later |
Parameters of a destination Oracle tenant of OceanBase Database
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| enablePartitionBucket | BOOLEAN | false | Specifies whether to enable the feature of writing data to partitions. | OMS V3.3.1 and later |
| sysInstance | STRING | null | The database service connection information of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| obSupportNlsSet | BOOLEAN | true | Specifies whether to reset the NLS format. This parameter is available only when the destination is an Oracle tenant of OceanBase Database. | OMS V3.3.1 and later |
| maxRetryTimes | LONG | 1000 | The maximum number of retries allowed in the case of a destination writing failure. | OMS V3.3.1 and later |
| useSSL | BOOLEAN | false | Specifies whether to use SSL. | OMS V3.3.1 and later |
| retryIntervalTime | LONG | 1000 | The retry interval, in milliseconds. | OMS V3.3.1 and later |
| transactionTableReserveDays | LONG | 7 | The retention period for transaction tables, in days. | OMS V3.3.1 and later |
| shardFuncClassName | STRING | com.oceanbase.oms.connector.jdbc.sink.JdbcSinkRouteComputeFunc |
The implementation of the partition function. This parameter is used together with ShuffleDispatcher. This parameter is not specified by default. |
OMS V3.3.1 and later |
| workerNum | LONG | 16 | The number of threads for data writing. | OMS V3.3.1 and later |
| ignoreRedunantColumnsReplicate | BOOLEAN | false | Specifies whether to ignore redundant columns in the source to be written. If the parameter is set to true, redundant columns in the source are ignored. If the parameter is set to true, data cannot be written when the source has redundant columns. |
OMS V3.3.1 and later |
| correctMode | LIST | null | The description of the compatible format for writing incompatible data to the destination. Currently, this parameter only supports the Oracle mode of OceanBase Database. Valid values are the following ones. Separate different values with a comma (,).
|
OMS V4.2.2 and later |
| delayThresholdToCompareAllColumns | LONG | 10 | The incremental delay threshold for switching to primary key matching when isCompareAllColumns is set to true. |
OMS V3.3.1 and later |
| jdbcUrl | STRING | null | The connection string of the destination. | OMS V3.3.1 and later |
| timezone | STRING | +08:00 | The time zone. | OMS V3.3.1 and later |
| useTxId | BOOLEAN | false | Specifies whether to use a transaction ID. | OMS V3.3.1 and later |
| canGeometryBatched | BOOLEAN | The system provides a recommended value based on the sinkType parameter. |
Specifies whether the geometry format supports batch writing. | OMS V3.3.1 and later |
| binlogOffsetDb | STRING | null | The database for storing binlog offsets when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| binlogOffsetTable | STRING | binlog_offset | The table for storing binlog offsets when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| isCompareAllColumns | BOOLEAN | false | Specifies whether to enable full column matching, that is, whether to add all columns to the WHERE condition for incremental UPDATE and DELETE operations. | OMS V3.3.1 and later |
| password | STRING | null | The JDBC password. | OMS V3.3.1 and later |
| sysPassword | STRING | null | The password of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| disableForeignKeyCheck | BOOLEAN | true | Specifies whether to disable the foreign key check. | OMS V3.3.1 and later |
| enableTransactionWriter | BOOLEAN | true | Specifies whether to enable transaction writing capabilities. | OMS V3.3.1 and later |
| isUkDependSource | BOOLEAN | false | An Oracle or PostgreSQL source may have incomplete incremental logs. If the source table does not have a primary key, all source columns can be selected by using the index in the destination. The value of this parameter is automatically calculated internally. | OMS V3.3.1 and later |
| clientQueryPartitionWithInnerTable | BOOLEAN | true | Specifies whether to use internal tables to query partition metadata. | OMS V4.2.1 and later |
| txnDb | STRING | * | The database in which the transaction table that prevents cyclical replication and use is located. | OMS V3.3.1 and later |
| dependTargetSchema | BOOLEAN | true | Specifies whether to write data to the destination based on its schema. The default value is true. If the parameter is set to false, the destination schema is not queried. In some cases where the destination is an OceanBase database, you may need to set this parameter to false. |
OMS V3.3.1 and later |
| enableNoUniqueConstraintTableReplicate | BOOLEAN | false | Specifies whether to migrate tables without a primary key during data migration from a MySQL database, to a PyMySQL or PostgreSQL database, and then to an Oracle tenant of OceanBase Database, or from an Oracle tenant of OceanBase Database to a PyMySQL or PostgreSQL database, and then to a MySQL database. | OMS V4.2.2 and later |
| sysUsername | STRING | null | The username of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| sinkType | STRING | null | The type of the destination data source. | OMS V3.3.1 and later |
| localRegionNo | STRING | null | The number of the region to which data is written. | OMS V3.3.1 and later |
| dryRun | BOOLEAN | false | Specifies whether to enable dry-run without executing any SQL statements. | OMS V3.3.1 and later |
| username | STRING | null | The JDBC username. | OMS V3.3.1 and later |
| filterHiddenPK | BOOLEAN | false | This parameter takes effect only when enableNoUniqueConstraintTableReplicate is set to true and the source and destination are of a certain combination. This parameter specifies whether to ignore the hidden primary keys of tables without primary keys in OceanBase Database. The hidden columns will be filtered out during data writing to the destination. |
OMS V3.3.1 and later |
| sqlMode | STRING | null | The SQL mode. | OMS V3.3.1 and later |
| charset | STRING | The system provides a recommended value based on the jdbcUrl, username, password, and sinkType parameters. |
The character set. | OMS V3.3.1 and later |
| writeMode | STRING | batch | The write mode of the destination. Batch writing and non-batch writing are supported. | OMS V3.3.1 and later |
| isPreLoadAllTableSchema | BOOLEAN | false | Specifies whether to preload all table schemas in the destination to the memory for a query when dependTargetSchema is set to true. |
OMS V4.2.2 and later |
| Booster class | STRING | null | The startup class, which is com.oceanbase.oms.connector.jdbc.sink.oboracle.OBOracleJDBCSinkBooster. |
OMS V3.3.1 and later |
| isMysqlReplaceMode | BOOLEAN | false | Specifies whether to use the replace mode to write data to the destination MySQL database or the MySQL tenant of OceanBase Database. | OMS V3.3.1 and later |
| canLobBatched | BOOLEAN | The system provides a recommended value based on the sinkTypeparameter. |
Specifies whether LOB data can be rewritten in batch. This feature does not support Oracle or the Oracle mode of OceanBase Database, because null values of the LOB type are written by using the empty_clob and empty_blob functions. |
OMS V3.3.1 and later |
| slowSqlThreshold | LONG | 15 | The threshold of a slow SQL query, which controls the printing and execution of SQL statements. | OMS V3.3.1 and later |
| printSql | BOOLEAN | false | Specifies whether to print executed SQL statements. | OMS V3.3.1 and later |
| tenant | STRING | null | The tenant. | OMS V3.3.1 and later |
| obFreeMemoryMin | LONG | 20 | The threshold of the available memory of OceanBase Database for suspending data writing. If the value is greater than or equal to 0, OMS checks the available memory of OceanBase Database in the destination against the threshold, and suspends data writing if the available memory is less than the threshold until it exceeds the specified value in the next detection cycle. | OMS V3.3.1 and later |
| highWatermarkQueryIntervalSec | LONG | 1 | The interval, in seconds, at which the program queries the CPU idle rate or available memory in OceanBase Database if the value of obFreeCpuMin is larger than 0 or the value of obFreeMemoryMin is larger than 0. |
OMS V3.3.1 and later |
| obFreeCpuMin | LONG | 0 | The threshold of the idle CPU rate of OceanBase Database for suspending data writing. If the value is greater than 0, OMS checks the idle CPU rate of OceanBase Database in the destination against the threshold, and suspends data writing if the idle CPU rate is less than the threshold until it exceeds the specified value in the next detection cycle. Note that CPU statistics may be inaccurate. Therefore, we recommend that you do not specify this parameter. | OMS V3.3.1 and later |
| retryErrorMessage | STRING | null | The message for the error for which a retry is performed. | OMS V3.3.1 and later |
| skipErrorMessage | STRING | null | The message for the error to skip. | OMS V3.3.1 and later |
| skipErrorCode | STRING | null | The code for the error to skip. | OMS V3.3.1 and later |
| retryErrorCode | STRING | null | The code for the error for which a retry is performed. | OMS V3.3.1 and later |
Parameters of a destination MySQL or PostgreSQL instance
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| enablePartitionBucket | BOOLEAN | false | Specifies whether to enable the feature of writing data to partitions. | OMS V3.3.1 and later |
| sysInstance | STRING | null | The database service connection information of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| maxRetryTimes | LONG | 1000 | The maximum number of retries allowed in the case of a destination writing failure. | OMS V3.3.1 and later |
| useSSL | BOOLEAN | false | Specifies whether to use SSL. | OMS V3.3.1 and later |
| retryIntervalTime | LONG | 1000 | The retry interval, in milliseconds. | OMS V3.3.1 and later |
| transactionTableReserveDays | LONG | 7 | The retention period for transaction tables, in days. | OMS V3.3.1 and later |
| shardFuncClassName | STRING | com.oceanbase.oms.connector.jdbc.sink.JdbcSinkRouteComputeFunc |
The implementation of the partition function. This parameter is used together with ShuffleDispatcher. This parameter is not specified by default. |
OMS V3.3.1 and later |
| workerNum | LONG | 16 | The number of threads for data writing. | OMS V3.3.1 and later |
| ignoreRedunantColumnsReplicate | BOOLEAN | false | Specifies whether to ignore redundant columns in the source to be written. If the parameter is set to true, redundant columns in the source are ignored. If the parameter is set to true, data cannot be written when the source has redundant columns. |
OMS V3.3.1 and later |
| correctMode | LIST | null | The description of the compatible format for writing incompatible data to the destination. Currently, this parameter only supports the Oracle mode of OceanBase Database. Valid values are the following ones. Separate different values with a comma (,).
|
OMS V4.2.2 and later |
| delayThresholdToCompareAllColumns | LONG | 10 | The incremental delay threshold for switching to primary key matching when isCompareAllColumns is set to true. |
OMS V3.3.1 and later |
| jdbcUrl | STRING | null | The connection string of the destination. | OMS V3.3.1 and later |
| timezone | STRING | +08:00 | The time zone. | OMS V3.3.1 and later |
| useTxId | BOOLEAN | false | Specifies whether to use a transaction ID. | OMS V3.3.1 and later |
| canGeometryBatched | BOOLEAN | The system provides a recommended value based on the sinkTypeparameter. |
Specifies whether the geometry format supports batch writing. | OMS V3.3.1 and later |
| binlogOffsetDb | STRING | null | The database for storing binlog offsets when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| binlogOffsetTable | STRING | binlog_offset | The table for storing binlog offsets when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| isCompareAllColumns | BOOLEAN | false | Specifies whether to enable full column matching, that is, whether to add all columns to the WHERE condition for incremental UPDATE and DELETE operations. | OMS V3.3.1 and later |
| password | STRING | null | The JDBC password. | OMS V3.3.1 and later |
| sysPassword | STRING | null | The password of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| disableForeignKeyCheck | BOOLEAN | true | Specifies whether to disable the foreign key check. | OMS V3.3.1 and later |
| enableTransactionWriter | BOOLEAN | true | Specifies whether to enable transaction writing capabilities. | OMS V3.3.1 and later |
| isUkDependSource | BOOLEAN | false | An Oracle or PostgreSQL source may have incomplete incremental logs. If the source table does not have a primary key, all source columns can be selected by using the index in the destination. The value of this parameter is automatically calculated internally. | OMS V3.3.1 and later |
| clientQueryPartitionWithInnerTable | BOOLEAN | true | Specifies whether to use internal tables to query partition metadata. | OMS V4.2.1 and later |
| txnDb | STRING | * | The database in which the transaction table that prevents cyclical replication and use is located. | OMS V3.3.1 and later |
| dependTargetSchema | BOOLEAN | true | Specifies whether to write data to the destination based on its schema. The default value is true. If the parameter is set to false, the destination schema is not queried. In some cases where the destination is an OceanBase database, you may need to set this parameter to false. |
OMS V3.3.1 and later |
| enableNoUniqueConstraintTableReplicate | BOOLEAN | false | Specifies whether to migrate tables without a primary key during data migration from a MySQL database, to a PyMySQL or PostgreSQL database, and then to an Oracle tenant of OceanBase Database, or from an Oracle tenant of OceanBase Database to a PyMySQL or PostgreSQL database, and then to a MySQL database. | OMS V4.2.2 and later |
| sysUsername | STRING | null | The username of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| sinkType | STRING | null | The type of the destination data source. | OMS V3.3.1 and later |
| localRegionNo | STRING | null | The number of the region to which data is written. | OMS V3.3.1 and later |
| dryRun | BOOLEAN | false | Specifies whether to enable dry-run without executing any SQL statements. | OMS V3.3.1 and later |
| username | STRING | null | The JDBC username. | OMS V3.3.1 and later |
| filterHiddenPK | BOOLEAN | false | This parameter takes effect only when enableNoUniqueConstraintTableReplicate is set to true and the source and destination are of a certain combination. This parameter specifies whether to ignore the hidden primary keys of tables without primary keys in OceanBase Database. The hidden columns will be filtered out during data writing to the destination. |
OMS V3.3.1 and later |
| sqlMode | STRING | null | The SQL mode. | OMS V3.3.1 and later |
| charset | STRING | The system provides a recommended value based on the jdbcUrl, username, password, and sinkType parameters. |
The character set. | OMS V3.3.1 and later |
| writeMode | STRING | batch | The write mode of the destination. Batch writing and non-batch writing are supported. | OMS V3.3.1 and later |
| isPreLoadAllTableSchema | BOOLEAN | false | Specifies whether to preload all table schemas in the destination to the memory for a query when dependTargetSchema is set to true. |
OMS V4.2.2 and later |
| Booster class | STRING | null | The starter class. The starter class of a MySQL database is com.oceanbase.oms.connector.jdbc.sink.mysql.MySQLJDBCSinkBooster. The starter class of a PostgreSQL database is com.oceanbase.oms.connector.jdbc.sink.pg.PgJDBCSinkBooster. |
OMS V3.3.1 and later |
| isMysqlReplaceMode | BOOLEAN | false | Specifies whether to use the replace mode to write data to the destination MySQL database or the MySQL tenant of OceanBase Database. | OMS V3.3.1 and later |
| canLobBatched | BOOLEAN | The system provides a recommended value based on the sinkTypeparameter. |
Specifies whether LOB data can be rewritten in batch. This feature does not support Oracle or the Oracle mode of OceanBase Database, because null values of the LOB type are written by using the empty_clob and empty_blob functions. |
OMS V3.3.1 and later |
| slowSqlThreshold | LONG | 15 | The threshold of a slow SQL query, which controls the printing and execution of SQL statements. | OMS V3.3.1 and later |
| printSql | BOOLEAN | false | Specifies whether to print executed SQL statements. | OMS V3.3.1 and later |
| tenant | STRING | null | The tenant. | OMS V3.3.1 and later |
| obFreeMemoryMin | LONG | 20 | The threshold of the available memory of OceanBase Database for suspending data writing. If the value is greater than or equal to 0, OMS checks the available memory of OceanBase Database in the destination against the threshold, and suspends data writing if the available memory is less than the threshold until it exceeds the specified value in the next detection cycle. | OMS V3.3.1 and later |
| highWatermarkQueryIntervalSec | LONG | 1 | The interval, in seconds, at which the program queries the CPU idle rate or available memory in OceanBase Database if the value of obFreeCpuMin is larger than 0 or the value of obFreeMemoryMin is larger than 0. |
OMS V3.3.1 and later |
| obFreeCpuMin | LONG | 0 | The threshold of the idle CPU rate of OceanBase Database for suspending data writing. If the value is greater than 0, OMS checks the idle CPU rate of OceanBase Database in the destination against the threshold, and suspends data writing if the idle CPU rate is less than the threshold until it exceeds the specified value in the next detection cycle. Note that CPU statistics may be inaccurate. Therefore, we recommend that you do not specify this parameter. | OMS V3.3.1 and later |
| retryErrorMessage | STRING | null | The message for the error for which a retry is performed. | OMS V3.3.1 and later |
| skipErrorMessage | STRING | null | The message for the error to skip. | OMS V3.3.1 and later |
| skipErrorCode | STRING | null | The code for the error to skip. | OMS V3.3.1 and later |
| retryErrorCode | STRING | null | The code for the error for which a retry is performed. | OMS V3.3.1 and later |
Parameters of a destination Oracle instance
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| enablePartitionBucket | BOOLEAN | false | Specifies whether to enable the feature of writing data to partitions. | OMS V3.3.1 and later |
| sysInstance | STRING | null | The database service connection information of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| maxRetryTimes | LONG | 1000 | The maximum number of retries allowed in the case of a destination writing failure. | OMS V3.3.1 and later |
| useSSL | BOOLEAN | false | Specifies whether to use SSL. | OMS V3.3.1 and later |
| retryIntervalTime | LONG | 1000 | The retry interval, in milliseconds. | OMS V3.3.1 and later |
| transactionTableReserveDays | LONG | 7 | The retention period for transaction tables, in days. | OMS V3.3.1 and later |
| shardFuncClassName | STRING | com.oceanbase.oms.connector.jdbc.sink.JdbcSinkRouteComputeFunc |
The implementation of the partition function. This parameter is used together with ShuffleDispatcher. This parameter is not specified by default. |
OMS V3.3.1 and later |
| workerNum | LONG | 16 | The number of threads for data writing. | OMS V3.3.1 and later |
| ignoreRedunantColumnsReplicate | BOOLEAN | false | Specifies whether to ignore redundant columns in the source to be written. If the parameter is set to true, redundant columns in the source are ignored. If the parameter is set to true, data cannot be written when the source has redundant columns. |
OMS V3.3.1 and later |
| correctMode | LIST | null | The description of the compatible format for writing incompatible data to the destination. Currently, this parameter only supports the Oracle mode of OceanBase Database. Valid values are the following ones. Separate different values with a comma (,).
|
OMS V4.2.2 and later |
| delayThresholdToCompareAllColumns | LONG | 10 | The incremental delay threshold for switching to primary key matching when isCompareAllColumns is set to true. |
OMS V3.3.1 and later |
| jdbcUrl | STRING | null | The connection string of the destination. | OMS V3.3.1 and later |
| timezone | STRING | +08:00 | The time zone. | OMS V3.3.1 and later |
| useTxId | BOOLEAN | false | Specifies whether to use a transaction ID. | OMS V3.3.1 and later |
| canGeometryBatched | BOOLEAN | The system provides a recommended value based on the sinkTypeparameter. |
Specifies whether the geometry format supports batch writing. | OMS V3.3.1 and later |
| binlogOffsetDb | STRING | null | The database for storing binlog offsets when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| binlogOffsetTable | STRING | binlog_offset | The table for storing binlog offsets when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| isCompareAllColumns | BOOLEAN | false | Specifies whether to enable full column matching, that is, whether to add all columns to the WHERE condition for incremental UPDATE and DELETE operations. | OMS V3.3.1 and later |
| password | STRING | null | The JDBC password. | OMS V3.3.1 and later |
| sysPassword | STRING | null | The password of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| disableForeignKeyCheck | BOOLEAN | true | Specifies whether to disable the foreign key check. | OMS V3.3.1 and later |
| enableTransactionWriter | BOOLEAN | true | Specifies whether to enable transaction writing capabilities. | OMS V3.3.1 and later |
| isUkDependSource | BOOLEAN | false | An Oracle or PostgreSQL source may have incomplete incremental logs. If the source table does not have a primary key, all source columns can be selected by using the index in the destination. The value of this parameter is automatically calculated internally. | OMS V3.3.1 and later |
| clientQueryPartitionWithInnerTable | BOOLEAN | true | Specifies whether to use internal tables to query partition metadata. | OMS V4.2.1 and later |
| txnDb | STRING | * | The database in which the transaction table that prevents cyclical replication and use is located. | OMS V3.3.1 and later |
| dependTargetSchema | BOOLEAN | true | Specifies whether to write data to the destination based on its schema. The default value is true. If the parameter is set to false, the destination schema is not queried. In some cases where the destination is an OceanBase database, you may need to set this parameter to false. |
OMS V3.3.1 and later |
| enableNoUniqueConstraintTableReplicate | BOOLEAN | false | Specifies whether to migrate tables without a primary key during data migration from a MySQL database, to a PyMySQL or PostgreSQL database, and then to an Oracle tenant of OceanBase Database, or from an Oracle tenant of OceanBase Database to a PyMySQL or PostgreSQL database, and then to a MySQL database. | OMS V4.2.2 and later |
| sysUsername | STRING | null | The username of the sys tenant of OceanBase Database. | OMS V3.3.1 and later |
| sinkType | STRING | null | The type of the destination data source. | OMS V3.3.1 and later |
| localRegionNo | STRING | null | The number of the region to which data is written. | OMS V3.3.1 and later |
| dryRun | BOOLEAN | false | Specifies whether to enable dry-run without executing any SQL statements. | OMS V3.3.1 and later |
| username | STRING | null | The JDBC username. | OMS V3.3.1 and later |
| filterHiddenPK | BOOLEAN | false | This parameter takes effect only when enableNoUniqueConstraintTableReplicate is set to true and the source and destination are of a certain combination. This parameter specifies whether to ignore the hidden primary keys of tables without primary keys in OceanBase Database. The hidden columns will be filtered out during data writing to the destination. |
OMS V3.3.1 and later |
| sqlMode | STRING | null | The SQL mode. | OMS V3.3.1 and later |
| charset | STRING | The system provides a recommended value based on the jdbcUrl, username, password, and sinkType parameters. |
The character set. | OMS V3.3.1 and later |
| writeMode | STRING | batch | The write mode of the destination. Batch writing and non-batch writing are supported. | OMS V3.3.1 and later |
| isPreLoadAllTableSchema | BOOLEAN | false | Specifies whether to preload all table schemas in the destination to the memory for a query when dependTargetSchema is set to true. |
OMS V4.2.2 and later |
| oracleSocketTimeout | LONG | 50,000 | The timeout period for an Oracle sink, in milliseconds. | OMS V3.3.1 and later |
| Booster class | STRING | null | The startup class, which is com.oceanbase.oms.connector.jdbc.sink.oracle.OracleJDBCSinkBooster. |
OMS V3.3.1 and later |
| isMysqlReplaceMode | BOOLEAN | false | Specifies whether to use the replace mode to write data to the destination MySQL database or the MySQL tenant of OceanBase Database. | OMS V3.3.1 and later |
| canLobBatched | BOOLEAN | The system provides a recommended value based on the sinkTypeparameter. |
Specifies whether LOB data can be rewritten in batch. This feature does not support Oracle or the Oracle mode of OceanBase Database, because null values of the LOB type are written by using the empty_clob and empty_blob functions. |
OMS V3.3.1 and later |
| slowSqlThreshold | LONG | 15 | The threshold of a slow SQL query, which controls the printing and execution of SQL statements. | OMS V3.3.1 and later |
| printSql | BOOLEAN | false | Specifies whether to print executed SQL statements. | OMS V3.3.1 and later |
| tenant | STRING | null | The tenant. | OMS V3.3.1 and later |
| obFreeMemoryMin | LONG | 20 | The threshold of the available memory of OceanBase Database for suspending data writing. If the value is greater than or equal to 0, OMS checks the available memory of OceanBase Database in the destination against the threshold, and suspends data writing if the available memory is less than the threshold until it exceeds the specified value in the next detection cycle. | OMS V3.3.1 and later |
| highWatermarkQueryIntervalSec | LONG | 1 | The interval, in seconds, at which the program queries the CPU idle rate or available memory in OceanBase Database if the value of obFreeCpuMin is larger than 0 or the value of obFreeMemoryMin is larger than 0. |
OMS V3.3.1 and later |
| obFreeCpuMin | LONG | 0 | The threshold of the idle CPU rate of OceanBase Database for suspending data writing. If the value is greater than 0, OMS checks the idle CPU rate of OceanBase Database in the destination against the threshold, and suspends data writing if the idle CPU rate is less than the threshold until it exceeds the specified value in the next detection cycle. Note that CPU statistics may be inaccurate. Therefore, we recommend that you do not specify this parameter. | OMS V3.3.1 and later |
| retryErrorMessage | STRING | null | The message for the error for which a retry is performed. | OMS V3.3.1 and later |
| skipErrorMessage | STRING | null | The message for the error to skip. | OMS V3.3.1 and later |
| skipErrorCode | STRING | null | The code for the error to skip. | OMS V3.3.1 and later |
| retryErrorCode | STRING | null | The code for the error for which a retry is performed. | OMS V3.3.1 and later |
Parameters of a destination RocketMQ instance
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| clientThreadCount | LONG | -1 | The size of the asynchronous sending thread pool of RocketMQ. The default value depends on the CPU size of the machine. | OMS V3.3.1 and later |
| tags | STRING | null | The tags that are written to messages. If a message contains a table name, the default value is tableName + _tag. Otherwise, the default value is default. |
OMS V3.3.1 and later |
| workerNum | LONG | 16 | The number of threads for data writing. | OMS V3.3.1 and later |
| clientCallbackExecutorThreadsNum | LONG | -1 | The number of threads for request processing in RocketMQ. The default value depends on the CPU size of the machine. | OMS V3.3.1 and later |
| enableTrace | BOOLEAN | false | The enableTrace setting of the RocketMQ client. enableTrace specifies whether to enable message tracing. |
OMS V3.3.1 and later |
| sinkType | STRING | null | The type of the destination data source. | OMS V3.3.1 and later |
| nameSvrAddr | STRING | null | The nameSvrAddr setting in RocketMQ, which specifies the list of NameServer addresses. |
OMS V3.3.1 and later |
| Booster class | STRING | null | The startup class, which is com.oceanbase.oms.connector.rocketmq.sink.RocketMQSinkBooster. |
OMS V3.3.1 and later |
| producerGroup | STRING | OMS | The producer group in RocketMQ, which is a collection of message producer instances that share the same features or responsibilities. | OMS V3.3.1 and later |
| sendMsgTimeout | LONG | 30000 | The sending timeout period. | OMS V3.3.1 and later |
| namespace | STRING | - | The Namespace in RocketMQ. |
OMS V3.3.1 and later |
| shardFuncClassName | STRING | com.oceanbase.oms.connector.rocketmq.sink.RocketMQShardFunc |
The implementation of the partition function. | OMS V3.3.1 and later |
| partitionMode | STRING | HASH | The distribution logic that determines the partition into which a data record is to be written. Valid values: HASH (primary key HASH) and TABLE (TABLE name HASH). | OMS V3.3.1 and later |
| secretKey | STRING | null | The Secret Key of RocketMQ, which is used for authentication. | OMS V3.3.1 and later |
| accessKey | STRING | null | The Access Key ID of RocketMQ, which is a unique string used to identify a user. | OMS V3.3.1 and later |
| serializerType | STRING | DEFAULT | The serialization format for writing messages to the queue. Valid values: DEFAULT, DEFAULT_WITH_SCHEMA, SHAREPLEX, CANAL, DATAWORKS_V2, and DATAWORKS_V1. | OMS V3.3.1 and later |
| metaExtend | JSON | null | The JSON extension data, which is used to write data to the extend field in DataWorks V2. |
OMS V3.3.1 and later |
| serializerDataTypeMapper | JSON | null | The mapping from SchemaType to STRING. This parameter is for compatibility purpose and solves INT64-related issues. | OMS V4.1.0 and later |
| maxRetryTimes | LONG | 1000 | The maximum number of retries allowed. | OMS V3.3.1 and later |
| retryIntervalTime | LONG | 1000 | The retry interval. | OMS V3.3.1 and later |
| retryErrorMessage | STRING | null | The message for the error for which a retry is performed. | OMS V3.3.1 and later |
| skipErrorMessage | STRING | null | The message for the error to skip. | OMS V3.3.1 and later |
| skipErrorCode | STRING | null | The code for the error to skip. | OMS V3.3.1 and later |
| retryErrorCode | STRING | null | The code for the error for which a retry is performed. | OMS V3.3.1 and later |
Parameters of a destination Kafka instance
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| shardFuncClassName | STRING | com.oceanbase.oms.connector.kafka.sink.KafkaShardFunc | The implementation of the partition function. This parameter is used together with ShuffleDispatcher. This parameter is not specified by default. |
OMS V3.3.1 and later |
| workerNum | LONG | The system provides a recommended value based on the partitionMode parameter. |
The number of threads for data writing. | OMS V3.3.1 and later |
| sinkType | STRING | null | The type of the destination data source. | OMS V3.3.1 and later |
| dryRun | BOOLEAN | false | Specifies whether to enable dry-run without executing any SQL statements. | OMS V3.3.1 and later |
| batchSize | LONG | 1048576 | The producer parameter batch.size in Kafka, which controls the size of the message batches. |
OMS V3.3.1 and later |
| properties | JSON | null | The properties file in Kafka. | OMS V3.3.1 and later |
| consumerGroup | STRING | test | The Kafka consumer to be created for topic existence detection. This parameter corresponds to the consumer parameter group.id. |
OMS V3.3.1 and later |
| servers | STRING | null | The producer parameter bootstrap.servers in Kafka, which specifies a list of servers that the producer needs to connect to when establishing the initial connection to the Kafka cluster. |
OMS V3.3.1 and later |
| Booster class | STRING | null | The startup class, which is com.oceanbase.oms.connector.kafka.sink.KafkaSinkBooster. |
OMS V3.3.1 and later |
| partition | LONG | 0 | The name of the partition to write data to when partitionMode is set to ONE. |
OMS V3.3.1 and later |
| kafkaVersion | STRING | 2.0 | The version of the Kafka client. Valid values: 0.9 and 2.0. | OMS V3.3.1 and later |
| lingerMs | LONG | 100 | The producer parameter linger.ms in Kafka, which controls the message batch processing behavior. |
OMS V3.3.1 and later |
| enableAsync | BOOLEAN | true | Specifies whether to write data by using an asynchronous interface. | OMS V3.3.1 and later |
| isTopicMandatory | BOOLEAN | true | Specifies whether to check the existence of a topic. | OMS V3.3.1 and later |
| partitionMode | STRING | HASH | The distribution logic that determines the partition into which a data record is to be written. Valid values: HASH (primary key HASH) and TABLE (TABLE name HASH). | OMS V3.3.1 and later |
| sslTruststoreLocation | STRING | null | The address of the SSL authentication file. | OMS V3.3.1 and later |
| saslConfig | JSON | null | The SASL configuration. Details: {"mechanism": "The authentication mechanism, which can be GSSAPI, PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.","username": "An account username, which is required for PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.","password": "The account password, which is required for PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.","kerberosPrincipal": "","kerberosKeyTabFileLocation": "An authentication configuration file, which is required for the GSSAPI mechanism.","jaasConfFileLocation": "An authentication configuration file, which is required for the GSSAPI mechanism.","kerberosConfFileLocation": "The address of the Kerberos configuration file, which is required for the GSSAPI mechanism."} |
OMS V3.3.1 and later |
| [Authentication] securityProtocol | STRING | null | The authentication protocol, which can be SASL_PLAINTEXT, SASL_SSL, PLAINTEXTSASL, or SSLSASL. | OMS V3.3.1 and later |
| serializerType | STRING | DEFAULT | The serialization format for writing messages to the queue. Valid values: DEFAULT, DEFAULT_WITH_SCHEMA, SHAREPLEX, CANAL, DATAWORKS_V2, and DATAWORKS_V1. | OMS V3.3.1 and later |
| metaExtend | JSON | null | The JSON extension data, which is used to write data to the extend field in DataWorks V2. |
OMS V3.3.1 and later |
| serializerDataTypeMapper | JSON | null | The mapping from SchemaType to STRING. This parameter is for compatibility purpose and solves INT64-related issues. | OMS V4.1.0 and later |
| retryErrorMessage | STRING | null | The message for the error for which a retry is performed. | OMS V3.3.1 and later |
| skipErrorMessage | STRING | null | The message for the error to skip. | OMS V3.3.1 and later |
| skipErrorCode | STRING | null | The code for the error to skip. | OMS V3.3.1 and later |
| retryErrorCode | STRING | null | The code for the error for which a retry is performed. | OMS V3.3.1 and later |
Parameters of a destination DataHub instance
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| sequenceEnable | BOOLEAN | false | Specifies whether to enable the sorting feature of DataWorks. | OMS V3.3.1 and later |
| requestRetrySec | LONG | 3600 | The retry interval for a failed write request, in seconds. | OMS V3.3.1 and later |
| shardFuncClassName | STRING | com.oceanbase.oms.connector.datahub.sink.DatahubShardFunc |
The implementation of the partition function. This parameter is used together with ShuffleDispatcher. This parameter is not specified by default. |
OMS V3.3.1 and later |
| enableBinary | BOOLEAN | true | Specifies whether binary writing is enabled on the DataHub client. | OMS V3.3.1 and later |
| workerNum | LONG | 16 | The number of threads for data writing. | OMS V3.3.1 and later |
| accessKey | STRING | null | The DataHub access key ID, which is a unique string used to identify a user. | OMS V3.3.1 and later |
| connectionTimeoutMs | LONG | 10000 | The timeout period of the DataHub client. | OMS V3.3.1 and later |
| sinkType | STRING | null | The data source type of the destination. | OMS V3.3.1 and later |
| sequenceStartTimestamp | LONG | 0 | The initial value of a sequence, which is used for machine migration. If the clock is inconsistent among different machines, you do not need to specify this parameter. | OMS V3.3.1 and later |
| Booster class | STRING | null | The startup class, which is com.oceanbase.oms.connector.datahub.sink.DatahubSinkBooster. |
OMS V3.3.1 and later |
| accessId | STRING | null | The AccessKey ID of DataHub. | OMS V3.3.1 and later |
| endpoint | STRING | null | The endpoint of the DataHub instance. | OMS V3.3.1 and later |
| sequenceIgnoreLocal | BOOLEAN | false | Specifies whether to ignore local sequences. | OMS V3.3.1 and later |
| tooLargeRequestErrorCodes | LIST | [RequestEntityTooLarge, TooLargePayload] | The error code that indicates that a request is too large. | OMS V3.3.1 and later |
| protocol | STRING | TUPLE | The topic protocol. Valid values: BLOB and TUPLE. | OMS V3.3.1 and later |
| requestLimitSize | LONG | 3575104 | The maximum size of a writing request, in bytes, which is dynamically adjusted based on errors. | OMS V3.3.1 and later |
| partitionMode | STRING | HASH | The distribution logic that determines the partition into which a data record is to be written. Valid values: HASH (primary key HASH) and TABLE (TABLE name HASH). | OMS V3.3.1 and later |
| serializerType | STRING | DEFAULT | The serialization format for writing messages to the queue. Valid values: DEFAULT, DEFAULT_WITH_SCHEMA, SHAREPLEX, CANAL, DATAWORKS_V2, and DATAWORKS_V1. | OMS V3.3.1 and later |
| metaExtend | JSON | null | The JSON extension data, which is used to write data to the extend field in DataWorks V2. |
OMS V3.3.1 and later |
| serializerDataTypeMapper | JSON | null | The mapping from SchemaType to STRING. This parameter is for compatibility purpose and solves INT64-related issues. | OMS V4.1.0 and later |
| retryErrorMessage | STRING | null | The message for the error for which a retry is performed. | OMS V3.3.1 and later |
| skipErrorMessage | STRING | null | The message for the error to skip. | OMS V3.3.1 and later |
| skipErrorCode | STRING | null | The code for the error to skip. | OMS V3.3.1 and later |
| retryErrorCode | STRING | null | The code for the error for which a retry is performed. | OMS V3.3.1 and later |
Parameters of a source Kafka instance
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| kafkaAddr | STRING | null | The Kafka address read by the Kafka source. This parameter is now deprecated and replaced by the parameter kafkaServer. |
OMS V4.0.0 and later |
| srcSubTopic | STRING | null | The unique ID of the store client. | OMS V4.2.2 and later |
| kafkaTopic | STRING | null | The Kafka topic. | OMS V4.2.2 and later |
| Booster class | STRING | null | The startup class, which is com.oceanbase.oms.connector.source.kafka.KafkaSourceBooster. |
OMS V3.3.1 and later |
| checkpoint | STRING | null | The checkpoint, in the Map<String, String> format. | OMS V4.2.2 and later |
| sourceType | STRING | null | The type of the source data source. | OMS V3.3.1 and later |
| kafkaServer | STRING | null | The address of the Kafka server. | OMS V4.2.2 and later |
| cdcDataFormat | STRING | tidbOpenProtocol | The TiDB data format. Valid values: tidbOpenProtocol and tidbBinlog. |
OMS V4.0.0 and later |
| timestamp | LONG | null | The old checkpoint and topic timestamp in Kafka. | OMS V4.2.2 and later |
| kafkaPort | STRING | null | The port of the Kafka server. This parameter is now deprecated and replaced by the parameter kafkaServer. |
OMS V4.0.0 and later |
| forceConsumeBegin | BOOLEAN | false | Specifies whether to force the Kafka source to read data from the starting offset of the topic. | OMS V4.2.2 and later |
| properties | JSON | null | The Kafka properties. | OMS V3.3.1 and later |
| isSourceDataWriteToFile | BOOLEAN | false | Specifies whether to write data to a file. This parameter is used for testing only. Do not use it in a production environment. | OMS V4.2.2 and later |
| sslTruststoreLocation | STRING | null | The address of the SSL authentication file. | OMS V3.3.1 and later |
| saslConfig | JSON | null | The SASL configuration. Details: {"mechanism": "The authentication mechanism, which can be GSSAPI, PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.","username": "An account username, which is required for PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.","password": "The account password, which is required for PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.","kerberosPrincipal": "","kerberosKeyTabFileLocation": "An authentication configuration file, which is required for the GSSAPI mechanism.","jaasConfFileLocation": "An authentication configuration file, which is required for the GSSAPI mechanism.","kerberosConfFileLocation": "The address of the Kerberos configuration file, which is required for the GSSAPI mechanism."} |
OMS V3.3.1 and later |
| securityProtocol | STRING | null | The authentication protocol, which can be SASL_PLAINTEXT, SASL_SSL, PLAINTEXTSASL, or SSLSASL. | OMS V3.3.1 and later |
logproxy parameters
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| sys.password | STRING | null | The password of the sys tenant of OceanBase Database. | OMS V4.2.1 and later |
| wait.queue.size | LONG | null | The size of the logproxy queue. | OMS V3.3.1 and later |
| protocol.version | LONG | 1 | The logproxy protocol version. | OMS V3.3.1 and later |
| retry.interval.ms | LONG | 2 | The connection retry interval. | OMS V3.3.1 and later |
| useSchemaCache | BOOLEAN | false | Specifies whether to use the schema cache during record processing. | OMS V3.3.1 and later |
| sourceType | STRING | null | The type of the source data source. | OMS V3.3.1 and later |
| logproxy.address | STRING | null | The address of logproxy. | OMS V3.3.1 and later |
| username | STRING | null | The name of the OceanBase Database user. | OMS V3.3.1 and later |
| splitThreshold | LONG | 128 | The maximum amount of data allowed for a single record batch. | OMS V3.3.1 and later |
| properties | STRING | null | The parameters that are transparently transmitted to OBCDC. | OMS V4.2.1 and later |
| oceanbase.cluster.id | STRING | null | The ID of the OceanBase cluster. | OMS V3.3.1 and later |
| useBetaListener | BOOLEAN | false | Specifies whether to use an experimental parser for parsing data replication center (DRC). | OMS V3.3.1 and later |
| timezone | STRING | +08:00 | The time zone of the source data source. | OMS V3.3.1 and later |
| conn.timeout.ms | LONG | 60000 | The connection timeout period. | OMS V3.3.1 and later |
| sequence.start.timestamp | LONG | 0 | The sequence generation time. | |
| ignore.unknown.record.type | BOOLEAN | true | Specifies whether to ignore unknown record types. | OMS V3.3.1 and later |
| sourceBatchMemorySize | LONG | The system provides a recommended value. | The maximum memory size allowed for a record batch. | OMS V3.3.1 and later |
| Booster class | STRING | null | The startup class, which is com.oceanbase.oms.connector.source.logproxy.LogProxySourceBooster. |
OMS V3.3.1 and later |
| ignoreDdl | BOOLEAN | true | Specifies whether to ignore DDL statements. | OMS V3.3.1 and later |
| sys.username | STRING | null | The name of the OceanBase Database tenant. | OMS V4.2.1 and later |
| checkpoint | STRING | null | The checkpoint of OceanBase Database. | OMS V3.3.1 and later |
| enabled.compression.table.white.list | BOOLEAN | false | Specifies whether to pass the compression allowlist. | OMS V3.3.1 and later |
| password | STRING | null | The password of the account for accessing OceanBase Database. | OMS V3.3.1 and later |
| idle.timeout.ms | LONG | 15 | The wait time before a timeout. | OMS V3.3.1 and later |
| oceanbase.cluster.url | STRING | null | The URL of the OceanBase cluster. | OMS V3.3.1 and later |
| reconnect.times | LONG | -1 | The logproxy reconnection time. | OMS V3.3.1 and later |
| client.id | STRING | default | The client ID of logproxy. | OMS V3.3.1 and later |
| read.timeout.ms | LONG | 60000 | The timeout period for reading data. | OMS V3.3.1 and later |
DataFlowSource parameters
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| taskIdentity | STRING | 1 | The project ID. | OMS V3.3.1 and later |
| connTimeoutSec | LONG | 120 | The timeout period. | OMS V3.3.1 and later |
| nonePkUkQueryWholeTable | BOOLEAN | true | Specifies whether to query a whole table without sharding if the table has no primary key. | OMS V3.3.1 and later |
| rtrimOracleChar | BOOLEAN | false | Specifies whether to remove trailing spaces when the data type of the source is char. If the data is all spaces, reserve one space. The maximum length of the char type is 2000. When the character sets of the source and destination are inconsistent, the data may exceed the length limit and cannot be written to the destination. | OMS V3.3.1 and later |
| maxSqlNoActiveTime | LONG | 700 | The JDBC timeout period. | OMS V3.3.1 and later |
| taskSubId | STRING | 1 | The directory of the verification result file of the Checker component. | OMS V3.3.1 and later |
| stringEncodingCharset | STRING | null | The character set for encoding and decoding strings. When the source is a DB2 database, if you use the getString method and have configured stringEncodingCharset, the byte stream returned by the database is decoded into a string of the specified character set. When the source is an Oracle database, if stringEncodingCharset is configured, the system directly calls the getBytes method to get the raw bytes of CHAR, VARCHAR, and CLOB data and decodes the byte data into the corresponding string of the specified character set. |
OMS V4.2.1 and later |
| queryMetaInfoBySysTenant | BOOLEAN | false | For OceanBase Database V1.4.77, the partition information must be obtained from the sys tenant. | OMS V4.2.1 and later |
| timestamptzConvert | BOOLEAN | false | Specifies whether to convert the city-based time zone to the hour-based time zone during data reading from an Oracle database or an Oracle tenant of OceanBase Database if the data type is TIMESTAMP WITH TIME ZONE. Example: Asia/Shanghai -> GMT+08:00. | OMS V3.3.1 and later |
| retryIntervalTime | LONG | 2000 | The retry interval. | OMS V3.3.1 and later |
| workerNum | LONG | 16 | The number of threads for data reading. | OMS V3.3.1 and later |
| nonePkUkTruncateDstTable | BOOLEAN | false | Specifies whether to allow tables without a primary key to be truncated in the destination. | OMS V3.3.1 and later |
| consistencyMigration | BOOLEAN | false | Specifies whether to lock full migration snapshots when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| nullReplaceEnable | BOOLEAN | false | Specifies whether to enable character replacement in a scenario where the ORACLE gb18030 field value is in the range of chr(129)-chr(254), a NOT NULL constraint is present, and the value read by JDBC is NULL. |
OMS V3.3.1 and later |
| sliceByPkIncrementMinMax | BOOLEAN | true | Specifies whether to use pk_increment sharding on tables without a primary key in the OceanBase Database source (in either MySQL or Oracle mode). If this parameter is set to true, the min/max value of pk_increment is used for sharding. If this parameter is set to false, normal sharding is used. The index order is specified by limit. |
OMS V3.3.1 and later |
| enableSplitByPartition | BOOLEAN | true | Specifies whether to split the data in a partitioned table by partition. | OMS V4.2.2 and later |
| tablePartition | STRING | null | The read-only subpartitions of the partitioned table. The value is in the {"fullName":"regex", ....} JSON format. Assume that in the DB1 database, the Tab1 and Tab2 tables are expected to read the data that matches the p1. and x1. partitions, respectively. To split data that matches the partition names, the value of this parameter should be {"DB1.Tab1": "p1.", "DB1.Tab2": "x1."}. Note that the database, table, and column names should be the same as the actual ones, including the letter case. Otherwise, a mismatch error occurs. |
OMS V3.3.1 and later |
| sliceWorkerNum | LONG | 16 | The number of threads for data sharding. | OMS V3.3.1 and later |
| stringOriginalCharset | STRING | null | When the source is an Oracle database, if this parameter is specified, the system calls the getBytes method directly on CHAR, VARCHAR, and CLOB data and converts the data to a string in the specified encoding format. If stringEncodingCharset is specified, the setting of stringOriginalCharset is overwritten. |
OMS V4.2.1 and later |
| timezone | STRING | +08:00 | The time zone of the extract-transform-load (ETL) operation. | OMS V3.3.1 and later |
| databaseMaxConnection | LONG | 128 | The maximum number of connections. | OMS V3.3.1 and later |
| forceReadWholeTableWithoutSlice | BOOLEAN | false | Specifies whether to read the full table directly without sharding. When some tables in the source cannot be sharded, you can set this parameter to true for direct reading. Note that this parameter takes effect only for MySQL databases. |
OMS V3.3.1 and later |
| sliceQueueSize | LONG | 128 | The maximum number of data sharding queues. | OMS V3.3.1 and later |
| binlogOffsetDb | STRING | test | The database for storing binlog offsets when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| sourceBatchMemorySize | LONG | 16777216 | The maximum memory size allowed for a record batch. | OMS V3.3.1 and later |
| sliceIndex | JSON | null | The sharding index in the format of Map<String, String>. The key is in the database.table format. |
OMS V4.1.0 and later |
| sliceBlocks | LONG | 128 | The number of blocks in each block shard in an Oracle database. | OMS V3.3.1 and later |
| binlogOffsetTable | STRING | binlog_offset | The table for storing binlog offsets when incremental synchronization and full migration are performed for tables without a primary key in a MySQL database. | OMS V3.3.1 and later |
| intervalTypeFmt | STRING | N/A | Specifies whether to adjust the interval type format for Oracle in the following cases:
|
OMS V3.3.1 and later |
| controlFlag | STRING | NOOP | The lowercase sending mode. Valid values: NOOP (do not send), SYNC (send in synchronous-blocking mode), and ASYNC (send asynchronously). | OMS V4.2.2 and later |
| clientQueryPartitionWithInnerTable | BOOLEAN | true | Specifies whether to use an internal table to query the partition information of OceanBase Database. | OMS V4.2.1 and later |
| indexHint | BOOLEAN | true | Specifies whether to use hints during sharding. | OMS V3.3.1 and later |
| sliceBatchSize | LONG | 2000 | The data amount in each shard. | OMS V3.3.1 and later |
| enableNoUniqueConstraintTableReplicate | BOOLEAN | false | Specifies whether to migrate tables without a primary key during data migration from a MySQL or PostgreSQL database to a MySQL tenant of OceanBase Database, or from a MySQL tenant of OceanBase Database to a MySQL or PostgreSQL database. | OMS V3.3.1 and later |
| fetchSize | LONG | 1000 | The fetch size of the JDBC driver. The JDBC driver obtains the specified number of result set rows from the database at one time and puts them in the memory when executing a query. | OMS V3.3.1 and later |
| oracleRowidSliceHint | STRING | N/A | Specifies whether to use hints for block shard queries. | OMS V3.3.1 and later |
| sliceTableBatchSize | LONG | 100 | The number of tables processed at a time for full migration. | OMS V3.3.1 and later |
| nullReplaceString | STRING | N/A | Specifies whether to replace the queried value with the specified string in a scenario where the ORACLE gb18030 field value is in the range of chr(129)-chr(254), a NOT NULL constraint is present, and the value read by JDBC is NULL, when nullReplaceEnable is set to true. |
OMS V3.3.1 and later |
| stringDecodingCharset | STRING | null | The character set for decoding data from a DB2 source. The data is then converted to strings by using stringEncodingCharset. Usually, the character set is latin1. If stringDecodingCharset is specified, stringEncodingCharset is required. |
OMS V4.2.1 and later |
| sourceMaxRetryTimes | LONG | 1000 | The maximum number of retries allowed to obtain data. | OMS V3.3.1 and later |
| sourceType | STRING | null | The type of the source data source. | OMS V3.3.1 and later |
| maxSqlRetryTime | LONG | 3600 | The maximum amount of time allowed for retries to pull data. | OMS V3.3.1 and later |
| batchSize | LONG | 1000 | The maximum number of records in a data shard for batch writing. | OMS V3.3.1 and later |
| rtrimDB2Graphic | BOOLEAN | false | Specifies whether to remove trailing spaces for data of the graphic type read from a DB2 database. | OMS V3.3.1 and later |
| oracleRowidSliceRowsMin | LONG | -1 | The block size returned by each shard. | OMS V3.3.1 and later |
| oracleLobFetchSize | LONG | null | The amount of data that the Oracle fetches for LOB columns. | OMS V3.3.1 and later |
| extConfig | JSON | null | The data source configuration in the Map<String, String> format. | OMS V3.3.1 and later |
| charset | STRING | utf8 | The character set. | OMS V3.3.1 and later |
| clients | JSON | null | The connection information of the source. | OMS V3.3.1 and later |
| Booster class | STRING | null | The startup class, which is com.oceanbase.oms.connector.source.dataflow.DataFlowSourceBooster. |
OMS V3.3.1 and later |
| userIntervalTypeFmt | BOOLEAN | true | Specifies whether to return data of the interval type. | OMS V3.3.1 and later |
| taskType | STRING | migrate | The type of the job. | OMS V3.3.1 and later |
| taskResume | BOOLEAN | true | Specifies whether to restore data by using a checkpoint. | OMS V3.3.1 and later |
| sliceOracleOptimizerFeatures | STRING | N/A | The block sharding feature of an Oracle database. This parameter is set by executing alter session set OPTIMIZER_FEATURES_ENABLE=''. In some scenarios, you need to set the parameter to 10.2.0.3 to optimize performance. |
OMS V3.3.1 and later |
JDBC-Sink startup parameters
| Database type | Plug-in Jar | Booster class |
|---|---|---|
| MYSQL | jdbc-sink-mysql.jar | com.oceanbase.oms.connector.jdbc.sink.mysql.MySQLJDBCSinkBooster |
| DB2 | jdbc-sink-db2.jar | com.oceanbase.oms.connector.jdbc.sink.db2.DB2JDBCSinkBooster |
| OB_MYSQL | jdbc-sink-ob-mysql.jar | com.oceanbase.oms.connector.jdbc.sink.obmysql.OBMySQLJDBCSinkBooster |
| OB_ORACLE | jdbc-sink-ob-oracle.jar | com.oceanbase.oms.connector.jdbc.sink.oboracle.OBOracleJDBCSinkBooster |
| ORACLE | jdbc-sink-oracle.jar | com.oceanbase.oms.connector.jdbc.sink.oracle.OracleJDBCSinkBooster |
| POSTGRESQL | jdbc-sink-pg.jar | com.oceanbase.oms.connector.jdbc.sink.pg.PgJDBCSinkBooster |
STORE_MULTI_SOURCE parameters
| Parameter | Type | Default value | Description | Applicable version |
|---|---|---|---|---|
| useDRCNet | BOOLEAN | true | Specifies whether to use the data replication center (DRC) protocol. | OMS V3.3.1 and later |
| blackRegionNo | STRING | null | The number of the region that is not involved in data migration, which is used to prevent cyclical replication. | OMS V3.3.1 and later |
| drcMarkDb | STRING | null | The database in which the transaction table that prevents cyclical replication and use is located. | OMS V3.3.1 and later |
| useBetaListener | BOOLEAN | false | Specifies whether to use an experimental parser for parsing DRC. | OMS V3.3.1 and later |
| timezone | STRING | +08:00 | The time zone of the source. | OMS V3.3.1 and later |
| clients | JSON | null | The connection information of the Store component. Details: {"clientId": "the unique ID, which is also the source_id specified by Condition to determine the client whose generated heartbeats are passed. We recommend that you use Subtopic","subtopic": "store subtopic","clusterManagerUrl": "store url/cm url","filterConditions": "store allowlist"} |
OMS V3.3.1 and later |
| sourceBatchMemorySize | LONG | 0 | The memory size for assembling records. | OMS V3.3.1 and later |
| Booster class | STRING | null | The booster class, which is com.oceanbase.oms.connector.source.store.MultiStoreSourceBooster. |
OMS V3.3.1 and later |
| ignoreDdl | BOOLEAN | true | Specifies whether to ignore DDL statements. | OMS V3.3.1 and later |
| askSelfUnit | BOOLEAN | false | Specifies whether to request only the data generated by the DRC. | OMS V3.3.1 and later |
| checkpoint | JSON | null | The startup checkpoint of the store. | OMS V3.3.1 and later |
| connectionTimeoutSec | LONG | 120 | The connection timeout period of the Store component. | OMS V3.3.1 and later |
| maxRetries | LONG | 100 | The maximum number of retries allowed for connecting to the Store component. | OMS V3.3.1 and later |
| useSchemaCache | BOOLEAN | false | Specifies whether to use the schema cache during record processing. | OMS V3.3.1 and later |
| password | STRING | null | The account and password required to connect to the Store component. | OMS V3.3.1 and later |
| sourceType | STRING | null | The type of the source data source. | OMS V3.3.1 and later |
| isSourceDataWriteToFile | BOOLEAN | false | Specifies whether to write the source data to a file. This parameter is used only for testing. | OMS V3.3.1 and later |
| buildRecordConcurrent | BOOLEAN | false | Specifies whether to enable concurrent assembling when the source assembles records. If this parameter is set to true, four concurrent threads are allocated to record assembling. |
OMS V3.3.1 and later |
| username | STRING | null | The name of the user. | OMS V3.3.1 and later |
| filterOpts | LIST | null | The filter store options, which are valid only in a file writing simulation. | OMS V3.3.1 and later |
| drcProperties | JSON | null | The store client parameters that are transparently transmitted. | OMS V3.3.1 and later |
| splitThreshold | LONG | 128 | The threshold for splitting a large transaction. | OMS V3.3.1 and later |
| drcMarkTable | STRING | null | The transaction table that prevents cyclical replication and use. | OMS V3.3.1 and later |