Full-Import parameters|V4.2.4|OceanBase Migration Service|OMS docs|Distributed Database

Coordinator

Parameter	Type	Default value	Description	Applicable version
taskTwoActive	BOOLEAN	false	Specifies whether the task is a disaster recovery dual-live task. This parameter is used only for full migration.	OMS V3.3.1 and later
taskIdentity	STRING	1	The task ID.	OMS V3.3.1 and later
checkDstTableEmpty	BOOLEAN	true	Specifies whether to check whether the destination table is empty.	OMS V3.3.1 and later
precheckSinkSchema	BOOLEAN	true	Specifies whether to query the destination table during precheck. If you do not query the destination table, you cannot perform the precheck.	OMS V3.3.1 and later
selectEmptyTableParallel	LONG	16	The number of concurrent queries for checking empty tables during full migration (DATAFLOW-SOURCE).	OMS V4.2.0 and later
timezone	STRING	+08:00	The time zone for ETL processing.	OMS V3.3.1 and later
systemDefaultTimezone	STRING	null	The time zone for reading data. By default, the current time zone is used.	OMS V3.3.1 and later
sourceType	STRING	null	The type of the source data source.	OMS V3.3.1 and later
sinkType	STRING	null	The type of the destination data source.	OMS V3.3.1 and later
consistencyMigration	BOOLEAN	false	Specifies whether to lock the full migration snapshot when you migrate MySQL data without primary keys in full and incremental modes.	OMS V3.3.1 and later
allowRecordTypes	STRING	null	The message types to be filtered at the task level. The message types are separated by commas (,). The message types include ROW, DELETE, UPDATE, INSERT, HEARTBEAT, REPLACE, and DDL.	OMS V3.3.1 and later
tableFilterCacheSize	LONG	512	The size of the LRU cache for CreateTableIfNotExistDDLFilter.	OMS V3.3.1 and later
throttleMemoryBound	STRING	null	The maximum memory size, in bytes.	OMS V3.3.1 and later
throttleRecordBound	STRING	null	The maximum number of records in the framework.	OMS V3.3.1 and later
throttleIOPS	STRING	null	The maximum traffic size, in bytes.	OMS V3.3.1 and later
throttleRps	STRING	null	The maximum RPS.	OMS V3.3.1 and later
enablePreprocessConfig	BOOLEAN	true	Specifies whether to allow the system to automatically calculate the queue length based on the memory.	OMS V4.0.2 and later
connectorJvmParam	STRING	null	The JVM parameters for starting the connector.	OMS V3.3.1 and later
bridgeQueueSize	LONG	256	The capacity of the intermediate cache queue. Note that the unit of this capacity is RecordBatch.	OMS V3.3.1 and later
shuffleBucketSize	LONG	128	The number of buckets for `ShuffleRecordDispatcher`. The number of buckets indicates the maximum number of concurrent operations.	OMS V3.3.1 and later
batchAllInOneTable	BOOLEAN	false	Specifies whether all messages in a batch are from the same table. In full migration, this parameter is set to `true` by default.	OMS V3.3.1 and later
shuffleFlushIntervalMs	LONG	The recommended value is used.	The interval between batches, in milliseconds.	OMS V3.3.1 and later
shuffleMinBatchSize	LONG	20	The minimum batch size for `ShuffleRecordDispatcher`. `ShuffleRecordDispatcher` is a message dispatcher that synchronizes messages to a message queue (Kafka/RocketMQ/DataHub). The main purpose is to improve the concurrency of messages.	OMS V3.3.1 and later
shuffleDdlToAllRoute	BOOLEAN	false	Specifies whether to synchronize DDL operations to all queues when `shuffleRecordDispatcher` is used.	OMS V4.0.1 and later
shuffleMaxBatchSize	LONG	64	The maximum batch size for `ShuffleRecordDispatcher`.	OMS V3.3.1 and later
shuffleBucketMaxInFlightBatchSize	LONG	2147483647	The maximum number of unacknowledged messages in a bucket for `ShuffleRecordDispatcher`.	OMS V3.3.1 and later
printTraceMsg	BOOLEAN	The recommended value is used.	Specifies whether to print message logs. In incremental synchronization, message logs are printed by default. In full migration, message logs are not printed by default.	OMS V3.3.1 and later
loggerLevel	STRING	INFO	The log printing level. Valid values: DEBUG, INFO, WARN, and ERROR.	OMS V3.3.1 and later
destSubId	STRING	null	The subid of the destination data source. This parameter is compatible with JDBCWriter.	OMS V3.3.1 and later
daemonIp	STRING	null	The IP address of the connector. This parameter is compatible with JDBCWriter.	OMS V3.3.1 and later
destSubTopic	STRING	null	The subtopic of the destination data source. This parameter is compatible with JDBCWriter.	OMS V3.3.1 and later
checkpointUseTaskName	BOOLEAN	false	Specifies whether to use the light-connector format for checkpoint files.	OMS V3.3.1 and later
daemonPort	STRING	null	The port number. By default, the port number is 9000. This parameter is compatible with JDBCWriter.	OMS V3.3.1 and later
enableMetricReportTask	BOOLEAN	true	Specifies whether to report checkpoint files. In OMS V4.0.0 and later, if you set `enableOmsConnectorV2Report` to `true`, you can set this parameter to `false`.	OMS V3.3.1 and later
listenPort	LONG	0	The HTTP listening port.	OMS V3.3.1 and later
httpBacklog	LONG	16	The length of the TCP connection queue.	OMS V3.3.1 and later
taskName	STRING	null	The name of the task.	OMS V3.3.1 and later
caseStrategy	STRING	FOLLOW-SOURCE	The case sensitivity strategy. Valid values: DEFAULT (match after query), FOLLOW_SOURCE (same as the source), UPPER (convert all to uppercase), and LOWER (convert all to lowercase).	OMS V3.3.1 and later
isCaseSensitive	BOOLEAN	false	Specifies whether to use case-sensitive matching.	OMS V3.3.1 and later
dbTableCaseStrategy	STRING	FOLLOW-SOURCE	The case sensitivity strategy for databases and tables. Valid values: DEFAULT (match after query), FOLLOW_SOURCE (same as the source), UPPER (convert all to uppercase), and LOWER (convert all to lowercase).	OMS V4.2.2 and later
columnCaseStrategy	STRING	FOLLOW-SOURCE	The case sensitivity strategy for columns. Valid values: DEFAULT (match after query), FOLLOW_SOURCE (same as the source), UPPER (convert all to uppercase), and LOWER (convert all to lowercase).	OMS V4.2.2 and later
noneedRetryException	STRING	null	The exceptions that are not retried during full precheck.	OMS V3.3.1 and later
runIgnoreError	BOOLEAN	false	Specifies whether to continue the task when a table synchronization fails.	OMS V3.3.1 and later
maxRecordCapacity	LONG	16384	The size of the dispatcher queue.	OMS V3.3.1 and later
dispatcherClassName	STRING	The recommended value is used. The value depends on the `sinkType` and `hotKeyMerge` parameters.	The type of the dispatcher. By default, the system selects the type. In full migration, the default value is `MetricDispatcher`. In incremental synchronization, the default value is `DefaultTransactionScheduler`. If the destination is a message queue, the default value is `ShuffleRecordDispatcher`. The selection of the dispatcher is affected by hot key merging and partition writing.	OMS V3.3.1 and later
enableActiveReportTask	BOOLEAN	true	Specifies whether to report `drc.active.xxx` files. In OMS V4.0.0 and later, if you set `enableOmsConnectorV2Report` to `true`, you can set this parameter to `false`.	OMS V3.3.1 and later
reportIntervalSec	LONG	5	The interval between outputs of checkpoint, heartbeat, and diagnose files.	OMS V3.3.1 and later
enableDmlDdlTypeStatistics	BOOLEAN	false	Specifies whether to enable DDL/DML statistics.	OMS V3.3.1 and later
enableOmsConnectorV2Report	BOOLEAN	false	After component integration, the new checkpoint format is provided, and the connector file name is changed to oms.connector.checkpoint. To ensure compatibility with previous connectors, the default value is `false`. This value is changed to `true` after most components are upgraded.	OMS V4.0.1 and later
reportMetricsIntervalSec	LONG	10	The interval between outputs of metrics logs.	OMS V3.3.1 and later
castConfig	STRING	null	The configuration for serialization. Bytes are written in base64 format. Timestamps are written in sec.mill format. Datetimes are written in year-month-day-hour-minute-second format. Bits are written in base64 format. For more information, see the following example:{"bytesType": "BASE64: Convert bytes to base64 strings.","timestampType": "SEC_NANO: The maximum timestamp with 9 digits in sec. The timestamp is represented as a string. SEC: The timestamp is represented as a long. MILLI: The timestamp is represented as a long. MILLI_STR: The timestamp is represented as a string. DATETIME_NORMAL: The timestamp is represented as a string. ","dateTimeType": "ISO: The timestamp is represented as a string with T between the year, month, day, hour, minute, and second. NORMAL: The timestamp is represented as a string. TIMESTAMP_MILL: The timestamp is represented as a long. ","bitType": "NUMBER: Convert bits to longs. BASE64: Convert bits to base64 strings.","appendZoneId": "Whether to add the zone ID. Default value: true"}	OMS V3.3.1 and later

Condition

The format of the Condition module is as follows:

{
  "briefWhiteCondition": [], // Simplified white list configuration. If this parameter exists, the filtering rules of the Store are built based on this parameter.
  "whiteCondition": [], // White list configuration
  "blackCondition": [] // Black list configuration
}

For example, in the scenario where data is synchronized from source_db.source_table to sink_db.sink_table, the format of the Condition module is as follows.

{
  "whiteCondition": [
    {
      "name": "source_db",
      "map": "sink_db",
      "all": false,
      "subs": [
        {
          "name": "source_table",
          "map": "sink_table"
        }
      ]
    }
  ],
  "blackCondition": [

  ]
}

The Condition module is a nested configuration that consists of three layers: db, table, and column.

Parameter	Description
name	The name of the source object at the current level. For example, the name of the database at the first level and the name of the table at the second level.
map	The name of the object to be written to the destination. If the destination is a database, the meaning is the same as that of the source. If the destination is a message queue, such as Kafka or RocketMQ, which only has a Topic namespace, the table name at the table level is mapped to the destination. If the destination is a message queue, such as DataHub, which has two levels of mapping, such as Project and Topic, the database and table names are mapped to the destination. Note that the columns map indicates the renaming mechanism of columns.
all	Whether to synchronize objects at the lower level. For example, if all is set to true at the database level, all tables under the database are synchronized. Otherwise, only the tables defined by subs are synchronized.

The above parameters can meet the needs of general migration scenarios. In addition to these parameters, OMS also supports wildcard and regular expression matching and the attrs capability.

(Wildcard/regular expression) matching capability

{
  "whiteCondition": [
    {
      "name": "source_*",
      "map": "sink_db",
      "func": "fn",
      "all": false,
      "subs": [
        {
          "name": "source_*",
          "map": "sink_table",
          "func": "fn"
        }
      ]
    }
  ],
  "blackCondition": [
    {
      "name": "source_db_[0-10]",
      "map": "sink_db",
      "func": "reg",
      "all": false,
      "subs": [
        {
          "name": "source_table_[0-10]",
          "map": "sink_table",
          "func": "reg"
        }
      ]
    }
  ]
}

The func parameter specifies the matching function, which defaults to string equals. reg indicates regular expression matching, and fn indicates fnmatch.

The preceding example indicates that all tables in all databases starting with source need to be synchronized to sink_db.sink_table, but the tables source_table_0 to source_table_10 in the databases source_db_0 to source_db_10 need to be excluded.

attrs capability

Filtering DML operations

{
"whiteCondition": [
    {
    "name": "source_db",
    "map": "sink_db",
    "all": false,
    "attr": {
        "dml": "i,u,d"
    },
    "subs": [
        {
        "name": "source_table",
        "map": "sink_table"
        }
    ]
    }
],
"blackCondition": []
}

The dml:i,u,d parameter in attr indicates that the task synchronizes only INSERT, DELETE, and UPDATE messages. Note that full messages are not considered INSERT messages.

By default, the attr attributes are inherited. For example, if the attr parameter at the table level does not exist, the system attempts to obtain the attr parameter at the parent level.

Filtering DDL operations

{
"whiteCondition": [
    {
    "name": "source_db",
    "map": "sink_db",
    "all": false,
    "subs": [
        {
        "name": "source_table",
        "map": "sink_table",
        "attr": {
            "ddl": "ct,dt"
        }
        }
    ]
    }
],
"blackCondition": []
}

The ddl: ct,dt parameter in attr indicates that only CREATE TABLE and DROP TABLE DDL operations are accepted.

Data conversion using SQL syntax
```
{
"whiteCondition": [
    {
    "name": "source_db",
    "map": "sink_db",
    "all": false,
    "subs": [
        {
        "name": "source_table",
        "map": "sink_table",
        "attr": {
            "op": "select substr(c1, 1, 2) as c3 from record where c2 = x"
        }
        }
    ]
    }
],
"blackCondition": []
}
```
The op parameter in attr uses SQL to specify column calculations and column filtering. For example, in the preceding example, select substr(c1, 1, 2) as c3 from record where c2 = x indicates that the following operations need to be performed on source_db.source_table:
1. Filter the data where c2 = x.
2. Extract the first two characters from the c1 column and rename it to c3, then pass it down. Note that the c1 column still exists at this point.

Some attributes in message queue scenarios

{
"whiteCondition": [
    {
    "name": "source_db",
    "all": false,
    "subs": [
        {
        "name": "source_table",
        "map": "topic_1",
        "attr": {
            "shard_fields": "c1,c2",
            "ddl_map": "topic_2"
        }
        }
    ]
    }
],
"blackCondition": []
}

shard_fields specifies the c1 and c2 columns as the shard columns of the source_table. This is mainly used to determine which partition/shard/queue the data is delivered to.
ddl_map specifies that when a DDL change occurs in the source_table, the change message needs to be delivered to the topic_2 topic.

Target-end parameters of OceanBase Database in MySQL compatible mode

Parameter	Type	Default value	Description	Applicable version
enablePartitionBucket	BOOLEAN	false	Whether to enable the partition write capability.	OMS V3.3.1 and later
sysInstance	STRING	null	The database service connection information of the system tenant of OceanBase Database.	OMS V3.3.1 and later
maxRetryTimes	LONG	1000	The maximum number of retries for sink write failures.	OMS V3.3.1 and later
useSSL	BOOLEAN	false	Whether to use SSL.	OMS V3.3.1 and later
retryIntervalTime	LONG	1000	The retry interval, in milliseconds.	OMS V3.3.1 and later
transactionTableReserveDays	LONG	7	The number of days for which transaction tables are retained.	OMS V3.3.1 and later
shardFuncClassName	STRING	`com.oceanbase.oms.connector.jdbc.sink.JdbcSinkRouteComputeFunc`	The partition function implementation that needs to be used in conjunction with `ShuffleDispatcher`. This parameter is not required to be set by default.	OMS V3.3.1 and later
workerNum	LONG	16	The number of write threads.	OMS V3.3.1 and later
ignoreRedunantColumnsReplicate	BOOLEAN	false	Whether to allow writing when the source end has more columns than the target end. If set to true, the extra columns at the source end are ignored. If set to false, an error is returned when the source end has more columns than the target end.	OMS V3.3.1 and later
correctMode	LIST	null	The compatible format for writing incompatible data to the target end. Currently, only OBORACLE is implemented. The following enumeration values are separated by commas (,). USE_DATE_LIMIT_VALUE: For time types, if the value exceeds the limit of the target end, the limit value is written. For example, the maximum value of OBORACLE is 9999-12-31 23:59:59. If the value exceeds this limit, the value is written as 9999-12-31 23:59:59 to the target end. USE_SPACE_REPLACE_EMPTY_STRING: Use a space to replace an empty string. This setting mainly solves the scenario where the target end of OBORACLE has a NOT_NULLABLE constraint, but the source end passes an empty string.	OMS V4.2.2 and later
delayThresholdToCompareAllColumns	LONG	10	When `isCompareAllColumns` is `true`, the incremental delay drops to how many seconds will switch to primary key matching.	OMS V3.3.1 and later
jdbcUrl	STRING	null	The connection string of the target end.	OMS V3.3.1 and later
timezone	STRING	+08:00	The time zone.	OMS V3.3.1 and later
useTxId	BOOLEAN	false	Whether to use the transaction ID.	OMS V3.3.1 and later
canGeometryBatched	BOOLEAN	The value is determined by the `sinkType` parameter.	Whether to support batch writing of geometry data.	OMS V3.3.1 and later
binlogOffsetDb	STRING	null	The database where the binlogOffset is stored for full and incremental migration of MySQL without a primary key.	OMS V3.3.1 and later
binlogOffsetTable	STRING	binlog_offset	The table where the binlogOffset is stored for full and incremental migration of MySQL without a primary key.	OMS V3.3.1 and later
isCompareAllColumns	BOOLEAN	false	Whether to enable the full column matching scenario, that is, whether to add all columns to the WHERE clause for incremental UPDATE and DELETE operations.	OMS V3.3.1 and later
password	STRING	null	The JDBC password.	OMS V3.3.1 and later
sysPassword	STRING	null	The password of the system tenant of OceanBase Database.	OMS V3.3.1 and later
disableForeignKeyCheck	BOOLEAN	true	Whether to disable foreign key checks.	OMS V3.3.1 and later
enableTransactionWriter	BOOLEAN	true	Whether to enable transaction writing.	OMS V3.3.1 and later
isUkDependSource	BOOLEAN	false	For incremental logs of Oracle and PostgreSQL sources, the logs may not be complete. In the absence of a primary key, all columns that can be selected from the source end using the target end's index are used. This value is automatically calculated internally.	OMS V3.3.1 and later
clientQueryPartitionWithInnerTable	BOOLEAN	true	Whether to query the partition metadata using an internal table.	OMS V4.2.1 and later
txnDb	STRING	*	The database where the transaction table is located to prevent circular replication.	OMS V3.3.1 and later
dependTargetSchema	BOOLEAN	true	Whether to depend on the schema of the target end for writing. The default value is true. If set to false, the target end will not be queried. In some cases where the target end is OceanBase Database, this parameter may need to be turned off.	OMS V3.3.1 and later
enableNoUniqueConstraintTableReplicate	BOOLEAN	false	Whether to allow synchronization of tables without a primary key. This parameter only affects the forward and reverse links of MySQL->pbMySQL/PG->OBOracle.	null
sysUsername	STRING	null	The username of the system tenant of OceanBase Database.	OMS V3.3.1 and later
sinkType	STRING	null	The type of the target data source.	OMS V3.3.1 and later
localRegionNo	STRING	null	The `regionNo` for writing.	OMS V3.3.1 and later
dryRun	BOOLEAN	false	Dry run, without executing SQL.	OMS V3.3.1 and later
username	STRING	null	The JDBC username.	OMS V3.3.1 and later
filterHiddenPK	BOOLEAN	false	If `enableNoUniqueConstraintTableReplicate` is `true` and the source and target ends meet certain conditions, this parameter takes effect. This parameter specifies to ignore the hidden primary key of tables without a primary key in OceanBase Database and filter out the hidden columns during writing to the target end.	OMS V3.3.1 and later
sqlMode	STRING	null	The SQL mode.	OMS V3.3.1 and later
charset	STRING	The value is determined by the `jdbcUrl`, `username`, `password`, and `sinkType` parameters.	The connection encoding.	OMS V3.3.1 and later
writeMode	STRING	batch	The write mode of the target end, including batch writing and non-batch writing.	OMS V3.3.1 and later
isPreLoadAllTableSchema	BOOLEAN	false	When `dependTargetSchema` is `true`, whether to pre-load the schema of all tables of the target end to memory for query.	OMS V4.2.2 and later
boosterClass	STRING	null	The startup class, `com.oceanbase.oms.connector.jdbc.sink.obmysql.OBMySQLJDBCSinkBooster`.	OMS V3.3.1 and later
isMysqlReplaceMode	BOOLEAN	false	Whether to use the replace mode for writing MySQL or OceanBase Database in MySQL compatible mode.	OMS V3.3.1 and later
canLobBatched	BOOLEAN	The value is determined by the `sinkType` parameter.	Whether to support batch writing of LOB data. Currently, ORACLE and OBORACLE do not support batch writing of LOB data because empty LOB values need to be written using the `empty_clob` and `empty_blob` functions.	OMS V3.3.1 and later
slowSqlThreshold	LONG	15	The threshold for slow SQL, which controls the printing of executed SQL.	OMS V3.3.1 and later
printSql	BOOLEAN	false	Whether to print executed SQL.	OMS V3.3.1 and later
tenant	STRING	null	The tenant.	OMS V3.3.1 and later
obFreeMemoryMin	LONG	20	If this value is greater than or equal to 0, it will be written to the target end to detect the free memory of OceanBase Database. If the free memory of OceanBase Database is less than this value, the write will be paused. It will resume when the next detection finds that this value meets the condition.	OMS V3.3.1 and later
highWatermarkQueryIntervalSec	LONG	1	If obFreeCpuMin > 0 or obFreeMemoryMin > 0, the program will query the free CPU or memory of OceanBase Database to the local every this value (in seconds).	OMS V3.3.1 and later
obFreeCpuMin	LONG	0	If this value is greater than 0, it will be written to the target end to detect the free CPU of OceanBase Database. If the free CPU of OceanBase Database is less than this value, the write will be paused. It will resume when the next detection finds that this value meets the condition. Please note that CPU collection may be inaccurate, so it is not recommended to set this value.	OMS V3.3.1 and later
retryErrorMessage	STRING	null	The error message that needs to be retried.	OMS V3.3.1 and later
skipErrorMessage	STRING	null	The error message that needs to be skipped.	OMS V3.3.1 and later
skipErrorCode	STRING	null	The error code that needs to be skipped.	OMS V3.3.1 and later
retryErrorCode	STRING	null	The error code that needs to be retried.	OMS V3.3.1 and later

Parameters of OceanBase Database in Oracle compatible mode in the sink

Parameter	Type	Default value	Description	Supported version
enablePartitionBucket	BOOLEAN	false	Enables partition write.	V3.3.1 and later versions of OMS
sysInstance	STRING	null	The connection information for the database service of the database tenant in OceanBase Database.	OMS V3.3.1 and later
obSupportNlsSet	BOOLEAN	true	Specifies whether to reset the NLS format. This option is valid only when the destination is OceanBase Database in Oracle compatible mode.	OceanBase Database Management System V3.3.1 and later
maxRetryTimes	LONG	1000	The maximum number of times the sink retries write operations when the writes fail.	OMS V3.3.1 and later
useSSL	BOOLEAN	false	Whether to use SSL.	OMS V3.3.1 and later
retryIntervalTime	LONG	1000	The time interval for retry, in milliseconds.	V3.3.1 and later of OMS
transactionTableReserveDays	LONG	7	The number of days the transaction table is retained.	OMS V3.3.1 and later
shardFuncClassName	STRING	`com.oceanbase.oms.connector.jdbc.sink.JdbcSinkRouteComputeFunc`	The partitioning function to be used in conjunction with `ShuffleDispatcher`.	OMS V3.3.1 and later
workerNum	LONG	16	The number of write threads.	OMS V3.3.1 and later versions
ignoreRedundantColumnsReplicate	BOOLEAN	false	Specifies whether to allow source data to have columns that are not in the destination. If this parameter is set to true, the extra columns in the source data are ignored. If this parameter is set to false, a data replication error is returned because the source data contains more columns than the destination.	OMS V3.3.1 and later versions
correctMode	LIST	null	Description of a compatible format of an incompatible data written to a destination. Only OBORACLE has been implemented. The following enum values are separated by commas. USE_DATE_LIMIT_VALUE: For time types, if the value exceeds the limit on the destination side, the limit value will be written to the destination. For example, the maximum value of OBORACLE is 9999-12-31 23:59:59. If the value exceeds this limit, the limit value will be written to the destination side. USE_SPACE_REPLACE_EMPTY_STRING: The empty string will be replaced with a space. This is mainly for the scenario where NOT_NULLABLE constraints may exist on the destination side but the source side sends an empty string.	OMS V4.2.2 and later
delayThresholdToCompareAllColumns	LONG	10	When `isCompareAllColumns` is `true`, the incremental delay threshold that the value is compared in seconds.	OMS V3.3.1 and later
jdbcUrl	STRING	null	The endpoint connection string.	OMS V3.3.1 and later versions
timezone	STRING	+08:00	The time zone.	OMS V3.3.1 and later versions
useTxId	BOOLEAN	false	Specifies whether to use a transaction ID.	OMS V3.3.1 and later
canGeometryBatched	BOOLEAN	The `sinkType` configuration item affects the recommended values logic	Whether geometry format supports batch write.	OMS V3.3.1 and later
binlogOffsetDb	STRING	null	The database in which the binlogOffset is stored. This database must be created for full and incremental migrations of MySQL tables without primary keys.	OMS V3.3.1 and later
binlogOffsetTable	STRING	binlog_offset	The table in which binlogOffset is stored. This table is used for full or incremental migration of MySQL tables without primary keys.	OMS V3.3.1 and later
isCompareAllColumns	BOOLEAN	false	Specifies whether to enable full-column comparison scenarios, that is, whether to add the entire column to the WHERE condition for incremental UPDATE and DELETE.	OMS V3.3.1 and later
password	STRING	null	JDBC Password.	OMS V3.3.1 and later

Target-end parameters of MySQL and PostgreSQL

Parameter	Type	Default value	Description	Applicable version
enablePartitionBucket	BOOLEAN	false	Specifies whether to enable the partition write capability.	OMS V3.3.1 and later
sysInstance	STRING	null	The database connection information of the system tenant of OceanBase Database.	OMS V3.3.1 and later
maxRetryTimes	LONG	1000	The maximum number of retries for sink write failures.	OMS V3.3.1 and later
useSSL	BOOLEAN	false	Specifies whether to use SSL.	OMS V3.3.1 and later
retryIntervalTime	LONG	1000	The retry interval, in milliseconds.	OMS V3.3.1 and later
transactionTableReserveDays	LONG	7	The number of days to retain the transaction table.	OMS V3.3.1 and later
shardFuncClassName	STRING	`com.oceanbase.oms.connector.jdbc.sink.JdbcSinkRouteComputeFunc`	The partition function implementation. This parameter is used in conjunction with `ShuffleDispatcher`. By default, this parameter is not set.	OMS V3.3.1 and later
workerNum	LONG	16	The number of write threads.	OMS V3.3.1 and later
ignoreRedunantColumnsReplicate	BOOLEAN	false	Specifies whether to allow writing when the source end has more columns than the target end. If set to true, the extra columns at the source end are ignored. If set to false, an error is returned when the source end has more columns than the target end.	OMS V3.3.1 and later
correctMode	LIST	null	Describes the compatible format for writing incompatible data to the target end. Currently, only OBORACLE is supported. The following values are available, separated by commas (,): USE_DATE_LIMIT_VALUE: If the value of a time type exceeds the limit of the target end, the limit value is written to the target end. For example, the maximum value of OBORACLE is 9999-12-31 23:59:59. If the value exceeds this limit, 9999-12-31 23:59:59 is written to the target end. USE_SPACE_REPLACE_EMPTY_STRING: Replaces empty strings with spaces. This setting is used to handle scenarios where the target end has NOT NULL constraints, but the source end passes empty strings.	OMS V4.2.2 and later
delayThresholdToCompareAllColumns	LONG	10	When `isCompareAllColumns` is `true`, the incremental delay drops to how many seconds will switch to primary key matching.	OMS V3.3.1 and later
jdbcUrl	STRING	null	The connection string of the target end.	OMS V3.3.1 and later
timezone	STRING	+08:00	The time zone.	OMS V3.3.1 and later
useTxId	BOOLEAN	false	Specifies whether to use the transaction ID.	OMS V3.3.1 and later
canGeometryBatched	BOOLEAN	Depends on the `sinkType` parameter	Specifies whether to support batch writing of geometry data.	OMS V3.3.1 and later
binlogOffsetDb	STRING	null	The database where binlogOffset is stored for full and incremental migration of MySQL tables without a primary key.	OMS V3.3.1 and later
binlogOffsetTable	STRING	binlog_offset	The table where binlogOffset is stored for full and incremental migration of MySQL tables without a primary key.	OMS V3.3.1 and later
isCompareAllColumns	BOOLEAN	false	Specifies whether to enable the full column matching scenario, that is, whether to add all columns to the WHERE clause for incremental UPDATE and DELETE operations.	OMS V3.3.1 and later
password	STRING	null	The JDBC password.	OMS V3.3.1 and later
sysPassword	STRING	null	The password of the system tenant of OceanBase Database.	OMS V3.3.1 and later
disableForeignKeyCheck	BOOLEAN	true	Specifies whether to disable foreign key checks.	OMS V3.3.1 and later
enableTransactionWriter	BOOLEAN	true	Specifies whether to enable the transaction write capability.	OMS V3.3.1 and later
isUkDependSource	BOOLEAN	false	For Oracle and PostgreSQL source ends, incremental logs may be incomplete. In this case, if the primary key is missing, all columns that can be selected from the source end using the target end's index are used. This value is automatically calculated internally.	OMS V3.3.1 and later
clientQueryPartitionWithInnerTable	BOOLEAN	true	Specifies whether to query the partition metadata from the internal table.	OMS V4.2.1 and later
txnDb	STRING	*	The schema of the transaction table, which is used to prevent circular replication.	OMS V3.3.1 and later
dependTargetSchema	BOOLEAN	true	Specifies whether to write data to the target end based on the schema of the target end. By default, this parameter is set to true. If this parameter is set to false, the system does not query the target end. In some cases, you may need to disable this parameter when the target end is OceanBase Database.	OMS V3.3.1 and later
enableNoUniqueConstraintTableReplicate	BOOLEAN	false	Specifies whether to allow synchronization of tables without a primary key. This parameter only affects the forward and reverse links between MySQL and pbMySQL, and between PostgreSQL and OBOracle.	OMS V4.2.2 and later
sysUsername	STRING	null	The username of the system tenant of OceanBase Database.	OMS V3.3.1 and later
sinkType	STRING	null	The type of the target data source.	OMS V3.3.1 and later
localRegionNo	STRING	null	The `regionNo` for writing.	OMS V3.3.1 and later
dryRun	BOOLEAN	false	Specifies whether to perform a dry run without executing SQL statements.	OMS V3.3.1 and later
username	STRING	null	The JDBC username.	OMS V3.3.1 and later
filterHiddenPK	BOOLEAN	false	If `enableNoUniqueConstraintTableReplicate` is `true` and certain conditions are met, this parameter takes effect. This parameter specifies whether to ignore the hidden primary key of a table without a primary key in OceanBase Database and filter out the hidden column when writing to the target end.	OMS V3.3.1 and later
sqlMode	STRING	null	The SQL mode.	OMS V3.3.1 and later
charset	STRING	Depends on the `jdbcUrl`, `username`, `password`, and `sinkType` parameters	The connection encoding.	OMS V3.3.1 and later
writeMode	STRING	batch	The write mode of the target end, which can be batch or non-batch.	OMS V3.3.1 and later
isPreLoadAllTableSchema	BOOLEAN	false	When `dependTargetSchema` is `true`, whether to pre-load the schema of all tables of the target end to the memory for querying.	OMS V4.2.2 and later
boosterClass	STRING	null	The startup class. For a MySQL database, the startup class is `com.oceanbase.oms.connector.jdbc.sink.mysql.MySQLJDBCSinkBooster`. For a PostgreSQL database, the startup class is `com.oceanbase.oms.connector.jdbc.sink.pg.PgJDBCSinkBooster`.	OMS V3.3.1 and later
isMysqlReplaceMode	BOOLEAN	false	Specifies whether to write data to MySQL or OceanBase Database in MySQL compatible mode in replace mode.	OMS V3.3.1 and later
canLobBatched	BOOLEAN	Depends on the `sinkType` parameter	Specifies whether to support batch writing of LOB data. Currently, LOB data cannot be batch written to ORACLE and OBORACLE. This is because empty LOB values need to be written using the `empty_clob` and `empty_blob` functions.	OMS V3.3.1 and later
slowSqlThreshold	LONG	15	The threshold for slow SQL statements.	OMS V3.3.1 and later
printSql	BOOLEAN	false	Specifies whether to print the executed SQL statements.	OMS V3.3.1 and later
tenant	STRING	null	The tenant.	OMS V3.3.1 and later
obFreeMemoryMin	LONG	20	If this value is greater than or equal to 0, the system checks the free memory of OceanBase Database at the target end. If the free memory of OceanBase Database is less than this value, the write operation is paused. The write operation is resumed when the value is detected to meet the condition in the next round.	OMS V3.3.1 and later
highWatermarkQueryIntervalSec	LONG	1	If obFreeCpuMin > 0 or obFreeMemoryMin > 0, the system queries the free CPU or memory of OceanBase Database at the target end every specified interval (in seconds).	OMS V3.3.1 and later
obFreeCpuMin	LONG	0	If this value is greater than 0, the system checks the free CPU of OceanBase Database at the target end. If the free CPU of OceanBase Database is less than this value, the write operation is paused. The write operation is resumed when the value is detected to meet the condition in the next round. Note that CPU collection may be inaccurate. Therefore, we recommend that you do not set this parameter.	OMS V3.3.1 and later
retryErrorMessage	STRING	null	The error message that needs to be retried.	OMS V3.3.1 and later
skipErrorMessage	STRING	null	The error message that needs to be skipped.	OMS V3.3.1 and later
skipErrorCode	STRING	null	The error code that needs to be skipped.	OMS V3.3.1 and later
retryErrorCode	STRING	null	The error code that needs to be retried.	OMS V3.3.1 and later

Oracle target parameters

Parameter	Type	Default value	Description	Supported versions
enablePartitionBucket	BOOLEAN	false	Specifies whether to enable the partition write feature.	OMS V3.3.1 and later
sysInstance	STRING	null	The database service connection information for the system tenant of OceanBase Database.	OMS V3.3.1 and later
maxRetryTimes	LONG	1000	The maximum number of retries for sink write failures.	OMS V3.3.1 and later
useSSL	BOOLEAN	false	Specifies whether to use SSL.	OMS V3.3.1 and later
retryIntervalTime	LONG	1000	The retry interval, in milliseconds.	OMS V3.3.1 and later
transactionTableReserveDays	LONG	7	The number of days to retain transaction tables.	OMS V3.3.1 and later
shardFuncClassName	STRING	`com.oceanbase.oms.connector.jdbc.sink.JdbcSinkRouteComputeFunc`	The partition function to be used with `ShuffleDispatcher`. This parameter is not required by default.	OMS V3.3.1 and later
workerNum	LONG	16	The number of write threads.	OMS V3.3.1 and later
ignoreRedunantColumnsReplicate	BOOLEAN	false	Specifies whether to allow writing when the source contains more columns than the target. If set to true, the extra columns in the source are ignored. If set to false, an error is returned when the source contains more columns than the target.	OMS V3.3.1 and later
correctMode	LIST	null	Describes the compatible format for writing incompatible data to the target. Currently, only OBORACLE is supported. The following values are supported, separated by commas (,): USE_DATE_LIMIT_VALUE: If the time value exceeds the limit value of the target, the limit value is written. For example, the maximum value of OBORACLE is 9999-12-31 23:59:59. If the value exceeds this limit, it is written as 9999-12-31 23:59:59 to the target. USE_SPACE_REPLACE_EMPTY_STRING: Replaces empty strings with spaces. This setting is mainly used to address scenarios where the target has NOT_NULLABLE constraints but the source passes empty strings.	OMS V4.2.2 and later
delayThresholdToCompareAllColumns	LONG	10	When `isCompareAllColumns` is `true`, the incremental delay must be reduced to how many seconds before it switches to primary key matching.	OMS V3.3.1 and later
jdbcUrl	STRING	null	The target connection string.	OMS V3.3.1 and later
timezone	STRING	+08:00	The time zone.	OMS V3.3.1 and later
useTxId	BOOLEAN	false	Specifies whether to use the transaction ID.	OMS V3.3.1 and later
canGeometryBatched	BOOLEAN	Depends on the `sinkType` parameter.	Specifies whether to support batch writes for geometry data.	OMS V3.3.1 and later
binlogOffsetDb	STRING	null	The database where binlogOffset is stored for full and incremental migration of MySQL tables without a primary key.	OMS V3.3.1 and later
binlogOffsetTable	STRING	binlog_offset	The table where binlogOffset is stored for full and incremental migration of MySQL tables without a primary key.	OMS V3.3.1 and later
isCompareAllColumns	BOOLEAN	false	Specifies whether to enable full column matching. That is, whether to add all columns to the WHERE clause for incremental UPDATE and DELETE operations.	OMS V3.3.1 and later
password	STRING	null	The JDBC password.	OMS V3.3.1 and later
sysPassword	STRING	null	The password of the system tenant of OceanBase Database.	OMS V3.3.1 and later
disableForeignKeyCheck	BOOLEAN	true	Disables foreign key checks.	OMS V3.3.1 and later
enableTransactionWriter	BOOLEAN	true	Specifies whether to enable the transaction write feature.	OMS V3.3.1 and later
isUkDependSource	BOOLEAN	false	For incremental logs from Oracle and PostgreSQL sources, not all logs may be available. In the absence of a primary key, all columns in the source can be selected using indexes from the target. This value is automatically calculated internally.	OMS V3.3.1 and later
clientQueryPartitionWithInnerTable	BOOLEAN	true	Specifies whether to use an internal table to query partition metadata.	OMS V4.2.1 and later
txnDb	STRING	*	The schema for transaction tables, used to prevent cyclic replication.	OMS V3.3.1 and later
dependTargetSchema	BOOLEAN	true	Specifies whether to write to the target based on the schema of the target. The default value is true. If set to false, the target schema is not queried. In some cases where the target is OceanBase Database, this parameter may need to be disabled.	OMS V3.3.1 and later
enableNoUniqueConstraintTableReplicate	BOOLEAN	false	Specifies whether to synchronize tables without a primary key. This parameter only affects the forward and reverse links between MySQL and pbMySQL, and between PostgreSQL and OBORACLE.	OMS V4.2.2 and later
sysUsername	STRING	null	The username of the system tenant of OceanBase Database.	OMS V3.3.1 and later
sinkType	STRING	null	The type of the target data source.	OMS V3.3.1 and later
localRegionNo	STRING	null	The `regionNo` for writing.	OMS V3.3.1 and later
dryRun	BOOLEAN	false	A dry run that does not execute SQL statements.	OMS V3.3.1 and later
username	STRING	null	The JDBC username.	OMS V3.3.1 and later
filterHiddenPK	BOOLEAN	false	If `enableNoUniqueConstraintTableReplicate` is `true` and certain conditions are met between the source and target, this parameter takes effect. This parameter specifies whether to ignore the hidden primary key of a table without a primary key in OceanBase Database when writing to the target. Hidden columns are filtered out during the write process.	OMS V3.3.1 and later
sqlMode	STRING	null	The SQL mode.	OMS V3.3.1 and later
charset	STRING	Depends on the `jdbcUrl`, `username`, `password`, and `sinkType` parameters.	The connection character set.	OMS V3.3.1 and later
writeMode	STRING	batch	The write mode for the target, which can be batch or non-batch.	OMS V3.3.1 and later
isPreLoadAllTableSchema	BOOLEAN	false	When `dependTargetSchema` is `true`, whether to pre-load all table schemas of the target to memory for query.	OMS V4.2.2 and later
oracleSocketTimeout	LONG	50000	The timeout for the sink when the target is an Oracle database, in milliseconds.	OMS V3.3.1 and later
boosterClass	STRING	null	The startup class, `com.oceanbase.oms.connector.jdbc.sink.oracle.OracleJDBCSinkBooster`.	OMS V3.3.1 and later
canLobBatched	BOOLEAN	Depends on the `sinkType` parameter.	Specifies whether to support batch writes for LOB data. Currently, batch writes for LOB data are not supported for ORACLE and OBORACLE. This is because LOB null values must be written using the `empty_clob` and `empty_blob` functions.	OMS V3.3.1 and later
slowSqlThreshold	LONG	15	The threshold for slow SQL statements, controlling the printing of executed SQL statements.	OMS V3.3.1 and later
printSql	BOOLEAN	false	Specifies whether to print executed SQL statements.	OMS V3.3.1 and later
tenant	STRING	null	The tenant.	OMS V3.3.1 and later
obFreeMemoryMin	LONG	20	If this value is greater than or equal to 0, the free memory of the target OceanBase Database is detected. If the free memory of OceanBase Database is less than this value, the write process is paused until the next detection finds that the value meets the condition.	OMS V3.3.1 and later
highWatermarkQueryIntervalSec	LONG	1	If obFreeCpuMin or obFreeMemoryMin is greater than 0, the program queries the free CPU or memory of OceanBase Database to the local system at this interval, in seconds.	OMS V3.3.1 and later
obFreeCpuMin	LONG	0	If this value is greater than 0, the free CPU of the target OceanBase Database is detected. If the free CPU of OceanBase Database is less than this value, the write process is paused until the next detection finds that the value meets the condition. Please note that CPU collection may be inaccurate, so it is not recommended to set this parameter.	OMS V3.3.1 and later
retryErrorMessage	STRING	null	The error message that needs to be retried.	OMS V3.3.1 and later
skipErrorMessage	STRING	null	The error message that needs to be skipped.	OMS V3.3.1 and later
skipErrorCode	STRING	null	The error code that needs to be skipped.	OMS V3.3.1 and later
retryErrorCode	STRING	null	The error code that needs to be retried.	OMS V3.3.1 and later

Target parameters of RocketMQ

Parameter	Type	Default value	Description	Applicable version
clientThreadCount	LONG	-1	The size of the asynchronous sending thread pool of RocketMQ. By default, it is determined based on the CPU cores of the server.	OMS V3.3.1 and later
tags	STRING	null	The tags of the message. If the message contains a table name, the default value is `tableName + _tag`. Otherwise, it is `default`.	OMS V3.3.1 and later
workerNum	LONG	16	The number of write threads.	OMS V3.3.1 and later
clientCallbackExecutorThreadsNum	LONG	-1	The number of threads for handling requests in RocketMQ. By default, it is determined based on the CPU cores of the server.	OMS V3.3.1 and later
enableTrace	BOOLEAN	false	The enableTrace configuration of the RocketMQ client. This parameter specifies whether to enable message trace tracking.	OMS V3.3.1 and later
sinkType	STRING	null	The type of the target data source.	OMS V3.3.1 and later
nameSvrAddr	STRING	null	The `nameSvrAddr` configuration of RocketMQ, which specifies the list of NameServer addresses.	OMS V3.3.1 and later
boosterClass	STRING	null	The startup class, `com.oceanbase.oms.connector.rocketmq.sink.RocketMQSinkBooster`.	OMS V3.3.1 and later
producerGroup	STRING	OMS	The Producer Group of RocketMQ, which identifies and manages a group of message producer instances with the same functionality or responsibilities.	OMS V3.3.1 and later
sendMsgTimeout	LONG	30000	The timeout period for sending messages.	OMS V3.3.1 and later
nameSpace	STRING	-	The `Namespace` of RocketMQ.	OMS V3.3.1 and later
shardFuncClassName	STRING	`com.oceanbase.oms.connector.rocketmq.sink.RocketMQShardFunc`	The implementation of the partitioning function.	OMS V3.3.1 and later
partitionMode	STRING	HASH	The distribution logic that determines which data is written to which partition. Valid values: HASH (primary key hash) and TABLE (table name hash).	OMS V3.3.1 and later
secretKey	STRING	null	The Secret Key of RocketMQ, which is used for authentication.	OMS V3.3.1 and later
accessKey	STRING	null	The Access Key ID (short for Access Key) of RocketMQ, which is a unique string that identifies the user.	OMS V3.3.1 and later
serializerType	STRING	DEFAULT	The serialization format for writing messages to the message queue. Valid values: DEFAULT, DEFAULT_WITH_SCHEMA, SHAREPLEX, CANAL, DATAWORKS_V2, and DATAWORKS_V1.	OMS V3.3.1 and later
metaExtend	JSON	null	The JSON extension data, which is used only for writing the `extend` field in DataWorks V2.	OMS V3.3.1 and later
serializerDataTypeMapper	JSON	null	The mapping from SchemaType to STRING. This parameter is used to resolve the issue of the INT64 type.	OMS V4.1.0 and later
maxRetryTimes	LONG	1000	The maximum number of retries.	OMS V3.3.1 and later
retryIntervalTime	LONG	1000	The interval between retries.	OMS V3.3.1 and later
retryErrorMessage	STRING	null	The error message that requires a retry.	OMS V3.3.1 and later
skipErrorMessage	STRING	null	The error message that needs to be skipped.	OMS V3.3.1 and later
skipErrorCode	STRING	null	The error code that needs to be skipped.	OMS V3.3.1 and later
retryErrorCode	STRING	null	The error code that requires a retry.	OMS V3.3.1 and later

Target-end parameters

Parameter	Type	Default value	Description	Applicable version
shardFuncClassName	STRING	com.oceanbase.oms.connector.kafka.sink.KafkaShardFunc	The parameter must be used in conjunction with `ShuffleDispatcher`. The default value does not require any settings.	OMS V3.3.1 and later
workerNum	LONG	The value depends on the `partitionMode` parameter.	The number of write threads.	OMS V3.3.1 and later
sinkType	STRING	null	The type of the target-end data source.	OMS V3.3.1 and later
dryRun	BOOLEAN	false	Indicates whether to write data to Kafka.	OMS V3.3.1 and later
batchSize	LONG	1048576	The `batch.size` parameter of the producer (Producer) in Kafka, which specifies the size of a message batch.	OMS V3.3.1 and later
properties	JSON	null	The properties (configuration file) in Kafka.	OMS V3.3.1 and later
consumerGroup	STRING	test	The `group.id` parameter of the Kafka consumer.	OMS V3.3.1 and later
servers	STRING	null	The `bootstrap.servers` parameter of the producer (Producer) in Kafka, which specifies the list of servers to which the producer connects to establish an initial connection with the Kafka cluster.	OMS V3.3.1 and later
boosterClass	STRING	null	The startup class, `com.oceanbase.oms.connector.kafka.sink.KafkaSinkBooster`.	OMS V3.3.1 and later
partition	LONG	0	If `partitionMode` is set to `ONE`, this parameter specifies the name of the fixed partition to which data is written.	OMS V3.3.1 and later
kafkaVersion	STRING	2.0	The version of the Kafka client, which can be 0.9 or 2.0.	OMS V3.3.1 and later
lingerMs	LONG	100	The `linger.ms` parameter of the producer (Producer) in Kafka, which specifies the batch processing behavior of messages.	OMS V3.3.1 and later
enableAsync	BOOLEAN	true	Indicates whether to use an asynchronous interface for writing data.	OMS V3.3.1 and later
isTopicMandatory	BOOLEAN	true	Indicates whether to check the existence of a topic.	OMS V3.3.1 and later
partitionMode	STRING	HASH	The distribution logic that determines which data is written to which partition. Valid values: `HASH` (based on the primary key) and `TABLE` (based on the table name).	OMS V3.3.1 and later
sslTruststoreLocation	STRING	null	The address of the SSL authentication file.	OMS V3.3.1 and later
saslConfig	JSON	null	The sasl configuration.{"mechanism": "The authentication mechanism, which can be GSSAPI, PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.","username": "The username. If the authentication mechanism is PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512, you must specify the username.","password": "The password. If the authentication mechanism is PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512, you must specify the password.","kerberosPrincipal": "","kerberosKeyTabFileLocation": "The path of the Kerberos keytab file. This parameter is required when the authentication mechanism is GSSAPI.","jaasConfFileLocation": "The path of the JAAS configuration file. This parameter is required when the authentication mechanism is GSSAPI.","kerberosConfFileLocation": "The path of the krb conf file. This parameter is required when the authentication mechanism is GSSAPI."}	OMS V3.3.1 and later
securityProtocol	STRING	null	The authentication protocol. Valid values: `SASL_PLAINTEXT`, `SASL_SSL`, `PLAINTEXTSASL`, and `SSLSASL`.	OMS V3.3.1 and later
serializerType	STRING	DEFAULT	The serialization format for writing data to a message queue. Valid values: `DEFAULT`, `DEFAULT_WITH_SCHEMA`, `SHAREPLEX`, `CANAL`, `DATAWORKS_V2`, and `DATAWORKS_V1`.	OMS V3.3.1 and later
metaExtend	JSON	null	The JSON extension data that is written to the `extend` field in DataWorks V2.	OMS V3.3.1 and later
serializerDataTypeMapper	JSON	null	The mapping from SchemaType to STRING. This parameter is used to resolve the issue of the INT64 type.	OMS V4.1.0 and later
retryErrorMessage	STRING	null	The error message that needs to be retried.	OMS V3.3.1 and later
skipErrorMessage	STRING	null	The error message that needs to be skipped.	OMS V3.3.1 and later
skipErrorCode	STRING	null	The error code that needs to be skipped.	OMS V3.3.1 and later
retryErrorCode	STRING	null	The error code that needs to be retried.	OMS V3.3.1 and later

DataHub sink parameters

Parameter	Type	Default value	Description	Supported versions
sequenceEnable	BOOLEAN	false	Specifies whether to enable the sorting feature of DataWorks.	OMS V3.3.1 and later
requestRetrySec	LONG	3600	The retry time for failed write requests, in seconds.	OMS V3.3.1 and later
shardFuncClassName	STRING	`com.oceanbase.oms.connector.datahub.sink.DatahubShardFunc`	The partitioning function class that must be used with `ShuffleDispatcher`. By default, this parameter is not set.	OMS V3.3.1 and later
enableBinary	BOOLEAN	true	Specifies whether to enable binary write for the DataHub client.	OMS V3.3.1 and later
workerNum	LONG	16	The number of write threads.	OMS V3.3.1 and later
accessKey	STRING	null	The Access Key ID of DataHub. An Access Key is a unique string that identifies the user.	OMS V3.3.1 and later
connectionTimeoutMs	LONG	10000	The timeout period for the DataHub client.	OMS V3.3.1 and later
sinkType	STRING	null	The type of the target data source.	OMS V3.3.1 and later
sequenceStartTimestamp	LONG	0	The initial value of the sequence, which is used for machine migration. If the time on different machines is inconsistent, you do not need to set this parameter.	OMS V3.3.1 and later
boosterClass	STRING	null	The startup class, `com.oceanbase.oms.connector.datahub.sink.DatahubSinkBooster`.	OMS V3.3.1 and later
accessId	STRING	null	The AccessKey ID of DataHub.	OMS V3.3.1 and later
endpoint	STRING	null	The endpoint of DataHub.	OMS V3.3.1 and later
sequenceIgnoreLocal	BOOLEAN	false	Specifies whether to ignore the local sequence.	OMS V3.3.1 and later
tooLargeRequestErrorCodes	LIST	[RequestEntityTooLarge, TooLargePayload]	The error codes indicating that the request size is too large.	OMS V3.3.1 and later
protocol	STRING	TUPLE	The topic protocol. Valid values: BLOB and TUPLE.	OMS V3.3.1 and later
requestLimitSize	LONG	3575104	The size limit for write requests, which is dynamically adjusted based on errors. The unit is bytes.	OMS V3.3.1 and later
partitionMode	STRING	HASH	The distribution logic that determines which data is written to which partition. Valid values: HASH (based on the primary key) and TABLE (based on the table name).	OMS V3.3.1 and later
serializerType	STRING	DEFAULT	The serialization format for writing to the message queue. Valid values: DEFAULT, DEFAULT_WITH_SCHEMA, SHAREPLEX, CANAL, DATAWORKS_V2, and DATAWORKS_V1.	OMS V3.3.1 and later
metaExtend	JSON	null	JSON extended data that is applied only to the `extend` field when writing to DataWorks V2.	OMS V3.3.1 and later
serializerDataTypeMapper	JSON	null	The mapping from SchemaType to STRING, which resolves the issue of INT64.	OMS V4.1.0 and later
retryErrorMessage	STRING	null	The error message that needs to be retried.	OMS V3.3.1 and later
skipErrorMessage	STRING	null	The error message that needs to be skipped.	OMS V3.3.1 and later
skipErrorCode	STRING	null	The error code that needs to be skipped.	OMS V3.3.1 and later
retryErrorCode	STRING	null	The error code that needs to be retried.	OMS V3.3.1 and later

Parameters of the Kafka source

Parameter	Type	Default value	Description	Applicable version
kafkaAddr	STRING	null	The address of the Kafka server. This parameter is deprecated. Use the `kafkaServer` parameter instead.	OMS V4.0.0 and later
srcSubTopic	STRING	null	The unique identifier.	OMS V4.2.2 and later
kafkaTopic	STRING	null	The topic in Kafka.	OMS V4.2.2 and later
boosterClass	STRING	null	The startup class, `com.oceanbase.oms.connector.source.kafka.KafkaSourceBooster`.	OMS V3.3.1 and later
checkpoint	STRING	null	The checkpoint, which is a Map<String, String>.	OMS V4.2.2 and later
sourceType	STRING	null	The type of the source data.	OMS V3.3.1 and later
kafkaServer	STRING	null	The address of the Kafka server.	OMS V4.2.2 and later
cdcDataFormat	STRING	tidbOpenProtocol	The data format of TiDB, which can be `tidbOpenProtocol` or `tidbBinlog`.	OMS V4.0.0 and later
timestamp	LONG	null	The old checkpoint and topic timestamp in Kafka.	OMS V4.2.2 and later
kafkaPort	STRING	null	The port of the Kafka server. This parameter is deprecated. Use the `kafkaServer` parameter instead.	OMS V4.0.0 and later
forceConsumeBegin	BOOLEAN	false	Specifies whether to read data from the beginning offset of the topic.	OMS V4.2.2 and later
properties	JSON	null	The properties of Kafka.	OMS V3.3.1 and later
isSourceDataWriteToFile	BOOLEAN	false	Specifies whether to write data to a file. This parameter is for testing only and cannot be used in a production environment.	OMS V4.2.2 and later
sslTruststoreLocation	STRING	null	The address of the SSL authentication file.	OMS V3.3.1 and later
saslConfig	JSON	null	The sasl configuration.{"mechanism": "The authentication mechanism, which can be GSSAPI, PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.","username": "The username. This parameter is required if the authentication mechanism is PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.","password": "The password. This parameter is required if the authentication mechanism is PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.","kerberosPrincipal": "","kerberosKeyTabFileLocation": "The path of the Kerberos keytab file. This parameter is required if the authentication mechanism is GSSAPI.","jaasConfFileLocation": "The path of the JAAS configuration file. This parameter is required if the authentication mechanism is GSSAPI.","kerberosConfFileLocation": "The path of the krb conf file. This parameter is required if the authentication mechanism is GSSAPI."}	OMS V3.3.1 and later
securityProtocol	STRING	null	The authentication protocol, which can be SASL_PLAINTEXT, SASL_SSL, PLAINTEXTSASL, or SSLSASL.	OMS V3.3.1 and later

logproxy source parameters

Parameter	Type	Default value	Description	Supported version
sys.password	STRING	null	The password of the system tenant of OceanBase Database.	OMS V4.2.1 and later
wait.queue.size	LONG	null	The size of the logproxy wait queue.	OMS V3.3.1 and later
protocol.version	LONG	1	The protocol version of logproxy.	OMS V3.3.1 and later
retry.interval.ms	LONG	2	The interval between two retry attempts.	OMS V3.3.1 and later
useSchemaCache	BOOLEAN	false	Specifies whether to use the schema cache when processing records.	OMS V3.3.1 and later
sourceType	STRING	null	The type of the source data source.	OMS V3.3.1 and later
logproxy.address	STRING	null	The address of logproxy.	OMS V3.3.1 and later
username	STRING	null	The username of OceanBase Database.	OMS V3.3.1 and later
splitThreshold	LONG	128	The maximum data size supported by a single record batch.	OMS V3.3.1 and later
properties	STRING	null	The parameters passed to OBCDC.	OMS V4.2.1 and later
oceanbase.cluster.id	STRING	null	The cluster ID of OceanBase Database.	OMS V3.3.1 and later
useBetaListener	BOOLEAN	false	Specifies whether to use the experimental parser for parsing.	OMS V3.3.1 and later
timezone	STRING	+08:00	The time zone of the source data source.	OMS V3.3.1 and later
conn.timeout.ms	LONG	60000	The connection timeout period.	OMS V3.3.1 and later
sequence.start.timestamp	LONG	0	The start time of the sequence.
ignore.unknown.record.type	BOOLEAN	true	Specifies whether to ignore unknown record types.	OMS V3.3.1 and later
sourceBatchMemorySize	LONG	Logical recommendation	The memory size limit of a record batch.	OMS V3.3.1 and later
boosterClass	STRING	null	The startup class, `com.oceanbase.oms.connector.source.logproxy.LogProxySourceBooster`.	OMS V3.3.1 and later
ignoreDdl	BOOLEAN	true	Specifies whether to ignore DDL operations.	OMS V3.3.1 and later
sys.username	STRING	null	The name of the system tenant of OceanBase Database.	OMS V4.2.1 and later
checkpoint	STRING	null	The checkpoint of OceanBase Database.	OMS V3.3.1 and later
enabled.compression.table.white.list	BOOLEAN	false	Specifies whether to pass the compression white list.	OMS V3.3.1 and later
password	STRING	null	The password of OceanBase Database.	OMS V3.3.1 and later
idle.timeout.ms	LONG	15	The idle wait time.	OMS V3.3.1 and later
oceanbase.cluster.url	STRING	null	The URL of the OceanBase Database cluster.	OMS V3.3.1 and later
reconnect.times	LONG	-1	The reconnection time of logproxy.	OMS V3.3.1 and later
client.id	STRING	default	The `client.id` of logproxy.	OMS V3.3.1 and later
read.timeout.ms	LONG	60000	:The data read timeout period.	OMS V3.3.1 and later

DataFlowSource parameters

Parameter	Type	Default	Description	Supported versions
taskIdentity	STRING	1	Task identifier.	OMS V3.3.1 and later
connTimeoutSec	LONG	120	The timeout period.	OMS V3.3.1 and later versions
nonePkUkQueryWholeTable	BOOLEAN	true	A table without a primary key can query the whole table.	OMS V3.3.1 and later
rtrimOracleChar	BOOLEAN	false	Specifies whether to remove trailing spaces when the source data type is CHAR. When all characters in a CHAR-type field are spaces, one space is reserved. The maximum length of a CHAR-type field is 2000 bytes. If the source and target character sets do not match, the data may exceed this length limit, resulting in write errors to the target.	OMS V3.3.1 and later
maxSqlNoActiveTime	LONG	700	The value of the JDBC timeout period.	OMS V3.3.1 and later
taskSubId	STRING	1	Fully generates the progress file directory.	OMS V3.3.1 and later
stringEncodingCharset	STRING	null	The character set used for encoding or decoding strings. If the source is a DB2 database and the `getString` method is called while `stringEncodingCharset` is configured, the database returns a byte stream that will be decoded into a string based on the specified character set. If the source is an Oracle database, and `stringEncodingCharset` is configured, then for CHAR, VARCHAR, and CLOB type data, the `getBytes` method will be called directly to obtain the original byte stream, and then the byte data will be decoded into the corresponding string based on the character set specified by the parameter.	OMS V4.2.1 and later versions
queryMetaInfoBySysTenant	BOOLEAN	false	OceanBase Database V1.4.77 requires metadata information to be obtained from the sys tenant.	OMS V4.2.1 and later versions
timestamptzConvert	BOOLEAN	false	When you read data from an Oracle database or OceanBase database in Oracle compatible mode, timestamps with time zones are converted from city time zones to time zones represented by hours. For example, a timestamp with a city time zone of Asia/Shanghai is converted to a timestamp with a time zone of +08:00.	OMS V3.3.1 and later
retryIntervalTime	LONG	2000	The retry interval time.	OMS V3.3.1 and later
workerNum	LONG	16	Number of data read threads.	OMS V3.3.1 and later
nonePkUkTruncateDstTable	BOOLEAN	false	Whether to allow TRUNCATE on the table in the destination database that has no primary or unique key.	OMS V3.3.1 and later
consistencyMigration	BOOLEAN	false	This parameter specifies whether to perform a full migration with no primary keys in conjunction with incremental data migration. The full migration will be blocked until the migration snapshot is locked.	OMS V3.3.1 and later
nullReplaceEnable	BOOLEAN	false	Specifies whether to replace the value of an ORACLE gb18030 field with values between `chr(129)` and `chr(254)` as NULL if the field does not allow NULL values and a JDBC connection reads it as NULL.	V3.3.1 and later
sliceByPkIncrementMinMax	BOOLEAN	true	Specifies whether to use pk_increment to divide a non-primary key table of the source OceanBase Database (MySQL or Oracle compatible) into slices. If this option is set to true, the slice is divided based on the minimum and maximum values of pk_increment. If this option is set to false, slices are divided based on the values of the specified column, and the index is ordered by limit.	OMS V3.3.1 and later versions
enableSplitByPartition	BOOLEAN	true	Specifies whether to split the data of a partitioned table by partitions.	OMS V4.2.2 and later versions
tablePartition	STRING	null	The name of the partitioned table. The configuration format is in JSON format, `{"fullName":"regex", ....}`. If a database named DB1 exists, Tab1 is expected to read data of partitions whose names match the partition name `p1.`, and Tab2 is expected to use the partition name `x1.`. To divide data of partitions whose names match the partition names, set the value of the parameter to `{"DB1.Tab1": "p1.", "DB1.Tab2": "x1."}`. Note that the names of the database, table, and column must be actual and case-sensitive. If the cases do not match, a mismatch will occur.	OMS V3.3.1 and later
sliceWorkerNum	LONG	16	The number of threads in the OOS slice worker.	OMS V3.3.1 and later versions
stringOriginalCharset	STRING	null	When the source is an Oracle database, if this value is configured, the `getBytes` method will be called for char, varchar, and clob data types, and the result will be converted to the specified encoding. If `stringEncodingCharset` is configured, it will override `stringOriginalCharset`.	OMS V4.2.1 and later versions
timezone	STRING	+08:00	The time zone for ETL processing.	OMS V3.3.1 and later

Source database: Oracle database; Target database: non-Oracle or OceanBase database in Oracle compatible mode
Target database: Oracle database; Source database: non-Oracle database or OceanBase database in Oracle compatible mode
{"intervalDayDef":1,"intervalToSecondDef":1,"intervalYearDef":1,"intervalToMonthDef":2,"intervalSign":true,"intervalDayFmttime":2}
Source database: OceanBase database in Oracle compatible mode; Target database: non-Oracle database or OceanBase database in Oracle compatible mode
Target database: OceanBase database in Oracle compatible mode; Source database: non-Oracle database or OceanBase database in Oracle compatible mode
{"intervalDayDef":2,"intervalToSecondDef":6,"intervalYearDef":1,"intervalToMonthDef":2,"intervalSign":true,"intervalDayFmttime":2}

Parameters of the JDBC-SINK startup class

DBType	Plugin Jar	BoosterClass
MYSQL	jdbc-sink-mysql.jar	com.oceanbase.oms.connector.jdbc.sink.mysql.MySQLJDBCSinkBooster
DB2	jdbc-sink-db2.jar	com.oceanbase.oms.connector.jdbc.sink.db2.DB2JDBCSinkBooster
OB_MYSQL	jdbc-sink-ob-mysql.jar	com.oceanbase.oms.connector.jdbc.sink.obmysql.OBMySQLJDBCSinkBooster
OB_ORACLE	jdbc-sink-ob-oracle.jar	com.oceanbase.oms.connector.jdbc.sink.oboracle.OBOracleJDBCSinkBooster
ORACLE	jdbc-sink-oracle.jar	com.oceanbase.oms.connector.jdbc.sink.oracle.OracleJDBCSinkBooster
POSTGRESQL	jdbc-sink-pg.jar	com.oceanbase.oms.connector.jdbc.sink.pg.PgJDBCSinkBooster

STORE_MULTI_SOURCE parameter description

Parameter	Type	Default value	Description	Applicable version
useDRCNet	BOOLEAN	true	Specifies whether to use the DRC protocol.	OMS V3.3.1 and later
blackRegionNo	STRING	null	The regionNo that does not need to be synchronized. This parameter is used to prevent circular replication.	OMS V3.3.1 and later
drcMarkDb	STRING	null	The database where the table used for preventing circular replication is located.	OMS V3.3.1 and later
useBetaListener	BOOLEAN	false	Specifies whether to use the experimental parser for parsing.	OMS V3.3.1 and later
timezone	STRING	+08:00	The time zone of the source.	OMS V3.3.1 and later
clients	JSON	null	The connection information of the Store component.`{"clientId": "The unique identifier, which is also the source_id filled in the Condition. This parameter is used to determine the heartbeat of the Client. It is recommended to use Subtopic","subtopic": "store subtopic","clusterManagerUrl": "store url/cm url","filterConditions": "store whitelist"}`	OMS V3.3.1 and later
sourceBatchMemorySize	LONG	0	The memory size used for record assembly.	OMS V3.3.1 and later
boosterClass	STRING	null	The startup class, `com.oceanbase.oms.connector.source.store.MultiStoreSourceBooster`.	OMS V3.3.1 and later
ignoreDdl	BOOLEAN	true	Specifies whether to ignore DDL operations.	OMS V3.3.1 and later
askSelfUnit	BOOLEAN	false	Specifies whether to request only data generated by DRC.	OMS V3.3.1 and later
checkpoint	JSON	null	The startup point information of Store.	OMS V3.3.1 and later
connectionTimeoutSec	LONG	120	The connection timeout period of the Store component.	OMS V3.3.1 and later
maxRetries	LONG	100	The maximum number of retries for connecting to the Store component.	OMS V3.3.1 and later
useSchemaCache	BOOLEAN	false	Specifies whether to use the schema cache when processing records.	OMS V3.3.1 and later
password	STRING	null	The account and password for connecting to the Store component.	OMS V3.3.1 and later
sourceType	STRING	null	The type of the data source at the source end.	OMS V3.3.1 and later
isSourceDataWriteToFile	BOOLEAN	false	Specifies whether to write the data of the source to a file. This parameter is for testing only.	OMS V3.3.1 and later
buildRecordConcurrent	BOOLEAN	false	Specifies whether to assemble records in parallel. If this parameter is set to true, four threads are used to concurrently assemble records.	OMS V3.3.1 and later
username	STRING	null	The username.	OMS V3.3.1 and later
filterOpts	LIST	null	This parameter is valid only when data is written to a file.	OMS V3.3.1 and later
drcProperties	JSON	null	The parameters of the Store client.	OMS V3.3.1 and later
splitThreshold	LONG	128	The threshold for splitting large transactions.	OMS V3.3.1 and later
drcMarkTable	STRING	null	The table used for preventing circular replication.	OMS V3.3.1 and later

Customer Stories

Documentation

Enterprise Edition

Community Edition

Full-Import parameters