OceanBase Migration Service

V4.0.2Enterprise Edition

OMS Documentation
What's new
OMS Introduction
- What is OMS?
- Terms
- OMS HA
- Architecture
  - Overview
  - Hierarchical functional system
  - Basic components
- Limits
Quick Start
- Data migration process
- Data synchronization process
Deploy OMS
- Deployment types
- System and network requirements
- Memory and disk requirements
- Environment preparations
- Deploy OMS on a single node
- Deploy OMS on multiple nodes in a single region
- Deploy OMS on multiple nodes in multiple regions
- Integrate the OIDC protocol to OMS to implement SSO
- Scale-out OMS
- Check the deployment
- Deploy a time-series database (Optional)
OMS console
- Log on to the OMS console
- Overview
- User center
  - Configure user information
  - Change your logon password
  - Log off
Data migration
- Data migration overview
- Migrate data from a MySQL database to a MySQL tenant of OceanBase Database
- Migrate data from a MySQL tenant of OceanBase Database to a MySQL database
- Migrate data from an Oracle database to a MySQL tenant of OceanBase Database
- Migrate data from an Oracle tenant of OceanBase Database to an Oracle database
- Migrate data from an Oracle database to an Oracle tenant of OceanBase Database
- Migrate data from a DB2 LUW database to an Oracle tenant of OceanBase Database
- Migrate data from an Oracle tenant of OceanBase Database to a DB2 LUW database
- Migrate data from a DB2 LUW database to a MySQL tenant of OceanBase Database
- Migrate data from a MySQL tenant of OceanBase Database to a DB2 LUW database
- Migrate data within OceanBase Database
- Active-active disaster recovery between OceanBase databases
- Migrate data from a TiDB database to a MySQL tenant of OceanBase Database
- Migrate data from a PostgreSQL database to a MySQL tenant of OceanBase Database
- Manage data migration projects
  - View details of a data migration project
  - Change the name of a data migration project
  - View and modify migration objects
  - Use tags to manage data migration projects
  - Download and import the settings of migration objects
  - Start and pause a data migration project
  - Release and delete a data migration project
- Features
  - DML filtering
  - Synchronize DDL operations
  - Configure matching rules for migration objects
  - Wildcard rules
  - Rename a database table
  - Use SQL conditions to filter data
  - Create and update a heartbeat table
  - Schema migration mechanisms
  - Schema migration operations
  - Set an incremental synchronization timestamp
- Supported DDL operations and limits for synchronization
  - DDL synchronization from a MySQL database to a MySQL tenant of OceanBase Database
    - Overview of DDL synchronization from a MySQL database to a MySQL tenant of OceanBase Database
    - CREATE TABLE
      - Create a table
      - Create a column
      - Create an index or a constraint
      - Create partitions
    - Data type conversion
    - ALTER TABLE
      - Modify a table
      - Operations on columns
      - Operations on constraints and indexes
      - Operations on partitions
    - TRUNCATE TABLE
    - RENAME TABLE
    - DROP TABLE
    - CREATE INDEX
    - DROP INDEX
    - DDL incompatibilities between a MySQL database and a MySQL tenant of OceanBase Database
      - Overview
      - Incompatibilities of the CREATE TABLE statement
        Incompatibilities of CREATE TABLE
        Column types that are supported to create indexes or constraints
      - Incompatibilities of the ALTER TABLE statement
        Incompatibilities of ALTER TABLE
        Change the type of a constrained column
        Change the type of an unconstrained column
        Change the length of a constrained column
        Change the length of an unconstrained column
        Delete a constrained column
      - Incompatibilities of DROP INDEX operations
  - Synchronize DDL operations from a MySQL tenant of OceanBase Database to a MySQL database
  - DDL operations for synchronizing data from an Oracle database to an Oracle tenant of OceanBase Database
    - Overview
    - CREATE TABLE
      - Overview
      - Create a relational table
        Create a relational table
        Define columns of a relational table
      - Virtual columns
      - Regular columns
      - Create partitions
        Overview
        Partitioning
        Subpartitioning
        Composite partitioning
        User-defined partitioning
        Subpartition templates
      - Constraints
        Overview
        Inline constraints
        Out-of-line constraints
    - CREATE INDEX
      - Overview
      - Normal indexes
    - ALTER TABLE
      - Modify tables
      - Modify, drop, and add table attributes
      - Column attribute management
        Modify, drop, and add column attributes
        Rename a column
        Add columns and column attributes
        Modify column attributes
        Drop columns
      - Modify, drop, and add constraints
      - Partition management
        Modify, drop, and add partitions
        Drop partitions
        Drop subpartitions
        Add partitions and subpartitions
        Modify partitions
        Truncate partitions
    - DROP TABLE
    - COMMENT
    - RENAME OBJECT
    - TRUNCATE TABLE
    - DROP INDEX
    - DDL incompatibilities between an Oracle database and an Oracle tenant of OceanBase Database
      - Overview
      - Incompatibilities of CREATE TABLE
      - Incompatibilities in table modification operations
        Incompatibilities of ALTER TABLE
        Change the type of a constrained column
        Change the type of an unconstrained column
        Change the length of a constrained column
        Change the length of an unconstrained column
  - Synchronize DDL operations from an Oracle tenant of OceanBase Database to an Oracle database
  - Synchronize DDL operations from an Oracle tenant of OceanBase Database to a DB2 LUW database
  - Synchronize DDL operations from a DB2 LUW database to a MySQL tenant of OceanBase Database
  - Synchronize DDL operations from a MySQL tenant of OceanBase Database to a DB2 LUW database
  - DDL synchronization between MySQL tenants of OceanBase Database
  - DDL synchronization between Oracle tenants of OceanBase Database
Data synchronization
- Overview
- Synchronize data from OceanBase Database to a Kafka instance
- Synchronize data from an OceanBase database to a RocketMQ instance
- Synchronize data from OceanBase Database to a DataHub instance
- Synchronize data from an ODP logical table to a physical table in a MySQL tenant of OceanBase Database
- Synchronize data from an ODP logical table to a DataHub instance
- Synchronize data from an IDB logical table to a physical table in a MySQL tenant of OceanBase Database
- Synchronize data from an IDB logical table to a DataHub instance
- Synchronize data from a MySQL database to a DataHub instance
- Synchronize data from an Oracle database to a DataHub instance
- Manage data synchronization projects
  - View details of a data synchronization project
  - Change the name of a data synchronization project
  - View and modify synchronization objects
  - Use tags to manage data synchronization projects
  - Download and import the settings of synchronization objects
  - Start and pause a data synchronization project
  - Release and delete a data synchronization project
- Features
  - DML filtering
  - Synchronize DDL operations
  - Rename databases and tables
  - Rename a topic
  - Use SQL conditions to filter data
  - Column filtering
  - Data formats
Create and manage data sources
- Create data sources
  - Create an OceanBase data source
    - Create a physical OceanBase data source
    - Create a DBP data source
    - Create an IDB data source
  - Create a MySQL data source
  - Create an Oracle data source
  - Create a TiDB data source
  - Create a Kafka data source
  - Create a RocketMQ data source
  - Create a DataHub data source
  - Create a DB2 LUW data source
  - Create a PostgreSQL data source
- Manage data sources
  - View data source information
  - Copy a data source
  - Edit a data source
  - Delete a data source
- Create a database user
- User privileges
- Enable binlogs for the MySQL database
- Minimum privileges required when an Oracle database serves as the source
OPS & Monitoring
- O&M overview
- Go to the overview page
- Server
  - View server information
  - Update the quota
  - View server logs
- Components
  - Store
    - Create a store
    - View details of a store
    - Update the configurations of a store
    - Start and pause a store
    - Delete a store
  - Incr-Sync
    - View details of an Incr-Sync component
    - Start and pause an Incr-Sync component
    - Migrate an Incr-Sync component
    - Update the configurations of an Incr-Sync component
    - Batch O&M
    - Delete an Incr-Sync component
  - Full-Import
    - View details of a Full-Import component
    - Pause a Full-Import component
    - Rerun and resume a Full-Import component
    - Update the configurations of a Full-Import component
    - Delete a Full-Import component
  - Full-Verification
    - View details of a Full-Verification component
    - Pause a Full-Verification component
    - Rerun and resume a Full-Verification component
    - Update the configurations of a Full-Verification component
    - Delete a Full-Verification component
- O&M tickets
  - View details of an O&M ticket
  - Skip a ticket or sub-ticket
  - Retry a ticket or sub-ticket
System management
- Permission Management
  - Overview
  - Manage users
  - Manage departments
- Alert center
  - View project alerts
  - View system alerts
  - Manage alert settings
- Associate with OCP
- System parameters
  - Modify system parameters
  - Modify HA configurations
  - oblogproxy parameters
- Operation audit
OMS O&M
- Manage OMS services
- OMS logs
- Component O&M
  - O&M operations for the Supervisor component
  - CLI-based O&M for the Connector component
  - O&M operations for the Store component
- Component tuning
  - Incr-Sync/Full-Import tuning
  - Oracle store tuning
- Component parameters
  - Coordinator
  - Condition
  - Source Plugin
    - Overview
    - StoreSource
    - DataFlowSource
    - LogProxySource
    - KafkaSource (TiDB)
  - Sink Plugin
    - Overview
    - JDBC-Sink
    - KafkaSink
    - DatahubSink
    - RocketMQSink
  - Store parameters
    - Parameters of an Oracle store
    - Parameters of a DB2 store
    - Parameters of a MySQL store
    - Parameters of an OceanBase store
  - Parameters of the CM component
  - Parameters of the Supervisor component
- Set throttling
Reference Guide
- API Reference
  - Obtain the status of a migration project
  - Obtain the status of a synchronization project
- OMS error codes
- Alert Reference
  - oms_host_down
  - oms_host_down_migrate_resource
  - oms_host_threshold
  - oms_migration_failed
  - oms_migration_delay
  - oms_sync_failed
  - oms_sync_status_inconsistent
  - oms_sync_delay
Upgrade Guide
- Overview
- Upgrade OMS in single-node deployment mode
- Upgrade OMS in multi-node deployment mode
- FAQ
FAQ
- General O&M
  - How do I modify the resource quotas of an OMS container?
  - How do I troubleshoot the OMS server down issue?
  - Deploy InfluxDB for OMS
  - Increase the disk space of the OMS host
- Project diagnostics
  - How do I troubleshoot common problems with Oracle Store?
  - How do I perform performance tuning for Oracle Store?
  - What do I do when Oracle Store reports an error at the isUpdatePK stack?
  - What do I do when a store does not have data of the timestamp requested by the downstream?
  - What do I do when OceanBase Store failed to access an OceanBase cluster through RPC?
  - How do I use LogMiner to pull data from an Oracle database?
- OPS & monitoring
  - What are the alert rules?
- Data synchronization
  - FAQ about synchronization to a message queue
    - What are the strategies for ensuring the message order in incremental data synchronization to Kafka
- Data migration
  - User privileges
    - What privileges do I need to grant to a user during data migration to or from an Oracle database?
  - Full migration
    - How do I query the ID of a checker?
    - How do I query log files of the Checker component of OMS?
    - How do I query the verification result files of the Checker component of OMS?
    - What do I do if the destination table does not exist?
    - What can I do when the full migration failed due to LOB fields?
    - What do I do if garbled characters cannot be written into OceanBase Database V3.1.2?
  - Incremental synchronization
    - How do I skip DDL statements?
    - How do I update whitelists and blacklists?
    - What are the application scope and limits of ETL?
- Installation and deployment
  - How do I upgrade Store?
Release Note
- V4.0
  - OMS V4.0.2
  - OMS V4.0.1
- V3.4
  - OMS V3.4.0
- V3.3
  - OMS V3.3.1
  - OMS V3.3.0
- V3.2
  - OMS V3.2.2
  - OMS V3.2.1
- V3.1
  - OMS V3.1.0
- V2.1
  - OMS V2.1.2
  - OMS V2.1.0

Download PDF

OceanBase

OceanBase

OceanBase, A Highly Scalable Database for Transactional, Analytical, and AI Workloads.

Company

About OceanBase Contact Us Partner

Product

OceanBase Cloud OceanBase Enterprise Quick Start Download Pricing

Resources

Docs Blog White Paper Trust Center

Follow us

© OceanBase 2024. All rights reserved | Cloud Service Agreement | Privacy Policy | Legal | Security

Document Feedback

Coordinator

Last Updated：2023-06-29 11:11:39 Updated

share

share

The configuration path of the Coordinator is conf/coordinator.json.

[JVM] connectorJvmParam

Description: specifies JVM parameters.
Default value: None.
Valid values: JVM parameters.
Supported version: OMS V3.3.1 and later.

[Batch distribution] maxRecordCapacity

Description: specifies the size of the dispatcher queue.
Default value: 16384.
Value type: int.
Supported version: OMS V3.3.1 and later.

[Batch distribution] bridgeQueueSize

Description: specifies the length of the intermediate queue.
Default value: 256.
Value type: int.
Supported version: OMS V3.3.1 and later.

[Batch distribution] shuffleMinBatchSize

Description: specifies the minimum batch size. It is used together with ShuffleRecordDispatcher.
Default value: 20.
Value type: int. The value must be smaller than that of shuffleMaxBatchSize.
Supported version: OMS V3.3.1 and later.

[Batch distribution] shuffleMaxBatchSize

Description: specifies the maximum batch size. It is used together with ShuffleRecordDispatcher.
Default value: 64.
Value type: int.
Supported version: OMS V3.3.1 and later.

[Batch distribution] shuffleFlushIntervalMs

Description: specifies the maximum time to wait before sending a batch. It is used together with ShuffleRecordDispatcher.
Default value: 100 ms.
Value type: int.
Supported version: OMS V3.3.1 and later.

[Batch distribution] shuffleBucketMaxInFlightBatchSize

Description: specifies the number of messages without an ACK in the same bucket. It is used together with ShuffleRecordDispatcher.
Default value: 1 for incremental synchronization and -1 for full migration. The value -1 indicates no limit.
Value type: int.
Supported version: OMS V3.3.1 and later.

[Batch distribution] shuffleBucketSize

Description: specifies the bucket size, which indicates the number of concurrent buckets. It is used together with ShuffleRecordDispatcher.
Default value: None. This parameter is required.
Value type: int.
Supported version: OMS V3.3.1 and later.

[Transaction writing] useTxId

Description: specifies whether to use transaction IDs.
Default value: false.
Value type: Boolean.
Supported version: OMS V3.3.1 and later.

[Transaction writing] antiLoopMode

Description: specifies whether to enable the anti-loop mode for the transaction table.
Default value: false.
Valid values: true and false.
Supported version: OMS V3.3.1 and later.

[Tables without a primary key] obHiddenPrimaryKeySyncDestColumn

Description: specifies the name of a column into which a hidden column in OceanBase Database is converted.
Default value: OMS_PK_INCRMT.
Value type: String.
Supported version: OMS V3.3.1 and later.

[Tables without a primary key] rowidExtendedColumns

Description: specifies the names of the four columns into which the Oracle ROWID column is converted.
Default value: OMS_OBJECT_NUMBER,OMS_RELATIVE_FNO,OMS_BLOCK_NUMBER,OMS_ROW_NUMBER.
Value type: String.
Supported version: OMS V3.3.1 and later.

[Tables without a primary key] noPkOrUkAllowed

Description: specifies whether to support synchronization without a PRIMARY KEY (PK), NOT NULL, or Unique Key (UK) constraint. It is the main control parameter.
Default value: true. Specify the value based on the types of the source and destination.
Value type: Boolean.
Supported version: OMS V3.3.1 and later.

[HTTP API] listenPort

Description: specifies the HTTP open port.
Default value: 0, indicating a random port.
Value range: A port number greater than 30000 is recommended to avoid conflicts.
Supported version: OMS V3.3.1 and later.

[DDL] skipDdl

Description: specifies the skipped DDL statements.
Default value: None.
Valid values: JSON array.
Supported version: OMS V3.3.1 and later.

[DDL] filterComment

Description: specifies whether to filter comments in DDL statements.
Default value: false.
Value type: Boolean.
Supported version: OMS V3.3.1 and later.

[DDL] shuffleDdlToAllRoute

Description: specifies whether to synchronize DDL statements to all queues when shuffleRecordDispatcher is specified.
Default value: false.
Value type: Boolean.
Supported version: OMS V3.4.0 and later.

[DDL] charsetMapping

Description: specifies the DDL character set mapping.
Default value: Null.
Supported version: OMS V3.3.1 and later.

[DDL] collationMapping

Description: specifies the DDL collation mapping.
Default value: Null.
Supported version: OMS V3.3.1 and later.

[Error handling] runIgnoreError

Description: specifies whether to exit when table synchronization fails due to an exception.
Default value: false.
Value type: Boolean.
Supported version: OMS V3.4.0 and later.

[Filtering] allowRecordTypes

Description: specifies the message type for link-level filtering.
Default value: By default, messages are not filtered.
Valid values: ROW, DELETE, UPDATE, INSERT, HEARTBEAT, and REPLACE. The value is a comma-delimited string.
Supported version: OMS V3.3.1 and later.

[Specific scenarios] intervalTypeFmt

Description: specifies the format of the interval type.
Default value: None. The format of the interval type is obtained by querying the database.
Supported version: OMS V3.3.1 and later.

[Specific scenarios] timestamptzConvert

Description: specifies whether to convert the Asia/Shanghai time zone to GMT+8:00.
Default value: true if the source is an Oracle tenant of OceanBase Database or an Oracle database, and false if otherwise.
Value type: Boolean.
Supported version: OMS V3.3.1 and later.

[Case sensitivity] caseStrategy

Description: specifies the case-sensitivity strategy.
Default value: FOLLOW_SOURCE, which specifies to follow the case in the source database.
Valid values: DEFAULT, FOLLOW_SOURCE, UPPER, and LOWER.
Supported version: OMS V3.3.1 and later.

[Case sensitivity] isCaseSensitive

Description: specifies whether to enable case sensitivity.
Default value: false.
Value type: Boolean.
Supported version: OMS V3.3.1 and later.

[Hotkey merge] hotKeyMerge

Description: specifies whether to enable hotkey merge.
Default value: false.
Value type: Boolean.
Supported version: OMS V3.3.1 and later.

[Hotkey merge] hotKeyTable

Description: specifies to initialize the hotkey table.
Default value: None.
Valid values: JSON array.

Example: [{"schema":"db1","table":"table1"},{"schema":"db1","table":"table1"}].
Supported version: OMS V3.3.1 and later.

[Hotkey merge] mergeDelayThreshold

Description: specifies the number of accumulated keys for determining a hotkey.
Default value: 120.
Value type: int.
Supported version: OMS V3.3.1 and later.

[Statistics] enableDmlDdlTypeStatistics

Description: specifies whether to enable DDL/DML statistics collection.
Default value: false.
Value type: Boolean.
Supported version: OMS V3.3.1 and later.

[Statistics] reportIntervalSec

Description: specifies the interval for generating checkpoints, heartbeat packets, or diagnose files.
Default value: 5.
Value type: int.
Supported version: OMS V3.3.1 and later.

[Special scenarios] systemDefaultTimezone

Description: specifies the time zone for reading the time.
Default value: the system time zone.
Valid values: time zones.
Supported version: OMS V3.3.1 and later.

[Specific scenarios] db2Mode

Description: specifies the mode of the DB2 LUW database.
Default value: normal.
Valid values: oracle and normal.
Supported version: OMS V3.3.1 and later.

[Specific scenarios] mysqlVersion

Description: specifies the version of the source MySQL database.
Default value: None.
Valid values: MySQL database versions.
Supported version: OMS V3.3.1 and later.

[Throttling] throttleRps

Description: specifies whether to limit the RPS.
Default value: -1, which specifies not to limit the RPS.
Value type: int.
Supported version: OMS V3.3.1 and later.

[Throttling] throttleIOPS

Description: specifies whether to limit the IOPS, in bytes.
Default value: -1, which specifies not to limit the IOPS.
Value type: int.
Supported version: OMS V3.3.1 and later.

[Throttling] throttleRecordBound

Description: specifies the number of records within the framework.
Default value: -1, which specifies not to limit the number of records.
Value type: int.
Supported version: OMS V3.3.1 and later.

[Throttling] throttleMemoryBound

Description: specifies whether to limit the memory, in bytes.
Default value: -1, which specifies not to limit the memory.
Value type: int.
Supported version: OMS V3.3.1 and later.

Previous topic

Oracle store tuning

Next topic