OceanBase Migration Service

V4.0.2Enterprise Edition

OMS Documentation
What's new
OMS Introduction
- What is OMS?
- Terms
- OMS HA
- Architecture
  - Overview
  - Hierarchical functional system
  - Basic components
- Limits
Quick Start
- Data migration process
- Data synchronization process
Deploy OMS
- Deployment types
- System and network requirements
- Memory and disk requirements
- Environment preparations
- Deploy OMS on a single node
- Deploy OMS on multiple nodes in a single region
- Deploy OMS on multiple nodes in multiple regions
- Integrate the OIDC protocol to OMS to implement SSO
- Scale-out OMS
- Check the deployment
- Deploy a time-series database (Optional)
OMS console
- Log on to the OMS console
- Overview
- User center
  - Configure user information
  - Change your logon password
  - Log off
Data migration
- Data migration overview
- Migrate data from a MySQL database to a MySQL tenant of OceanBase Database
- Migrate data from a MySQL tenant of OceanBase Database to a MySQL database
- Migrate data from an Oracle database to a MySQL tenant of OceanBase Database
- Migrate data from an Oracle tenant of OceanBase Database to an Oracle database
- Migrate data from an Oracle database to an Oracle tenant of OceanBase Database
- Migrate data from a DB2 LUW database to an Oracle tenant of OceanBase Database
- Migrate data from an Oracle tenant of OceanBase Database to a DB2 LUW database
- Migrate data from a DB2 LUW database to a MySQL tenant of OceanBase Database
- Migrate data from a MySQL tenant of OceanBase Database to a DB2 LUW database
- Migrate data within OceanBase Database
- Active-active disaster recovery between OceanBase databases
- Migrate data from a TiDB database to a MySQL tenant of OceanBase Database
- Migrate data from a PostgreSQL database to a MySQL tenant of OceanBase Database
- Manage data migration projects
  - View details of a data migration project
  - Change the name of a data migration project
  - View and modify migration objects
  - Use tags to manage data migration projects
  - Download and import the settings of migration objects
  - Start and pause a data migration project
  - Release and delete a data migration project
- Features
  - DML filtering
  - Synchronize DDL operations
  - Configure matching rules for migration objects
  - Wildcard rules
  - Rename a database table
  - Use SQL conditions to filter data
  - Create and update a heartbeat table
  - Schema migration mechanisms
  - Schema migration operations
  - Set an incremental synchronization timestamp
- Supported DDL operations and limits for synchronization
  - DDL synchronization from a MySQL database to a MySQL tenant of OceanBase Database
    - Overview of DDL synchronization from a MySQL database to a MySQL tenant of OceanBase Database
    - CREATE TABLE
      - Create a table
      - Create a column
      - Create an index or a constraint
      - Create partitions
    - Data type conversion
    - ALTER TABLE
      - Modify a table
      - Operations on columns
      - Operations on constraints and indexes
      - Operations on partitions
    - TRUNCATE TABLE
    - RENAME TABLE
    - DROP TABLE
    - CREATE INDEX
    - DROP INDEX
    - DDL incompatibilities between a MySQL database and a MySQL tenant of OceanBase Database
      - Overview
      - Incompatibilities of the CREATE TABLE statement
        Incompatibilities of CREATE TABLE
        Column types that are supported to create indexes or constraints
      - Incompatibilities of the ALTER TABLE statement
        Incompatibilities of ALTER TABLE
        Change the type of a constrained column
        Change the type of an unconstrained column
        Change the length of a constrained column
        Change the length of an unconstrained column
        Delete a constrained column
      - Incompatibilities of DROP INDEX operations
  - Synchronize DDL operations from a MySQL tenant of OceanBase Database to a MySQL database
  - DDL operations for synchronizing data from an Oracle database to an Oracle tenant of OceanBase Database
    - Overview
    - CREATE TABLE
      - Overview
      - Create a relational table
        Create a relational table
        Define columns of a relational table
      - Virtual columns
      - Regular columns
      - Create partitions
        Overview
        Partitioning
        Subpartitioning
        Composite partitioning
        User-defined partitioning
        Subpartition templates
      - Constraints
        Overview
        Inline constraints
        Out-of-line constraints
    - CREATE INDEX
      - Overview
      - Normal indexes
    - ALTER TABLE
      - Modify tables
      - Modify, drop, and add table attributes
      - Column attribute management
        Modify, drop, and add column attributes
        Rename a column
        Add columns and column attributes
        Modify column attributes
        Drop columns
      - Modify, drop, and add constraints
      - Partition management
        Modify, drop, and add partitions
        Drop partitions
        Drop subpartitions
        Add partitions and subpartitions
        Modify partitions
        Truncate partitions
    - DROP TABLE
    - COMMENT
    - RENAME OBJECT
    - TRUNCATE TABLE
    - DROP INDEX
    - DDL incompatibilities between an Oracle database and an Oracle tenant of OceanBase Database
      - Overview
      - Incompatibilities of CREATE TABLE
      - Incompatibilities in table modification operations
        Incompatibilities of ALTER TABLE
        Change the type of a constrained column
        Change the type of an unconstrained column
        Change the length of a constrained column
        Change the length of an unconstrained column
  - Synchronize DDL operations from an Oracle tenant of OceanBase Database to an Oracle database
  - Synchronize DDL operations from an Oracle tenant of OceanBase Database to a DB2 LUW database
  - Synchronize DDL operations from a DB2 LUW database to a MySQL tenant of OceanBase Database
  - Synchronize DDL operations from a MySQL tenant of OceanBase Database to a DB2 LUW database
  - DDL synchronization between MySQL tenants of OceanBase Database
  - DDL synchronization between Oracle tenants of OceanBase Database
Data synchronization
- Overview
- Synchronize data from OceanBase Database to a Kafka instance
- Synchronize data from an OceanBase database to a RocketMQ instance
- Synchronize data from OceanBase Database to a DataHub instance
- Synchronize data from an ODP logical table to a physical table in a MySQL tenant of OceanBase Database
- Synchronize data from an ODP logical table to a DataHub instance
- Synchronize data from an IDB logical table to a physical table in a MySQL tenant of OceanBase Database
- Synchronize data from an IDB logical table to a DataHub instance
- Synchronize data from a MySQL database to a DataHub instance
- Synchronize data from an Oracle database to a DataHub instance
- Manage data synchronization projects
  - View details of a data synchronization project
  - Change the name of a data synchronization project
  - View and modify synchronization objects
  - Use tags to manage data synchronization projects
  - Download and import the settings of synchronization objects
  - Start and pause a data synchronization project
  - Release and delete a data synchronization project
- Features
  - DML filtering
  - Synchronize DDL operations
  - Rename databases and tables
  - Rename a topic
  - Use SQL conditions to filter data
  - Column filtering
  - Data formats
Create and manage data sources
- Create data sources
  - Create an OceanBase data source
    - Create a physical OceanBase data source
    - Create a DBP data source
    - Create an IDB data source
  - Create a MySQL data source
  - Create an Oracle data source
  - Create a TiDB data source
  - Create a Kafka data source
  - Create a RocketMQ data source
  - Create a DataHub data source
  - Create a DB2 LUW data source
  - Create a PostgreSQL data source
- Manage data sources
  - View data source information
  - Copy a data source
  - Edit a data source
  - Delete a data source
- Create a database user
- User privileges
- Enable binlogs for the MySQL database
- Minimum privileges required when an Oracle database serves as the source
OPS & Monitoring
- O&M overview
- Go to the overview page
- Server
  - View server information
  - Update the quota
  - View server logs
- Components
  - Store
    - Create a store
    - View details of a store
    - Update the configurations of a store
    - Start and pause a store
    - Delete a store
  - Incr-Sync
    - View details of an Incr-Sync component
    - Start and pause an Incr-Sync component
    - Migrate an Incr-Sync component
    - Update the configurations of an Incr-Sync component
    - Batch O&M
    - Delete an Incr-Sync component
  - Full-Import
    - View details of a Full-Import component
    - Pause a Full-Import component
    - Rerun and resume a Full-Import component
    - Update the configurations of a Full-Import component
    - Delete a Full-Import component
  - Full-Verification
    - View details of a Full-Verification component
    - Pause a Full-Verification component
    - Rerun and resume a Full-Verification component
    - Update the configurations of a Full-Verification component
    - Delete a Full-Verification component
- O&M tickets
  - View details of an O&M ticket
  - Skip a ticket or sub-ticket
  - Retry a ticket or sub-ticket
System management
- Permission Management
  - Overview
  - Manage users
  - Manage departments
- Alert center
  - View project alerts
  - View system alerts
  - Manage alert settings
- Associate with OCP
- System parameters
  - Modify system parameters
  - Modify HA configurations
  - oblogproxy parameters
- Operation audit
OMS O&M
- Manage OMS services
- OMS logs
- Component O&M
  - O&M operations for the Supervisor component
  - CLI-based O&M for the Connector component
  - O&M operations for the Store component
- Component tuning
  - Incr-Sync/Full-Import tuning
  - Oracle store tuning
- Component parameters
  - Coordinator
  - Condition
  - Source Plugin
    - Overview
    - StoreSource
    - DataFlowSource
    - LogProxySource
    - KafkaSource (TiDB)
  - Sink Plugin
    - Overview
    - JDBC-Sink
    - KafkaSink
    - DatahubSink
    - RocketMQSink
  - Store parameters
    - Parameters of an Oracle store
    - Parameters of a DB2 store
    - Parameters of a MySQL store
    - Parameters of an OceanBase store
  - Parameters of the CM component
  - Parameters of the Supervisor component
- Set throttling
Reference Guide
- API Reference
  - Obtain the status of a migration project
  - Obtain the status of a synchronization project
- OMS error codes
- Alert Reference
  - oms_host_down
  - oms_host_down_migrate_resource
  - oms_host_threshold
  - oms_migration_failed
  - oms_migration_delay
  - oms_sync_failed
  - oms_sync_status_inconsistent
  - oms_sync_delay
Upgrade Guide
- Overview
- Upgrade OMS in single-node deployment mode
- Upgrade OMS in multi-node deployment mode
- FAQ
FAQ
- General O&M
  - How do I modify the resource quotas of an OMS container?
  - How do I troubleshoot the OMS server down issue?
  - Deploy InfluxDB for OMS
  - Increase the disk space of the OMS host
- Project diagnostics
  - How do I troubleshoot common problems with Oracle Store?
  - How do I perform performance tuning for Oracle Store?
  - What do I do when Oracle Store reports an error at the isUpdatePK stack?
  - What do I do when a store does not have data of the timestamp requested by the downstream?
  - What do I do when OceanBase Store failed to access an OceanBase cluster through RPC?
  - How do I use LogMiner to pull data from an Oracle database?
- OPS & monitoring
  - What are the alert rules?
- Data synchronization
  - FAQ about synchronization to a message queue
    - What are the strategies for ensuring the message order in incremental data synchronization to Kafka
- Data migration
  - User privileges
    - What privileges do I need to grant to a user during data migration to or from an Oracle database?
  - Full migration
    - How do I query the ID of a checker?
    - How do I query log files of the Checker component of OMS?
    - How do I query the verification result files of the Checker component of OMS?
    - What do I do if the destination table does not exist?
    - What can I do when the full migration failed due to LOB fields?
    - What do I do if garbled characters cannot be written into OceanBase Database V3.1.2?
  - Incremental synchronization
    - How do I skip DDL statements?
    - How do I update whitelists and blacklists?
    - What are the application scope and limits of ETL?
- Installation and deployment
  - How do I upgrade Store?
Release Note
- V4.0
  - OMS V4.0.2
  - OMS V4.0.1
- V3.4
  - OMS V3.4.0
- V3.3
  - OMS V3.3.1
  - OMS V3.3.0
- V3.2
  - OMS V3.2.2
  - OMS V3.2.1
- V3.1
  - OMS V3.1.0
- V2.1
  - OMS V2.1.2
  - OMS V2.1.0

Download PDF

OceanBase

OceanBase

OceanBase, A Highly Scalable Database for Transactional, Analytical, and AI Workloads.

Company

About OceanBase Contact Us Partner

Product

OceanBase Cloud OceanBase Enterprise Quick Start Download Pricing

Resources

Docs Blog White Paper Trust Center

Follow us

© OceanBase 2024. All rights reserved | Cloud Service Agreement | Privacy Policy | Legal | Security

Document Feedback

LogProxySource

Last Updated：2023-06-29 11:11:39 Updated

share

share

Starter class

logproxy-source.jar

com.oceanbase.oms.connector.source.logproxy.LogProxySourceBooster

[Metadata] logproxy.address

Description: specifies the address of the LogProxy.
Default value: None. This parameter is required.
Value type: String.
Supported version: OMS V3.3.1 and later.

[Metadata] username

Description: specifies the username.
Default value: None.
Supported version: OMS V3.3.1 and later.

[Metadata] password

Description: specifies the password.
Default value: None.
Supported version: OMS V3.3.1 and later.

[Metadata] dbType

Description: specifies the type of the database.
Default value: None. This parameter is required.
Value range: The value of DbTypeEnum.
Supported version: OMS V3.3.1 and later.

[Checkpoint information] checkpoint

Description: specifies the checkpoint in JSON format. The key is logproxy-client-id and the value is the start timestamp.
Default value: None. This parameter is required.
Value range:

{ "clientId": "12345" }
Supported version: OMS V3.3.1 and later.

[LogProxy parameters] client.id

Description: specifies the user defined client ID (ClientConf.USER_DEFINED_CLIENTID).
Default value: None.
Value type: String.
Supported version: OMS V3.3.1 and later.

[LogProxy parameters] table.white.list

Description: specifies the whitelist of tables, the format of which is the same as that in the store.
Default value: None. This parameter is required.
Value type: String.
Supported version: OMS V3.3.1 and later.

[LogProxy parameters] oceanbase.cluster.url

Description: specifies the instance URL in an OceanBase cluster.
Default value: None. This parameter is required.
Value type: String.
Supported version: OMS V3.3.1 and later.

[LogProxy parameters] wait.queue.size

Description: specifies the transfer queue size (ClientConf.TRANSFER_QUEUE_SIZE).
Default value: None.
Value type: int.
Supported version: OMS V3.3.1 and later.

[LogProxy parameters] properties

Description: specifies the ObReaderConfig parameter.
Default value: None.
Value type: JSON.
Supported version: OMS V3.3.1 and later.

[DDL] ignoreDdl

Description: specifies whether to ignore DDL statements.
Default value: true.
Valid values: true and false.
Supported version: OMS V3.3.1 and later.

[Transaction splitting] splitThreshold

Description: specifies the threshold for splitting a large transaction.
Default value: 128.
Value type: int.
Supported version: OMS V3.3.1 and later.

sourceBatchMemorySize

Description: specifies the maximum number of records for batch execution (recordBatch) in a transaction. The value is set based on the JVM memory. A transaction will be split for synchronization when either recordBatch or recordBatch is reached.
Default value: 4194304, in bytes.
Value type: int.
Supported version: OMS V4.0.1 and later.

Previous topic

Next topic

KafkaSource (TiDB)