This topic describes the components of OceanBase Migration Service (OMS).
DBCat
DBCat is the core schema migration component of OMS. As the native schema conversion engine of OceanBase, the DBCat component can accurately map or convert data based on the source data type, destination data type, and character encoding type. The DBCat component can convert and migrate various database objects such as tables, constraints, indexes, and views.
In addition, the DBCat component can be maximally compatible with the tenant types of OceanBase Database. For example, if a version of OceanBase Database does not support some data types in the source database, the DBCat component selects the most compatible data types for conversion and mapping.
The following problems may occur when you migrate data between heterogeneous databases. In this case, you can use the DBCat component to automatically convert the schema of the source database.
The database engines in the source and destination databases are different.
The definitions of data objects vary greatly between databases.
The destination database does not support some data objects in the source database.
The precision settings of the same data type do not match.
In a complex data migration between heterogeneous databases, you can use the DBCat component to generate basic scripts, which are then manually processed by the database administrator (DBA) to generate a schema in the destination database.
Store
The Store component pulls data in different ways for different types of databases. For example, the Store component of an OceanBase database is implemented based on Liboblog.
Liboblog is an incremental data synchronization tool of OceanBase Database. It pulls the redo logs from each partition of an OceanBase database by using remote procedure calls (RPCs), converts the redo logs into an intermediate data format based on the schema information of each table and column, and then outputs the modified data as a transaction.
Full-Import
The Full-Import component migrates the static data and some incremental data from the source database to the destination database. Each migrated table will undergo a series of steps including data sharding, data reading, data processing, and data application.
The Full-Import component selects a proper sharding method based on the schema of a table to concurrently migrate data in the table to the destination. For most data sources, full data is read in streaming mode to reduce the reading pressure on the source database.
Full data is batch-inserted into the destination database. For partitioned tables in OceanBase Database, data is written and aggregated by partition to improve the writing efficiency. The selected tables are concurrently migrated. The sharding checkpoint ACK mechanism is used for tables to ensure data integrity.
Incr-Sync
The Incr-Sync component synchronizes the incremental data generated by the Store component to the destination database. OMS abstracts the synchronized incremental records as DML, DDL, and HB operations. DML operations are INSERT, UPDATE, and DELETE. OMS efficiently and correctly synchronizes these operations to the destination database to ensure final data consistency.
For a table with a UNIQUE constraint, OMS ensures final data consistency based on the idempotence of operations performed on the primary key columns.
For a table without a primary key, OMS ensures final data consistency by using the transaction table mechanism to avoid transaction replay.
Currently, OMS does not guarantee final data consistency when you migrate data from a MySQL database to a table without a primary key in a MySQL tenant of OceanBase Database.