OceanBase Migration Service (OMS) Community Edition supports real-time data migration between OceanBase Community Edition and different types of data sources, such as MySQL, TiDB, Kafka, and RocketMQ, and supports data migration between MySQL tenants of OceanBase Community Edition.
Background
OMS Community Edition supports online migration. The server deployed with OMS Community Edition must be able to connect to both source and destination databases at the same time. You can start a data migration project after you configure the source and destination databases and select the objects to be migrated.
OMS Community Edition automatically performs all the procedures in a data migration project, replicating the database objects, existing data, or incremental data in the source database (data table) to the destination database (data table). It also ensures the real-time synchronization of incremental data from the source database to the destination database.
The server deployed with OMS Community Edition must be able to connect to both source and destination databases at the same time. To start a data migration project, you only need to configure the source and destination databases and select the tables to be migrated.
Data migration process
To create and manage a data migration project, perform the following steps:
Prepare for data migration.
Before you migrate data by using OMS Community Edition, create a user in the source or destination database and authorize the user. For more information, see Create and authorize a database user.
Add a data source.
In the OMS Community Edition console, add the source and destination databases as data sources. For more information, see Add a data source.
Create a data migration project.
Specify the source database, destination database, migration type, and tables to be migrated in the migration project. A data migration project consists of the following steps:
Data migration
OMS Community Edition automatically runs the data migration project without any human intervention in the interim. You do not need to start, suspend, or interrupt the process.
Data verification
After the data migration is completed, you can verify the data integrity and accuracy.
Synchronization link switching
Before your business is applied to the destination database, you can switch the synchronization link to create a synchronization link from the destination database to the source database.
Switch your business to the source database.
Create a rollback plan before you switch your business, to reduce the impact of data migration on your business.
Optional. View the status of the data migration project.
On the migration project monitoring page, you can view the link details and monitor the migration project status in real time. For more information, see View details of a data migration project.
Optional. Stop and release the data migration project.
After the data migration project is completed and data no longer needs to be synchronized from the source database to the destination database, you can clear the current data migration project. For more information, see Delete a data migration project.
Supported project types
OMS Community Edition supports the following types of online data migration projects:
Data migration from a MySQL database to an OceanBase Community Edition instance
Data migration from an OceanBase Community Edition instance to a MySQL database
Data migration between two OceanBase Community Edition instances
Data migration from a TiDB database to an OceanBase Community Edition instance
Supported migration types
For more information about the operations for each type of data migration project, see Start a data migration project.
| Migration type | Description |
|---|---|
| Schema migration | The migration of the definitions of data objects (tables, indexes, constraints, comments, and views) from the source database to the destination OceanBase database. Temporary tables are automatically filtered out. If the source database is not an OceanBase database, OMS Community Edition performs format conversion and encapsulation based on the syntax definition and standard of the tenant type of the destination OceanBase database, and then OMS Community Edition replicates the data to the destination OceanBase database. |
| Full migration | The migration of the existing data from tables in the source database to corresponding tables in the OceanBase database. You can view table objects and table indexes on the Full Migration page. The status of the full migration changes to Completed only after migration of the table objects and table indexes is completed. On the Table Indexes page, you can click View Creation Syntax next to the target table object to view its index creation syntax. You can combine full migration with incremental migration to ensure data consistency between the source and destination databases. If any objects failed to be migrated during a full migration, the causes of the failure are displayed. |
| Incremental migration | The migration of changed data of the source database to the corresponding table in OceanBase Database. Data changes include data addition, modification, and deletion. When services are continuously writing data to the source database, OMS Community Edition starts the incremental data pull module to pull incremental data from the source instance, parses and encapsulates the incremental data, and then stores the data in OMS Community Edition, before it starts the full data migration. After a full data migration task is completed, OMS Community Edition starts the incremental data replay module to pull incremental data from the incremental data pull module. The incremental data is synchronized to the destination instance after being filtered, mapped, and converted. |
| Full verification | After the full data migration and incremental data migration are completed, OMS Community Edition automatically initiates a full data verification task to verify the data tables in the source database and the tables in the destination database. OMS Community Edition also provides corresponding APIs for you to initiate custom data verification tasks in the incremental data synchronization process. You can view the information of specific columns where inconsistent data is detected. OMS Community Edition runs SQL scripts to correct the data in the destination database based on the data in the source database. The correction operation is not supported if the source database has no corresponding data. |
| Forward switchover | Forward switchover is the abstraction and standardization of the conventional system cutover process, a type of switchover that does not operate business application connections, and a task flow that needs to be executed before and after the data migration link of OMS Community Edition is used for application switchover. You need to make sure that the entire forward switchover process is completed before the application connections are switched over to OceanBase Database. Forward switchover will be included if you choose to perform data migration. You need to terminate forward incremental synchronization, delete the additional columns and unique indexes that the migration depends on, add the filtered CHECK constraint, and activate the triggers and foreign keys in the destination database to ensure the data integrity and availability of the destination database. If reverse incremental migration is configured, the subtasks for starting reverse incremental migration and disabling triggers and foreign keys in the source database are included in the forward switchover process to start real-time incremental synchronization from the destination database to the source database. |
| Reverse incremental migration | After the migration is completed, you can start the incremental data synchronization link on the OMS Community Edition to synchronize the incremental data from the destination database to the source database (that is, in the reverse direction) before the business database switchover is completed for business cutover scenarios. Then, the changed data that occur in the destination OceanBase database after the business cutover are synchronized to the source business database in real time. You can check the performance of the reverse synchronization on OMS Community Edition, including:
|