This topic describes how to use OceanBase Migration Service (OMS) to migrate incremental data from an Oracle tenant of OceanBase Database, which can be a physical data source or an ApsaraDB for OceanBase data source, to a MySQL database.
Prerequisites
You have created dedicated database users for data migration in the Oracle tenant of OceanBase Database and the MySQL database, and granted required privileges to the users. For more information, see Create a database user.
You have created data sources for the source and destination. For more information, see Create an OceanBase data source and Create a MySQL data source.
Limitations
At present, MySQL 5.5, 5.6, 5.7, and 8.0 are supported.
OMS does not support the migration of tables without primary keys from an Oracle tenant of OceanBase Database to a MySQL database.
If the destination is a database, OMS does not support triggers in the destination database. If triggers exist in the destination database, the data migration may fail.
OMS supports the migration of only objects whose database name, table name, and column name are ASCII-encoded and do not contain special characters. The special characters are spaces, line breaks, and the following characters:
. | " ' ` ( ) = ; / & \.OMS does not support a standby OceanBase database as the source.
Considerations
If you use OceanBase Database V4.x, we recommend that you enable log archiving. If you enable log archiving, OMS implements incremental synchronization by consuming archive logs after clogs are recycled. For more information about how to enable log archiving, see the Log archive chapter.
If the UTF-8 character set is used in the source, we recommend that you use a compatible character set, such as UTF-8 or UTF-16, in the destination to avoid garbled characters.
If the clocks between nodes or between the client and the server are out of synchronization, the latency may be inaccurate during incremental synchronization.
For example, if the clock is earlier than the standard time, the latency can be negative. If the clock is later than the standard time, the latency can be positive.
In a project for migrating data from an Oracle tenant of OceanBase Database of a version earlier than V3.2.x to a MySQL database, if the source table is a multi-partition table with a global unique index and you update the values of the partitioning key of the table, data may be lost during migration.
If you change the unique index in the destination, you must restart the data migration project to avoid data inconsistency.
A difference between the source and destination table schemas may result in data consistency. Some known scenarios are described as follows:
When you manually create a table schema at the destination, if the data types of any columns are not supported by OMS, implicit data type conversion may occur at the destination, which causes inconsistent column types between the source and destination.
If the length of a column at the destination is shorter than that at the source, the data of this column may be automatically truncated, which causes data inconsistency between the source and destination.
If you select only Incremental Synchronization when you create a data migration project, OMS requires that the local incremental logs in the source database be retained for more than 48 hours.
Procedure
Create a data migration project.
Log on to the OMS console.
In the left-side navigation pane, click Data Migration.
On the Data Migration page, click Create Migration Project in the upper-right corner.
On the Select Source and Destination page, configure the parameters.
Parameter Description Migration Project Name We recommend that you set it to a combination of digits and letters. It must not contain any spaces and cannot exceed 64 characters in length. Tag (Optional) Click the field and select a target tag from the drop-down list. You can also click Manage Tags to create, modify, and delete tags. For more information, see Use tags to manage data migration projects. Source If you have created a data source for the Oracle tenant of OceanBase Database, which can be a physical data source or an ApsaraDB for OceanBase data source, select it from the drop-down list. If not, click New Data Source in the drop-down list and create one in the dialog box that appears on the right. For more information about the parameters, see Create a physical OceanBase data source or Create a public cloud OceanBase data source. Destination If you have created a MySQL data source, select it from the drop-down list. If not, click New Data Source in the drop-down list and create one in the dialog box that appears on the right. For more information about the parameters, see Create a MySQL data source. Click Next.
On the Select Migration Type page, specify the migration type for the migration project.
At present, only DML Synchronization is supported for Incremental Synchronization. For more information, see DML filtering.
On the Select Migration Objects page, select the migration objects.
You can select Specify Objects or Match Rules to specify the migration objects.
Select Specify Objects. Then select the objects to be migrated on the left and click > to add them to the list on the right. You can select tables and views of one or more databases as the migration objects.
Notice
The names of tables to be migrated, as well as the names of columns in the tables, must not contain Chinese characters.
If the database or table name contains a double dollar sign ($$), you cannot create the migration project.
OMS automatically filters out unsupported tables.
OMS also allows you to import objects from text, rename objects, set row filters, view column information, and remove a single object or all objects to be migrated.
Operation Description Import objects - In the list on the right of the Specify Migration Scope section, click Import Objects in the upper-right corner.
- In the dialog box that appears, click OK.
Notice
This operation will overwrite previous selections. Proceed with caution. - In the Import Objects dialog box, import the objects to be migrated.
You can import CSV files to rename databases/tables and set row filtering conditions. For more information, see Download and import the settings of migration objects. - Click Validate.
- After the validation succeeds, click OK.
Rename objects OMS allows you to rename the migration objects. For more information, see Rename a migration or synchronization object. Configure settings OMS allows you to use the WHEREclause to filter rows. For more information, see Use SQL conditions to filter data.
You can also view column information of the migration object in the View Column section.Remove one or all objects During data mapping, OMS allows you to remove one or more selected objects to be migrated or synchronized to the destination. - To remove a single migration object:
In the list on the right of the Specify Migration Scope section, move the pointer over the target object, and click Remove. - To remove all migration objects:
In the list on the right of the Specify Migration Scope section, click Remove All in the upper-right corner. In the dialog box that appears, click OK.
Select Match Rules. For more information, see Configure matching rules for migration objects.
Click Next. On the Migration Options page, configure the incremental synchronization parameters.
Parameter Description Incremental Log Pull Resource Configuration You can select Small, Medium, or Large to use the corresponding default value of Memory. You can also customize the resource configurations for incremental log pull. Through resource configuration for the Store component, you can limit the resource consumption of a project in log pull in the incremental synchronization phase. Notice
In the case of custom configurations, the minimum value is 1, and only integers are supported.
Incremental Data Write Resource Configuration You can select Small, Medium, or Large to use the corresponding default values of Write Concurrency and Memory. You can also customize the resource configurations for incremental data write. Through resource configuration for the Incr-Sync component, you can limit the resource consumption of a project in data writes in the incremental synchronization phase. Notice
In the case of custom configurations, the minimum value is 1, and only integers are supported.
Incremental Record Retention Time The duration that incremental parsed files are cached in OMS. A longer retention period results in more disk space occupied by the Store component. Incremental Synchronization Start Timestamp Specify a point in time after which data is to be synchronized. The default value is the current system time. For more information, see Set an incremental synchronization timestamp. Click Precheck to start a precheck on the data migration project.
During the precheck, OMS checks the read and write privileges of the database users and the network connections of the databases. The data migration project can be started only after it passes all check items. You can perform one of the following operations if an error is returned during the precheck:
Identify and troubleshoot the problem and then perform the precheck again.
Click Skip in the Actions column of a failed precheck item. In the dialog box that appears, you can view the prompt for the consequences of the operation and click OK.
Click Start Project. If you do not need to start the project now, click Save to go to the details page of the data migration project. You can start the project later as needed.
OMS allows you to modify the migration objects when the data migration project is running. For more information, see View and modify migration objects. After a data migration project is started, the migration subtasks will be executed based on the selected migration types. For more information, see the "View migration details" section in the View details of a data migration project topic.