This topic describes the background, limits, and configuration method of matching rules for migration objects, as well as the wildcard rules supported by OceanBase Migration Service (OMS) Community Edition.
Background
OMS Community Edition supports DDL statements for incremental migration. However, if a new table is created in the source database during migration, the data in this table cannot be migrated because this table is not among the specified data migration objects.
A wildcard is a short text that represents a group of paths. You can use a wildcard as an elementary regular expression. When you create a data migration project, you can configure data migration objects based on wildcard rules. For more information, see the "Wildcard rules" section in this topic.
OMS Community Edition allows you to configure object migration and exclusion rules for all types of data migration projects. Object migration and exclusion rules apply in all stages of a data migration project. Object migration and exclusion rules take effect on a data migration project only if you configure or update the rules before the project starts.
Notice
If you select DDL Synchronization when you create a data migration project, you must configure the data migration objects based on wildcard rules.
When you use wildcards to configure object migration and exclusion rules during the creation of a data migration project, you can:
Specify the migration objects by configuring wildcard rules.
Edit and verify the matching rules, and preview matched objects.
View the differences between objects before and after a wildcard rule is updated.
You can configure database name mappings and table name mappings based on wildcard rules.
Limits
You must specify an object migration rule for tables.
If you configure multiple schemas to match one type of objects in a rule, reverse incremental synchronization is not supported.
If multi-table aggregation is enabled, reverse incremental synchronization is not supported.
If DML filtering is enabled, full data verification is not supported.
You cannot perform DDL operations on the source database during the period between schema migration and full migration.
You can configure object migration and exclusion rules to match schemas and tables, but not to synchronize DDL operations.
You cannot change a table name that matches the object exclusion rule to a name that does not match the object exclusion rule by using a DDL operation.
If you use gh-ost to synchronize DDL operations for incremental migration from a MySQL database to OceanBase Community Edition, set Object Exclusion Rule to
{database_name}.*_ghc.
Configure a matching rule
Create a migration project.
Log on to the OMS Community Edition console.
In the left-side navigation pane, click Data Migration.
On the Data Migration page, click Create Migration Project in the upper-right corner.
On the Select Source and Destination page, configure the parameters. For more information, see the procedure for creating a data migration project of the corresponding type.
Parameter Description Migration Project Name We recommend that you set it to a combination of Chinese characters, digits, and letters. It must not contain any spaces and cannot exceed 64 characters in length. Label Click the field and select the target tag from the drop-down list. You can click Manage Tags to create, modify, and delete tags. For more information, see Use tags to manage data migration projects. Source Select a created source data source. Destination Select a created destination data source. Click Next.
(Optional) In the dialog box that appears, click OK.
On the Select Migration Type page, specify related parameters.
(Optional) Click Next.
Click Next. On the Select Objects page, select the migration objects and migration scope.
You can select one of the following two modes to migrate objects: Specify Objects or Match Rules. This section describes how to configure a matching rule.
Select Match Rules.
In the Specify Migration Scope section, specify object migration rules in the Object Migration Rule field and object exclusion rules in the Object Exclusion Rule field. The Object Exclusion Rule field is optional. For more information about supported matching rules, see the "Wildcard rules" section in this topic.
Notice
If you use gh-ost to synchronize DDL operations for incremental migration from a MySQL database to OceanBase Community Edition, Object Exclusion Rule is required. Set it to
{database_name}.*_ghc.
Pay attention to the privilege settings of the source database user. If you do not grant all required privileges to the migration user, some objects are not displayed in the frontend by OMS Community Edition, and you cannot correctly configure matching rules. In this case, you need to add these objects to Object Exclusion Rule to prevent the data migration project from being interrupted because OMS Community Edition cannot find the target objects.After you specify the migration object rule and object exclusion rule, you can view the migratable objects and matching results on the right side. The matching results are displayed on the Final Objects, New Object, and Removed Objects tabs. The wildcard rules and exclusion rules apply to tables and views.
You can also rename an object. Specifically, on the Final Objects tab, hover the pointer over the target object, click Rename, enter the new name, and click OK to rename the target object.
Click Verify and Preview Objects.
After the verification succeeds, click Next.
On the Migration Options page, configure the parameters. For more information, see the procedure for creating a data migration project of the selected migration type.
Click Precheck to start a precheck on the data migration project.
Click Start Project.
Wildcard rules
The wildcard rules supported by OMS Community Edition include "*", "?", "[]", and "[!]". Each rule occupies a single row and spaces are not allowed before or after a rule.
After you configure the object migration and exclusion rules, a new table created in the source database can be synchronized by OMS Community Edition in real time if the table name matches the migration rule and does not match the exclusion rule.
The rules to include or exclude migration objects are case-sensitive. A rule is hit only when the object name exactly matches the rule, including the case.
Asterisk (*)
You can use an asterisk to match a string of characters in any length, or a null character. You can also use an asterisk to select all objects in a database or schema. For example, if you set Object Migration Rule to <Schema>.*, all tables and views in the specified schema are displayed on the Matching Result tab in the right-side panel.
In the Object Exclusion Rule field, you can set the objects that do not need to be migrated, and then click Verify. After verification is passed, click Preview Objects and view the objects on the Matching Result tab.
Question mark (?)
You can use a question mark to match a single character. To match more than one character, use multiple consecutive question marks. You can also use a question mark to select the objects in a database or schema that meet specified conditions. For example, if you set Object Migration Rule to <Schema>.t?, tables and views that conform to the format in the specified schema are displayed on the Matching Result tab in the right-side panel.
Notice
You cannot use a question mark to match a null character. In other words, the place held by the question mark must have a character.
In the Object Exclusion Rule field, you can set the objects that do not need to be migrated, and then click Verify. After verification is passed, click Preview Objects and view the objects on the Matching Result tab.
Square brackets ([])
You can use a pair of square brackets to include any characters that must be matched, or use [start-end] to match a continuous range. For example, if you set Object Migration Rule to <Schema>.[]*, tables and views that conform to the format in the specified schema are displayed on the Matching Result tab in the right-side panel.
In the Object Exclusion Rule field, you can set the objects that do not need to be migrated, and then click Verify. After verification is passed, click Preview Objects and view the objects on the Matching Result tab.
An exclamation point in square brackets ([!])
You can use this wildcard rule to match any non-null characters that are not included in the square brackets, or use [!start-end] to indicate a continuous range. For example, if you set Object Migration Rule to <Schema>.[!]*, tables and views that conform to the format in the specified schema are displayed on the Matching Result tab in the right-side panel.
In the Object Exclusion Rule field, you can set the objects that do not need to be migrated, and then click Verify. After verification is passed, click Preview Objects and view the objects on the Matching Result tab.