View details of a data migration task

After a data migration task starts, you can view the task metrics on the details page of the task, such as the basic information, progress, and status of the task.

Go to the details page

1. Log in to the OMS console.

2. In the left-side navigation pane, click Data Migration.

3. On the Data Migration page, click the name of the target task to go to the details page, where you can view its Basic Information and Migration Details.

   On the Data Migration page, you can search for the target data migration task based on tags, status, type, and keywords. The states of a data migration task include:

   • Not Started: indicates that the data migration task has not started. You can click Start in the Actions column to start the task.

   • Running: The data migration task is in progress. The plan and current progress of data migration are displayed on the right side of the status.

   • Modifying : The data migration task is modifying the migration objects.

   • Integrating: The process of merging the data migration task of the modified migration object with the task of modifying the migration object.

   • Stopped: The data migration task is manually stopped. You can click Resume in the Actions column to resume the migration from the point where it was stopped.

   • Failed: The data migration task failed. The specific reason for the failure is displayed on the right side of the status. You can click the name of the data migration task to view the specific error message on the details page.

   • Completed: The data migration task has been successfully executed. OMS has migrated the specified data content to the target database in the migration mode you preset.

   • Releasing: The data migration task is being released. A data migration task in the Releasing state cannot be operated.

   • Released: The data migration task has been successfully released. After a task is released, OMS terminates the current migration and incremental synchronization tasks.

View basic information

The Basic Information section displays the basic information about the current data migration task.

You can perform the following operations:

• View migration objects

  Click View Objects in the upper-right corner of the page to view the list of migration objects for the current data migration task. You can also modify the migration objects during the execution of the data migration task. For more information, see View and modify migration objects.

• View component monitoring metrics

  Click View Component Monitoring in the upper-right corner of the page to view information about components such as Store, Incr-Sync, Full-Import, and Full-Verification. You can also perform the following operations:

  • Start a component: Click Start next to the target component that is not running. In the dialog box that appears, click OK.

  • Stop a component: Click Stop next to the target component that is running. In the dialog box that appears, click OK.

  • Update a component: Click Update next to the target component. On the Update Configuration page, modify the relevant configurations and click Update.

    Notice

    Update involves a restart. Proceed with caution.

  • View logs: Click View Logs next to the target component. The View Logs page displays the logs generated in the most recent run. You can search for logs, Download, and Copy logs.

• View or modify parameter configurations

  Data migration tasks in the Running, Modifying, Integrating, Completed, Ending, and Ended state support viewing parameter configurations. Data migration tasks in the Not Started, Stopped, and Failed state support modifying parameter configurations. For more information, see View and modify migration parameters.

  The specific parameters that support modification depend on the type of the data migration task and the phase in which the task is running. Different phases support modifying different parameters.

• Download object configuration information

  OMS allows you to download the configurations of a data migration task in a batch import format. For more information, see Download and import migration object configurations.

• Download conflict log files

  During full migration, incremental synchronization, or reverse incremental synchronization of a data migration task, if conflicts are found, OMS allows you to download conflict log files. For more information, see Download conflict logs.

View migration details

The Migration Details section displays the status, progress, start time, completion time, and total duration of all subtasks.

Schema migration

The schema migration task migrates data object definitions, such as tables, indexes, constraints, comments, and views, from the source database to the destination database. Temporary tables are automatically filtered. If the source database is not an OceanBase database, the data types and SQL syntaxes are automatically converted and assembled based on the syntax definitions for the destination OceanBase tenant type, and then replicated to the destination database. If an object with the same name exists in the destination database, the object is skipped by default. You must ensure that the schemas of the source and destination tables are consistent.

If your data migration task enters the forward switchover step, the OMS instance automatically drops hidden columns and unique indexes based on the type of your data migration task. For more information, see Description of data migration service hidden column mechanism.

You can view the overall status, start time, end time, and total duration of the schema migration, as well as the migration progress of the destination objects on the Schema Migration page. If your data migration task migrates data from an Oracle database to the Oracle compatible mode of an OceanBase database, you can also view whether to proceed to the next phase after the schema migration is complete, on the right side of the progress bar.

You can also perform the following operations on the destination objects:

• View the creation syntax: You can click View in the Actions column of the target object to view the creation syntax of a database, table, or index.

  If the syntax is fully compatible, the DDL syntax executed on the OBServer node is displayed. If the syntax is not fully compatible, the converted creation syntax is displayed.

• Modify the creation syntax and retry: You can check the failed DDL statements and errors, modify the definition of a failed DDL conversion result, and migrate it to the destination database again.

• Retry or retry all failed objects: For a failed schema migration task, you can retry the conversion individually or click Retry All Failed Objects in the upper-right corner of the corresponding tab.

• Skip or batch skip: For a failed schema migration task, you can skip the conversion of the failed objects individually or select the objects that need to be skipped, and click Batch Skip in the upper-right corner. When you skip an object, its index is also skipped.

• Remove or batch remove: For a failed schema migration task, you can remove the failed objects individually or select the objects that need to be removed, and click Batch Remove in the upper-right corner. When you remove an object, its index is also removed.

• View details: For a failed schema migration task, you can view the DDL statements and error information executed on the OBServer node.

• After the schema migration is complete, you can download the conversion information of completed or converted tables as a CSV file.

  1. Click the ... in the upper-right corner of the tab and select Download Schema Conversion Information.

  2. In the Download Schema Conversion Information dialog box, select Conversion Information Only or Complete DDL statements and conversion information at the destination.

  3. Click Download.

Full migration

Full migration involves migrating existing data from tables in the source database to corresponding tables in the target database. On the Full Migration page, you can filter objects by source and target databases, or select View Objects with Errors to filter out objects that hinder the overall migration progress. You can also view related details on the Table Objects, Table Indexes, and Migration Performance tabs. The status of a full migration task changes to Completed only after the table objects and table indexes are migrated.

• On the Table Objects tab, you can view the names, source and target databases, estimated data volume, migrated data volume, and status of tables.

• On the Table Indexes tab, you can view the table objects, source and target databases, creation time, end time, time consumed, and status. You can also view the index creation syntax and remove unwanted indexes.

• On the Migration Performance tab, you can view graphs of Source RPS, Target RPS, Source migration traffic, Target migration traffic, Average Read Time at Source, Average Slicing Time at Source, and Average Write Time at Target to identify performance-related issues, if any.

You can combine full migration with incremental synchronization to ensure data consistency between the source and target databases. If any objects fail to be migrated during full migration, the causes of the failure are displayed.

Incremental Synchronization

During incremental synchronization, the data that has changed in the source database, including new data, modified data, and deleted data, is synchronized to the corresponding table in the target database. Before full migration starts, OMS starts the incremental pull module to pull incremental updates from the source instance, parses and packages the incremental updates, and stores them in OMS. After full migration is completed, OMS starts the incremental data replay module to obtain incremental data from the incremental data pull module. The incremental data is filtered, mapped, and converted, and then synchronized to the target instance.

If an exception occurs in Incr-Sync, for example, if a DDL statement executed in the source database causes the data migration task to fail, the failed DDL statement is displayed on the page, along with a Skip button. In this case, you can click Skip in the dialog box and confirm the operation.

For a data migration task in the Running state, you can view its latency, current timestamp, or Kafka consumption timestamp, and incremental synchronization performance in the Incremental Synchronization section. The latency is displayed in the following format: X seconds (updated Y seconds ago). Normally, Y is less than 20.

If you have configured Obtain Incremental Data through Kafka for the Oracle data source, the latency of incremental synchronization from the Oracle database to the OceanBase Database Oracle compatible mode or the reverse incremental synchronization from the OceanBase Database Oracle compatible mode to the Oracle database is displayed in the following format: Kafka message X seconds/Incremental data Y.

• Kafka message latency refers to the latency between the timestamp when the incremental data is written to Kafka and the current time when the incremental data is read from Kafka and written to the target database.

• Incremental data latency refers to the latency between the timestamp when the incremental data is generated in the source database and the current time when the incremental data is read from Kafka and written to the target database. Note that since the order of the messages delivered from OGG to Kafka are not guaranteed, the latency is for reference only.

For a data migration task in the Stopped or Failed state, you can enable the DDL/DML statistics collection feature to collect statistics on database operations performed after this feature is enabled. You can also view the specific information about incremental synchronization objects and the performance of incremental synchronization.

• The Synchronization Object Statistics tab displays the table-level statistics about DML statements executed in the source and target databases. The numbers displayed in the Change Sum, Delete, Insert, and Update fields in the section above the Synchronization Object Statistics tab are the sums of the corresponding columns on this tab.

• The Incremental Synchronization Performance tab displays the following content:

  • Latency: the latency in synchronizing incremental changes from the source database to the target database, in seconds.

  • Migration traffic: the traffic throughput of incremental changes synchronized from the source database to the target database, in Kbit/s.

  • Average execution time: the average execution time of an SQL statement, in ms.

  • Average commit time: the average commit time of a transaction, in ms.

  • RPS: the number of rows written to the target database per second.

  When you create a data migration task, we recommend that you specify the alert level and alert frequency to always keep track of the task status. OMS is in low-protection mode by default. You can change the alert level based on your business requirements. For more information, see Configure alerts.

  • If the latency of incremental synchronization in a data migration task exceeds the configured alert threshold and incremental synchronization is always in the running state, the system does not trigger an alert.

  • If the latency of incremental synchronization in a data migration task does not exceed the configured alert threshold, the state of incremental synchronization changes from running to monitoring. Once the state of incremental synchronization changes to monitoring, it will not change back to running even if the latency exceeds the alert threshold.

Full verification

After full migration and incremental migration are completed, OMS automatically initiates a full verification task to verify the data tables in the source and target databases.

You can also initiate custom data verification tasks during incremental synchronization. On the Full Verification page, you can view the overall status, start time, end time, total consumed time, estimated total number of rows, number of migrated rows, real-time traffic, and RPS of the full verification task.

The Full Verification page contains the Verified objects and Full Verification Performance tabs.

• On the Verified objects tab, you can view the verification progress and verification object list.

  • You can view the names, source and target databases, full verification progress and results, and result summary of all migration objects.

  • You can filter migration objects by source or target database.

  • You can select View Completed Objects Only to view the basic information of objects that have completed schema migration, such as the object names.

  • You can choose Reverify > Restart Full Verification to run full verification again for all migration objects.

  • Take note of the following points about tables with inconsistent verification results:

    If you need to reverify all data in the tables, choose Reverify > Reverify Abnormal Table.

    If you need to reverify only inconsistent data, choose Reverify > Only Inconsistent Records.

    Notice

    Correction operations are not supported if the source database does not have any data.

• On the Full Verification Performance tab, you can view the performance data such as the graphs of Source RPS, Target RPS, Source traffic verification, and Target verification traffic, and performance benchmarks to identify performance-related issues.

  OMS allows you to skip full verification for tasks that are in progress or have failed. On the Full Verification page, click Skip Full Verification in the upper-right corner. In the dialog box that appears, click OK.

  Notice

  After you skip full verification, you cannot resume the verification task for data comparison and correction. You can only clone the current task to initiate full verification again. Therefore, proceed with caution.

  You can click Go To Next Stage after full verification is completed to perform forward switchover. After you enter the switchover process, you cannot recheck the current verification task or make any data comparisons or corrections.

Forward switchover

Forward switchover (abstracted and standardized as the system cutover process) does not involve the switching of business application connections. Instead, it is a task flow that needs to be executed when you use OMS to migrate data during an application cutover. Make sure that the forward switchover is completed before you switch the business application connections to the target endpoint.

The forward switchover is an essential step in data migration. It ensures that the related work for data migration is completed, and allows you to start the reverse incremental component based on your business needs. The main tasks involved in the forward switchover are as follows:

1. You need to confirm that the data migration is completed and wait for the synchronization latency to be leveled.

2. OMS automatically adds the check constraints and foreign key constraints that were ignored during schema migration to the target endpoint if the target endpoint is an Oracle database, an Oracle-compatible tenant of OceanBase Database, or a DB2 LUW database.

3. OMS automatically deletes the additional hidden columns and unique indexes that are used for migration.

   This operation is performed only for data migration tasks between an Oracle database and OceanBase Database or between two OceanBase Databases. For more information, see Data migration service hidden column mechanism.

4. You need to manually migrate database objects such as triggers, functions, and stored procedures from the source endpoint to the target endpoint, as OMS does not support these objects.

5. If the data migration task involves reverse incremental migration, you need to disable the triggers at the source endpoint and delete the foreign key constraints.

The following figure shows the steps of a forward switchover.

1. Start Forward Switchover

   This step confirms that you want to start the forward switchover. No related operations are performed in the background. Before you start the forward switchover, make sure that the data migration is completed and a business cutover is required. Then, click Start Forward Switchover to start the OMS forward switchover process.

   Notice

   Before you start the forward switchover, make sure that the source endpoint has stopped writing data.

2. Precheck for forward switchover

   This step performs a precheck for the forward switchover. The following content is checked:

   • The synchronization latency between the source and target endpoints. The synchronization latency is considered acceptable if it is less than or equal to 15 seconds.

   • The write permission of the source endpoint account. If the data migration task involves reverse incremental migration, the source endpoint account must have the write permission. This ensures that data can be normally written to the target endpoint during reverse incremental migration.

   • The read permission of the target endpoint account. If the data migration task involves reverse incremental migration, the target endpoint account must have the read permission. This ensures that data can be normally read from the target endpoint to the source endpoint during reverse incremental migration.

   • The incremental log configuration of the target endpoint. If the data migration task involves reverse incremental migration, make sure that the incremental log configuration of the target endpoint meets the requirements for extracting reverse incremental logs.

   If the precheck is passed, OMS automatically proceeds to the next step. If the precheck fails, OMS displays the Retry and Skip options.

   Notice

   If you choose Skip, missing data in the target endpoint or failure of reverse incremental migration may occur. Proceed with caution.

3. Start the Store component of the target

   Note

   • This step is displayed only when the data migration task involves reverse incremental migration.

   • If you have configured Obtain Incremental Data through Kafka for the Oracle data source, you do not need to start the Store component of the target for reverse incremental migration in a data migration task from the Oracle compatible mode of OceanBase Database to an Oracle database.

   If the precheck for the forward switchover is passed, OMS automatically starts the incremental log pulling service for the target endpoint to obtain DML and DDL operations, parse and save the related log data, and prepare for reverse incremental migration. This step is expected to take 3 to 5 minutes.

4. Confirm the stop of writes at the source endpoint

   This step checks whether the source endpoint has stopped writing data. If the source endpoint has stopped writing data, click OK.

5. Confirm the leveling of the synchronization checkpoint

   This step checks whether the synchronization checkpoint at the target endpoint has been leveled to the point at which data writing has stopped at the source endpoint. If the step is in the running or failed state and the synchronization checkpoint has not been leveled for a long time, you can click Skip.

   Notice

   If you choose Skip, the data at the source and target endpoints may be inconsistent. Proceed with caution.

6. Stop forward synchronization

   This step stops the forward synchronization service. After the service is stopped, any database changes at the source endpoint will no longer be synchronized to the target endpoint for this task. If the service stop fails, OMS displays the Retry or Skip option.

   Notice

   The Skip option can be selected only when the background task of stopping the forward synchronization service is completed. Otherwise, unexpected writing of source data to the target endpoint may occur. Proceed with caution.

7. Execute Database Object Processing

   This step processes objects that were ignored during data migration or are not supported by OMS.

   • Migrate database objects to the target endpoint: You need to manually migrate database objects such as triggers, functions, and stored procedures from the source endpoint to the target endpoint. If you have completed the migration, click Mark as Completed.

   • Disable the triggers and foreign key constraints at the source endpoint: You need to perform this operation only if the data migration task involves reverse incremental migration. This ensures that data will not be affected by triggers or foreign key constraints during reverse incremental synchronization, which could cause the synchronization to fail. You need to complete this operation manually. If you have completed the operation, click Mark as Completed.

   • Migrate objects that were ignored during schema migration to the target endpoint: You need to perform this operation only if the target endpoint is an Oracle database, an Oracle-compatible tenant of OceanBase Database, or a DB2 LUW database. This automatically adds the check constraints and foreign key constraints that were ignored during schema migration to the target endpoint. For other target endpoints, these objects are migrated by default during schema migration, without the need for additional operations.

   • Delete the hidden columns and unique indexes added by OMS: This operation is performed only for data migration tasks between an Oracle database and OceanBase Database or between two OceanBase Databases. It automatically deletes the hidden columns and unique indexes added by OMS to ensure data migration consistency at the target endpoint. This operation is automatically performed, and the time it takes depends on the amount of data at the target endpoint. During this step, OMS provides the Skip option. However, if you choose to skip this step, you will need to manually delete the hidden columns and unique indexes. Proceed with caution. For more information, see Data migration service hidden column mechanism.

8. Start Reverse Increment

   Note

   This step is displayed only when the data migration task involves reverse incremental migration.

   This step starts the incremental synchronization service at the target endpoint to synchronize incremental DML or DDL changes at the target endpoint to the source endpoint in real time. The incremental synchronization is configured in the same way as that in the Create Incremental Synchronization task. For more information about incremental DDL, see Incremental DDL operations supported for migration.

Reverse incremental migration

For a data migration task in the Running state, you can view its latency, current timestamp, and performance of reverse incremental migration in the Reverse Incremental Migration section. The latency is displayed in the following format: X seconds (updated Y seconds ago). Normally, Y is less than 20.

For a data migration task in the Stopped or Failed state, you can enable the DDL/DML statistics collection feature to collect statistics on database operations performed after this feature is enabled. You can also view the specific information about the objects and performance of reverse incremental migration.

• The Synchronization Object Statistics tab displays the statistics about table-level DML statements executed in the current data migration task. The numbers displayed in the Change Sum, Delete, Insert, and Update fields in the section above the Synchronization Object Statistics tab are the sums of the corresponding columns on this tab.

• The Reverse Incremental Performance tab displays the following content:

  • Latency: the latency in synchronizing incremental data from the target database to the source database, in seconds.

  • Migration traffic: the traffic throughput of incremental data synchronization from the target database to the source database, in Kbit/s.

  • Average execution time: the average execution time of an SQL statement, in ms.

  • Average commit time: the average commit time of a transaction, in ms.

  • RPS: the number of rows written to the target database per second.