Migrate incremental data from OceanBase Database in HBase API compatibility mode to a Lindorm/HBase database

You can create a data migration task to seamlessly migrate incremental data from the source database to the target database by migrating data from OceanBase Database in HBase API compatibility mode to a Lindorm/HBase database.

Prerequisites

• You have created a cluster and a tenant in the source OceanBase database. For more information, see Create an instance and Create a tenant.

• You have created a target database instance.

• You have created dedicated database users for data migration on both the source and target sides and granted them the required permissions. For more information, see User privileges.

Limitations

• Only users with the project role of Project Owner, Project Admin, or Data Services Admin can create a data migration task.

• Currently, data transmission supports Lindorm/HBase wide-table engine version V2.x and OceanBase Database HBase API compatibility mode V4.2.5 or later.

• When migrating data from OceanBase Database in HBase API compatibility mode to a Lindorm/HBase database, only incremental synchronization is supported; DDL synchronization is not supported.

• When migrating data from OceanBase Database in HBase API compatibility mode to a Lindorm/HBase database, only table objects of the HBase model are supported. Selecting other types of table objects may result in data quality issues.

Considerations

• If the source character set is UTF-8, it is recommended that the target use a character set compatible with the source (for example, UTF-8, UTF-16, etc.) to avoid issues such as garbled characters at the target due to character set incompatibility.

• If you modify a unique index at the target, you need to restart the data migration task; otherwise, data inconsistency may occur.

• Clock desynchronization between nodes, or between the client terminal and the server, can cause inaccurate latency reporting during incremental synchronization.

  For example, if a clock is ahead of standard time, the reported latency may be negative. If a clock is behind standard time, it may cause a positive latency.

• Verify whether the data migration service's precision for column types such as DECIMAL, FLOAT, and DATETIME meets your expectations. If the precision of the target field type is lower than that of the source field type, truncation may occur, leading to data inconsistency between the source and target.

• When creating a data migration task, if you configure only Incremental Synchronization, the data migration requires that local incremental logs in the source database be retained for more than 48 hours.

Supported source and target instance types

Procedure

1. Create a data migration task.

   1. Log in to the OceanBase Cloud console.

   2. In the left-side navigation pane, select Data Services > Migrations.

   3. On the Migrations page, click the Migrate Data tab.

   4. In the upper-right corner of the Migrate Data tab, click Create Task.

2. In the Edit Task Name text box, enter a custom name for the migration task.

   We recommend that you use a combination of Chinese characters, numbers, and letters. The name must not contain spaces and must not exceed 64 characters in length.

3. On the Configure Source & Target page, configure the parameters.

   1. In the Source section, configure the parameters.

      If you need to reference an existing data source, click Quick Fill next to Source and select the target data source from the drop-down list. After selection, the configurations for the source section will be automatically populated. If you want to save the current configuration as a new data source, click the Save icon in the upper-right corner of the Source section.

      You can also click Manage Data Source in the Quick Fill drop-down list to go to the Data Sources page to view and manage data sources. This page provides centralized management for different types of data sources. For more information, see Data sources.

      Parameter	Description
      Cloud Vendor	Currently only supports Alibaba Cloud.
      Region	Select the region where the source database is located.
      Database Type	Select the source database type as OceanBase HBase API Compatible.
      Instance Type	Select the source instance type as Dedicated (Key-Value) or Cluster Instance (Ultimate).
      Instance	The ID or name of the instance where the OceanBase Database HBase API compatible mode tenant resides. You can view the instance ID or name on the Instances page. Note When the cloud vendor is Alibaba Cloud, you can also select instances of Alibaba Cloud root accounts with cross-account authorization. For details, see Alibaba Cloud account authorization.
      Tenant	The ID or name of the OceanBase Database HBase API compatible mode tenant. You can expand an instance on the Instances page to view the tenants' IDs or names under that instance.
      Database Account	The username of the OceanBase Database HBase API compatible mode tenant used for data migration.
      Password	The password of the database user.

   2. In the Configure Source & Target step, configure the parameters in the Target section.

      If you need to reference an existing data source, click Quick Fill next to Target and select the target data source from the drop-down list. After selection, the configurations for the destination section will be automatically populated. If you want to save the current configuration as a new data source, click the Save icon in the upper-right corner of the Target section.

      You can also click Manage Data Source in the Quick Fill drop-down list to go to the Data Sources page to view and manage data sources. This page provides centralized management for different types of data sources. For more information, see Data sources.

      Parameter	Description
      Cloud Vendor	Currently supports Alibaba Cloud.
      Database Type	Select the target database type as HBase or Lindorm based on your actual situation.
      Region	Select the region where the target database is located.
      Instance Type	Select Alibaba Cloud HBase Enhanced Edition or Alibaba Cloud Lindorm.
      Connection Type	Select Public IP.
      Connection Information	Enter the address or IP of the target database.
      Port	Enter the port of the database.
      Database Account	The database account used for data migration.
      Password	The password of the database account.

4. Click Test and Continue. On the Select Type & Objects page, configure the parameters.

   1. In the Migration Type section, select the migration type for the current data migration task.

      Currently, only incremental synchronization is supported. This migration type supports configuring DML synchronization. For details, see Configure DML/DDL synchronization.

   2. In the Select Migration Objects section, configure the method for selecting migration objects.

      Currently, you can select migration objects by using Specify Objects.

   3. In the Select Migration Scope section, select the objects to migrate.

      • When you select Specify Objects, you can select one or more tables and views from a database as migration objects. Select the objects to migrate on the left and click > to add them to the list on the right.

        For data migration, you can import objects by text, rename target objects, and remove one or all migration objects.

        Operation	Description
        Import objects	In the list on the right side of the selection area, click Import Object in the upper-right corner. For more information, see Import migration objects.
        Rename	You can rename migration objects. For more information, see Rename a database or table.
        Remove/Clear All	During data mapping, you can remove one or more temporarily selected objects to the target. Remove a single migration object In the list on the right side of the selection area, click the Remove icon next to the target object to remove it. Remove all migration objects In the list on the right side of the selection area, click Clear All in the upper-right corner. In the dialog box, click OK to remove all migration objects.

5. Click Next. On the Migration Options page, configure the parameters for incremental synchronization.

   Parameter	Description
   Write Concurrency	This parameter specifies the number of concurrent writes to the target during incremental synchronization. The maximum value is 512. A high concurrency value may cause excessive load on the target and affect business operations.
   Incremental Synchronization Start Timestamp	Specifies the data to be migrated after a specific point in time. The default is the current system time. For more information, see Set an incremental synchronization timestamp.
   Incremental Migration Rate Limit	You can decide whether to enable the incremental synchronization rate limit based on your actual needs. If enabled, set the RPS (the maximum number of data rows that can be synchronized to the target per second during incremental synchronization) and BPS (the maximum amount of data that can be synchronized to the target per second during incremental synchronization). Note The RPS and BPS values set here are only for rate limiting. The actual achievable performance during incremental synchronization is subject to factors such as the source, target, and instance specifications.

6. Click Pre-check to perform a pre-check on the data migration task.

   In the Pre-check step, the system checks whether the read and write permissions of the database user and the network connection meet the requirements. You can only start the data migration task after all checks pass. If an error occurs during the pre-check:

   • You can troubleshoot and fix the issue, then rerun the pre-check until it succeeds.

   • Alternatively, you can click Skip in the Actions column of the failed pre-check item. A dialog box will appear, informing you of the specific impact of skipping this operation. After confirming that it is acceptable, click OK in the dialog box.

7. After the pre-check succeeds, click Purchase to go to the Purchase Data Migration Instance page.

   After the purchase succeeds, you can start the data migration task. For more information about how to purchase a data migration instance, see Purchase a data migration instance. If you do not need to purchase a data migration instance at this time, click Save to go to the details page of the data migration task. You can manually purchase a data migration instance later as needed.

   You can click Configure Validation Task in the upper-right corner of the details page to compare the data differences between the source database and the target database. For more information, see Create a data validation task.

   The data migration service allows you to modify the migration objects when the task is running. For more information, see View and modify migration objects. After the data migration task is started, it is executed based on the selected migration types. For more information, see the "View migration details" section in View details of a data migration task.