Direct load|V4.2.6|OceanBase Migration Service| docs|Distributed Database

Direct load

Last Updated：2025-04-16 02:21:13 Updated

OceanBase Migration Service (OMS) Community Edition supports data writes by using direct load when you migrate data. This topic describes the limitations and parameters of direct load, and shows you how to query the status of a direct load task.

Background information

OceanBase Database allows you to write data into a database by using direct load. The direct load feature can get around conventional SQL-layer APIs and directly write data into files in a database, thereby improving the data import efficiency. For more information, see Overview.

Notice

Direct load does not support resumable transmission. After a migration task is interrupted and resumed, OMS Community Edition re-reads the source, but does not read the tables that are already migrated before the interruption.

OMS Community Edition allows you to specify the direct load method when you create the following types of data migration tasks.

direct-load-en

Limitations

This data write method is supported for full data migration only when you migrate data to an instance of OceanBase Database Community Edition V4.2.1 or later by using OMS Community Edition V4.2.2 or later.
The size of data in a single row cannot exceed 2 MB.
Only data duplication caused by primary key constraints can be handled, and that caused by unique key constraints cannot be handled.
Generated columns, triggers, and user-defined types (UDTs) are not supported.
Direct load applies to empty tables and cannot be interrupted. In other words, resumable transmission is not supported. Therefore, if direct load is used for full data migration and is interrupted, the data in the source is read again even after the task is resumed.
If an imported table contains large object (LOB) fields, execute the following statement to disable the enable_rebalance parameter for OceanBase Database Community Edition:
```
SHOW PARAMETERS LIKE 'enable_rebalance';
ALTER SYSTEM SET enable_rebalance = false;
```
Direct load does not support concurrency for destination databases. Make sure that you use the direct load method to write data to only one table in the destination database at a time.
The number and order of fields in the source table must be the same as those in the destination table.
Direct load does not support tables that contain prefixed indexes.

Parameters

The following table describes the direct load parameters, which are in the Sink module of parameters of the Incr-Sync component.

Parameter	Description
dbVersion	The version of the database. Only OceanBase Database Community Edition V4.2.1 and later support direct load. If you do not specify this parameter, OMS Community Edition automatically queries the database version by using the following statement: `SHOW VARIABLES LIKE 'version_comment'`.
host	The IP address of the remote procedure call (RPC) server. By default, OMS Community Edition automatically queries the address by using the following statement: `SHOW PARAMETERS LIKE 'rpc_port'`, and uses the first `svr_ip` value returned.
port	The RPC port of the server. By default, OMS Community Edition automatically queries the port number by using the following statement: `SHOW PARAMETERS LIKE 'rpc_port'`, and uses the first `svr_port` value returned.
username	Reuse the username specified for the destination database in the format of `Username@Tenant name#Cluster name`. The username and tenant name are parsed based on this parameter and are used for `TableAPI`.
password	Reuse the password configured for the destination database.
serverParallel	The number of concurrent worker threads of the destination database user to process the data writes of direct load. The default value is 8.
maxErrors	The maximum number of errors allowed. The default value is 0.
replaceData	Specifies whether to replace duplicate primary keys. The default value is `true`. If you set this parameter to `false`, duplicate primary keys are ignored.
serverTimeout	The timeout period of the direct load task on OBServer nodes. Unit: μs. The default value is `1209600L * 1000 * 1000`, which equals 14 days.
rpcExecuteTimeout	The timeout period for executing an RPC request. Unit: ms. The default value is `600000`, which equals 10 minutes.

Query the status of a direct load task

Execute the following statement in the sys tenant to query the system table that records direct load statistics:
```
SELECT * FROM oceanbase.__all_virtual_load_data_stat;
```
Execute the following statement to query the OBServer log and troubleshoot issues of the direct load task:
```
grep table_load observer.log*
```

Enterprise Edition