Background information
OceanBase Database allows you to insert data into a database by using direct load. In other words, OceanBase Database can directly write data into data files. With the direct load feature, you can bypass interfaces in the SQL layer and directly allocate space and insert data into data files, thereby improving the data import efficiency.
Scenarios
The direct load feature applies to the following scenarios:
Data migration and synchronization: In data migration or synchronization, a large amount of data of different types must be migrated from different data sources to OceanBase Database. Conventional SQL interfaces cannot meet the requirement on timeliness.
Conventional extract, transform, and load (ETL): After data is extracted and transformed in the source, a large amount of data must be loaded to the destination within a short time. The direct load technology can improve the import performance.
Data loading from text files or other data sources to OceanBase Database: direct load can accelerate the data loading process.
Considerations
In direct load, the RPC port instead of the SQL port is required to transmit data.
Data to be imported is submitted at the table level instead of the session or transaction level.
Retry or resumable transmission is not supported.
Bit data types are not supported.
Virtual generated columns are not supported.
Command-line options for direct load
Notice
OBLOADER can connect to an OBServer node or OceanBase Database Proxy (ODP) for direct load. The version requirements are as follows:
- OBServer node: V4.2.0 or later.
- ODP: V4.1.3 while the OBServer node version must be V4.2.1 or later.
| Option | Description | ApsaraDB for OceanBase & ODP | OceanBase Database & ODP | OceanBase Database & OBServer node |
|---|---|---|---|---|
| --direct | Specifies to use direct load. | Required | Required | Required |
| --parallel | The degree of parallelism (DOP) on the server. | Optional | Optional | Optional |
| --rpc-port | The inner RPC port of the server. You can obtain the RPC port as follows:
|
Required | Required | Required |
| -u(--user) | The username that you use to log on to the database. | Required | Required | Required |
| -P(--port) | The SQL port number. | Required | Required | Required |
| -t(--tenant) | The tenant name of the cluster. | Optional If this option is not specified, partition calculation may be skipped. |
Required | Required |
| -c(--cluster) | The cluster name of the database. | Optional | Required | - |
| --public-cloud | Indicates that the database environment is ApsaraDB for OceanBase. | Required | - | - |
| --no-sys | Specifies that the import does not rely on the sys tenant. This option applies only to OceanBase Database of a version earlier than V4.0.0. | Optional | Optional | Optional |
| --sys-user | The user on which the import relies in the sys tenant. If this option is not specified, the default value root takes effect. This option applies only to OceanBase Database of a version earlier than V4.0.0. |
Optional This option is mutually exclusive with the --no-sys option. |
Optional This option is mutually exclusive with the --no-sys option. |
Optional This option is mutually exclusive with the --no-sys option. |
| --sys-password | The password of the user on which the import relies on in the sys tenant. This option applies only to OceanBase Database of a version earlier than V4.0.0. | Optional This option is mutually exclusive with the --no-sys option. |
Optional This option is mutually exclusive with the --no-sys option. |
Optional This option is mutually exclusive with the --no-sys option. |