OceanBase Database allows you to insert data into a database by using direct load. In other words, OceanBase Database can directly write data into data files. With the direct load feature, you can bypass interfaces in the SQL layer and directly allocate space and insert data into data files, thereby improving the data import efficiency.
Direct load features
Full direct load
Full direct load is to directly write a complete data set into the data file of a database at one time. It can bypass SQL layer interfaces and directly allocate space and insert data in the data file, improving the data import efficiency. Full direct load is generally used for database initialization, data migration, or quick loading of a large amount of data.
Incremental direct load
Incremental direct load is to directly write new data into the data file of a database without using the SQL interface, especially when a large amount of data already exists. It can bypass data processing at the SQL layer and directly write new data into the data file, improving the data write efficiency. Incremental direct load is generally used in data write scenarios with a high throughput, such as large-scale real-time data collection and log write.
Scenarios
The direct load feature applies to the following scenarios:
Data migration and synchronization: In data migration or synchronization, a large amount of data of different types must be migrated from different data sources to OceanBase Database. Conventional SQL interfaces cannot meet the requirement on timeliness.
Conventional extraction, transformation, and loading (ETL): After data is extracted and transformed in the source, a large amount of data must be loaded to the destination within a short time. The direct load technology can improve the import performance. As for the extract, load, and transform (ELT) technology, the data loading efficiency can also be improved by using direct load.
Data loading from text files or other data sources to OceanBase Database: Direct load can accelerate the data loading process.
Supported statements for direct load
At present, OceanBase Database supports the following statements for direct load: