OceanBase Database allows you to insert data into a database in direct load mode. In other words, OceanBase Database can directly write data to data files. With the direct load feature, you can get around APIs at the SQL layer and directly allocate space and insert data into data files, thereby improving the data import efficiency.
Scenarios
The direct load feature applies to the following scenarios:
Data migration and synchronization: In data migration or synchronization, a large amount of data in different formats must be migrated from different data sources to OceanBase Database. Conventional SQL APIs cannot meet the requirement on timeliness.
Conventional extract, transform, and load (ETL): After data is extracted and transformed in the source, a large amount of data must be loaded to the target within a short time. The direct load technology can improve the import performance. As for the ELT technology, the data load efficiency can also be improved by using direct load.
Data load from text files or other data sources to OceanBase Database: Direct load can accelerate the data load process.
Considerations
Direct load will write all existing data. If the source table contains a large amount of data but only a small amount of data needs to be imported, direct load is inappropriate in this case.
Temporary files are generated during direct load. When a large amount of data is imported, the generated temporary files will occupy much disk space. In this case, you must pay attention to the disk usage and prevent the disk space from being used up by these temporary files. When the degree of parallelism (DOP) for direct load is greater than or equal to 8, the system will compress the generated temporary files to reduce the occupied disk space.
Supported statements for direct load
At present, OceanBase Database supports the following statements for direct load: