This topic describes how to import external data into OceanBase Database.
OceanBase Database provides flexible data import methods to import data from various data sources. Different import methods are suitable for different scenarios. You can choose an appropriate data import tool based on the data source type and business scenario. As scenarios become more complex, multiple import methods can be used in combination. When importing data, you need to consider the data source, data file format, and the support of the data import tool. If the data source and data file format are clear, you need to design the data import solution based on the data source and the data import tool. If the business has a familiar data import tool, you need to consider the tool's support and design the data import solution based on the business scenario. This topic focuses on the data source type and tool from two perspectives to describe the data import solution of OceanBase Database.
Considerations for data source type
This topic describes the import methods supported by common data sources. This topic helps you select an appropriate import method based on your business scenarios.
| Data source type | Import method | More information |
|---|---|---|
|
|
|
Database:
|
You can use the following tools:
|
|
| Spark | OceanBase Spark Connector | For more information about OceanBase Spark Connector, see OceanBase Spark Connector. |
Considerations for importing data from derivative tools
This section describes common derivative tools and supported import options to help you choose the appropriate import option based on your specific scenario.
| Tool | Scenario | Supported data sources | More information |
|---|---|---|---|
| LOAD DATA syntax | For large-scale data import. | CSV/ORC/Apache Parquet/compressed files stored in the following locations:
|
For more information about the LOAD DATA syntax, see LOAD DATA (Oracle mode) and LOAD DATA (MySQL mode). |
| obloader | For large-scale data import. | CSV/Apache Parquet/Apache ORC/SQL files stored in the following locations:
|
For more information about obloader, see obloader documentation. |
| External table | For log analysis, historical data analysis, multi-source data integration, data lake analysis, and lakehouse acceleration. | CSV/ORC/Apache Parquet/compressed files stored in the following locations:
|
For more information about external tables, see External tables (Oracle mode) and External tables (MySQL mode). |
| INSERT SQL statement |
|
Internal tables or external tables | For more information about INSERT SQL statements, see INSERT (Oracle mode) and INSERT (MySQL mode). |
| Flink | For real-time data import from Flink. | For more information, see the Flink documentation. | For more information, see Synchronize data from a MySQL database to an OceanBase database by using Flink CDC. |
| Canal | For real-time data import from Canal. | For more information, see the Canal documentation. | For more information, see Synchronize data from a MySQL database to an OceanBase database by using Canal. |
| OMS | For large-scale database-to-database migration and synchronization. |
|
|
| DataX | For data synchronization between various data sources, including relational databases such as MySQL and Oracle, HDFS, and Hive. DataX does not limit the amount of data and allows you to migrate tables one by one. The migration speed can reach 60 MB/s. | For more information, see the DataX documentation. | For more information, see Migrate data from a MySQL database to an OceanBase database by using DataX |
| CloudCanal | Migrates or synchronizes data from databases such as MySQL, Oracle, and PostgreSQL to OceanBase Database. | Visit the CloudCanal website. | For more information, see Migrate data from a MySQL database to OceanBase Database by using CloudCanal. |