This topic introduces how to import external data into OceanBase Database.
OceanBase Database provides a variety of flexible data import methods, allowing data from multiple sources to be imported into the database. Different import methods are suitable for different scenarios. You can choose the appropriate import tool based on the type of data source and business scenario. As scenarios become more complex and diverse, multiple import methods can be used together. When importing data, in addition to considering the data source and data file format, you should also consider the support provided by the import tool. If your business scenario already has a specific data source and data file format, you need to start from the data source and design your import solution in combination with the available tools. If your business is already proficient with a certain import tool, you need to consider the tool’s capabilities and evaluate the feasibility of different import solutions based on your business scenario. Here, we mainly look at OceanBase data import solutions from the perspectives of data source types and tools.
Considerations based on data source type
This section describes the import solutions supported by common data sources to help you quickly select an appropriate solution based on your scenario.
| Data source type | Import solution | For more information |
|---|---|---|
|
|
|
Database:
|
You can use one of the following tools to import data. For more information about the compatibility of the tools, see the official website of the tool.
|
|
| Spark | Connect to OceanBase Database by using the JDBC interface. | For more information about the JDBC interface, see Connect to OceanBase Database by using the JDBC interface. |
Consider data import solutions from the perspective of derivative tools
This section describes common derivative tools and import solutions supported by the tools. It helps you quickly select an appropriate import solution based on your business scenarios.
| Tool | Applicable scenario | Supported data sources | For more information |
|---|---|---|---|
| LOAD DATA syntax | Suitable for large-scale data import. | CSV, ORC, Apache Parquet, and compressed files stored in the following locations:
|
For more information about the LOAD DATA syntax, see LOAD DATA (Oracle-compatible mode) and LOAD DATA (MySQL-compatible mode). |
| obloader | Suitable for large-scale data import. | CSV, Apache Parquet, Apache ORC, and SQL files stored in the following locations:
|
For more information about obloader, see obloader documentation. |
| External table | Suitable for log analysis, historical data analysis, multi-source data integration, data lake analysis, and data warehouse acceleration. | CSV, ORC, and Apache Parquet files and compressed files stored in the following locations:
|
For more information about external tables, see Create an external table (Oracle-compatible mode) and Create an external table (MySQL-compatible mode). |
| INSERT SQL statements |
|
Internal tables and external tables | For more information about INSERT SQL statements, see INSERT (Oracle-compatible mode) and INSERT (MySQL-compatible mode). |
| Flink | Suitable for importing data from Flink in real time. | Visit the Flink website for more information. | For more information, see Use Flink CDC to synchronize data from a MySQL database to OceanBase Database. |
| Canal | Suitable for importing data from Canal in real time. | Visit the Canal website for more information. | For more information, see Use Canal to synchronize data from a MySQL database to OceanBase Database. |
| OMS | Suitable for large-scale database-to-database migration and synchronization. |
|
For more information about OMS, see OMS documentation. |
| DataX | Suitable for synchronizing data across various sources, including HDFS, Hive, and relational databases like MySQL and Oracle. DataX supports table-by-table migration without any limits on data volume and can achieve speeds of up to 60 MB/s. | Visit the DataX website for more information. | For more information, see Use DataX to migrate table data from a MySQL database to OceanBase Database. |
| CloudCanal | Suitable for migrating or synchronizing data from MySQL, Oracle, and PostgreSQL databases to OceanBase Database. | Visit the CloudCanal website for more information. | For more information, see Use CloudCanal to migrate data from a MySQL database to OceanBase Database. |