As a high-performance distributed relational database, OceanBase Database provides a comprehensive ecosystem that integrates many tools to simplify database management, improve operational efficiency, and seamlessly connect to a variety of IT environments. This topic describes data integration, orchestration, and visualization tools used in analytical processing (AP) scenarios.
Note
For more information about all ecosystem tools integrated in OceanBase Database, see Integrations.
Data integration
Data integration is an IT process that integrates data from different sources into one view to support analysis, reporting, and business decision-making. Data can be scattered across multiple systems, such as relational databases, files, applications, NoSQL databases, and cloud storage services.
Change Data Capture (CDC) is an important technology in the data integration field to help you identify data that has changed since the last extraction. You can use the data obtained by using CDC to create history databases or perform near-real-time caching. You can also provide the data to message queues (MQs) and use the data for analysis and auditing.
The binlog service of OceanBase Database is compatible with MySQL binlogs. You can reuse the existing incremental parsing system for MySQL binlogs to synchronize incremental data of MySQL tenants of OceanBase Database without changing the existing data links. For more information about the binlog service, see Binlog service.
OceanBase Database is compatible with mainstream data integration tools, including but not limited to those described in the following table.
| Tool | Description | References |
|---|---|---|
| Flink | An open-source framework for large-scale data processing and analysis. | For more information, see Use Flink CDC to synchronize data from a MySQL database to OceanBase Database. |
| Canal | A CDC framework for database synchronization and integration. | For more information, see Use Canal to synchronize data from a MySQL database to OceanBase Database. |
| Maxwell | A CDC tool for MySQL databases. | For more information, see Read data from OceanBase Database by using Maxwell. |
| SeaTunnel | A distributed, high-performance data integration platform for data migration and real-time stream processing. | For more information, see Read and synchronize data from OceanBase Database by using SeaTunnel.
NoticeSeaTunnel is in the Beta stage. It has been preliminarily tested but requires further comprehensive adaptation. |
| Debezium | An open-source, enterprise-level general-purpose computing framework that supports task scheduling and management across multiple computing engines. | For more information, see Read and synchronize data from OceanBase Database by using Debezium. |
| DataX | An offline data synchronization tool and platform. | For more information, see Use DataX to migrate CSV files to OceanBase Database. |
Orchestration
Orchestration tools in the data integration field are software tools that manage, schedule, and coordinate different data processing tasks and data processes. These tools provide advanced ways to organize and perform multiple data integration activities, such as data extract-transform-load (ETL) processes as well as data cleansing, verification, and publishing tasks.
| Tool | Description | References |
|---|---|---|
| DolphinScheduler | An open-source, distributed workflow task scheduling system that supports multiple types of tasks. | For more information, see Configure OceanBase Database as the data source for DolphinScheduler.
NoticeThis tool is under beta testing. Preliminary testing has been completed, pending comprehensive adaptation. |
| Linkis | An open-source general-purpose computing framework for enterprises, which supports task scheduling and management of multiple computing engines. | For more information, see Integrate OceanBase Database with Linkis. |
Observability
Observability is the ability to monitor, track, analyze, and interpret data processing flows and system status. High observability is crucial for complex data channels and systems. It allows the O&M team to identify and respond to any problems as soon as possible, to ensure system health and performance.
In addition, data quality is a key component of the data integration process and also requires high observability. Data quality observability tools allow the O&M team to verify data accuracy, consistency, and integrity, and check whether the data is as expected. This ensures that business decisions are made based on trusted data.
| Tool | Description | References |
|---|---|---|
| Prometheus | A powerful open-source system monitoring and alerting toolkit designed for reliability and multidimensional data collection. It is widely used to monitor the health of services and applications. | For more information, see Integrate OCP monitoring into Prometheus. |
Visualization
Visualization tools in the data integration field are software tools that display, interpret, and transmit data by using graphs and tables. These tools obtain data from data integration processes and transform the data into views. This way, you can gain insight into the trends and patterns behind the data even without in-depth technical knowledge.
| Tool | Description | References |
|---|---|---|
| Superset | An open-source business intelligence tool for data exploration and visualization. | For more information, see Analyze data by using Superset and OceanBase Cloud Database. |
| PowerBI | A tool that allows you to easily connect to data sources, visualize and discover important content, and share the content with anyone as needed. | For more information, see Connect to OceanBase Database in Power BI. |
| QuickBI | An easy-to-use data visualization and analysis tool that supports quick generation of data reports. | For more information, see Connect to an OceanBase database in Quick BI. |
| Tableau | A data visualization tool for generating interactive and shareable reports. | For more information, see Connect to an OceanBase database in Tableau. |
| Guanyuan BI | A tool that allows you to easily connect to OceanBase Database for data display. | - |
| Yonghong BI | A tool that allows you to connect to OceanBase Database for data display in vividime V11. | - |