OceanBase Change Data Capture (CDC) helps you identify data that has changed since the last extraction. You can use the data to make history databases or perform near real-time caching. You can also provide the CDC data to message queues (MQs), so you can use MQs for analysis and auditing.
Introduction to CDC
OceanBase CDC has two components:
- obcdc
- oblogmsg
obcdc is a fundamental component of OceanBase CDC. oblogproxy is derived from obcdc and obcdc depends on oblogmsg.
obcdc
obcdc provides incremental data of OceanBase Database in real time as a dynamic-link library.
obcdc requests clogs, also known as redo logs, of each partition from OceanBase Database by using remote procedure calls (RPCs). It converts clogs to LogMsg data based on the table and column schemas that are obtained by querying OBServer nodes. After obcdc aggregates data changes of each partition involved in a transaction, it provides the changed data of the transaction to an external process row by row.
obcdc application
obcdc provides the incremental transaction data of OceanBase Database to external processes. Processes can call obcdc for data synchronization and subscription.
Data synchronization: Data from OceanBase Database is synchronized to other homogeneous or heterogeneous databases for purposes such as logical migration, logical backup, and logical upgrade.
Data subscription: Data from OceanBase Database is synchronized to message queues or data warehouses for purposes such as real-time computing, auditing, and monitoring.
Deployment and use
For more information about how to deploy and use obcdc, see Install and deploy obcdc and obcdc development instructions.
oblogproxy
As the proxy of obcdc, oblogproxy calls obcdc to obtain the incremental transaction data of OceanBase Database as a consumer.
oblogproxy provides links for applications to access and manage real-time incremental logs of OceanBase Database.
Note
You can synchronize data from an OceanBase database to a MySQL database in real time by using Canal and oblogproxy. For more information, see Use Canal to synchronize data from OceanBase Database to a MySQL database.
oblogmsg
oblogmsg is a message format used by obcdc, the incremental data collection module of OceanBase Database, to export incremental data. oblogmsg supports writing incremental data of different types to OceanBase Database. It also supports data serialization and deserialization.
For more information about how to install and deploy oblogmsg, see Install and deploy oblogmsg.