Overview

OceanBase Change Data Capture (CDC) helps you identify data that has changed since the last extraction. You can use the data to make history databases or perform near real-time caching. You can also provide the CDC data to message queues (MQs), so you can use MQs for analysis and auditing.

Introduction to CDC

OceanBase CDC has two components:

• obcdc
• oblogmsg

obcdc is a fundamental component of OceanBase CDC. oblogproxy is derived from obcdc and obcdc depends on oblogmsg.

obcdc

obcdc provides incremental data of OceanBase Database in real time as a dynamic-link library.

obcdc requests clogs, also known as redo logs, of each partition from OceanBase Database by using remote procedure calls (RPCs). It converts clogs to LogMsg data based on the table and column schemas that are obtained by querying OBServer nodes. After obcdc aggregates data changes of each partition involved in a transaction, it provides the changed data of the transaction to an external process row by row.

obcdc application

obcdc provides the incremental transaction data of OceanBase Database to external processes. Processes can call obcdc for data synchronization and subscription.

• Data synchronization: Data from OceanBase Database is synchronized to other homogeneous or heterogeneous databases for purposes such as logical migration, logical backup, and logical upgrade.

• Data subscription: Data from OceanBase Database is synchronized to message queues or data warehouses for purposes such as real-time computing, auditing, and monitoring.

Deployment and use

For more information about how to deploy and use obcdc, see Install and deploy obcdc and obcdc development instructions.

oblogproxy

As the proxy of obcdc, oblogproxy calls obcdc to obtain the incremental transaction data of OceanBase Database as a consumer.

oblogproxy provides links for applications to access and manage real-time incremental logs of OceanBase Database.

For more information about how to deploy and use oblogproxy, see Install and deploy oblogproxy by using the installation package and Build oblogproxy by using the source code.

oblogmsg

oblogmsg is a message format used by obcdc, the incremental data collection module of OceanBase Database, to export incremental data. oblogmsg supports writing incremental data of different types to OceanBase Database. It also supports data serialization and deserialization.

For more information about how to install and deploy oblogmsg, see Install and deploy oblogmsg.