Data subscription

Data subscription uses Change Data Capture (CDC) to continuously read database logs from upstream sources (such as MySQL Binlog or OceanBase Binlog services). After parsing, sorting, or transforming the incremental changes, it replays them at the target side. Common targets include OceanBase, Kafka, and then writes the data to analytics databases or wide tables via Flink/custom consumers.

Typical scenarios: Real-time synchronization to BI/data warehouses, cache and search engine refresh, incremental data catch-up during migration and cutover, traffic throttling via Kafka for multi-team consumption, and data export from OceanBase Binlog.

Subscribe to MySQL Binlog using Canal

Canal simulates a MySQL Slave node to subscribe to Binlog and delivers row-level changes to the client. It can be configured to directly synchronize data to OceanBase, or first write data to a message system via modes such as TCP / Kafka / RocketMQ / RabbitMQ, and then be consumed by downstream components and written to the target database (including OceanBase).

Subscribe to multiple-source databases using Flink CDC

Flink CDC provides a Database CDC Source on top of Flink, which can pull stock + incremental data from various databases. Combined with Flink SQL, it performs joins, widening, and aggregation, then writes the data to sinks such as Kafka, JDBC (including OceanBase), and Hive. It is suitable for pipelines requiring multi-source integration, real-time processing, and high Exactly-Once compliance.

Data subscription and writing to Kafka / OceanBase scenarios

A common pattern is: Log parsing by CDC or Canal → Writing to Kafka (buffering, multiple subscriptions) → Flink/OMS/custom consumer → Persistence in OceanBase or other storage. Alternatively, Flink CDC can directly sink data to OceanBase JDBC, omitting Kafka, depending on throughput requirements and coupling considerations.