In OceanBase Database V4.X, partitions are separate from log streams. All partition transaction logs of a tenant unit can be recorded in one log stream. This reduces the number of log streams. However, a single log stream may contain substantial table data, requiring OceanBase Change Data Capture (obcdc) to pull all log stream data.
By default, tenant-level data synchronization is the finest granularity in OceanBase Database V4.X. However, for some users, this granularity is too coarse and may cause high downstream consumption load. To resolve this issue, obcdc has introduced table-level allowlist and blocklist in OceanBase Database V4.X to reduce unnecessary data storage. In addition, the sizes of the allowlist and blocklist parameters of obcdc in OceanBase Database V4.X are unlimited. The parameters are no longer subject to the current size limitation of 64 KB.
In OceanBase Database V4.X, the allowlist and blocklist feature of obcdc also allows you to rename tables. You can rename a table by performing a DDL operation to dynamically switch the table from the allowlist to the blocklist, without the need to modify the allowlist and blocklist parameters. After that, the table is no longer synchronized.
Parameters
obcdc is compatible with the allowlist and blocklist feature of OceanBase Database V3.X and supports data synchronization at the tenant, database, and table levels. obcdc has the following three parameters.
| Parameter | Description | Default value | Real-time modification supported | Remarks |
|---|---|---|---|---|
tb_white_list |
The allowlist of tables to synchronize. | *.*.* |
No | The allowlist is in the triplet format of tenant name.database name. table name. By default, *.*.* specifies to synchronize all tables in the cluster. Multiple triplets must be separated with vertical bars (\|), and individual triplets support regular expressions. |
tb_black_list |
The blocklist of tables not to synchronize. | No | The blocklist is in the triplet format of tenant name.database name. table name. By default, *.*.* specifies to synchronize none of the tables in the cluster. Multiple triplets must be separated with vertical bars (\|), and individual triplets support regular expressions. |
|
enable_white_black_list |
Specifies whether to enable the allowlist and blocklist. | 1 |
No | N/A |
Examples
The following examples demonstrate how to configure the allowlist:
Synchronize the tb1 table in the db1 database of the mytenant1 tenant and all the tables in the db2 database of the mytenant2 tenant.
mytenant1.db1.tb1|mytenant2.db2.*
Synchronize the tb1 and tb2 tables in the db1 database of the mytenant1 tenant.
mytenant1.db1.tb1|mytenant1.db1.tb2
Synchronize the tables prefixed with tb in the db1 database of the mytenant1 tenant.
mytenant1.db1.tb*
Note
- Only tables in the allowlist and not in the blocklist can be synchronized. By default, the allowlist includes all tables of all tenants, and the blocklist is empty.
- Regular expressions based on fnmatch are supported. By default, matching rules are case-insensitive. If you specify
enable_oracle_mode_match_case_sensitive=1, matching rules for tenants in Oracle mode are case-sensitive.
obcdc supports the following regular expressions: *, ?, [seq], and [!seq].
Use the allowlist and blocklist
The earlier versions of obcdc have limited the size of an individual parameter to 64 KB. At present, the allowlist and blocklist parameters are not subject to this limitation. You can configure the allowlist and blocklist in the libobcdc.conf file.
######### [working_mode] #########
#working_mode=memory
meta_data_refresh_mode=data_dict
#meta_data_refresh_mode=online
######### [synchronous information] #########
tb_white_list=*.*.*
#tb_white_list=sys.oblog*.*|oblog_tt.*.*
tb_black_list=t1.d1.t1
instance_index=0
instance_num=1
Run the following command to apply the allowlist and blocklist:
./obcdc_tailf -V -f libobcdc.conf -T $TIMESTAMP
obcdc also allows you to specify whether to enable the allowlist and blocklist by configuring the enable_white_black_list parameter. The default value is 1, which means to enable the allowlist and blocklist.
#libobcdc.conf
enable_white_black_list=0