Task management|V4.3.6|OceanBase Binlog Service|oblogproxy docs|Distributed Database

Create a binlog task

This command enables binlog for the specified OceanBase Database tenant. The Binlog Server asynchronously creates N binlog instances. After you execute the command, resources are allocated. If allocation succeeds, an OK packet is returned immediately. If fewer than N instances are created successfully, a Warning packet is returned.

Syntax

CREATE BINLOG FOR TENANT cluster.tenant 
       [FROM timestamp] [TO USER user PASSWORD pwd] 
       WITH (CLUSTER URL cluster_url | ROOTSERVER_LIST rootserver_list) [, binlog_option [, binlog_option...]];

binlog_option:
      INITIAL_TRX_XID
    | INITIAL_TRX_GTID_SEQ
    | REPLICATE NUM
    | SERVER UUID
    | BINLOG_TENANT_FAILOVER
    | BINLOG_TENANT_REMOTE_FAILOVER
    | BINLOG_TENANT_EXPIRE_LOGS_SECONDS
    | BINLOG_TENANT_EXPIRE_LOGS_SIZE
    | BINLOG_TENANT_LOGS_AUTO_SCALE
    | BINLOG_TENANT_SCOPE_CFG

BINLOG_TENANT_LOGS_AUTO_SCALE: {
    "enable": value,
    "max_scale_up_capacity_gb": value,
    "min_scale_down_capacity_gb": value,
    "scale_down_threshold": "value",
    "scale_down_step": "value",
    "scale_up_threshold": "value", 
    "scale_up_step": "value",
    "computed_binlog_expire_logs_size": value
}

Parameters

Parameter	Description
cluster.tenant	The tenant for which the binlog task is to be created, and the cluster to which the tenant belongs.
timestamp	Optional. Start timestamp. If omitted, starts from the current timestamp.
user	Optional. Username of the OceanBase cluster system tenant (sys tenant). If the Binlog Server was started with system tenant credentials, you can omit this.
pwd	Optional. Password for the user specified by `user`. If the Binlog Server was started with system tenant credentials, you can omit this.
cluster_url	The OceanBase cluster config URL. Must be enclosed in quotation marks. After connecting to the cluster, run `SHOW PARAMETERS LIKE 'obconfig_url';` to view the value (`value` field). Note You must specify exactly one of `cluster_url` and `rootserver_list`. `cluster_url` has higher priority than `rootserver_list`. If both are specified, `cluster_url` takes effect by default.
rootserver_list	Root Service host information in the format `server_ip:server_rpc_port:server_sql_port`. For scenarios without OCP deployment. Note You must specify exactly one of `cluster_url` and `rootserver_list`. `cluster_url` has higher priority than `rootserver_list`. If both are specified, `cluster_url` takes effect by default.
binlog_option	Optional. Binlog task options. Separate multiple options with commas (`,`). Use a comma (`,`) to separate this clause from `cluster_url`. For supported options, see binlog_option below.

binlog_option

The following binlog_option settings are important:

INITIAL_TRX_XID: Specifies the unique xid of the starting transaction.
INITIAL_TRX_GTID_SEQ: Specifies the GTID sequence number of the starting transaction.
REPLICATE NUM: Number of binlog instance replicas for the tenant. When the value is greater than 1, the binlog service randomly selects one replica as the primary instance.
SERVER UUID: Manually specifies the server UUID.
BINLOG_TENANT_FAILOVER: Controls whether failover recovery for the binlog instance itself is enabled. When enabled, if the machine node is healthy but the binlog instance fails, the binlog instance is automatically recovered. Default is 1 (enabled).

Note

This parameter is available from V4.3.0.
BINLOG_TENANT_REMOTE_FAILOVER: Controls whether remote-node failover recovery for binlog instances is enabled. When enabled, if the machine node where the binlog instance resides fails, the binlog instance is automatically migrated to another healthy machine node and recovered. Default is 0 (disabled).
Note
- This parameter is available from V4.3.0.
- Remote-node failover is jointly controlled by BINLOG_TENANT_REMOTE_FAILOVER and BINLOG_TENANT_FAILOVER:
  - When BINLOG_TENANT_FAILOVER is 0, both local-node and remote-node failover are disabled.
  - When BINLOG_TENANT_FAILOVER is 1 and BINLOG_TENANT_REMOTE_FAILOVER is 0, local-node failover is enabled and remote-node failover is disabled (default).
  - When BINLOG_TENANT_FAILOVER is 1 and BINLOG_TENANT_REMOTE_FAILOVER is 1, both local-node and remote-node failover are enabled.
BINLOG_TENANT_EXPIRE_LOGS_SECONDS: Binlog file expiration time. Binlog files whose last update timestamp is less than the current timestamp minus BINLOG_TENANT_EXPIRE_LOGS_SECONDS, together with their index entries, are automatically purged. Default is 259200 seconds. Value range: [0, 4294967295]. 0 disables automatic cleanup by expiration time.

Note

This parameter is available from V4.3.0.
BINLOG_TENANT_EXPIRE_LOGS_SIZE: Binlog file expiration by size. During binlog rotation, the system checks whether the difference between the total size of all binlog files and the size of the first binlog file is greater than BINLOG_TENANT_EXPIRE_LOGS_SIZE. If so, it deletes the first binlog file and its index entries, repeating until the difference is less than or equal to BINLOG_TENANT_EXPIRE_LOGS_SIZE or only one binlog file remains, then rotates to the next binlog file. Default is 53687091200 bytes. Value range: [0, 9223372036854775807]. 0 disables automatic cleanup by size.

Note

This parameter is available from V4.3.0.
BINLOG_TENANT_LOGS_AUTO_SCALE: Auto scale-out/scale-in settings in JSON format. See BINLOG_TENANT_LOGS_AUTO_SCALE below.

Note

This parameter is available from V4.3.0.
BINLOG_TENANT_SCOPE_CFG: Tenant-level configuration, mainly for high-performance tuning. The value is a whitespace-separated list of obcdc_cfg_key=value pairs. Do not use whitespace around =. See BINLOG_TENANT_SCOPE_CFG below.

Note

This parameter is available from V4.3.0.

BINLOG_TENANT_LOGS_AUTO_SCALE

This parameter configures auto scale-out/scale-in and must be set in JSON format. Example:

CREATE BINLOG FOR TENANT obdemo.test1 
    WITH CLUSTER URL 'http://10.10.10.1:8080/services?Action=ObRootServiceInfo&ObCluster=obdemo', 
    BINLOG_TENANT_EXPIRE_LOGS_SECONDS 254400,
    BINLOG_TENANT_EXPIRE_LOGS_SIZE 53687094400, 
    BINLOG_TENANT_LOGS_AUTO_SCALE '{
      "enable":0,
      "scale_up_threshold":"20%",
      "scale_up_step":"20%",
      "max_scale_up_capacity_gb":5000,
      "scale_down_threshold":"50%",
      "scale_down_step":"20%",
      "min_scale_down_capacity_gb":10,
      "computed_binlog_expire_logs_size":10737418240
      }';

The following table describes each field in BINLOG_TENANT_LOGS_AUTO_SCALE:

Field	Default	Description
enable	0	Whether to enable auto scale-out/scale-in. Default `0` disables it.
max_scale_up_capacity_gb	5000	Upper limit for scale-out: maximum binlog storage capacity. No further automatic scale-out after this value. Unit: `GB`. Must be a positive integer.
min_scale_down_capacity_gb	10	Lower limit for scale-in: minimum binlog storage capacity. No further automatic scale-in after this value. Unit: `GB`. Must be a positive integer.
scale_down_threshold	50%	Scale-in threshold: scale-in is triggered when free binlog storage exceeds this value. The unit must match `scale_up_threshold`.
scale_down_step	20%	Disk space reclaimed per scale-in. Units: `GB` or `%`. Value must be a positive integer.
scale_up_threshold	20%	Scale-out threshold: scale-out is triggered when free binlog storage falls below this value. Units: `GB` or `%`. Value must be a positive integer.
scale_up_step	20%	Disk space added per scale-out. Units: `GB` or `%`. Value must be a positive integer.
computed_binlog_expire_logs_size	10737418240	Baseline disk capacity after auto scale-out/scale-in is enabled. Default 10 GB (10737418240).

BINLOG_TENANT_SCOPE_CFG

This parameter sets tenant-level options, mainly for high-performance tuning. The value is a whitespace-separated list of obcdc_cfg_key=value pairs. Do not use whitespace around =.

Note

This parameter is available from V4.3.0. Only commonly used settings are listed here.

Option	Default	Description
record_queue_size	200000	Data send queue size.
read_wait_num	20000	Batch wait count for the data read queue.
storage_wait_num	20000	Maximum batch size for binlog events.
binlog_serialize_thread_size	3	Serialization thread pool size; must be greater than `binlog_serialize_parallel_size`. `binlog_serialize_parallel_size` is the actual serialization parallelism.
throttle_convert_rps	0	Limit on binlog events persisted per second. Integer. Default `0` disables RPS throttling for binlog conversion. Range: [0, 4294967295].
throttle_convert_iops	0	Limit on bytes persisted per second for binlog events. Integer. Default `0` disables IOPS throttling for binlog conversion. Range: [0, 9223372036854775807].
throttle_dump_rps	0	Limit on binlog events sent downstream per second. Integer. Default `0` disables RPS throttling for binlog send. Range: [0, 4294967295].
throttle_dump_iops	0	Limit on bytes sent downstream per second for binlog events. Integer. Default `0` disables IOPS throttling for binlog send. Range: [0, 9223372036854775807].
throttle_dump_conn	0	Maximum concurrent subscription connections per binlog instance. Integer. Default `0` disables the limit. Range: [0, 512].

Examples

Create a binlog task for tenant test1 in cluster obdemo, with three binlog instances.

CREATE BINLOG FOR TENANT `obdemo`.`test1` 
    TO USER `root` PASSWORD `********` 
    WITH CLUSTER URL `http://10.10.10.1:8080/services?Action=ObRootServiceInfo&ObCluster=demo`, 
    REPLICATE NUM 3;

Create a binlog task for tenant test1 in cluster obdemo using ROOTSERVER_LIST.

CREATE BINLOG FOR TENANT `obdemo`.`test1` 
    TO USER `root` PASSWORD `********` 
    WITH ROOTSERVER_LIST '10.10.10.1:2882:2881';

Create a binlog task for tenant test1 in cluster obdemo with auto scale-out/scale-in and high-performance-related settings.

Note

Auto scale-out/scale-in is generally not recommended. This example sets enable to 0 to disable it.

CREATE BINLOG FOR TENANT obdemo.test1 
    TO USER `root` PASSWORD `********`
    WITH CLUSTER URL 'http://10.10.10.1:8080/services?Action=ObRootServiceInfo&ObCluster=obdemo', 
    BINLOG_TENANT_EXPIRE_LOGS_SECONDS 254400,
    BINLOG_TENANT_EXPIRE_LOGS_SIZE 53687094400,
    BINLOG_TENANT_SCOPE_CFG 'record_queue_size=100000 read_wait_num=2000 storage_wait_num=2000 binlog_serialize_thread_size=7 throttle_convert_rps=99 throttle_convert_iops=2097152 throttle_dump_rps=111 throttle_dump_iops=2097152 throttle_dump_conn=64', 
    BINLOG_TENANT_LOGS_AUTO_SCALE '{
      "enable":0,
      "scale_up_threshold":"20%",
      "scale_up_step":"20%",
      "max_scale_up_capacity_gb":5000,
      "scale_down_threshold":"50%",
      "scale_down_step":"20%",
      "min_scale_down_capacity_gb":10,
      "computed_binlog_expire_logs_size":10737418240
      }';

Query a binlog task

Queries binlog files and resource metrics for the primary binlog instance of a binlog task. If the primary binlog instance is unavailable, an empty set is returned.

Syntax

SHOW BINLOG STATUS [FOR TENANT cluster.tenant];

cluster.tenant is the tenant name and cluster name of the binlog task to query. FOR TENANT cluster.tenant is optional. SHOW BINLOG STATUS without it returns primary-instance information for all binlog tasks; with it, returns information for the specified tenant. Example:

SHOW BINLOG STATUS FOR TENANT obdemo.test1;

Sample output:


+-----------+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| cluster   | tenant | status                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
+-----------+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| obdemo    | test1  | {"binlog_instance":"gldwt7bnco","resources_metrics":{"client_id":"/home/admin/oblogproxy/run/gldwt7bnco","fd_count":78,"pid":12058,"cpu_status":{"cpu_count":8,"cpu_used_ratio":0.006535947788506746},"disk_status":{"disk_total_size_mb":307048,"disk_usage_size_process_mb":0,"disk_used_ratio":0.0,"disk_used_size_mb":78830},"memory_status":{"mem_total_size_mb":63882,"mem_used_ratio":0.016514824703335763,"mem_used_size_mb":1055},"network_status":{"network_rx_bytes":288853,"network_wx_bytes":289725}},"binlog_files":[{"binlog_name":"mysql-bin.000001","binlog_size":155}],"computed_binlog_expire_logs_size":53687094400}      |
+-----------+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Result set columns:

Column	Description
cluster	Cluster name
tenant	Tenant name
status	Primary binlog instance information

Note

Starting with obbinlog V4.3.0, the status column includes computed_binlog_expire_logs_size, representing the configured disk size. If auto scale-out/scale-in is enabled, this value is computed from that feature; otherwise it is the value of the binlog instance parameter BINLOG_EXPIRE_LOGS_SIZE.

Modify a binlog task

Modifies parameter settings for all binlog instances under the specified tenant. The changes also apply to binlog instances created later.

Syntax

ALTER BINLOG cluster.tenant 
      SET binlog_instance_option [, binlog_instance_option...];

binlog_instance_option:
      BINLOG_EXPIRE_LOGS_SECONDS
    | BINLOG_EXPIRE_LOGS_SIZE
    | FAILOVER
    | EXTRA_OBCDC_CFG
    | BINLOG_TENANT_FAILOVER
    | BINLOG_TENANT_REMOTE_FAILOVER
    | BINLOG_TENANT_EXPIRE_LOGS_SECONDS
    | BINLOG_TENANT_EXPIRE_LOGS_SIZE
    | BINLOG_TENANT_LOGS_AUTO_SCALE
    | BINLOG_TENANT_SCOPE_CFG

BINLOG_TENANT_LOGS_AUTO_SCALE: {
    "enable": value,
    "max_scale_up_capacity_gb": value,
    "min_scale_down_capacity_gb": value,
    "scale_down_threshold": "value",
    "scale_down_step": "value",
    "scale_up_threshold": "value", 
    "scale_up_step": "value",
    "computed_binlog_expire_logs_size": value
}

Parameters

Parameter	Description
cluster.tenant	Tenant whose binlog instance parameters are to be modified, and its cluster.
binlog_instance_option	Binlog instance parameters to modify, separated by commas (`,`). See binlog_instance_option below.

binlog_instance_option

BINLOG_EXPIRE_LOGS_SECONDS: Binlog file validity period. Default 259200 seconds. Binlog files expire automatically after this duration.
BINLOG_EXPIRE_LOGS_SIZE: Binlog file size threshold. Default 53687091200 bytes.
FAILOVER: Whether the binlog instance is automatically recovered after going offline. Default 1 (auto recovery).
EXTRA_OBCDC_CFG: obcdc parameter settings for the binlog instance.
BINLOG_TENANT_FAILOVER: Controls failover recovery for the binlog instance itself. When enabled, if the node is healthy but the binlog instance fails, it is automatically recovered. Default 1. Similar to legacy FAILOVER; this new parameter is recommended.

Note

This parameter is available from V4.3.0.
BINLOG_TENANT_REMOTE_FAILOVER: Controls remote failover for binlog instances. When enabled, if the node hosting the instance fails, the instance is migrated to another healthy node and recovered. Default 0.
Note
- This parameter is available from V4.3.0.
- Remote-node failover is jointly controlled by BINLOG_TENANT_REMOTE_FAILOVER and BINLOG_TENANT_FAILOVER:
  - When BINLOG_TENANT_FAILOVER is 0, both local-node and remote-node failover are disabled.
  - When BINLOG_TENANT_FAILOVER is 1 and BINLOG_TENANT_REMOTE_FAILOVER is 0, local-node failover is enabled and remote-node failover is disabled (default).
  - When BINLOG_TENANT_FAILOVER is 1 and BINLOG_TENANT_REMOTE_FAILOVER is 1, both are enabled.
BINLOG_TENANT_EXPIRE_LOGS_SECONDS: Same semantics as when creating a task. Default 259200 seconds. Range [0, 4294967295]. 0 disables cleanup by time. Similar to legacy BINLOG_EXPIRE_LOGS_SECONDS; this new parameter is recommended.

Note

This parameter is available from V4.3.0.
BINLOG_TENANT_EXPIRE_LOGS_SIZE: Same semantics as when creating a task. Default 53687091200 bytes. Range [0, 9223372036854775807]. 0 disables cleanup by size. Similar to legacy BINLOG_EXPIRE_LOGS_SIZE; this new parameter is recommended.

Note

This parameter is available from V4.3.0.
BINLOG_TENANT_LOGS_AUTO_SCALE: Auto scale-out/scale-in in JSON format. See the BINLOG_TENANT_LOGS_AUTO_SCALE subsection under Create a binlog task.

Note

This parameter is available from V4.3.0.
BINLOG_TENANT_SCOPE_CFG: Tenant-level tuning string. See the BINLOG_TENANT_SCOPE_CFG subsection under Create a binlog task.

Note

This parameter is available from V4.3.0.

Example

Note

Auto scale-out/scale-in is generally not recommended. This example sets enable to 0 to disable it.

ALTER BINLOG obdemo.test1 SET 
    BINLOG_TENANT_EXPIRE_LOGS_SECONDS =254401,
    BINLOG_TENANT_EXPIRE_LOGS_SIZE=53687094401,
    BINLOG_TENANT_SCOPE_CFG='record_queue_size=100002 read_wait_num=2002 storage_wait_num=2002 binlog_serialize_thread_size=6', 
    BINLOG_TENANT_LOGS_AUTO_SCALE='{
      "enable":0,
      "scale_up_threshold":"22%",
      "scale_up_step":"22%",
      "max_scale_up_capacity_gb":5002,
      "scale_down_threshold":"52%",
      "scale_down_step":"22%",
      "min_scale_down_capacity_gb":22,
      "computed_binlog_expire_logs_size":10737418242
      }';

Release a binlog task

Stops all binlog instances for the specified tenant and deletes existing binlog files.

Syntax

DROP BINLOG [IF EXISTS] FOR TENANT cluster.tenant;

cluster.tenant is the tenant and cluster of the binlog task to release.

Example

Release the binlog task for tenant test1 in cluster obdemo.

DROP BINLOG FOR TENANT obdemo.test1;

OceanBase

Customer Stories

Documentation

Task management

Create a binlog task

Syntax

Parameters

Note

Note

binlog_option

Note

Note

Note

Note

Note

Note

BINLOG_TENANT_LOGS_AUTO_SCALE

BINLOG_TENANT_SCOPE_CFG

Note

Examples

Note

Query a binlog task

Syntax

Note

Modify a binlog task

Syntax

Parameters

binlog_instance_option

Note

Note

Note

Note

Note

Note

Example

Note

Release a binlog task

Syntax

Example