ob_cluster_recyclebin_disk_used_over_threshold

2025-03-21 06:01:07  Updated

Description

This alert is generated when the amount of data in the recycle bin exceeds 100 GB.

Alert parameters

Parameter Value
Metric ob_database_disk_used_gb
Source Collected from internal views of OceanBase Database
SQL statement for collecting the metric in OceanBase Database of a version earlier than V4.0.0:
select t1.tenant_id, t2.database_id, t3.database_name, sum(t1.data_size) as data_size, sum(t1.required_size) as required_size from (select tenant_id, table_id, sum(data_size) as data_size, sum(required_size) as required_size from __all_virtual_meta_table group by table_id) t1 left join (select tenant_id, table_id, database_id from __all_virtual_table) t2 on t1.tenant_id = t2.tenant_id and t1.table_id = t2.table_id left join (select tenant_id, database_id, database_name from gv$database) t3 on t2.tenant_id = t3.tenant_id and t2.database_id = t3.database_id group by t2.database_id
SQL statement for collecting the metric in OceanBase Database V4.0.0 and later:
select t1.tenant_id, t1.database_name, t3.object_id as database_id, sum(t2.data_size) as data_size, sum(t2.required_size) as required_size from (select tenant_id, database_name, table_id, tablet_id from cdb_ob_table_locations) t1 left join (select tenant_id, tablet_id, svr_ip, svr_port, data_size, required_size from cdb_ob_tablet_replicas) t2 on t1.tenant_id = t2.tenant_id and t1.tablet_id = t2.tablet_id left join (select con_id, object_name ,object_id from CDB_OBJECTS where object_type = 'DATABASE') t3 on t1.tenant_id = t3.con_id and t1.database_name = t3.object_name group by t1.database_name
Collected metric ob_database_disk_required_size
Metric expression sum(ob_database_disk_required_size{database_name="__recyclebin",@LABELS}) by (@GBLABELS) / 1073741824
Collection cycle 60 seconds

Alert rule

Metric expression Metric description Default threshold Detection cycle Time before clearance
ob_database_disk_used_gb > 100 Disk space occupied by data in the recycle bin 100 (GB) 20 seconds 5 minutes

Alert information

Trigger method Alert level Scope
Based on the expression of the metric Warning Cluster

Alert templates

  • Overview

    • Template: ${alarm_target} ${alarm_name}
    • Sample: alarm_template_id=0:ob_cluster=Test-1 ob_cluster_recyclebin_disk_used_over_threshold
  • Details

    • Template: Cluster: ${ob_cluster_name}. Alert: ${alarm_name} The disk space occupied by the data in the recycle bin is ${value_shown}, which exceeds ${alarm_threshold} GB.
    • Sample: Cluster: Test. Alert: ob_cluster_recyclebin_disk_used_over_threshold The disk space occupied by data in the recycle bin is 120 GB, which exceeds 100 GB.

Impact on the system

If a large amount of data is cleaned up and the recycle bin is enabled, a lot of system resources will be wasted.

Possible causes

  • The recycle bin is enabled and a large amount of data is truncated and moved to the recycle bin. However, the recycle bin has a long purging interval. For example, if you execute the alter system set recyclebin_object_expire_time='7d' statement, data in the recycle bin will be purged in seven days.

  • The recycle bin is enabled and a large amount of data is accumulated after some time. However, automatic purging is disabled for the recycle bin. In this case, you must set a purging interval.

Solutions

  1. Check whether the recycle bin needs to be enabled.

    • Statement for enabling or disabling the recycle bin: set GLOBAL recyclebin = on / off;
    • Statement for enabling or disabling flashback for the recycle bin: set GLOBAL ob_enable_truncate_flashback = on /off;

    If you want to enable the recycle bin, you can configure an appropriate purging interval. For example, you can execute the alter system set recyclebin_object_expire_time='7d' statement to set the purging interval to seven days. Data in the recycle bin will be purged in seven days.

  2. Manually purge the data in the recycle bin.

    • Execute the SHOW RECYCLEBIN; statement to view objects in the recycle bin.
    • Execute the PURGE DATABASE object_name; statement to purge specified objects from the recycle bin.
    • Execute the PURGE TABLE object_name; statement to purge specified tables from the recycle bin.
    • Execute the PURGE RECYCLEBIN; statement to purge the entire recycle bin.

Contact Us