ob_zone_sstable_percent_over_threshold

2023-08-15 02:30:56  Updated

Description

This alert is triggered when the proportion of the disk space used in the directory where all OBServers in a zone are located (the /data/1 mount point of the OS) among the capacity of the operation directory where all OBServers in the zone are located exceeds the threshold.

Alert information

Parameter Value
Metric ob_zone_disk_percent
Note The value of the metric represents the data disk usage rate of a zone.When this value is greater than the threshold, this alert is triggered. The default threshold is 85%.
Source select /*+ READ_CONSISTENCY(WEAK)*/ total_size, free_size from __all_virtual_disk_stat where svr_ip = @svr_ip and svr_port = rpc_port();
Collected metric total_size, free_size
Metric expression 100 * (sum(total_size{metric_group="all_virtual_disk_stat",@LABELS}) by (@GBLABELS) - sum(free_size{metric_group="all_virtual_disk_stat",@LABELS}) by (@GBLABELS)) / sum(total_size{metric_group="all_virtual_disk_stat",@LABELS}) by (@GBLABELS)
Collection cycle 60 seconds

Alert rule

Metric Default threshold(unit:) Duration Detection cycle Time before clearance
ob_zone_disk_percent 75 0 seconds 0 seconds 5 minutes

Alert information

Trigger method Alert level Scope
Metric expression Critical Cluster

Alert templates

  • Overview: ${alarm_target} ${alarm_name}

  • Details: Cluster: ${ob_cluster_name}, Host: ${host}, Alert: ${alarm_name}. The data disk usage is ${value}%, exceeding the threshold of ${alarm_threshold}%.

  • Overview example: ob_cluster=C1-1000:obzone=zone-2. The data disk usage of the zone of the OceanBase cluster exceeds the threshold.

  • Details example: Cluster: obcluster-1, Host: 192.168.0.1, Alert: The data disk usage of the zone of the OceanBase cluster exceeds the threshold. The data disk usage is 76.0%, exceeding the threshold of 75.0%.

Impact on the system

If the data disk space is insufficient, new data cannot be written to the data disk.

Possible cause

The application data size increases continuously.

Suggested solution

See ob_server_sstable_percent_over_threshold.

Contact Us