Description
This alert is triggered when a disk is mounted to the / root directory as read-only.
Principle
The following table describes the key parameters that are involved in the monitoring and alerting logic.
| Parameter | Value |
|---|---|
| Metric | disk_readonly_flag
|
| Source | The metrics are collected from node_exporter. The records of the detection on read-only mounted disks are stored in host.check.readonly.mountpoint. By default, only disks in the / mount point are detected. Add the tag of check_readonly="1". |
| Collected metric | node_filesystem_readonly |
| Metric expression | max(node_filesystem_readonly{@LABELS}) by (@GBLABELS) |
| Collection cycle | 1 second |
Alert rule
| Metric | Default threshold | Duration | Alert cycle | Elimination cycle |
|---|---|---|---|---|
| ob_host_disk_readonly_flag | 1 | 0 seconds | 60 seconds | 5 minutes |
Alert information
| Trigger method | Alert level | Scope |
|---|---|---|
| Based on the expression of the metric | Critical | Server |
Alert templates
Overview: ${alarm_target} ${alarm_name}
Details: Cluster: ${ob_cluster_name}, Host: ${host}, Alert: ${alarm_name}, The disk mount point ${mount_point} is read-only.
Overview example: svr_ip=xxx.xxx.xxx.xxx Read-only server disk
Details example: Cluster: obcluster-1, Host: xxx.xxx.xxx.xxx, Alert: Read-only server disk, The disk mount point /data/t is read-only.
Impact on the system
This alert is for read-only detection on disks mounted to the / directory. When the disk is mounted to the / directory as read-only, most processes cannot properly work. For example, logs cannot be recorded and configurations cannot be updated. This can lead to serious system errors.
Possible causes
The disk is mounted as read-only.
An error occurs in the file system.
The disk has a bad sector.
Suggested solutions
Run the following commands to verify the alert.
Run the
$mount | grep '(ro,'command to view the disk that is mounted as read-only.Run the
df -hcommand to check whether the disk is mounted to the root directory.
Contact the disk manufacturer to repair or replace the disk.