Description
This alert is triggered when a secondary data backup task initiated in the OceanBase Cloud Platform (OCP) failed.
Principle
OCP-Server runs a scheduled task to check the secondary log backup tasks every minute. When a secondary log backup task failed, this alert is triggered.
Alert rule
| Metric | Default threshold | Duration | Detection cycle | Elimination cycle |
|---|---|---|---|---|
| N/A | N/A | 0 seconds | 60 seconds | 5 minutes |
Alert information
| Trigger method | Alert level | Scope |
|---|---|---|
| Reminder of OCP | Critical | Services |
Alert template
- Alert overview
- Template: ${alarm_target} ${alarm_name}
- Sample: ob_cluster=obcluster-1 base_secondary_backup_fail
- Alert details
- Template: Cluster: ${ob_cluster_name}[${ob_cluster_id}], Alert: base_secondary_backup_fail, Task started at: ${start_time}, Task ended at: ${finish_time}
- Sample: Cluster: obcluster-1[10006], Alert: base_secondary_backup_fail, Task started at: 2020-01-05T02:00:00Z, Task ended at: 2020-01-05T03:00:00Z
Impact on the system
If a secondary backup task executed to recover a tenant failed, the tenant data cannot be recovered.
Possible causes
There are various causes. You must locate the problem based on the specific cause.
Solutions
Check the OBServer logs for details.
Log on to the OCP console. On the Clusters page, find the target cluster in the Clusters list and click its name. Then, the Overview page of the cluster appears.
Click the More icon in the upper-right corner and select Download Logs.
- Specify the time range, which must contain the alert time.
- For the log type, select OBServer Logs.
Click Download Logs to download the OBServer logs.
Open the observer.log file and search for WARN- and ERROR-level logs to view the details.
