This section introduces the incident handling feature in OCP alerts, which allows you to execute plans to quickly resolve alerts in the details of certain alerts.
Alert list with incident response plans
Alert name |
Alert description |
Incident response plan |
|---|---|---|
| os_observer_not_exist | The observer process does not exist. | The observer process is restarted if it unexpectedly exits. The process is restarted only once within 30 minutes. |
| arbitration_service_unavailable | The arbitration service is unavailable. | The observer process is restarted if it unexpectedly exits. The process is restarted only once within 30 minutes. |
| tenant_memstore_percent_over_threshold | The memory usage of an OceanBase tenant exceeds the threshold. | The tenant memory is scaled out after the memory usage exceeds the threshold. |
| ob_tenant_request_queue_over_threshold | The request queue of an OceanBase tenant exceeds the threshold. | The tenant CPU is scaled out after the CPU usage exceeds the threshold. |
| obproxy_client_connections_usage_over_threshold | The client connection usage of OBProxy exceeds the threshold. | The cluster-level parameters are automatically scaled out when the OBProxy connection count is insufficient. |
| oas_anomaly_sql_from_anomaly_event_analysis_perf_degradation | Root cause analysis of host CPU overutilization, and SQL performance degradation. | The PlanCache of the SQL statement is refreshed. |
| oas_anomaly_sql_from_anomaly_event_analysis_plan_changed | Root cause analysis of host CPU overutilization, and SQL execution plan degradation. | The PlanCache of the SQL statement is refreshed. |
| oas_anomaly_sql_from_sql_inspection_perf_degradation | SQL inspection, and SQL performance degradation. | The PlanCache of the SQL statement is refreshed. |
| oas_anomaly_sql_from_sql_inspection_plan_changed | SQL inspection, and SQL execution plan degradation. | The PlanCache of the SQL statement is refreshed. |
Procedure
The following example describes the procedure for alerting ob_tenant_request_queue_over_threshold (OceanBase tenant request queue exceeded).
In the list of events, click the corresponding event details.
On the event details page, after you understand the basic information, click Execute Now.
Note
Some alerts already have automatic execution plans and you can view the execution result directly on the event details page, see **step 6**. For example, the alerts
os_observer_not_existandarbitration_service_unavailable.
Click OK in the pop-up dialog box to execute the plan.
View the execution result.
The plan executed successfully.
The plan executed failed and the plan was retried or rolled back.
Retry the plan
Click Retry.
Roll back the plan, contact OceanBase technical support to roll back the plan.
Click Disable to return to the event details page after you confirm the execution result.
Click View Successful Execution or View Failed Execution to view the execution result of the plan for an alert that has executed the plan. For more information, see step 4.
