Description
This alert is triggered when obproxyd.sh stops. obproxyd.sh is the daemon process for OBProxy.
The ocp_agent process monitors the uptime of the processes of components in OceanBase Database. When the uptime of a process is 0, the process is stopped. This alert indicates that the process has stopped at the moment rather than for a long time. In the latter case, other status alerts are triggered.
Relevant alerts:
- observer process: observer_process_stop
- obproxy process: obproxy_process_stop
- obproxyd.sh process: obproxyd_process_stop
- ocp_agentd process: agentd_process_stop
- ocp_mgragent process: mgragent_process_stop
- ocp_monagent process: monagent_process_stop
Principle
| Parameter | Value |
|---|---|
| Metric | obproxyd_uptime_delta_seconds |
| Source | The difference between the current time and the process creation time displayed in the 14th column of the stat file in the /proc/[pid] directory. |
| Collected metric | process_uptime_seconds |
| Metric expression | 0 - min(delta(process_uptime_seconds{name="obproxyd.sh",@LABELS}[@INTERVAL])) by (@GBLABELS) |
| Collection cycle | 5 seconds |
Alert rule
| Metric expression | Metric description | Default threshold | Detection cycle | Time before clearance |
|---|---|---|---|---|
| obproxyd_uptime_delta_seconds > 0 | The negative offset value of uptime is used. A metric value greater than 0 indicates that the process has stopped. | 0 seconds | 10 seconds | 5 minutes |
Alert information
| Trigger method | Alert level | Scope |
|---|---|---|
| Based on the expression of the metric | Warning | OBProxy |
Alert templates
Overview
- Template:
${alarm_target} ${alarm_name} - Example:
alarm_template_id=0:obproxy_cluster=ODPT2-2000005:host=xxx.xxx.xxx.xxx. The obproxyd.sh process stopped.
- Template:
Details
- Template:
OBProxy cluster: ${obproxy_cluster}. Host: ${host}. Alert: ${alarm_name}. The process has been running for ${value_shown}. - Example:
OBProxy cluster: ODPT2. Host: xxx.xxx.xxx.xxx. Alert: The obproxyd.sh process stopped. The process has been running for 1 day 15 hours 12 minutes 51 seconds.
- Template:
Impact on the system
When the obproxy process stops or unexpectedly exits, it is restarted by the obproxyd.sh process. If the obproxyd.sh process cannot start, the obproxy process cannot be restarted. This results in prolonged service interruption.
You must ensure that OBProxy stays online as much as possible and provides services to minimize service interruption.
Possible causes
None.
Solutions
Manually load the obproxyd.sh process. The restart of OBProxy causes business interruptions. Therefore, we recommend that you do not restart OBProxy in the OceanBase Cloud Platform (OCP) console.
cd /opt/taobao/install/obproxy-3.4.0 && bin/obproxyd.sh -c start -r /home/admin/logs/obproxy -n ODPT2 -p 2883
