monitor_exporter_unavaliable

2023-08-22 02:46:05  Updated

Description

The OCP server obtains the status of the monitoring exporter address once every minute. This alert is triggered when the exporter address is in active state for more than five minutes.

Alert information

Trigger method Alert level Scope
Based on the expression of the metric Caution Server

Alert rule

Metric Default threshold Source Duration Detection cycle Elimination cycle
monitor_exporter_avaliable 0 Monitoring data collection on the OCP server 300s 60s 5 min

Alert templates

  • Overview: ${alarm_target} ${alarm_name}

  • Details: Host: ${host}, Alert: An exception occurs on the monitoring exporter <${exporter_addr}>(Type: ${exporter_type},Scrape interval: ${scrape_interval} seconds).

  • Overview example: exporter_addr=http://xxx.xxx.xxx.xxx:8089/metrics/ob/perSecond:exporter_type=OB_CLUSTER:scrape_interval=1. An exception occurs on the monitoring exporter.

  • Details example: Host: xxx.xxx.xxx.xxx, Alert: An exception occurs on the monitoring exporter < http://xxx.xxx.xxx.xxx:8089/metrics/ob/perSecond > (Type: OB_CLUSTER, scrape_interval: 3 seconds).

${alarm_target} indicates the object where the alert is generated, in the exporter_addr=xxx:exporter_type=xxx:scrape_interval=xxx format. exporter_addr indicates the monitoring exporter address. exporter_type indicates the monitoring exporter type. scrape_interval indicates the monitoring data collection interval.

Impact on the system

No monitoring data is found in OCP. Therefore, the real-time system running status is unavailable and the monitoring alerts are not reported.

Possible causes

  • OCP-Agent, the monitoring data collection program, is abnormal and fails to return monitoring data.

  • The network is disconnected, causing OCP to fail to access the monitoring address.

Solutions

Run the following command on the OCP server to check whether the monitoring address exporter_addr is accessible:

curl http://xxx.xxx.xxx.xxx:8089/metrics/ob/perSecond

Note

Replace http://xxx.xxx.xxx.xxx:8089/metrics/ob/perSecond with the actual value of exporter_addr in the alert.

  • If it is inaccessible, the problem is probably caused by a network issue.

    Check whether the network is faulty and ensure that the network between the OCP server and the monitoring server is connected.

  • If it is accessible, the collection process OCP-Agent is faulty. Troubleshoot and solve the problem by referring to OCP-Agent script.

Contact Us