obproxy_client_connections_usage_over_threshold

Description

This alert is triggered when the usage of client connections of an OBProxy exceeds the threshold.

Principle

Alert rule

Alert information

Alert templates

• Overview

  • Template: ${alarm_target} ${alarm_name}
  • Example: alarm_template_id=0:obproxy_cluster=TEST-1:host=xxx.xxx.xxx.xxx. The usage of client connections of the OBProxy exceeds the threshold.

• Details

  • Template: OBProxy cluster: ${obproxy_cluster}. Host: ${host}. Alert: ${alarm_name}. The usage of client connections reaches ${value_shown}, which exceeds the threshold ${alarm_threshold}%.
  • Example: OBProxy cluster: TEST. Host: xxx.xxx.xxx.xxx. Alert: The usage of client connections of the OBProxy exceeds the threshold. The usage of client connections reaches 90%, which exceeds the threshold 85%.

Impact on the system

• The response time of business requests increases.
• The backlog of business requests results in exceptions, such as no responses to SQL queries and connection timeout.

Possible causes

• A large number of business requests are initiated.
• Prolonged connection occupation is caused due to slow response of the OBServer node.

Solutions

1. Execute the alert clearance plan.

   1. Execute the alert clearance plan to fix the alerted issue. For more information, see Execute the alert clearance plan.
   2. Retry the alert clearance plan. Make sure that the plan is executed to solve the original issue.
   3. Roll back the alert clearance plan. You can contact OceanBase Technical Support for assistance.

2. Increase the maximum number of connections allowed for an OBProxy and increase the memory for the OBProxy by 20% by modifying the client_max_connections and proxy_mem_limited parameters.

   The sample commands are as follows:

   ALTER proxyconfig SET client_max_connections = xxx;
   ALTER proxyconfig SET proxy_mem_limited = xxx;