Call an alert API

This section provides script examples to help you integrate and test the alert API.

Call the OCP API

The following script describes how to call the API for pushing alert events.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import base64
import requests
import json

data = {
    "alarmType": "your_alarm_type", 
    "labels": {"key":"value"}, 
    "target":"alarm_target",
}

base64userpass = base64.b64encode('{0}:{1}'.format('username', 'password'))
headers = {
    'Content-Type': 'application/json',
    'Accept': 'application/json',
    "Authorization": "Basic %s" % base64userpass,
}

resp = requests.post(url='http://xxx.xxx.xxx.xxx:8080/api/v2/alarm/alarms', headers=headers, data=json.dumps(data))
jresp = json.loads(resp.text)
print(jresp)

OCP alert integration example

After an alert is generated by OCP, the alert message is sent through an alert channel. The channel can be configured with a shell script or a Python script. This example describes how to send an alert through a channel. The send_alarm function retrieves the alert-related variable values from the environment variables using os.environ.

Here is the Python script example:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

# Note: The first line of the script must specify the program to be executed using a shebang. Only Python and BASH are supported.

import json
import requests
import os
import sys

ACCESS_TOKEN = "Ding Talk Token"

def send_alarm():
    """
    You can obtain the alert-related variable values from the environment variables using os.environ.
    For more information, see the list of variables in the "OCP alert template variables" section of the User Guide.
    """
    message = os.environ['message']
    data = {
        "msgtype": "markdown",
        "markdown": {
            "text": message
        }
    }
    resp = requests.post("https://oapi.dingtalk.com/robot/send?access_token=" + ACCESS_TOKEN, json=data)
    resp.close()
    response = json.loads(resp.text)

    # The returned result is written to standard error (stderr) or standard output (stdout) for verification of whether the alert was sent. stderr takes precedence.
    if not response['errcode'] == 0:
        sys.stderr.write(resp.text)
    else:
        print(resp.text)

def main():
    """
    If the alert is sent, the return value is 0.
    If the alert fails to be sent, the error is output to stderr, and the return value is not 0.
    """
    try:
        send_alarm()
        return 0
    except Exception as e:
        sys.stderr.write(str(e))
        return 1

if __name__ == "__main__":
    sys.exit(main())

Example applications

If you want to test whether the alerting chain is normal, that is, whether alerts can be pushed to a third-party alerting platform, you can also:

• When an OCP alert is triggered:

  1. Temporarily stop OCP-Agent to trigger alerts related to its unavailability, such as insufficient exporter count.

  2. Check whether the alert was received by the third-party platform.

• OCP Channel has a test feature that allows you to directly press the Send Test Message button for testing. For more information, see Create an alert channel.

Additionally, you can construct alerts using the above "OCP Alert Push Interface" and send them to third-party platforms through alert channel scripts.

Access the Alert API

The following example describes how to access the alert API of OCP.

Background information

When you want to integrate OCP alerts into your own alert platform, you need to use the API to query alert events. For more information, see Query alert events.

Scenario 1: View real-time alerts

When you need to view real-time alerts in the system, you can follow the content in the code example to periodically request the alert event list interface to obtain the objects that are currently alerting.

Code sample

curl 'http://OCP-IP:8080/api/v2/alarm/alarms?isSubscribedByMe=false&status=Active&page=1&size=10' \
--user username:password \
--compressed \
--insecure

Here, data.page.totalPages indicates the total number of pages for real-time alerts. You can view the alerts on a specific page. For example, if you want to view the data on page 2, set the page parameter to 2:

curl 'http://OCP-IP:8080/api/v2/alarm/alarms?isSubscribedByMe=false&status=Active&page=2&size=10' \
--user username:password \
--compressed \
--insecure

The following table describes the key information:

Scenario 2: Integrating with the customer's alert platform

Based on the code example in Scenario 1, you must periodically request the alert event list query interface and update the status of the received alert objects (targets).

The updates include:

• Last alert time: The duration since the last alert can be calculated based on the alert trigger time.

• Alert status: If an alert for a target is not returned in a batch of requests (where each batch consists of multiple alerts obtained through paginated requests), the alert is marked as restored.

• Real-time statistics: You can monitor the number of alerts in each alert level that are currently active.

Scenario 3: Verify the availability of alerts

To prevent the risk of important risks not being exposed due to unavailable alerts, you can periodically trigger an alert based on the sample code to verify the availability of the alert chain.

Sample code

1. Manually trigger an alert, such as a log alert: Write an ERROR log to the running logs of an OBServer node with the following content.

   echo '[2035-01-02 15:04:05.666666] ERROR [CLOG] update_free_quota (ob_log_file_pool.cpp:413) [1994][2072][Y0-0000000000000000] [lt=19] [dc=0] test ob error for ocp alarm, just ignore. ret=-999999' >> /home/admin/oceanbase/log/observer.log.wf
   

2. Enter the following command to request the API to verify the log alert.

3. In the request parameters, set the keyword (keyword) to the log content:

4. If an alert is generated in the system, it indicates that the alert chain is available.