Integrate OCP monitoring with Prometheus

This topic describes how to integrate OceanBase business monitoring with Prometheus.

Application scenarios

You want to integrate business monitoring using your own Prometheus system.

Prerequisites

• Ensure the current OCP user is in the ADMIN role.
• The Prometheus service of version V2.28 or later is deployed.
• OCP of version V4.4.2 or later is deployed.

Procedure

Step 1: Create a Prometheus data source

1. Log in to the OCP console.

2. In the left-side navigation pane, click Administration > External Integration.

3. Click the Prometheus tab.

4. On the Prometheus Data Sources page, click Create Prometheus Data Source.

5. In the dialog box that appears, specify the information for the Prometheus Data Source.

   Parameter	Description
   Data Source Name	The name of the Prometheus data source. It must be 2 to 32 characters in length, can contain Chinese characters, English letters, numbers, and underscores (_), and must be unique.
   Access Address	The access IP address of the Prometheus service, starting with http/https. Only one IP address is allowed.
   Port	The access port of the Prometheus service. The default is 9090, but it can be customized.
   Username	The username of the Prometheus service authorization account. This parameter is required if available.
   Password	The password of the Prometheus service authorization account. This parameter is required if available.
   Remarks	Add any special notes about the Prometheus data source.

6. Click Verify to validate the connectivity of the data source.

7. After verification succeeds, click Create.

8. Copy the Prometheus configuration information from the dialog box.

Step 2: Modify the prometheus.yaml configuration file

Add the configuration information copied in Step 1 to the Prometheus configuration file prometheus.yaml:

scrape_configs:
  - job_name: 'ocp_sd'
    scheme: 'http'
    fallback_scrape_protocol: "PrometheusText1.0.0"  # This parameter is required only for Prometheus 3.x; not required for 2.x
    http_sd_configs:
      - url: 'http://xx.xx.xx.xx:8080/api/v2/monitor/prometheus_sd'
        basic_auth:   # Access OCP to obtain all targets in prometheus_sd
            username: "******"
            password: "******"
    basic_auth:   # Control access to all targets
        username: "******"
        password: "******"

Step 3: Restart the Prometheus service

1. Restart the Prometheus service for the configuration to take effect.

2. Log in to the Prometheus service console.

3. Select Status > Target health to check whether the connected jobs are working properly. Ensure that the status of all targets is UP.

4. Enter executor_active_threads and ob_sysstat in Query to check whether the monitoring data from OCP and Agent has been successfully accessed.

Step 4: Enable the Prometheus data source

1. Log in to the OCP console.

2. In the left-side navigation pane, click Administration > External Integration.

3. Click the Prometheus tab.

4. On the Prometheus Data Source page, click Enable in the Actions column.

   After you enable the Prometheus data source, OCP and Prometheus can synchronize monitoring data bidirectionally. That is, Prometheus can collect data from OCP, and OCP can read monitoring data from Prometheus.

More information

Basic authentication in Prometheus

For specific authentication methods, refer to the official Prometheus website: Basic authentication.

The created web.yml file can be placed in the same directory as the prometheus.yaml file. That is, add the --web.config.file=/etc/prometheus/web.yml field when starting Docker or the container.

In the following example, /home/admin is a non-container directory that uniformly stores the prometheus.yaml and web.yml files.

docker run  -d --name prometheus -p 9090:9090 -v /home/admin/docker/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml -v /home/admin/docker/prometheus/web.yml:/etc/prometheus/web.yml prom/prometheus:v3.3.0    --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/prometheus --web.config.file=/etc/prometheus/web.yml

OCP verifies the Prometheus connection mechanism

OCP sends an HTTP request to access the /api/v1/status/buildinfo interface of Prometheus.

Request example:

# Username and password:
curl -H "Authorization: Basic $(echo -n 'Prometheus account:Prometheus password' | base64)" \
http://{PrometheusIP}:{Port}/api/v1/status/buildinfo

# Passwordless login:
curl -L http://{PrometheusIP}:{Port}/api/v1/status/buildinfo

Response example:

{
  "status": "success",  // Whether the request was successful.
  "data": {
    "version": "3.3.0",  // The version number of Prometheus.
    "revision": "3c49e69330ceee9c587a1d994c78535695edff16",
    "branch": "HEAD",
    "buildUser": "root@b362765c931d",
    "buildDate": "20250415-13:32:00",
    "goVersion": "go1.24.1"
  }
}

OCP version upgrade

By default, OCP V4.4.2 is compatible with Prometheus configurations from previous versions. However, for ease of future maintenance, it is strongly recommended that you refer to this section to recreate and configure the Prometheus data source.

Prometheus configuration files

The http_sd_configs section contains two basic_auth fields. Their purpose is for OCP to assist Prometheus in generating an access token, enabling Prometheus to have limited access to OCP and to agent-related exporters.

Note: Do not modify this configuration information yourself. If modification is needed, you can refer to Modify the agent authorization token to make changes on OCP.

global:
  scrape_interval:  10s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'ocp_sd'
    scheme: 'http'
    fallback_scrape_protocol: "PrometheusText1.0.0"  # This parameter is required only for Prometheus 3.x. It is not required for 2.x.
    http_sd_configs:
      - url: 'http://xx.xx.xx.xx:8080/api/v2/monitor/prometheus_sd'
        basic_auth:   # Access OCP to obtain all targets in prometheus_sd.
            username: "******"
            password: "******"
    basic_auth:   # Control access to all targets.
        username: "******"
        password: "******"

Modify the agent authorization key

OCP provides an API for modifying the agent authorization key. You can refer to the following steps to perform the operation:

1. Connect to the metaDB and query the key ID using the following SQL statement:

   select id from profile_credential where access_target = 'PROMETHEUS'
   

2. Execute the following command on the OCP container to modify the key:

   curl --location --request PUT 'http://OCP-IP:OCP-port/api/v2/integration/prometheus/changeAuthPd/{id}?id={id}' \
   --header 'Content-Type: application/json' \
   -u 'admin:OCP password'  \
   --data '{
       "newAuthPd": "******"
   }'
   

   Here, {id} is the ID obtained from the previous step, {OCP-IP} is the IP address of OCP, {OCP-port} is the port of OCP, {admin:OCP password} is the account and password of OCP, and {******} is the new password after modification (it is recommended to keep it consistent with the OCP password format).

3. Check whether the return result indicates success.

4. Modify the prometheus.yaml file and update the two password fields in basic_auth to the new password obtained in Step 2.

5. Restart the Prometheus container, enter the Prometheus console, select Status > Target health, and check whether the connected job is working properly. If the status of all targets is UP, the modification is successful.

HTTPS authentication instructions

When you choose to access OCP and Prometheus over HTTPS, refer to HTTPS authentication instructions to add TLS and related certificates for OCP and Prometheus.

Related operations

• OCP V4.3.0 now supports connecting monitoring dashboards to Prometheus data sources. For details, see Connect monitoring queries to external time-series systems.

• If you are using the Grafana GUI dashboard for data visualization, you can download the OCP Grafana template JSON file from the following document: Grafana dashboard.