This topic introduces how to monitor OceanBase Cloud databases using OBCloud Exporter and Prometheus.
Background information
OBCloud Exporter is a data processing tool for databases provided by OceanBase Cloud, which can be used to authenticate and retrieve database data. It can be used together with Prometheus to monitor databases locally.
Prerequisites
Java 8 or higher runtime environment has been deployed.
Docker has been deployed.
Procedure
. Download OBCloud Exporter.
Note
Currently, direct download for OBCloud Exporter is not available. Please contact OceanBase Cloud technical support for assistance.
. Deploy OBCloud Exporter.
Unzip the OBCloud Exporter installation package, and the installation directory is
/opt/obcloud-exporter.mkdir /opt/obcloud-exporter tar -zxvf Obcloud_Exporter.tar.gz -C /opt/obcloud-exporter --strip-components=1The directory contains the following files:
- Dockerfile: Docker container build file.
- muticloud_metric_config.yaml: Metric configuration file, which does not require modification by default.
- application.yaml: Collection configuration file.
- obcloud-exporter-1.0-SNAPSHOT-jar-with-dependencies.jar: OBCloud Exporter program.
- readme.md: Project instructions.
Edit the configuration file
application.yaml.# Monitoring Name obcloud_monitor: # Credentials credential: access_key_id: xxx access_key_secret: xx # Endpoint end_point: xxx # Whether to periodically refresh metrics, default is false metric_meta_auto_refresh: false instances: # Instance ID - instance_id: xxx # Query tenant under specified cluster, tenant_id needs to be configured tenant_id: xxx # Instance type, cluster for cluster instance, tenant for tenant instance instance_type: cluster # Metrics metrics:access_key_id、access_key_secret: Authentication information, obtained by accessing My Account > AccessKey. For details, refer to Manage AccessKey.
end_point: Refer to Service access point.
metric_meta_auto_refresh: Whether to periodically refresh metrics, the default is false.
instances: Monitoring instances, including multiple cluster/tenant instances, entered with a hyphen (-).
instance_id: Instance ID.
Note
Using tenant ID under the cluster instance is not supported.
instance_type: Filled in according to the instance type, "cluster" for cluster instance, "tenant" for tenant instance.
tenant_id: To query tenants under a specified cluster, tenant_id needs to be configured.
metrics: Metric information. If it is left blank, it indicates that all metrics are collected. When specific metrics need to be collected, the format is as follows:
instances: -instance_id: xxx tenant_id: xxx instance_type: cluster metrics: - sql_all_rt - sql_delete_rtNote
Collecting too many metrics may affect the collection speed. Under appropriate circumstances, metrics can be split into multiple exporters.
. Run OBCloud Exporter through Java.
java -jar /opt/obcloud-exporter/obcloud-exporter-1.0-SNAPSHOT-jar-with-dependencies.jar
Verify the deployment result through the browser. Access Port 9400 of the corresponding IP address, and confirm whether monitoring data is obtained.
Note
You can access the IP address according to the actual port configured.
. Download Prometheus, find the required version of Prometheus installation package on the official website, and download it. Here, V2.50.1 of the linux version is used as an example.
wget https://github.com/prometheus/prometheus/releases/download/v2.50.1/prometheus-2.50.1.linux-amd64.tar.gz
. Deploy Prometheus.
Unzip and install Prometheus, and the installation directory is
/opt/prometheus.mkdir -p /opt/prometheus tar -zxvf prometheus-2.50.1.linux-amd64.tar.gz -C /opt/prometheus --strip-components=1Transfer the configuration file prometheus.yml to the working directory and edit it as needed.
mv /opt/prometheus/prometheus.yml /usr/local/bin/ vi /usr/local/bin/prometheus.ymlModify the parameters as follows:
global: scrape_interval: 30s scrape_timeout: 20s scrape_configs: - job_name: "nodes" static_configs: - targets: ['localhost:9400']Create a Prometheus service file and related data storage directory.
mkdir /var/lib/prometheus/ sudo vim /etc/systemd/system/Prometheus.serviceFill in the following content and save it.
Note
User and Group parameters are optional. To set user and group, refer to the step below.
[Unit] Description=Prometheus Wants=network-online.target After=network-online.target [Service] User=prometheus Group=prometheus Restart=on-failure ExecStart=/opt/prometheus/prometheus \ --config.file /usr/local/bin/prometheus.yml \ --storage.tsdb.path /var/lib/prometheus/ [Install] WantedBy=multi-user.target(Optional) Set user and group. If the user/group is already configured in the previous step, you can skip this step.
sudo groupadd prometheus sudo useradd -r -g prometheus prometheus sudo chown -R prometheus:prometheus /opt/prometheus /var/lib/prometheus sudo chmod -R 755 /opt/prometheus /var/lib/prometheus
. Start the Prometheus service.
# Load the new systemd configuration
sudo systemctl daemon-reload
# Start the Prometheus service
sudo systemctl start prometheus
# Enable auto-start on boot
sudo systemctl enable prometheus
# Check the service status
sudo systemctl status prometheus
. Confirm if Prometheus has started successfully.
sudo netstat -ntlp | grep 9090
Download OBCloud Exporter.
Note
Currently, direct download for OBCloud Exporter is not available. Please contact OceanBase Cloud technical support for assistance.
Deploy OBCloud Exporter.
- Unzip the OBCloud Exporter installation package, and the installation directory is
/opt/obcloud-exporter.
mkdir /opt/obcloud-exporter tar -zxvf Obcloud_Exporter.tar.gz -C /opt/obcloud-exporter --strip-components=1The directory contains the following files:
- Dockerfile: Docker container build file.
- muticloud_metric_config.yaml: Metric configuration file, which does not require modification by default.
- application.yaml: Collection configuration file.
- obcloud-exporter-1.0-SNAPSHOT-jar-with-dependencies.jar: OBCloud Exporter program.
- readme.md: Project instructions.
Edit the configuration file
application.yaml.# Monitoring Name obcloud_monitor: # Credentials credential: access_key_id: xxx access_key_secret: xx # Endpoint end_point: xxx # Whether to periodically refresh metrics, default is false metric_meta_auto_refresh: false instances: # Instance ID - instance_id: xxx # Query tenant under specified cluster, tenant_id needs to be configured tenant_id: xxx # Instance type, cluster for cluster instance, tenant for tenant instance instance_type: cluster # Metrics metrics:access_key_id、access_key_secret: Authentication information, obtained by accessing My Account > AccessKey. For details, refer to Manage AccessKey.
end_point: Refer to Service access point.
metric_meta_auto_refresh: Whether to periodically refresh metrics, the default is false.
instances: Monitoring instances, including multiple cluster/tenant instances, entered with a hyphen (-).
instance_id: Instance ID.
Note
Using tenant ID under the cluster instance is not supported.
instance_type: Filled in according to the instance type, "cluster" for cluster instance, "tenant" for tenant instance.
tenant_id: To query tenants under a specified cluster, tenant_id needs to be configured.
metrics: Metric information. If it is left blank, it indicates that all metrics are collected. When specific metrics need to be collected, the format is as follows:
instances: - instance_id: xxx tenant_id: xxx instance_type: cluster metrics: - sql_all_rt - sql_delete_rtNote
Collecting too many metrics may affect the collection speed. Under appropriate circumstances, metrics can be split into multiple exporters. The metric list is provided at the end of the topic under Monitoring metric list.
- Unzip the OBCloud Exporter installation package, and the installation directory is
Start OBCloud Exporter via Docker.
In the
/opt/obcloud-exporterdirectory, build the obcloud-exporter container.cd /opt/obcloud-exporter docker build -t obcloud-exporter:1.0 .Start the container directly or by mapping external configuration files.
Start the Docker container directly.
docker run -itd -p9400:9400 --name obcloud-exporter obcloud-exporter:1.0Start the Docker container by mapping external configuration files.
docker run -itd -p9400:9400 -v /opt/obcloud-exporter/application.yaml:/app/application.yaml --name obcloud-exporter obcloud-exporter:1.0
Verify the deployment result through the browser, access port 9400 of the corresponding IP address, and confirm whether monitoring data is obtained.
Note
You can access the IP address according to the actual port configured.
Create the Prometheus configuration file prometheus.yml.
vi /usr/local/bin/prometheus.ymlEdit the configuration file content:
global: scrape_interval: 30s evaluation_interval: 30s scrape_configs: - job_name: obcloud-exporter-test static_configs: # Local server plus port - targets: ['obcloud-exporter:9400'] labels: instance: obcloud-exporterStart the Prometheus service.
docker run -d \ -p 9090:9090 \ -v /usr/local/bin/prometheus.yml:/etc/prometheus/prometheus.yml \ --link obcloud-exporter \ --name prometheus \ prom/prometheusConfirm if Prometheus has started successfully.
sudo netstat -ntlp | grep 9090
View database monitoring with Prometheus
Access the Prometheus service using the browser to view the corresponding monitoring information: http://localhost:9090/targets.

Monitoring metric list
| Metric Name | Description |
|---|---|
| load1 | Average system load in the past 1 minute |
| load15 | Average system load in the past 15 minutes |
| load5 | Average system load in the past 5 minutes |
| cpu_percent | CPU usage rate |
| memory_buffers | Kernel BufferCache size |
| memory_cached | Memory used for caching |
| memory_free | Available physical memory size |
| memory_used | Used physical memory size |
| io | Average I/O operations per second |
| io_read | Number of reads per second |
| io_write | Number of writes per second |
| io_read_time | Average read time per I/O |
| io_time | I/O time |
| io_write_time | Average write time per I/O |
| io_byte | Average I/O data volume per second |
| io_read_byte | Data volume read per I/O |
| io_write_byte | Data volume written per I/O |
| net_recv | Data volume received per second |
| net_send | Data volume sent per second |
| net_throughput | Network throughput rate |
| ntp_offset_milliseconds | NTP clock offset |
| ob_data_disk_percent | OceanBase data disk usage rate |
| ob_clog_disk_percent | OceanBase log disk usage rate |
| ob_data_disk_used_size | Disk usage |
| ob_process_exists | OceanBase process survival status |
| sql_all_count | SQL statements processed per second |
| sql_delete_count | Delete statements processed per second |
| sql_insert_count | Insert statements processed per second |
| sql_other_count | Other statements such as DDL, DCL, DTL |
| sql_replace_count | Replace statements processed per second |
| sql_select_count | Select statements processed per second |
| sql_update_count | Update statements processed per second |
| sql_all_rt | Average processing time for SQL statements |
| sql_delete_rt | Average processing time for Delete statements |
| sql_insert_rt | Average processing time for Insert statements |
| sql_other_rt | Average processing time for other statements such as DDL, DCL, DTL |
| sql_replace_rt | Average processing time for Replace statements |
| sql_select_rt | Average processing time for Select statements |
| sql_update_rt | Average processing time for Update statements |
| active_session | Current active sessions |
| all_session | Current sessions |
| sql_distributed_count | Distributed execution plan processed per second |
| sql_local_count | Local execution processed per second |
| sql_remote_count | Remote execution plan processed per second |
| system_event_internal_total_waits | Internal wait events per second |
| wait_event_count | Wait events per second |
| system_event_internal_time_waited | Average waiting time for internal events |
| system_event_io_total_waits | I/O wait events per second |
| system_event_latch_total_waits | Latch wait events per second |
| system_event_other_total_waits | Other wait events per second |
| system_event_row_lock_wait_total_waits | Row lock wait events per second |
| system_event_sync_rpc_total_waits | Sync RPC wait events per second |
| wait_event_rt | Average waiting time for events |
| request_dequeue_count | Number of requests dequeued from the processing queue |
| request_enqueue_count | Number of requests enqueued into the processing queue |
| request_queue_time | Waiting time for SQL requests in the queue |
| ob_cpu_percent | Tenant thread CPU usage rate |
| memstore_percent | MemStore usage percentage |
| rpc_packet_in_rt | Average time to receive RPC packets |
| rpc_packet_out_rt | Average time to send RPC packets |
| rpc_packet_in | RPC packet receiving throughput |
| rpc_packet_out | RPC packet sending throughput |
| opened_cursors_count | Number of open cursors |
| transaction_commit_count | Number of transactions committed per second |
| transaction_count | Number of transactions processed per second |
| transaction_rollback_count | Number of transactions rolled back per second |
| transaction_timeout_count | Number of transactions timed out per second |
| transaction_commit_rt | Average commit time per transaction |
| transaction_rollback_rt | Average rollback time per transaction |
| transaction_rt | Average processing time per transaction on the server |
| trans_commit_log_count | Number of transaction logs committed per second |
| clog_trans_log_total_size | Size of transaction logs committed per second |
| memstore_write_lock_fail_count | Number of write lock wait failures |
| memstore_write_lock_succ_count | Number of write lock wait successes |
| memstore_write_lock_wait_time | Average write lock wait time |
| transaction_multi_partition_count | Number of distributed transactions per second |
| transaction_single_partition_count | Number of normal transactions per second |
| trans_commit_log_sync_rt | Average network synchronization time per transaction log |
| active_memstore_used | Active MemStore size |
| major_freeze_trigger | Major compaction threshold triggered |
| memstore_limit | MemStore limit |
| total_memstore_used | Total MemStore size |
| io_read_count | SSStore read count per second |
| io_write_count | SSStore write count per second |
| io_read_rt | Average read time per SSStore read |
| io_write_rt | Average write time per SSStore write |
| io_read_size | Data volume read per second in SSStore |
| io_write_size | Data volume written per second in SSStore |
| block_cache_size | Block cache size |
| bloom_filter_cache_size | Bloomfilter cache size |
| clog_cache_size | Clog cache size |
| location_cache_size | Location cache size |
| plan_cache_size | Execution plan cache size |
| row_cache_size | Row cache size |
| block_cache_hit_ratio | Block cache hit ratio |
| bloom_filter_cache_hit_ratio | Bloomfilter cache hit ratio |
| clog_cache_hit_ratio | Clog cache hit ratio |
| location_cache_hit_ratio | Location cache hit ratio |
| plan_cache_hit_ratio | Execution plan cache hit ratio |
| row_cache_hit_ratio | Row cache hit ratio |
| block_cache_req_total | Total block cache requests |
| bloom_filter_cache_req_total | Total Bloomfilter cache requests |
| clog_cache_req_total | Total Clog cache requests |
| location_cache_req_total | Total Location cache requests |
| row_cache_req_total | Total Row cache requests |
| ob_tenant_binlog_disk_used | Binary log disk usage |