This topic describes how to deploy Alertmanager by using an obd configuration file in a CentOS Linux 7.9 image for x86 architecture.
Prerequisites
You have installed V3.6.0 or later of obd. We recommend that you install the latest version. For more information, see Install obd.
The required components (such as OceanBase Database, OBAgent, Prometheus, and Alertmanager) are available in the obd image repository.
If the obd server can connect to the Internet, the required components will be automatically downloaded from the remote image repository.
If you deploy on an x86 server, make sure that the OBServer node supports the AVX instruction set. You can run the
lscpu | grep Flags | grep avxcommand to check whether the AVX instruction set is supported.Note
When you deploy on an x86 server, if the OceanBase Database version to be deployed is in the following list, the AVX instruction set is no longer required:
- For V4.2.x, the version must be V4.2.5.6 or later.
- For V4.3.x, the version must be V4.3.5.4 or later.
- For V4.4.x, the version must be V4.4.1.0 or later.
In an offline deployment scenario, if you deploy on an ARM server, make sure that the OBServer node supports the LSE instruction set. You can run the
lscpu | grep Flags | grep atomicscommand to check whether the LSE instruction set is supported. If the LSE instruction set is not supported, you must download an OceanBase Database installation package with thenonlseoption and use theobd mirror clonecommand to upload the installation package to the local image repository of obd.Note
The OceanBase Database installation package with the
nonlseoption is not included in the OceanBase All in One package.
Procedure
Note
This topic assumes that the environment supports public Internet access and the required software, such as OceanBase Database, OBAgent, Prometheus, and Alertmanager, is installed.
Step 1: Modify the configuration file
If obd is directly downloaded and installed on your machine, you can view the configuration file examples provided by obd in the /usr/obd/example directory.
If you installed obd by decompressing the all-in-one installation package, you can find sample configuration files provided by obd in the ~/.oceanbase-all-in-one/obd/usr/obd/example directory.
Differences between the sample configuration files are described in Download sample configuration files in Configuration File Description. Choose the appropriate configuration file based on your resource conditions.
Explanation
At the end of this topic, you will find a complete sample configuration file, which you can copy and modify as needed.
Create a configuration file.
[admin@test001 ~]$ vim alertmanager.yamlModify user information
## Only need to configure when remote login is required user: username: admin # password: your password if need key_file: /home/admin/.ssh/id_rsa # port: your ssh port, default 22 # timeout: ssh connection timeout (second), default 30usernamespecifies the login username for the target server where the OceanBase cluster is deployed. Ensure that the specified username has read and write privileges on the following paths:key_file,user,user_password,group, andgroupsin the directory specified by theuser_dirparameter. Bothpasswordandkey_fileare used to validate the user. You need to specify only one of them.Note
After you configure the secret key path, if your secret key does not require a password, comment or delete the
passwordstatement to avoid using thepasswordas the key password during login, which may cause the verification to fail.Configure OceanBase Database
Modify the IP address, port, and related directories of the server, and configure memory-related parameters and the user password under the sys tenant.
Note
OBServer supports deployment of the OceanBase Database Enterprise Edition. This example shows how to deploy the OceanBase Database Community Edition.
oceanbase-ce: # version: 4.3.5.0 servers: - name: server1 ip: 10.10.10.1 - name: server2 ip: 10.10.10.2 - name: server3 ip: 10.10.10.3 global: # Please set devname as the network adaptor's name whose ip is in the setting of severs. # if set severs as "127.0.0.1", please set devname as "lo" # if current ip is 192.168.1.10, and the ip's network adaptor's name is "eth0", please use "eth0" devname: eth0 cluster_id: 1 # please set memory limit to a suitable value which is matching resource. memory_limit: 8G # The maximum running memory for an observer system_memory: 2G # The reserved system memory. system_memory is reserved for general tenants. The default value is 30G. datafile_size: 20G # Size of the data file. datafile_next: 20G # the auto extend step. Please enter an capacity, such as 2G datafile_maxsize: 200G # the auto extend max size. Please enter an capacity, such as 20G log_disk_size: 20G # The size of disk space used by the clog files. cpu_count: 16 scenario: htap enable_auto_start: true mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started. rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started. obshell_port: 2886 # Operation and maintenance port for Oceanbase Database. The default value is 2886. This parameter is valid only when the version of oceanbase-ce is 4.2.2.0 or later. production_mode: false # The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field. home_path: /home/admin/observer # The directory for data storage. The default value is $home_path/store. data_dir: /data/observer # The directory for clog, ilog, and slog. The default value is the same as the data_dir value. redo_dir: /redo/observer ocp_agent_monitor_username: ocp_monitor ocp_agent_monitor_password: ****** # The password for obagent monitor user root_password: ****** # root user password, can be empty proxyro_password: ****** # proxyro user pasword, consistent with obproxy's observer_sys_password, can be empty server1: zone: zone1 server2: zone: zone2 server3: zone: zone3The configuration items in the configuration file are described in the Configuration Item Description section in Configuration File Description. Take note of the following points:
If no password is specified in the configuration file, OBd will automatically generate a random password. After the deployment is completed, you can use the
obd cluster edit-configcommand to view the corresponding configuration item in the configuration file to obtain the password.If the version of the OceanBase database to be deployed is V4.2.5 or later, and the
scenarioparameter is not specified in the configuration file, obd provides an interactive prompt to ask the user to specify the load type.Set
enable_auto_starttotrueto enable the auto-start feature for the observer process. This feature requires the deployment user to have sudo privileges and the deployment environment to not be a container environment.
Configure ODP
Note
OBOD supports the deployment of ODP. The following example shows how to deploy the ODP Community Edition.
obproxy-ce: # version: 4.3.2.0 depends: - oceanbase-ce servers: - 10.10.10.1 global: listen_port: 2883 # External port. The default value is 2883. prometheus_listen_port: 2884 # The Prometheus port. The default value is 2884. rpc_listen_port: 2885 enable_obproxy_rpc_service: true home_path: /home/admin/obproxy client_session_id_version: 2 # This parameter is used to specify whether to use the new logic to generate the client session ID. The parameter type is integer. The value range is [1, 2] and the default value is 2 (use the new logic). proxy_id: 5 # This parameter is used to set the ID for an ODP. The parameter type is integer. The default value is 0 and the value range is [0, 8191]. enable_cluster_checkout: false skip_proxy_sys_private_check: true enable_strict_kernel_release: false obproxy_sys_password: ****** # obproxy sys user password, can be empty. When a depends exists, obd gets this value from the oceanbase-ce of the depends. observer_sys_password: ***** # proxyro user pasword, consistent with oceanbase-ce's proxyro_password, can be empty. When a depends exists, obd gets this value from the oceanbase-ce of the depends.The meanings of the configurations in the configuration file can be found in Configuration Item Description of Configuration File Description. Select ODP to view the descriptions.
Configure components that relate to monitoring
obagent: depends: - oceanbase-ce servers: - name: server1 ip: 10.10.10.1 - name: server2 ip: 10.10.10.2 - name: server3 ip: 10.10.10.3 global: # The working directory for obagent. obagent is started under this directory. This is a required field. home_path: /home/admin/obagent # The port that pulls and manages the metrics. The default port number is 8088. monagent_http_port: 8088 mgragent_http_port: 8089 # Log path. The default value is log/monagent.log. log_path: log/monagent.log http_basic_auth_user: admin http_basic_auth_password: ****** prometheus: servers: - 10.10.10.1 depends: - obagent global: home_path: /home/admin/prometheus port: 9090 # The http port to use. Along with address, corresponds to the `web.listen-address` parameter. enable_lifecycle: true # Enable shutdown and reload via HTTP request. Corresponds to the `web.enable-lifecycle` parameter. data_dir: /home/admin/prometheus/data # Base path for metrics storage. Corresponds to the `storage.tsdb.path` parameter. basic_auth_users: # Usernames and passwords that have full access to the web server via basic authentication. Corresponds to the `basic_auth_users` parameter. <username>: <password> # The format of `basic_auth_users` : the key is the user name and the value is the password. alertmanager: servers: - 10.10.10.1 depends: - prometheus global: home_path: /home/admin/alertmanager log_dir: /home/admin/alertmanager/log # Alertmanager log storage directory data_retention: 120h #Alertmanager how long to keep data form, corresponds to the `data.retention` parameter. port: 9093 # The http port to use. Along with address, corresponds to the `web.listen-address` parameter. data_dir: /home/admin/alertmanager/data # Base path for data storage. Corresponds to the `storage.path` parameter. basic_auth_users: # Usernames and passwords that have full access to the web server via basic authentication. Corresponds to the `web.config.fil` parameter. <username>: <password> # The format of `basic_auth_users` : the key is the user name and the value is the password. receivers: #one or more notification list - email - mock_webhook email: receiver_type: email to: "<to_email>" from: "<from_email>" smarthost: "smtp.xx.com:xxx" auth_username: "smtp_username" auth_password: "smtp_password" send_resolved: true mock_webhook: receiver_type: webhook url: http://<host>:<port>/For more information about the configurations in the configuration file, see Configuration Item Description in Configuration File Description. This topic describes OBAgent, Prometheus, and Alertmanager.
Note
The following example shows how to configure
emailandwebhookfor notification. Replace the sample configuration with information that applies to your environment. For information about how to configure other notification methods, see the receivers section in the Alertmanager topic in Configure the configuration file.
Step 2: Deploy OceanBase Database
Tip
For more information about the commands mentioned in this section, see Cluster Command Groups.
Deploy an OceanBase Database cluster
[admin@test001 ~]$ obd cluster deploy obtest -c alertmanager.yamlFor example, the cluster name is
obtestand the relative path of the configuration file isalertmanager.yaml. You can customize the cluster name and specify the configuration file needed based on the actual situation.This command checks if the directory referenced by
home_pathanddata_diris empty. If not, it will return an error. If you are sure you want to delete all content in the directory, you can use the-foption to force the deletion.Start OceanBase Cluster
[admin@test001 ~]$ obd cluster start obtestThe cluster returns the deployment status of components and the corresponding connection strings, as follows:
Wait for observer init ok +-------------------------------------------------+ | oceanbase-ce | +---------------+---------+------+-------+--------+ | ip | version | port | zone | status | +---------------+---------+------+-------+--------+ | 10.10.10.1 | 4.3.5.3 | 2881 | zone1 | ACTIVE | | 10.10.10.2 | 4.3.5.3 | 2881 | zone1 | ACTIVE | | 10.10.10.3 | 4.3.5.3 | 2881 | zone1 | ACTIVE | +---------------+---------+------+-------+--------+ obclient -h10.10.10.1 -P2881 -uroot@sys -p'******' -Doceanbase -A cluster unique id: 95d6a939-8936-5848-95a1-3e21b483e67e-1990982668c-03050304 Connect to obproxy ok +-------------------------------------------------------------------+ | obproxy-ce | +---------------+------+-----------------+-----------------+--------+ | ip | port | prometheus_port | rpc_listen_port | status | +---------------+------+-----------------+-----------------+--------+ | 10.10.10.1 | 2883 | 2884 | 2885 | active | +---------------+------+-----------------+-----------------+--------+ obclient -h10.10.10.1 -P2883 -uroot@proxysys -p'******' -Doceanbase -A Connect to Obagent ok +------------------------------------------------------------------+ | obagent | +---------------+--------------------+--------------------+--------+ | ip | mgragent_http_port | monagent_http_port | status | +---------------+--------------------+--------------------+--------+ | 10.10.10.1 | 8089 | 8088 | active | | 10.10.10.2 | 8089 | 8088 | active | | 10.10.10.3 | 8089 | 8088 | active | +---------------+--------------------+--------------------+--------+ Connect to Prometheus ok +-----------------------------------------------------------+ | prometheus | +---------------------------+-------+--------------+--------+ | url | user | password | status | +---------------------------+-------+--------------+--------+ | http://10.10.10.1:9090 | admin | '******' | active | +---------------------------+-------+--------------+--------+ Connect to Alertmanager ok +-----------------------------------------------------------+ | alertmanager | +---------------------------+-------+--------------+--------+ | url | user | password | status | +---------------------------+-------+--------------+--------+ | http://10.10.10.1:9093 | admin | '******' | active | +---------------------------+-------+--------------+--------+ obshell program health check ok display obshell dashboard ok +----------------------------------------------------------+ | obshell Dashboard | +---------------------------+------+--------------+--------+ | url | user | password | status | +---------------------------+------+--------------+--------+ | http://10.10.10.1:2886 | root | '******' | active | +---------------------------+------+--------------+--------+ obtest running
Step 3: Verify the deployment results
After deployment, you can verify that the Alertmanager is working properly:
Validate the Alertmanager Web interface
Copy the Alertmanager URL from the output of your deployment, and use it to access the Alertmanager web interface in a browser:
http://10.10.10.1:9093In the Alibaba Cloud or other cloud environments, the system may fail to obtain a public IP address and instead output an intranet IP address. Please note that this IP address is not a public IP address and use the correct public IP address for access. After accessing, you need to log in using the username and password set in the configuration file (
basic_auth_users).Check the Alertmanager status
[admin@test001 ~]$ obd cluster display obtestThe output should be as follows, and the status of the Alertmanager component should be
active.Wait for observer init ok +-------------------------------------------------+ | oceanbase-ce | +---------------+---------+------+-------+--------+ | ip | version | port | zone | status | +---------------+---------+------+-------+--------+ | 10.10.10.1 | 4.3.5.3 | 2881 | zone1 | ACTIVE | | 10.10.10.2 | 4.3.5.3 | 2881 | zone1 | ACTIVE | | 10.10.10.3 | 4.3.5.3 | 2881 | zone1 | ACTIVE | +---------------+---------+------+-------+--------+ obclient -h10.10.10.1 -P2881 -uroot@sys -p'******' -Doceanbase -A cluster unique id: 95d6a939-8936-5848-95a1-3e21b483e67e-1990982668c-03050304 Connect to obproxy ok +-------------------------------------------------------------------+ | obproxy-ce | +---------------+------+-----------------+-----------------+--------+ | ip | port | prometheus_port | rpc_listen_port | status | +---------------+------+-----------------+-----------------+--------+ | 10.10.10.1 | 2883 | 2884 | 2885 | active | +---------------+------+-----------------+-----------------+--------+ obclient -h10.10.10.1 -P2883 -uroot@proxysys -p'******' -Doceanbase -A Connect to Obagent ok +------------------------------------------------------------------+ | obagent | +---------------+--------------------+--------------------+--------+ | ip | mgragent_http_port | monagent_http_port | status | +---------------+--------------------+--------------------+--------+ | 10.10.10.1 | 8089 | 8088 | active | | 10.10.10.2 | 8089 | 8088 | active | | 10.10.10.3 | 8089 | 8088 | active | +---------------+--------------------+--------------------+--------+ Connect to Prometheus ok +-----------------------------------------------------------+ | prometheus | +---------------------------+-------+--------------+--------+ | url | user | password | status | +---------------------------+-------+--------------+--------+ | http://10.10.10.1:9090 | admin | '******' | active | +---------------------------+-------+--------------+--------+ Connect to Alertmanager ok +-----------------------------------------------------------+ | alertmanager | +---------------------------+-------+--------------+--------+ | url | user | password | status | +---------------------------+-------+--------------+--------+ | http://10.10.10.1:9093 | admin | '******' | active | +---------------------------+-------+--------------+--------+ obshell program health check ok display obshell dashboard ok +----------------------------------------------------------+ | obshell Dashboard | +---------------------------+------+--------------+--------+ | url | user | password | status | +---------------------------+------+--------------+--------+ | http://10.10.10.1:2886 | root | '******' | active | +---------------------------+------+--------------+--------+ obtest running
Sample configuration file
Note
The following example shows how to configure email and webhook receivers. You need to modify the configuration based on your actual scenario. For information about how to configure other notification methods, see the receivers section in the Alertmanager section of Configuration file description.
## Only need to configure when remote login is required
user:
username: admin
# password: your password if need
key_file: /home/admin/.ssh/id_rsa
# port: your ssh port, default 22
# timeout: ssh connection timeout (second), default 30
oceanbase-ce:
# version: 4.3.5.0
servers:
- name: server1
ip: 10.10.10.1
- name: server2
ip: 10.10.10.2
- name: server3
ip: 10.10.10.3
global:
# Please set devname as the network adaptor's name whose ip is in the setting of severs.
# if set severs as "127.0.0.1", please set devname as "lo"
# if current ip is 192.168.1.10, and the ip's network adaptor's name is "eth0", please use "eth0"
devname: eth0
cluster_id: 1
# please set memory limit to a suitable value which is matching resource.
memory_limit: 8G # The maximum running memory for an observer
system_memory: 2G # The reserved system memory. system_memory is reserved for general tenants. The default value is 30G.
datafile_size: 20G # Size of the data file.
datafile_next: 20G # the auto extend step. Please enter an capacity, such as 2G
datafile_maxsize: 200G # the auto extend max size. Please enter an capacity, such as 20G
log_disk_size: 20G # The size of disk space used by the clog files.
cpu_count: 16
scenario: htap
mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started.
rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started.
obshell_port: 2886 # Operation and maintenance port for Oceanbase Database. The default value is 2886. This parameter is valid only when the version of oceanbase-ce is 4.2.2.0 or later.
production_mode: false
# The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
home_path: /home/admin/observer
# The directory for data storage. The default value is $home_path/store.
data_dir: /data/observer
# The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
redo_dir: /redo/observer
ocp_agent_monitor_username: ocp_monitor
ocp_agent_monitor_password: ****** # The password for obagent monitor user
root_password: ****** # root user password, can be empty
proxyro_password: ****** # proxyro user pasword, consistent with obproxy's observer_sys_password, can be empty
server1:
zone: zone1
server2:
zone: zone2
server3:
zone: zone3
obproxy-ce:
# version: 4.3.2.0
depends:
- oceanbase-ce
servers:
- 10.10.10.1
global:
listen_port: 2883 # External port. The default value is 2883.
prometheus_listen_port: 2884 # The Prometheus port. The default value is 2884.
rpc_listen_port: 2885
enable_obproxy_rpc_service: true
home_path: /home/admin/obproxy
client_session_id_version: 2 # This parameter is used to specify whether to use the new logic to generate the client session ID. The parameter type is integer. The value range is [1, 2] and the default value is 2 (use the new logic).
proxy_id: 5 # This parameter is used to set the ID for an ODP. The parameter type is integer. The default value is 0 and the value range is [0, 8191].
enable_cluster_checkout: false
skip_proxy_sys_private_check: true
enable_strict_kernel_release: false
obproxy_sys_password: ****** # obproxy sys user password, can be empty. When a depends exists, obd gets this value from the oceanbase-ce of the depends.
observer_sys_password: ***** # proxyro user pasword, consistent with oceanbase-ce's proxyro_password, can be empty. When a depends exists, obd gets this value from the oceanbase-ce of the depends.
obagent:
depends:
- oceanbase-ce
servers:
- name: server1
ip: 10.10.10.1
- name: server2
ip: 10.10.10.2
- name: server3
ip: 10.10.10.3
global:
# The working directory for obagent. obagent is started under this directory. This is a required field.
home_path: /home/admin/obagent
# The port that pulls and manages the metrics. The default port number is 8088.
monagent_http_port: 8088
mgragent_http_port: 8089
# Log path. The default value is log/monagent.log.
log_path: log/monagent.log
http_basic_auth_user: admin
http_basic_auth_password: ******
prometheus:
servers:
- 10.10.10.1
depends:
- obagent
global:
home_path: /home/admin/prometheus
port: 9090 # The http port to use. Along with address, corresponds to the `web.listen-address` parameter.
enable_lifecycle: true # Enable shutdown and reload via HTTP request. Corresponds to the `web.enable-lifecycle` parameter.
data_dir: /home/admin/prometheus/data # Base path for metrics storage. Corresponds to the `storage.tsdb.path` parameter.
basic_auth_users: # Usernames and passwords that have full access to the web server via basic authentication. Corresponds to the `basic_auth_users` parameter.
<username>: <password> # The format of `basic_auth_users` : the key is the user name and the value is the password.
alertmanager:
servers:
- 10.10.10.1
depends:
- prometheus
global:
home_path: /home/admin/alertmanager
log_dir: /home/admin/alertmanager/log # Alertmanager log storage directory
data_retention: 120h #Alertmanager how long to keep data form, corresponds to the `data.retention` parameter.
port: 9093 # The http port to use. Along with address, corresponds to the `web.listen-address` parameter.
data_dir: /home/admin/alertmanager/data # Base path for data storage. Corresponds to the `storage.path` parameter.
basic_auth_users: # Usernames and passwords that have full access to the web server via basic authentication. Corresponds to the `web.config.fil` parameter.
<username>: <password> # The format of `basic_auth_users` : the key is the user name and the value is the password.
receivers: #one or more notification list
- email
- mock_webhook
email:
receiver_type: email
to: "<to_email>"
from: "<from_email>"
smarthost: "smtp.xx.com:xxx"
auth_username: "smtp_username"
auth_password: "smtp_password"
send_resolved: true
mock_webhook:
receiver_type: webhook
url: http://<host>:<port>/