Before you deploy an OceanBase cluster, you must check the system environment and configurations.
View resources
If you are an enterprise user, we recommend that you deploy the OceanBase cluster on servers with the same hardware and software configurations. If you are deploying OceanBase Database on a single server, you can skip this section.
OceanBase Database supports operating systems such as Red Hat Enterprise Linux Server 7.X, CentOS 7.X, and Anolis OS 8.X. For more information, see Software and hardware requirements.
Note
The configuration files mentioned in this topic refer to files that are configured when you use OBD to deploy OceanBase Database. For more information, see Example configuration files.
View information about the operating system
Red Hat Enterprise Linux Server 7.X
Run the following command:
cat /etc/redhat-release
The following example shows the returned result of this command:
Red Hat Enterprise Linux Server release 7.2 (Maipo)
CentOS 7.X
Run the following command:
cat /etc/redhat-release
The following example shows the returned result of this command:
CentOS Linux release 7.2.1511 (Core)
Anolis OS 8.X
Run the following command:
cat /etc/os-release
The following example shows the returned result of this command:
NAME="Anolis OS"
VERSION="8.2"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="8.2"
PLATFORM_ID="platform:an8"
PRETTY_NAME="Anolis OS 8.2"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.cn/"
Unbutu
Run the following command:
cat /etc/os-release
The following example shows the returned result of this command:
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Debian9
Run the following command:
cat /etc/os-release
The following example shows the returned result of this command:
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
VERSION_CODENAME=stretch
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
View the kernel information
Run the following command:
uname -r
OceanBase Database requires that the OS kernel be of V3.10.0 or later.
View the memory information
Run the following command:
free -g
If the memory space available is less than the value of the memory_limit parameter, clear the cache or change the value of the memory_limitto less than the available memory space.
Run the following command to clear the cache:
echo 3 > /proc/sys/vm/drop_caches
View the disk information
Run the following command:
df -h
Check the configuration file to ensure that the corresponding disks are mounted to the data_dir, redo_dir, and home_path directories, that the data_dir and redo_dir directories are empty, and that the usage of the data disk is less than 4%.
View the NIC name
You need to specify the NIC for the devname parameter. For example:
# Please set devname as the network adaptor's name whose ip is in the setting of severs.
# if set severs as "127.0.0.1", please set devname as "lo"
# if current ip is 192.168.1.10, and the ip's network adaptor's name is "eth0", please use "eth0"
devname: eth0
Run the following command to view the NIC name:
ifconfig
Notice
To run the
ifconfigcommand, you must install thenet-toolsdependencies.
Enable password-free SSH logon
Terms
Central control server
The server that stores the installation package and cluster configuration information of OceanBase Database.
Target server
The server where you want to install OceanBase Database.
Prerequisites
You have the root user privileges on the target server.
Procedure
You can take the following steps to set up password-free logon through SSH:
Run the following command on the central control server to check whether the key exists:
ls ~/.ssh/id_rsa.pubIf yes, you do not need to generate a new private SSH key.
Optional. Run the following command on the central control server to generate the public and private SSH keys:
ssh-keygen -t rsaRun the following command on the central control server to log on to the target server:
ssh-copy-id root@<server_ip>
Appendix: Reference scripts
We recommend that you use the following script. To run this script, you must have the sudo privilege.
#!/usr/bin/bash
SERVERS=("root@172.19.x.x root@172.19.x.x" "root@172.19.x.x" "root@172.19.x.x")
PASSWORD="test@"
keygen() {
yum -y install expect
expect -c "
spawn ssh-keygen -t rsa
expect {
*(/root/.ssh/id_rsa):* { send -- \r;exp_continue}
*(y/n)* { send -- y\r;exp_continue}
*Enter* { send -- \r;exp_continue}
*(y/n)* { send -- y\r;exp_continue}
*Enter* { send -- \r;exp_continue}
eof {exit 0}
}
expect eof
"
}
copy(){
expect -c "
set timeout -1
spawn ssh-copy-id $1
expect {
*(yes/no)* { send -- yes\r; exp_continue }
*password:* { send -- $PASSWORD\r; exp_continue}
eof {exit 0}
}
expect eof
"
}
ssh_copy_id_to_all(){
keygen ;
for host in ${SERVERS[@]}
do
copy $host
done
}
ssh_copy_id_to_all
Notice
You must replace the list of hosts and the password in the script with the actual list of servers and password.
Configure the clock source
If you deploy OceanBase Database in a cluster, you must ensure that the time of all servers in the cluster is synchronized. Otherwise, the cluster cannot be started and exceptions will be thrown. If you have already configured the Network Time Protocol (NTP) clock source, no need to do it again.
The clocks of servers in the OceanBase cluster must be synchronized. Otherwise, the OceanBase cluster may not start or will encounter operational exceptions. A physical server is considered synchronous with the clock source server when the clock error between them is 50 ms or lower. The tolerable clock error of an OceanBase cluster cannot exceed 100 ms. When the clock error exceeds 100 ms, the cluster may become leaderless. You can synchronize the clock and restart the OceanBase cluster to restore it to normal.
Prerequisites
You have the root user privileges on all OBServers.
Procedure
Perform the following steps to configure the NTP clock source.
Run the following command on each server to install NTP:
yum install ntp ntpdate -yRun the following command to check the NTP connection to the server:
When the time difference between the server and the NTP server is within 50 ms, the clocks are considered synchronized.
[root@centos-01 /root]#ntpq -4p remote refid st t when poll reach delay offset jitter ============================================================================== *time6.aliyun.co xxx.xxx.xxx.xxx 2 u 712 1024 377 21.951 4.253 4.208(Optional) Enable the NTP server.
If you already have an NTP server, skip this step. The following example describes how to enable time synchronization by taking the server
xxx.xxx.xxx.xxxas the NTP server.Edit the configuration file.
Log on to the NTP server as a root user and run the following command to open the configuration file:
vi /etc/ntp.confAdd the following content to the configuration file:
server <yout_ntp_server_ip>Save and close the file.
Run the following command to restart the NTP server:
service ntpd restartRun the following command to check whether the NTP service is enabled:
ps -ef | grep -i ntpdRun the following command to enable the NTP service upon startup.
chkconfig ntpd on
Enable the NTP service on an OBServer.
Notice
You can enable the NTP service on an OBServer five minutes later after the NTP server is started. Otherwise, the system prompts "no server suitable for synchronization found".
Log on to OceanBase Database as the root user.
Run the following command to stop the NTP service:
service ntpd stopRun the following command to synchronize the time with the NTP server:
ntpdate xxx.xxx.xxx.xxxEdit the configuration file.
Run the following command to open the configuration file:
vi /etc/ntp.confAdd the following content to the configuration file:
server <yout_ntp_server_ip>Run the following command to start the NTP service:
/etc/init.d/ntpd start
Verify whether the configuration is successful:
Run the following command:
ntpstatThe following output is returned:
synchronised to NTP server (xxx.xxx.xxx.xxx) at stratum 3 time correct to within 2 ms polling server every 64 sRun the following command:
timedatectlThe following output is returned:
Local time: Thu 2021-04-22 11:02:32 CST Universal time: Thu 2021-04-22 03:02:32 UTC RTC time: Thu 2021-04-22 11:02:32 Time zone: Asia/Shanghai (CST, +0800) NTP enabled: yes NTP synchronized: yes RTC in local TZ: yes DST active: n/aIf the command output is as expected, the NTP service is running.
Disk planning
The running of an OBServer relies on the data disk, clog disk, and installation disk where OceanBase Database is installed. If you are an individual user, you can put all the data on a single disk and skip this section. If you are an enterprise user, you must separately mount your data to three disks.
If you do not have three disks for a server, or you are using Redundant Array of Independent Disks (RAID), you need to partition the disk or disk array into logical volumes. We recommend that you plan the following disks:
Data disk
The data disk is mounted to the
data_dirdirectory, and is used for storing the baseline data. When you start OceanBase Database for the first time, the${data_dir}/sstabledirectory is automatically created. The size of the data disk is determined by thedatafile_disk_percentageparameter. The default value of thedatafile_disk_percentageparameter is95. Once you have created thedata_dirdirectory, you cannot resize it. You can scale OceanBase Database out or in by adding or removing servers. OceanBase Database does not support disks that are added directly to a server.Clog disk
The clog disk is mounted to the
redo_dirdirectory. We recommend that you set the size of the clog disk to at least 3 times that of the memory for the OceanBase Database. The clog disk contains files of a fixed size. These files are located in the${redo_dir}/{clog,ilog,slog}installation directory, and are created and cleaned up as needed. When the total size of clog files reaches 80% of the total disk capacity, automatic cleanup is triggered. However, the clog files can be cleaned up only when the corresponding memory data has been compacted and merged with the baseline data. For a given size of memory data, the size of clog files is approximately three times larger. Therefore, the maximum space required for the clog disk is directly proportional to the total amount of data after two major compactions. You can use the empirical formula for evaluation: Size of clog files = Maximum memory space for incremental data × 3 or 4.OceanBase Database installation disk
The disk for the installation of OceanBase Database is mounted to the
home_pathdirectory. We recommend that you reserve at least 200 GB to save log files for 7 or more days. The RPM package of OceanBase Database is located in the${home_path}directory. The baseline data files and clog files are mapped to the preceding data disk and clog disk through soft links. The runtime logs of OceanBase Database are located in the${home_path}/logdirectory. The number of runtime logs will keep increasing because OceanBase Database cannot automatically delete them. Therefore, you need to regularly delete the runtime logs.
After you plan the disks, run the following command to view the disk information:
df -h
The following output is returned:
Filesystem Size Used Avail Use% Mounted on
devtmpfs 31G 0 31G 0% /dev
tmpfs 31G 0 31G 0% /dev/shm
tmpfs 31G 516K 31G 1% /run
tmpfs 31G 0 31G 0% /sys/fs/cgroup
/dev/vda1 493G 171G 302G 37% /
tmpfs 6.2G 0 6.2G 0% /run/user/0
/dev/vdd1 984G 77M 934G 1% /data
/dev/vdc1 196G 61M 186G 1% /redo
/dev/vdb1 492G 73M 467G 1% /home/admin/oceanbase
Result description:
The
/datastring indicates the information of the data disk, which is 1 TB in size.The
/redostring indicates the information of the log disk where the REDO logs are stored.The
/home/admin/oceanbasestring indicates the information of the installation disk where the binary files and runtime logs of OceanBase Database are stored.
Configure the limits.conf file
You can configure the limits.conf file to limit the number of processes. If you are an individual user, you can skip this section. If you are an enterprise user, you must configure the limits.conf file.
You can modify the limit on resources by using the following two methods:
Modify the limit at the session level when OceanBase Database is started.
Modify the limit globally by using the
limits.conffile in the/etc/security/directory.
OceanBase Database processes can be limited in terms of the following parameters: Stack, Open Files, and Core File Size. Stack refers to the maximum stack size. Open Files refers to the maximum number of file handles. Core File Size refers to the maximum size of core files.
Modify parameters
Note
You must perform this operation on each server.
The following example shows how set the value of the session-level Stack parameter to unlimited, the Open Files parameter to655350, and the Core File Size parameter to unlimited.
Run the following command to open the limits.conf file in the /etc/security/ directory:
vi /etc/security/limits.conf
Add the following content to the /limits.conf file in the /etc/security/ directory:
root soft nofile 655350
root hard nofile 655350
* soft nofile 655350
* hard nofile 655350
* soft stack 20480
* hard stack 20480
* soft nproc 655360
* hard nproc 655360
* soft core unlimited
* hard core unlimited
View parameters
Log off from the session and log on again. Run the following command to view whether the parameters have taken effect:
ulimit -a
Configure the sysctl.conf file
To improve the Linux performance, and ensure the normal running of OceanBase Database, you need to modify the parameters in the sysctl.conf file of each server before the installation of OceanBase Database. If you are an individual user, you can skip this section. If you are an enterprise user, you must configure the sysctl.conf file.
Note
You must perform this operation on each server.
Modify parameters
Run the following command to open the sysctl.conf file in the /etc/ directory:
vi /etc/sysctl.conf
Add the following content to the sysctl.conf file in the /etc/ directory:
# for oceanbase
## Modify the limit on the asynchronous I/O in the kernel.
fs.aio-max-nr=1048576
## Optimize the network.
net.core.somaxconn = 2048
net.core.netdev_max_backlog = 10000
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.ip_local_port_range = 3500 65535
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_slow_start_after_idle=0
vm.swappiness = 0
vm.min_free_kbytes = 2097152
# Specify the data directory of OceanBase Database.
kernel.core_pattern = /data/core-%e-%p-%t
Note that /data in the kernel.core_pattern field indicates the data directory of OceanBase Database. If you modify the configuration file for a test, you can set fs.aio-max-nr=1048576.
Load parameters
Run the following command to load the parameters and make the changes take effect.
sysctl -p
Disable the firewall and SELinux
If you are an individual user, you can skip this section. If you are an enterprise user, we recommend that you disable the firewall and SELinux.
Note
You must perform this operation on each server.
Disable the firewall
Run the following commands in the shown sequence to disable the firewall.
systemctl disable firewalld
systemctl stop firewalld
systemctl status firewalld
Disable SELinux
Run the following command to open the /etc/selinux/config configuration file:
vi /etc/selinux/config
In the /etc/selinux/config file, change the value of SELINUX to disabled.
SELINUX=disabled
Run the following command or restart the server for the change to take effect:
setenforce 0
Run the following command to check whether the change has taken effect:
sestatus
Create a user
If you are an individual user, you can use the root user and skip this section. If you are an enterprise user, we recommend that you create non-root users for the servers to ensure data security. You can create frequently used users as needed.
Note
You must perform this operation on each server.
Perform the following steps to create a user. The following example shows how to create an admin user.
Run the following commands to create the admin user:
useradd -U admin -d /home/admin -s /bin/bash mkdir -p /home/admin sudo chown -R admin:admin /home/adminRun the following command to set a password for the admin user:
passwd adminOptional. Set sudo privileges for the admin user.
Open the
/etc/sudoersfile. Add the following code to/etc/sudoers:## Same thing without a password # %wheel ALL=(ALL) NOPASSWD: ALL admin ALL=(ALL) NOPASSWD: ALLRun the following command to view the directory access privilege:
ls -alThe following output is returned:
drwxr-xr-x 2 admin admin 4096 Feb 9 18:43 drwxr-xr-x 2 admin admin 4096 Feb 9 18:43 log1If the admin user does not have the access privilege, run the following commands as the root user:
chown -R admin:admin /data chown -R admin:admin /redo chown -R admin:admin /home/adminNote that you need to specify your actual mount directories. The
/data,/redo, and/home/admindirectories in the preceding commands are only examples for your reference.