Configuration before deployment

Before you deploy an OceanBase cluster, you must check the system environment and configurations.

View resources

If you are an enterprise user, we recommend that you deploy the OceanBase cluster on servers with the same hardware and software configurations. If you are deploying OceanBase Database on a single server, you can skip this section.

OceanBase Database supports operating systems such as Red Hat Enterprise Linux Server 7.X, CentOS 7.X, and Anolis OS 8.X. For more information, see Software and hardware requirements.

Note

The configuration files mentioned in this topic refer to files that are configured when you use OBD to deploy OceanBase Database. For more information, see Example configuration files.

View information about the operating system

Red Hat Enterprise Linux Server 7.X

Run the following command:

cat /etc/redhat-release

The following example shows the returned result of this command:

 Red Hat Enterprise Linux Server release 7.2 (Maipo)

CentOS 7.X

Run the following command:

cat /etc/redhat-release

The following example shows the returned result of this command:

CentOS Linux release 7.2.1511 (Core)

Anolis OS 8.X

Run the following command:

cat /etc/os-release

The following example shows the returned result of this command:

NAME="Anolis OS"
VERSION="8.2"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="8.2"
PLATFORM_ID="platform:an8"
PRETTY_NAME="Anolis OS 8.2"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.cn/"

Unbutu

Run the following command:

cat /etc/os-release

The following example shows the returned result of this command:

NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

Debian9

Run the following command:

cat /etc/os-release

The following example shows the returned result of this command:

PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
VERSION_CODENAME=stretch
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

View the kernel information

Run the following command:

uname -r

OceanBase Database requires that the OS kernel be of V3.10.0 or later.

View the memory information

Run the following command:

free -g

If the memory space available is less than the value of the memory_limit parameter, clear the cache or change the value of the memory_limitto less than the available memory space.

Run the following command to clear the cache:

echo 3 > /proc/sys/vm/drop_caches

View the disk information

Run the following command:

df -h

Check the configuration file to ensure that the corresponding disks are mounted to the data_dir, redo_dir, and home_path directories, that the data_dir and redo_dir directories are empty, and that the usage of the data disk is less than 4%.

View the NIC name

You need to specify the NIC for the devname parameter. For example:

# Please set devname as the network adaptor's name whose ip is  in the setting of severs.
# if set severs as "127.0.0.1", please set devname as "lo"
# if current ip is 192.168.1.10, and the ip's network adaptor's name is "eth0", please use "eth0"
devname: eth0

Run the following command to view the NIC name:

ifconfig

Notice

To run the ifconfig command, you must install the net-tools dependencies.

Enable password-free SSH logon

Terms

• Central control server

  The server that stores the installation package and cluster configuration information of OceanBase Database.

• Target server

  The server where you want to install OceanBase Database.

Prerequisites

You have the root user privileges on the target server.

Procedure

You can take the following steps to set up password-free logon through SSH:

1. Run the following command on the central control server to check whether the key exists:

   ls ~/.ssh/id_rsa.pub
   

   If yes, you do not need to generate a new private SSH key.

2. Optional. Run the following command on the central control server to generate the public and private SSH keys:

   ssh-keygen -t rsa
   

3. Run the following command on the central control server to log on to the target server:

   ssh-copy-id  root@<server_ip>
   

Appendix: Reference scripts

We recommend that you use the following script. To run this script, you must have the sudo privilege.

#!/usr/bin/bash

   SERVERS=("root@172.19.x.x root@172.19.x.x" "root@172.19.x.x" "root@172.19.x.x")
   PASSWORD="test@"
   keygen() {
   yum -y install expect
   expect -c "
      spawn ssh-keygen -t rsa
      expect {
         *(/root/.ssh/id_rsa):* { send -- \r;exp_continue}
         *(y/n)* { send -- y\r;exp_continue}
         *Enter* { send -- \r;exp_continue}
         *(y/n)* { send -- y\r;exp_continue}
         *Enter* { send -- \r;exp_continue}
         eof {exit 0}
      }
      expect eof
   "
   }
   copy(){
   expect -c "
      set timeout -1
      spawn ssh-copy-id $1
      expect {
         *(yes/no)* { send -- yes\r; exp_continue }
         *password:* { send -- $PASSWORD\r; exp_continue}
         eof {exit 0}
      }
      expect eof
   "
   }
   ssh_copy_id_to_all(){
   keygen ;
   for host in ${SERVERS[@]}
   do
         copy $host
   done
   }
   ssh_copy_id_to_all

Notice

You must replace the list of hosts and the password in the script with the actual list of servers and password.

Configure the clock source

If you deploy OceanBase Database in a cluster, you must ensure that the time of all servers in the cluster is synchronized. Otherwise, the cluster cannot be started and exceptions will be thrown. If you have already configured the Network Time Protocol (NTP) clock source, no need to do it again.

The clocks of servers in the OceanBase cluster must be synchronized. Otherwise, the OceanBase cluster may not start or will encounter operational exceptions. A physical server is considered synchronous with the clock source server when the clock error between them is 50 ms or lower. The tolerable clock error of an OceanBase cluster cannot exceed 100 ms. When the clock error exceeds 100 ms, the cluster may become leaderless. You can synchronize the clock and restart the OceanBase cluster to restore it to normal.

Prerequisites

You have the root user privileges on all OBServers.

Procedure

Perform the following steps to configure the NTP clock source.

1. Run the following command on each server to install NTP:

   yum install ntp ntpdate -y
   

2. Run the following command to check the NTP connection to the server:

   When the time difference between the server and the NTP server is within 50 ms, the clocks are considered synchronized.

   [root@centos-01 /root]#ntpq -4p
   remote refid st t when poll reach delay offset jitter
   ==============================================================================
   *time6.aliyun.co xxx.xxx.xxx.xxx 2 u 712 1024 377 21.951 4.253 4.208
   

3. (Optional) Enable the NTP server.

   If you already have an NTP server, skip this step. The following example describes how to enable time synchronization by taking the server xxx.xxx.xxx.xxx as the NTP server.

   1. Edit the configuration file.

      Log on to the NTP server as a root user and run the following command to open the configuration file:

      vi /etc/ntp.conf
      

      Add the following content to the configuration file:

      server <yout_ntp_server_ip>
      

      Save and close the file.

   2. Run the following command to restart the NTP server:

      service ntpd restart
      

   3. Run the following command to check whether the NTP service is enabled:

      ps -ef | grep -i ntpd
      

   4. Run the following command to enable the NTP service upon startup.

      chkconfig ntpd on
      

4. Enable the NTP service on an OBServer.

   Notice

   You can enable the NTP service on an OBServer five minutes later after the NTP server is started. Otherwise, the system prompts "no server suitable for synchronization found".

   1. Log on to OceanBase Database as the root user.

   2. Run the following command to stop the NTP service:

      service ntpd stop
      

   3. Run the following command to synchronize the time with the NTP server:

      ntpdate xxx.xxx.xxx.xxx
      

   4. Edit the configuration file.

      Run the following command to open the configuration file:

      vi /etc/ntp.conf
      

      Add the following content to the configuration file:

      server <yout_ntp_server_ip>
      

   5. Run the following command to start the NTP service:

      /etc/init.d/ntpd start
      

5. Verify whether the configuration is successful:

   Run the following command:

   ntpstat
   

   The following output is returned:

   synchronised to NTP server (xxx.xxx.xxx.xxx) at stratum 3
   time correct to within 2 ms
   polling server every 64 s
   

   Run the following command:

   timedatectl
   

   The following output is returned:

         Local time: Thu 2021-04-22 11:02:32 CST
     Universal time: Thu 2021-04-22 03:02:32 UTC
           RTC time: Thu 2021-04-22 11:02:32
          Time zone: Asia/Shanghai (CST, +0800)
        NTP enabled: yes
   NTP synchronized: yes
    RTC in local TZ: yes
         DST active: n/a
   

   If the command output is as expected, the NTP service is running.

Disk planning

The running of an OBServer relies on the data disk, clog disk, and installation disk where OceanBase Database is installed. If you are an individual user, you can put all the data on a single disk and skip this section. If you are an enterprise user, you must separately mount your data to three disks.

If you do not have three disks for a server, or you are using Redundant Array of Independent Disks (RAID), you need to partition the disk or disk array into logical volumes. We recommend that you plan the following disks:

• Data disk

  The data disk is mounted to the data_dir directory, and is used for storing the baseline data. When you start OceanBase Database for the first time, the ${data_dir}/sstable directory is automatically created. The size of the data disk is determined by the datafile_disk_percentage parameter. The default value of the datafile_disk_percentage parameter is 95. Once you have created the data_dir directory, you cannot resize it. You can scale OceanBase Database out or in by adding or removing servers. OceanBase Database does not support disks that are added directly to a server.

• Clog disk

  The clog disk is mounted to the redo_dir directory. We recommend that you set the size of the clog disk to at least 3 times that of the memory for the OceanBase Database. The clog disk contains files of a fixed size. These files are located in the ${redo_dir}/{clog,ilog,slog} installation directory, and are created and cleaned up as needed. When the total size of clog files reaches 80% of the total disk capacity, automatic cleanup is triggered. However, the clog files can be cleaned up only when the corresponding memory data has been compacted and merged with the baseline data. For a given size of memory data, the size of clog files is approximately three times larger. Therefore, the maximum space required for the clog disk is directly proportional to the total amount of data after two major compactions. You can use the empirical formula for evaluation: Size of clog files = Maximum memory space for incremental data × 3 or 4.

• OceanBase Database installation disk

  The disk for the installation of OceanBase Database is mounted to the home_path directory. We recommend that you reserve at least 200 GB to save log files for 7 or more days. The RPM package of OceanBase Database is located in the ${home_path} directory. The baseline data files and clog files are mapped to the preceding data disk and clog disk through soft links. The runtime logs of OceanBase Database are located in the ${home_path}/log directory. The number of runtime logs will keep increasing because OceanBase Database cannot automatically delete them. Therefore, you need to regularly delete the runtime logs.

After you plan the disks, run the following command to view the disk information:

df -h

The following output is returned:

Filesystem      Size  Used Avail Use% Mounted on
devtmpfs         31G     0   31G   0% /dev
tmpfs            31G     0   31G   0% /dev/shm
tmpfs            31G  516K   31G   1% /run
tmpfs            31G     0   31G   0% /sys/fs/cgroup
/dev/vda1       493G  171G  302G  37% /
tmpfs           6.2G     0  6.2G   0% /run/user/0
/dev/vdd1     984G    77M  934G  1%  /data
/dev/vdc1     196G     61M   186G  1%  /redo
/dev/vdb1     492G    73M   467G  1%  /home/admin/oceanbase

Result description:

• The /data string indicates the information of the data disk, which is 1 TB in size.

• The /redo string indicates the information of the log disk where the REDO logs are stored.

• The /home/admin/oceanbase string indicates the information of the installation disk where the binary files and runtime logs of OceanBase Database are stored.

Configure the limits.conf file

You can configure the limits.conf file to limit the number of processes. If you are an individual user, you can skip this section. If you are an enterprise user, you must configure the limits.conf file.

You can modify the limit on resources by using the following two methods:

• Modify the limit at the session level when OceanBase Database is started.

• Modify the limit globally by using the limits.conf file in the /etc/security/ directory.

OceanBase Database processes can be limited in terms of the following parameters: Stack, Open Files, and Core File Size. Stack refers to the maximum stack size. Open Files refers to the maximum number of file handles. Core File Size refers to the maximum size of core files.

Modify parameters

Note

You must perform this operation on each server.

The following example shows how set the value of the session-level Stack parameter to unlimited, the Open Files parameter to655350, and the Core File Size parameter to unlimited.

Run the following command to open the limits.conf file in the /etc/security/ directory:

vi /etc/security/limits.conf

Add the following content to the /limits.conf file in the /etc/security/ directory:

root soft nofile 655350
root hard nofile 655350
* soft nofile 655350
* hard nofile 655350
* soft stack 20480
* hard stack 20480
* soft nproc 655360
* hard nproc 655360
* soft core unlimited
* hard core unlimited

View parameters

Log off from the session and log on again. Run the following command to view whether the parameters have taken effect:

ulimit -a

Configure the sysctl.conf file

To improve the Linux performance, and ensure the normal running of OceanBase Database, you need to modify the parameters in the sysctl.conf file of each server before the installation of OceanBase Database. If you are an individual user, you can skip this section. If you are an enterprise user, you must configure the sysctl.conf file.

Note

You must perform this operation on each server.

Modify parameters

Run the following command to open the sysctl.conf file in the /etc/ directory:

vi /etc/sysctl.conf

Add the following content to the sysctl.conf file in the /etc/ directory:

# for oceanbase
## Modify the limit on the asynchronous I/O in the kernel.
fs.aio-max-nr=1048576

## Optimize the network.
net.core.somaxconn = 2048
net.core.netdev_max_backlog = 10000
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

net.ipv4.ip_local_port_range = 3500 65535
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_slow_start_after_idle=0

vm.swappiness = 0
vm.min_free_kbytes = 2097152

# Specify the data directory of OceanBase Database.
kernel.core_pattern = /data/core-%e-%p-%t

Note that /data in the kernel.core_pattern field indicates the data directory of OceanBase Database. If you modify the configuration file for a test, you can set fs.aio-max-nr=1048576.

Load parameters

Run the following command to load the parameters and make the changes take effect.

sysctl -p

Disable the firewall and SELinux

If you are an individual user, you can skip this section. If you are an enterprise user, we recommend that you disable the firewall and SELinux.

Note

You must perform this operation on each server.

Disable the firewall

Run the following commands in the shown sequence to disable the firewall.

systemctl disable firewalld
systemctl stop firewalld
systemctl status firewalld

Disable SELinux

Run the following command to open the /etc/selinux/config configuration file:

vi /etc/selinux/config

In the /etc/selinux/config file, change the value of SELINUX to disabled.

SELINUX=disabled

Run the following command or restart the server for the change to take effect:

setenforce 0

Run the following command to check whether the change has taken effect:

sestatus

Create a user

If you are an individual user, you can use the root user and skip this section. If you are an enterprise user, we recommend that you create non-root users for the servers to ensure data security. You can create frequently used users as needed.

Note

You must perform this operation on each server.

Perform the following steps to create a user. The following example shows how to create an admin user.

• Run the following commands to create the admin user:

  useradd -U admin -d /home/admin -s /bin/bash
  mkdir -p /home/admin
  sudo chown -R admin:admin /home/admin
  

• Run the following command to set a password for the admin user:

  passwd admin
  

• Optional. Set sudo privileges for the admin user.

  Open the /etc/sudoers file. Add the following code to /etc/sudoers:

  ## Same thing without a password
  # %wheel        ALL=(ALL)       NOPASSWD: ALL
  admin       ALL=(ALL)       NOPASSWD: ALL
  

• Run the following command to view the directory access privilege:

  ls -al
  

  The following output is returned:

  drwxr-xr-x 2 admin admin 4096 Feb 9 18:43
  drwxr-xr-x 2 admin admin 4096 Feb 9 18:43 log1
  

  If the admin user does not have the access privilege, run the following commands as the root user:

  chown -R admin:admin /data
  chown -R admin:admin /redo
  chown -R admin:admin /home/admin
  

  Note that you need to specify your actual mount directories. The /data, /redo, and /home/admin directories in the preceding commands are only examples for your reference.