Deploy a three-replica OceanBase cluster

2023-08-01 06:02:28  Updated

A three-replica OceanBase cluster requires at least three hosts respectively located in three zones. OceanBase Database is a piece of single-process software. You only need to install the oceanbase-xxx.rpm and obproxy-xxx.rpm packages.

This topic describes how to install a three-replica OceanBase cluster through RPM packages.

Considerations

If you want to implement resource isolation for your OceanBase database, configure cgroups before the deployment.

For more information about resource isolation and cgroup, see Resource isolation and Configure cgroups.

Prerequisites

Before you install OceanBase Database, make sure that the following prerequisites are satisfied:

  • The OBServer node has been configured. For more information about how to configure an OBServer node, see Configure servers.

  • The deployment environment has been configured. For more information about how to configure the deployment environment, see Configure the deployment environment.

Procedure

Step 1: Install the RPM package

  1. Install the RPM package for OceanBase Database.

    $rpm_dir specifies the directory in which the RPM package is stored. $rpm_name specifies the name of the RPM package.

    [root@xxx /]# cd $rpm_dir
    [root@xxx $rpm_dir]# rpm -ivh $rpm_name
    

    Note

    The OceanBase Database software will be installed under the /home/admin/oceanbase directory.

    Example:

    [root@xxx admin]# rpm -ivh oceanbase-*-20220418212020.el7.x86_64.rpm
    Preparing...                          ################################# [100%]
    Updating / installing...
       1:oceanbase-*-20220418212020.################################# [100%]
    
  2. (Optional) Install OceanBase Client (OBClient).

OBClient is a command-line tool designed exclusively for OceanBase Database. It enables connectivity to both MySQL and Oracle tenants. If you only need to connect to the MySQL tenant, you can also use the MySQL client (mysql) to access OceanBase Database.

Note

In order to use versions of obclient before V2.2.1, install libobclient due to its dependency. To obtain the OBClient RPM package and the libobclient RPM package, contact OceanBase Technical Support.

Example:

[root@xxx /home/admin/rpm]# rpm -ivh obclient-2.2.1-20221122151945.el7.alios7.x86_64.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:obclient-2.2.1-20221122151945.el7################################# [100%]

##Check whether the installation is successful##
[root@xxx /home/admin/rpm]# which obclient
/usr/bin/obclient

Step 2: Configure directories

  1. Clear the OceanBase Database directory (not required at the first deployment).

    If you need to clear an old OceanBase Database environment or encounter issues during the installation and deployment process that disrupt the environment or generate files that may affect future installations, you can simply delete the existing OceanBase Database directory. The $cluster_name parameter specifies the name of the cluster.

    [root@xxx admin]# su - admin
    -bash-4.2$ kill -9 `pidof observer`
    -bash-4.2$ rm -rf /data/1/$cluster_name
    -bash-4.2$ rm -rf /data/log1/$cluster_name
    -bash-4.2$ rm -rf /home/admin/oceanbase/store/$cluster_name /home/admin/oceanbase/log/* /home/admin/oceanbase/etc/*config*
    -bash-4.2$ ps -ef|grep observer
    

    Example:

    [root@xxx admin]# su - admin
    -bash-4.2$ kill -9 `pidof observer`
    -bash-4.2$ rm -rf /data/1/obdemo
    -bash-4.2$ rm -rf /data/log1/obdemo
    -bash-4.2$ rm -rf /home/admin/oceanbase/store/obdemo /home/admin/oceanbase/log/* /home/admin/oceanbase/etc/*config*
    -bash-4.2$ ps -ef|grep observer
    
  2. Initialize the OceanBase Database directory.

    We recommend that you specify the data directory of OceanBase Database to an independent disk and link this directory to the home directory of OceanBase Database by using a soft link. $cluster_name specifies the cluster name.

    [root@xxx admin]# su - admin
    -bash-4.2$ mkdir -p /data/1/$cluster_name/{etc3,sort_dir,sstable,slog}
    -bash-4.2$ mkdir -p /data/log1/$cluster_name/{clog,etc2,ilog,oob_clog}
    -bash-4.2$ mkdir -p /home/admin/oceanbase/store/$cluster_name
    -bash-4.2$ for t in {etc3,sort_dir,sstable,slog};do ln -s /data/1/$cluster_name/$t /home/admin/oceanbase/store/$cluster_name/$t; done
    -bash-4.2$ for t in {clog,etc2,ilog,oob_clog};do ln -s /data/log1/$cluster_name/$t /home/admin/oceanbase/store/$cluster_name/$t; done
    

    Example:

    [root@xxx admin]# su - admin
    -bash-4.2$ mkdir -p /data/1/obdemo/{etc3,sort_dir,sstable,slog}
    -bash-4.2$ mkdir -p /data/log1/obdemo/{clog,etc2,ilog,oob_clog}
    -bash-4.2$ mkdir -p /home/admin/oceanbase/store/obdemo
    -bash-4.2$ for t in {etc3,sort_dir,sstable,slog};do ln -s /data/1/obdemo/$t /home/admin/oceanbase/store/obdemo/$t; done
    -bash-4.2$ for t in {clog,etc2,ilog,oob_clog};do ln -s /data/log1/obdemo/$t /home/admin/oceanbase/store/obdemo/$t; done
    

Note

The obdemo directory is named after the cluster and can be modified. It is required when the process starts.

The following sample code shows how to verify the result:

-bash-4.2$ cd /home/admin/oceanbase
-bash-4.2$ tree store/
store/
`-- obdemo
 |-- clog -> /data/log1/obdemo/clog
 |-- etc2 -> /data/log1/obdemo/etc2
 |-- etc3 -> /data/1/obdemo/etc3
 |-- ilog -> /data/log1/obdemo/ilog
 |-- oob_clog -> /data/log1/obdemo/oob_clog
 |-- slog -> /data/1/obdemo/slog
 |-- sort_dir -> /data/1/obdemo/sort_dir
 `-- sstable -> /data/1/obdemo/sstable

9 directories, 0 files

Step 3: Initialize the OceanBase cluster

Note

The IP addresses in the sample code are for reference only. You need to enter the actual server IP address during deployment.

  1. Start the observer process on the nodes.

    Start the observer process as the admin user on each node.

    Notice

    With three replicas, the startup parameters are different for each node. When you start the observer process, you only need to specify three or more servers that run Root Service, instead of all servers. You can add servers after the cluster is created.

    The parameters specify the data file size and memory size to accurately control the resource usage of OBServer nodes when resources are insufficient in certain environments. The following table describes these parameters:

    Parameter Description
    -i The NIC name. You can use the ifconfig command to view the NIC name.
    -p The service port number, which is usually set to 2881.
    -P The RPC port number, which is usually set to 2882.
    -n The name of the cluster. It can be modified. Cluster names must be unique.
    -z The zone where the started observer process belongs.
    -d The primary directory of the cluster. Do not modify other fields except for the cluster name obdemo.
    -c The cluster ID. It is a group of digits and can be modified. Cluster IDs must be unique.
    -l The log level.
    -r The Root Service list in the format of ip:2882:2881. Multiple items are separated with semicolons (;) to indicate Root Service information. This parameter is consistent on the three servers.
    -o The cluster startup parameters that need to be specified as needed.
    • system_memory specifies the internal reserved memory of OceanBase Database, which is 30 GB by default. You can adjust this parameter to a smaller value in the case of insufficient memory on the server. The impact is that memory may be insufficient during performance tests.
    • datafile_size specifies the size of the SSTable data file of OceanBase Database, which is determined based on the available space of /data/1/. You no longer need to specify the size again once specified in this step. We recommend that you specify the data file size to at least 100 GB and reserve some spaces.
      Note
      If /data/1 and /data/log1 belong to the same file system (share the same disk), ensure that the space reserved for /data/log1 is two to four times the size of the memory.
    • config_additional_dir specifies the redundancy directory for the parameter file.

    Example:

    zone1:

    [root@xxx admin]# su - admin
    -bash-4.2$ cd /home/admin/oceanbase && /home/admin/oceanbase/bin/observer -i eth0 -P 2882 -p 2881 -z zone1 -d /home/admin/oceanbase/store/obdemo -r '10.10.10.1:2882:2881;10.10.10.2:2882:2881;10.10.10.3:2882:2881' -c 10001 -n obdemo -o "system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2"
    devname: eth0
    rpc port: 2882
    mysql port: 2881
    zone: zone1
    data_dir: /home/admin/oceanbase/store/obdemo
    rs list: 10.10.10.1:2882:2881;10.10.10.2:2882:2881;10.10.10.3:2882:2881
    cluster id: 10001
    appname: obdemo
    optstr: system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2
    

    zone2:

    [root@xxx admin]# su - admin
    -bash-4.2$ cd /home/admin/oceanbase && /home/admin/oceanbase/bin/observer -i eth0 -P 2882 -p 2881 -z zone2 -d /home/admin/oceanbase/store/obdemo -r '10.10.10.1:2882:2881;10.10.10.2:2882:2881;10.10.10.3:2882:2881' -c 10001 -n obdemo -o "system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2"
    devname: eth0
    rpc port: 2882
    mysql port: 2881
    zone: zone2
    data_dir: /home/admin/oceanbase/store/obdemo
    rs list: 10.10.10.1:2882:2881;10.10.10.2:2882:2881;10.10.10.3:2882:2881
    cluster id: 10001
    appname: obdemo
    optstr: system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2
    

    zone3:

    [root@xxx admin]# su - admin
    -bash-4.2$ cd /home/admin/oceanbase && /home/admin/oceanbase/bin/observer -i eth0 -P 2882 -p 2881 -z zone3 -d /home/admin/oceanbase/store/obdemo -r '10.10.10.1:2882:2881;10.10.10.2:2882:2881;10.10.10.3:2882:2881' -c 10001 -n obdemo -o "system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2"
    devname: eth0
    rpc port: 2882
    mysql port: 2881
    zone: zone3
    data_dir: /home/admin/oceanbase/store/obdemo
    rs list: 10.10.10.1:2882:2881;10.10.10.2:2882:2881;10.10.10.3:2882:2881
    cluster id: 10001
    appname: obdemo
    optstr: system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2
    

    You can use the following commands to check whether the observer processhas started successfully:

    • netstat -ntlp: If the 2881 and 2882 ports are being listened to, then the process has started successfully.
    • ps -ef|grep observer: This command can return the observer process information.

    Example:

    -bash-4.2$ netstat -ntlp
    (Not all processes could be identified, non-owned process info
    will not be shown, you would have to be root to see it all.)
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
    tcp        0      0 0.0.0.0:2881            0.0.0.0:*               LISTEN      115745/observer
    tcp        0      0 0.0.0.0:2882            0.0.0.0:*               LISTEN      115745/observer
    ...        ...    ...                       ...                     ...         ...
    
    -bash-4.2$ ps -ef|grep observer
    admin     91335      0 40 16:18 ?        00:00:17 /home/admin/oceanbase/bin/observer -i eth0 -P 2882 -p 2881 -z zone1 -d /home/admin/oceanbase/store/obdemo -r 10.10.10.1:2882:2881;10.10.10.2:2882:2881 -c 10001 -n obdemo -o system_memory=30G,datafile_size=100G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2
    
  2. Perform the bootstrap operation on the cluster.

    Connect to any node by using the obclient command. The password is empty.

    [root@xxx admin]# obclient -h127.1 -uroot -P2881 -p
    Enter password:
    
    obclient> SET SESSION ob_query_timeout=1000000000;
    Query OK, 0 rows affected
    
    obclient> ALTER SYSTEM BOOTSTRAP ZONE 'zone1' SERVER '10.10.10.1:2882',ZONE 'zone2' SERVER '10.10.10.2:2882',ZONE 'zone3' SERVER '10.10.10.3:2882';
    Query OK, 0 rows affected
    

    Notice

    If an error is reported in this step, the reason may be that the startup parameter of the observer process is incorrect on the three nodes, the privileges on the directories related to the observer process are incorrect, the space of the log directory does not meet the required proportion, the time is not synchronous on the three nodes, or the node memory resources are insufficient. The log directory issue occurs because the log directory shares the same upper-level directory with the data directory, resulting in space occupation by the data directory. Check these problems and then clear the OceanBase Database directory. For more information, see Clear the OceanBase Database directory (not required at the first deployment).

  3. Verify whether the cluster is initialized.

    After you perform the bootstrap operation and execute the SHOW DATABASES statement, if oceanbase appears in the database list, the cluster has been initialized.

    obclient> SHOW DATABASES;
    +--------------------+
    | Database           |
    +--------------------+
    | oceanbase          |
    | information_schema |
    | mysql              |
    | SYS                |
    | LBACSYS            |
    | ORAAUDITOR         |
    | test               |
    +--------------------+
    7 rows in set
    
  4. Change the password.

    By default, the password of the root user under the sys tenant is empty. After successful initialization, you need to change the password.

    obclient> ALTER USER root IDENTIFIED BY '******';
    Query OK, 0 rows affected
    

More information

Contact Us