Add nodes

You can add nodes to a cluster that you manage.

Introduction to nodes

OceanBase Database is a single-process software solution. Its process is named observer. Generally, a server that runs an observer process is referred to as a node. A node is not equivalent to a physical server. In OceanBase Database, each observer process is uniquely identified by an IP address and a port number. Typically, a physical or virtual server runs one observer process. As the core process for OceanBase Database, the observer process provides almost all database kernel features, including the SQL engine, storage engine, and transaction engine. The process also provides distributed features such as RPC communication, load balancing, and partition management.

Add a node to a cluster by using a statement

Before you execute a statement to add a node, obtain the OceanBase Database RPM package of the corresponding version. For more information about how to obtain the RPM package for OceanBase Database, contact OceanBase Database Technical Support.

1. Log on to the sys tenant of the cluster as the root user.

2. Optional. Execute the following statements in sequence to add a zone and specify its region and IDC attributes.

   Note

   You can skip this step if you add a node to an existing zone. If you must add a zone and then add a node to it, this step cannot be skipped.

   This section describes how to add a zone named zone4 with the REGION='HANGZHOU',IDC='HZ1' attributes.

   obclient> ALTER SYSTEM ADD ZONE zone4 REGION HANGZHOU;
   
   obclient> ALTER SYSTEM START ZONE zone4;
   
   obclient> ALTER SYSTEM ALTER ZONE zone4 SET REGION='HANGZHOU',IDC='HZ1';
   

3. Log on to the server where the node to be added resides by using SSH and perform a pre-installation check.

   1. Go to the directory that stores the tool.

      [root@hostname /]# cd /root/t-oceanbase-antman/clonescripts
      

   2. Run the following command to start the check.

      We recommend that you use oat-cli. For more information about oat-cli, see OceanBase Database Deployment Guide of the corresponding version.

      [root@hostname clonescripts]# sh precheck.sh -m ob
      

   3. Run the following command to check whether the clocks are synchronized. Make sure that the clock difference between all nodes is within 2s.

      $IP indicates the IP address of another node in the cluster.

      [root@hostname /]# ntpdate $IP
      

4. Install the RPM package of OceanBase Database.

   rpm_dir specifies the directory that stores the RPM package. $rpm_name specifies the name of the RPM package.

   [root@hostname /]# cd $rpm_dir
   
   [root@hostname $rpm_dir]# rpm -i --prefix=/home/admin/oceanbase $rpm_name
   

5. Create the /data/1 and /data/log1 directories on the node to be added, and change the owner of the directories to admin.

   Generally, /data/1 is used as the data directory, and /data/log1 as the log directory.

   [root@xxx /]# mkdir -p /data/1 /data/log1
   [root@xxx /]# cd data
   [root@xxx data]# ll
   total 8
   drwxr-xr-x 2 root root 4096 Nov 18 14:24 1
   drwxr-xr-x 2 root root 4096 Nov 18 14:24 log1
   [root@xxx data]# cd ..
   [root@xxx /]# chown -R admin:admin data
   [root@xxx /]# cd data
   [root@xxx data]# ll
   total 8
   drwxr-xr-x 2 admin admin 4096 Nov 18 14:24 1
   drwxr-xr-x 2 admin admin 4096 Nov 18 14:24 log1
   

6. Initialize the directories of OceanBase Database.

   We recommend that you set the data directory of OceanBase Database on an independent disk and link this directory to the home directory of OceanBase Database by using a soft link.

   [root@xxx admin]# su - admin
   -bash-4.2$ mkdir -p /data/1/$cluster_name/{etc3,sort_dir,sstable,slog}
   -bash-4.2$ mkdir -p /data/log1/$cluster_name/{clog,etc2,ilog,oob_clog}
   -bash-4.2$ mkdir -p /home/admin/oceanbase/store/$cluster_name
   -bash-4.2$ for t in {etc3,sort_dir,sstable,slog};do ln -s /data/1/$cluster_name/$t /home/admin/oceanbase/store/$cluster_name/$t; done
   -bash-4.2$ for t in {clog,etc2,ilog,oob_clog};do ln -s /data/log1/$cluster_name/$t /home/admin/oceanbase/store/$cluster_name/$t; done
   

   $cluster_name indicates the cluster to which the node is to be added.

   Sample code:

   [root@xxx admin]# su - admin
   -bash-4.2$ mkdir -p /data/1/obdemo/{etc3,sort_dir,sstable,slog}
   -bash-4.2$ mkdir -p /data/log1/obdemo/{clog,etc2,ilog,oob_clog}
   -bash-4.2$ mkdir -p /home/admin/oceanbase/store/obdemo
   -bash-4.2$ for t in {etc3,sort_dir,sstable,slog};do ln -s /data/1/obdemo/$t /home/admin/oceanbase/store/obdemo/$t; done
   -bash-4.2$ for t in {clog,etc2,ilog,oob_clog};do ln -s /data/log1/obdemo/$t /home/admin/oceanbase/store/obdemo/$t; done
   

7. Start the node and register it with the cluster.

   1. Set the ulimit parameter for the admin user.

      [admin@hostname oceanbase]$ ulimit -s 10240; ulimit -c unlimited
      

   2. Configure environment variables.

      [admin@hostname oceanbase]$ export LD_LIBRARY_PATH=/home/admin/oceanbase/lib:$LD_LIBRARY_PATH LD_PRELOAD=''
      

   3. Start the node.

      Sample statement:

      [admin@hostname oceanbase]$ /home/admin/oceanbase/bin/observer -i eth0 -P XXXX -p YYYY -z zone1 -d /home/admin/oceanbase/store/obdemo -r 'xxx.xxx.xxx.xxx:xxxx:xxx.xxx.xxx.xxx:xxxx xxx.xxx.xxx.xxx:xxxx:yyyy' -c 20190716 -n obdemo -o "memory_limit_percentage=90,memstore_limit_percentage=60,datafile_disk_percentage=80,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2"
      

      Parameters:

      • -P specifies the RPC port number.

      • -p specifies the port number for direct connection.

      • -z specifies the zone to which the OBServer node is added.

      • -d specifies the storage directory of data.

      • -r specifies the IP address of the node to be started.

      • -c specifies the cluster ID.

      • -n specifies the cluster name.

      • -o specifies the startup parameter.

        For more information about the parameters, see System parameters.

8. Log on to the sys tenant of the cluster as the root user again and add a node to the cluster.

   $IP indicates the IP address of the node to be added. $PORT indicates the RPC port of the node. The default port is 2882.

   obclient> ALTER SYSTEM ADD SERVER '$IP:$PORT' ZONE 'zone4';
   

   This statement adds the node to the service list. Only nodes in the service list can provide services.

   Then, execute the following statement to check whether the node you added exists in the list. If yes, the node is successfully added to the cluster.

   obclient> SELECT * FROM oceanbase.DBA_OB_SERVERS;
   

Add a node to a cluster by using OBD

1. After you log on to the server where OBD is installed, write a new configuration file and run the following command to deploy a cluster:

   obd cluster deploy <deploy_name> -c <deploy_config_path>
   

   In the preceding command, the deploy_name parameter specifies the cluster name, which can be understood as the alias of the configuration file. deploy_config_path specifies the configuration file name.

   This command will check whether the directory files to which home_path and data_dir point are empty. If no, an error is returned. In this case, you can add the -f option to force clear the directory files.

   After you run the obd cluster deploy command, OBD checks the target server for the OceanBase Database installation package. If the package does not exist, OBD automatically obtains the package from the YUM repository.

2. Perform the following steps to add the content in the configuration file of the newly deployed cluster to the corresponding positions in the configuration file of the original cluster.

   1. Run the following command to view the directory of the configuration file:

      obd cluster list
      

   2. Open the configuration file of the original cluster and add the configuration of the newly deployed cluster.

      vim .obd/cluster/<deploy_name>/config.yaml
      

3. Run the following command to start the cluster again:

   obd cluster start <deploy_name>
   

4. Connect to OceanBase Database and execute the following statements in sequence to add a zone and specify its region and IDC attributes.

   Note

   You can skip this step if you add a node to an existing zone. If you must add a zone and then add a node to it, this step cannot be skipped.

   This section describes how to add a zone named zone4 with the REGION='HANGZHOU',IDC='HZ1' attributes.

   obclient> ALTER SYSTEM ADD ZONE zone4 REGION HANGZHOU;
   obclient> ALTER SYSTEM START ZONE zone4;
   obclient> ALTER SYSTEM ALTER ZONE zone4 SET REGION='HANGZHOU',IDC='HZ1';
   

5. Execute the following statement to add a node to the cluster:

   ALTER SYSTEM ADD SERVER 'ip:port' [,'ip:port'...] [ZONE='zone'];
   

   This statement adds the node to the service list. Only nodes in the service list can provide services.

   Then, execute the following statement to check whether the node you added exists in the list. If yes, the node is successfully added to the cluster.

   obclient> SELECT * FROM oceanbase.DBA_OB_SERVERS;