Add a node

You can add a node to an OceanBase cluster for elastic scaling or deployment adjustment.

• Elastic scaling: After a scale-out, more nodes are available in the zone to accommodate resource units. Then, you can perform a series of operations, such as migrating a unit, modifying the UNIT_NUM parameter for a tenant, and creating a tenant.

• Deployment adjustment: You can adjust the deployment mode of a cluster from three IDCs in the same region to five IDCs across three regions, and migrate a zone from one IDC to another when an IDC is to be demolished. In this scenario, you must first add a zone, then add nodes to the zone, and finally modify the locality of the tenant.

OceanBase Database outperforms conventional databases in scaling due to its distributed architecture. A conventional database stores its data on a single server, but OceanBase Database scatters its data on different servers, enhancing scalability. When the capacity of a cluster or tenant is insufficient, you only need to add more nodes. In this way, the cluster can accommodate more tenants and each tenant can accommodate more data and business traffic. When the capacity of a cluster or tenant is in surplus, you can remove nodes to reduce the cost.

The multi-replica deployment capability of OceanBase Database contributes to its architectural benefits against conventional databases. The multi-replica deployment capability is the basis for OceanBase Database to achieve lossless disaster recovery in different levels, including the server level, IDC level, and city level. OceanBase Database allows you to flexibly adjust the cluster deployment architecture to adapt to the technical evolution of business systems.

Prerequisites

• All OBServer nodes to be added are properly configured. For more information, see Configure servers and Configure the deployment environment.

• OceanBase Database is installed on all OBServer nodes to be added. You can download the installation package from the Download Center or request the installation package from OceanBase Technical Support.

Procedure

1. (Optional) In the case where you must first add a zone before you can add nodes to the zone, make sure that the zone has been added. For more information about how to add a zone, see Add a zone.

2. Log on to an OBServer node to be added by using SSH and perform a precheck.

   1. Log on as the root user to each OBServer node to be added one by one.

   2. Run the following command to start the check.

      [root@xxx /]# cd /root/t-oceanbase-antman/clonescripts
      
      [root@xxx clonescripts]# sh precheck.sh -m ob
      

   3. Run the following command to check whether the clocks are synchronized. Make sure that the clock difference between all nodes is within 100 ms.

      Note

      A physical server is considered to be synchronous with the clock source server when the clock offset between them is 50 ms or lower. The allowed clock offset of an OceanBase cluster cannot exceed 100 ms. When the clock offset exceeds 100 ms, no leader is available. For more information about how to configure clock synchronization, see Configure the clock source.

      [root@xxx clonescripts]#  ntpdate $IP
      

      In the command, $IP indicates the IP address of another node in the cluster.

3. Log on to an OBServer node to be added by using SSH and install the RPM package of OceanBase Database.

   Before you add OBServer nodes, you must install the OceanBase Database software on these nodes.

   The command is as follows:

   [root@xxx /]#cd $rpm_dir
   
   [root@xxx $rpm_dir]#rpm -i --prefix=/home/admin/oceanbase $rpm_name
   

   rpm_dir specifies the directory that stores the RPM package. $rpm_name specifies the name of the RPM package.

4. Create the /data/1 and /data/log1 directories on each OBServer node to be added and change the owner of the directories to admin.

   Generally, /data/1 is used as the data directory, and /data/log1 as the log directory.

   [root@xxx /]# mkdir -p /data/1 /data/log1
   
   [root@xxx /]# cd data
   
   [root@xxx data]# ll
   total 8
   drwxr-xr-x 2 root root 4096 Nov 18 14:24 1
   drwxr-xr-x 2 root root 4096 Nov 18 14:24 log1
   
   [root@xxx data]# cd ..
   
   [root@xxx /]#chown -R admin:admin data
   
   [root@xxx /]# cd data
   
   [root@xxx data]# ll
   total 8
   drwxr-xr-x 2 admin admin 4096 Nov 18 14:24 1
   drwxr-xr-x 2 admin admin 4096 Nov 18 14:24 log1
   

5. Initialize the directories of OceanBase Database.

   We recommend that you set the data directory of OceanBase Database on an independent disk and link this directory to the home directory of OceanBase Database by using a soft link.

   [root@xxx admin]#su - admin
   -bash-4.2$ mkdir -p /data/1/$cluster_name/{etc3,sort_dir,sstable,slog}
   -bash-4.2$ mkdir -p /data/log1/$cluster_name/{clog,etc2,ilog,oob_clog}
   -bash-4.2$ mkdir -p /home/admin/oceanbase/store/$cluster_name
   -bash-4.2$ for t in {etc3,sort_dir,sstable,slog};do ln -s /data/1/$cluster_name/$t /home/admin/oceanbase/store/$cluster_name/$t; done
   -bash-4.2$ for t in {clog,etc2,ilog,oob_clog};do ln -s /data/log1/$cluster_name/$t /home/admin/oceanbase/store/$cluster_name/$t; done
   

   cluster_name specifies the name of the OceanBase cluster to which OBServer nodes are to be added.

   Here is an example:

   [root@xxx admin]#su - admin
   -bash-4.2$ mkdir -p /data/1/obdemo/{etc3,sort_dir,sstable,slog}
   -bash-4.2$ mkdir -p /data/log1/obdemo/{clog,etc2,ilog,oob_clog}
   -bash-4.2$ mkdir -p /home/admin/oceanbase/store/obdemo
   -bash-4.2$ for t in {etc3,sort_dir,sstable,slog};do ln -s /data/1/obdemo/$t /home/admin/oceanbase/store/obdemo/$t; done
   -bash-4.2$ for t in {clog,etc2,ilog,oob_clog};do ln -s /data/log1/obdemo/$t /home/admin/oceanbase/store/obdemo/$t; done
   

6. Start the observer process.

   1. Log on as the admin user to each OBServer node to be added one by one.

   2. Set the ulimit parameter for the admin user.

      [admin@xxx /]$ulimit -s 10240; ulimit -c unlimited
      

   3. Configure environment variables.

      [admin@xxx /]$ export LD_LIBRARY_PATH=/home/admin/oceanbase/lib:$LD_LIBRARY_PATH LD_PRELOAD=''
      

   4. Run the following command to start the observer process.

      [admin@xxx /]$ cd /home/admin/oceanbase
      
      [admin@xxx oceanbase]$ ./bin/observer -i eth0 -c $cluster_id -p xxxx -P xxxx -z $zone_name -n $cluster_name -d ‘/home/admin/oceanbase/store/$cluster_name’ -r 'SVR_IP:SVR_PORT:SQL_PORT;SVR_IP:SVR_PORT:SQL_PORT;SVR_IP:SVR_PORT:SQL_PORT' -l WARN -o 'memory_limit=100GB,datafile_disk_percentage=85,config_additional_dir=/data/1/cluster_name/etc3;/data/log1/cluster_name/etc2'
      

      • -i: the NIC name, which varies with the server environment.

      • -c: the cluster ID. You can execute the SHOW PARAMETERS LIKE 'cluster_id'; statement to obtain the cluster ID.

      • -p: the SQL port number, which is generally 2881. We recommend that you do not modify the SQL port number unless necessary.

      • -P: the RPC port number, which is generally 2882. We recommend that you do not modify the RPC port number unless necessary.

      • -n: the name of the cluster. In this example, the cluster is named obdemo.

      • -z: the zone to which OBServer nodes are to be added.

      • -d: the data directory.

      • -r: the Root Service address list of the OceanBase cluster to which OBServer nodes are to be added.

      • -l: the level of logs to be printed. The value is WARN in this example, indicating that logs of the WARNING level are to be printed.

        For more information about the log levels in OceanBase Database, see Log generation levels.

      • -o: the startup parameters of the cluster. When the -o parameter is used, note that:

        • The parameters are case-insensitive. However, you are recommended to set them based on the names in observer.config.bin.

        • A parameter name cannot contain the following special characters: spaces, \r, \n, and \t.

        • An equal sign (=) is required between the name and the value of a parameter.

        • Separate the parameters with commas (,).

        In the example, memory_limit=100GB indicates that the maximum memory for starting the process is 100 GB. datafile_disk_percentage=85 indicates that the data disk usage is 85%.

      Here is an example:

      [admin@xxx /]$ cd /home/admin/oceanbase
      
      [admin@xxx oceanbase]$ ./bin/observer -i eth0 -c 20221216 -p 2881 -P 2882 -z 'zone4' -n 'obdemo' -d '/home/admin/oceanbase/store/obdemo' -r '10.10.10.1:2882:2881;10.10.10.2:2882:2881;10.10.10.3:2882:2881' -l WARN -o 'memory_limit=100GB,datafile_disk_percentage=85'
      

7. Add a node to the cluster.

   1. Log on to the sys tenant of the cluster as the root user.

      Run the following command to log on. You must replace the related information in the command based on the actual database environment.

      obclient -h10.xx.xx.xx -P2883 -uroot@sys -p***** -A
      

      For more information about how to connect to OceanBase Database, see Overview of the Connect to OceanBase Database chapter in Develop Applications in MySQL Mode and Overview of the Connect to OceanBase Database chapter in Develop Applications in Oracle Mode.

   2. Execute the following statement to add a node to a zone of the cluster.

      obclient [(none)]> ALTER SYSTEM ADD SERVER 'svr_ip:svr_port' [,'svr_ip:svr_port'...] [ZONE [=] 'zone_name'];
      

      • svr_ip: the IP address of the node to be added.

      • svr_port: the RPC port of the node to be added. Default value: 2882.

      • zone_name: the name of the zone to which the node is to be added.

      Here is an example:

      obclient [(none)]> ALTER SYSTEM ADD SERVER '10.xx.xx.xx:2882;10.xx.xx.xx:2882;10.xx.xx.xx:2882' ZONE 'zone4';
      

      This statement adds the OBServer node to the service list. Only OBServer nodes in the service list can provide services.

8. After the statement is executed, you can query the oceanbase.DBA_OB_SERVERS view to verify whether the node is added.

   Here is an example:

   obclient [(none)]> SELECT * FROM oceanbase.DBA_OB_SERVERS;
   

   If the OBServer node appears in the list, it is successfully added.

What to do next

You can add nodes for elastic scaling or deployment adjustment.

• Elastic scaling: After a node is added, more nodes are available in the zone to accommodate resource units. Then, you can perform a series of operations, such as migrating a unit, modifying the UNIT_NUM parameter for a tenant, and creating a tenant. For more information, see the following topics:

  • Unit migration

  • Adjust unit number

  • Create a tenant

• Deployment adjustment: You can adjust the deployment mode of a cluster from three IDCs in the same city to five IDCs across three regions, and migrate a zone from one IDC to another when an IDC is to be demolished. In this scenario, you must first add a zone, then add nodes to the zone, and finally modify the locality of the tenant. For more information about how to modify the locality of a tenant, see Change the locality.

More information

For more node-related O&M operations, see the following topics:

• View a node

• Restart a node

• Delete a node

• Isolate a node

• Replace a node