Online scale-out from standalone deployment to distributed deployment

OceanBase Database is built on a single-node distributed integrated architecture and supports online scale-out from standalone deployment to distributed deployment.

This topic describes how to use commands to install the RPM package of OceanBase Database to scale out to three-replica deployment.

Prerequisites

• The servers have been configured. For more information, see Configure servers.

• The deployment environment has been configured. For more information about how to configure the deployment environment, see Configure the deployment environment.

Procedure

Step 1: Deploy OBServer nodes

Perform the following steps separately on two servers.

1. Install the RPM package for OceanBase Database.

   For example:

   [root@xxx /home/admin/rpm]# rpm -ivh oceanbase-4.1.0.0-100000682023020119.el7.x86_64.rpm
   Preparing...                          ################################# [100%]
   Updating / installing...
      1:oceanbase-4.1.0.0-100000682023020################################# [100%]
   

2. (Optional) Clear directories.

   You do not need to clear directories for a server on which OceanBase Database is deployed for the first time.

   Clear the old OceanBase Database directories in the following cases:

   • An old OceanBase Database environment exists.
   • Problems occur during the installation and deployment process of OceanBase Database and thereby cause an environmental disorder or generate files that will affect the next installation.

   For example:

   [root@xxx /home/admin]# rm -rf /data/1/obdemo
   [root@xxx /home/admin]# rm -rf /data/log1/obdemo
   [root@xxx /home/admin]# rm -rf /home/admin/oceanbase/store/obdemo
   [root@xxx /home/admin]# rm -rf /home/admin/oceanbase/log/* /home/admin/oceanbase/etc/*config*
   

3. Initialize the OceanBase Database directory.

   We recommend that you specify the data directory of OceanBase Database to an independent disk and link this directory to the home directory of OceanBase Database by using a soft link.

   For example:

   [root@xxx /home/admin]# su - admin
   -bash-4.2$ mkdir -p /data/1/obdemo/{etc3,sort_dir,sstable,slog}
   -bash-4.2$ mkdir -p /data/log1/obdemo/{clog,etc2,ilog,oob_clog}
   -bash-4.2$ mkdir -p /home/admin/oceanbase/store/obdemo
   -bash-4.2$ for t in {etc3,sort_dir,sstable,slog};do ln -s /data/1/obdemo/$t /home/admin/oceanbase/store/obdemo/$t; done
   -bash-4.2$ for t in {clog,etc2,ilog,oob_clog};do ln -s /data/log1/obdemo/$t /home/admin/oceanbase/store/obdemo/$t; done
   

   Note

   The obdemo directory is named after the cluster and can be modified. It is required when the process starts.

4. Start the observer process.

   Start the observer process as the admin user.

   Parameters

   Parameter	Description
   -i	The NIC name. You can use the ifconfig command to view the NIC name.
   -p	The service port number, which is usually set to 2881.
   -P	The RPC port number, which is usually set to 2882.
   -n	The name of the cluster. It can be modified. Cluster names must be unique.
   -z	The zone where the started observer process belongs.
   -d	The primary directory of the cluster, which is created during directory initialization. Do not modify other fields except for the cluster name obdemo.
   -c	The cluster ID. It is a group of digits and can be modified. Cluster IDs must be unique.
   -l	The log level.
   -r	The Root Service list in the format of ip:2882:2881. Multiple items are separated with semicolons (;) to indicate Root Service information.
   -o	The cluster startup parameters, which need to be specified as needed. system_memory specifies the internal reserved memory of OceanBase Database, which is 30 GB by default. You can adjust this parameter to a smaller value in the case of insufficient memory on the server. The impact is that the memory may be insufficient during performance tests. datafile_size specifies the size of the SSTable data file of OceanBase Database, which is determined based on the available space of /data/1/. You no longer need to specify the size again once specified in this step. We recommend that you specify the data file size to at least 100 GB and reserve some spaces. Note If /data/1 and /data/log1 belong to the same file system (share the same disk), ensure that the space reserved for /data/log1 is two to four times the size of the memory. config_additional_dir specifies the redundancy directory for the parameter file.

   For example:

   1. Start the observer process on the node with the IP address of 10.10.10.2.

      [root@xxx /home/admin]# su - admin
      -bash-4.2$ cd /home/admin/oceanbase && /home/admin/oceanbase/bin/observer -i eth0 -P 2882 -p 2881 -z zone2 -d /home/admin/oceanbase/store/obdemo -r '10.10.10.2:2882:2881' -c 10001 -n obdemo -o "system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2"
      /home/admin/oceanbase/bin/observer -i eth0 -P 2882 -p 2881 -z zone2 -d /home/admin/oceanbase/store/obdemo -r 10.10.10.2:2882:2881 -c 10001 -n obdemo -o system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2
      devname: eth0
      rpc port: 2882
      mysql port: 2881
      zone: zone2
      data_dir: /home/admin/oceanbase/store/obdemo
      rs list: 10.10.10.2:2882:2881
      cluster id: 10001
      appname: obdemo
      optstr: system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2
      

   2. Start the observer process on the node with the IP address of 10.10.10.3.

      [root@xxx /home/admin]# su - admin
      -bash-4.2$ cd /home/admin/oceanbase && /home/admin/oceanbase/bin/observer -i eth0 -P 2882 -p 2881 -z zone3 -d /home/admin/oceanbase/store/obdemo -r '10.10.10.3:2882:2881' -c 10001 -n obdemo -o "system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2"
      /home/admin/oceanbase/bin/observer -i eth0 -P 2882 -p 2881 -z zone3 -d /home/admin/oceanbase/store/obdemo -r 10.10.10.3:2882:2881 -c 10001 -n obdemo -o system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2
      devname: eth0
      rpc port: 2882
      mysql port: 2881
      zone: zone3
      data_dir: /home/admin/oceanbase/store/obdemo
      rs list: 10.10.10.3:2882:2881
      cluster id: 10001
      appname: obdemo
      optstr: system_memory=30G,datafile_size=500G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2
      

5. Check whether the observer process is started.

   Run the following command to check whether the observer process is started.

   • For the netstat -ntlp command, if Ports 2881 and 2882 are listened to, the observer process is started.
   • The ps -ef|grep observer command can return the information about the observer process.

   For example:

   -bash-4.2$ netstat -ntlp
   (Not all processes could be identified, non-owned process info
   will not be shown, you would have to be root to see it all.)
   Active Internet connections (only servers)
   Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
   tcp        0      0 0.0.0.0:2881            0.0.0.0:*               LISTEN      115745/observer
   tcp        0      0 0.0.0.0:2882            0.0.0.0:*               LISTEN      115745/observer
   ...        ...    ...                       ...                     ...         ...
   
   -bash-4.2$ ps -ef|grep observer
   admin     60492      0 37 17:14 ?        00:00:37 /home/admin/oceanbase/bin/observer -i eth0 -P 2882 -p 2881 -z zone3 -d /home/admin/oceanbase/store/obdemo -r 10.10.10.3:2882:2881 -c 10001 -n obdemo -o system_memory=30G,datafile_size=100G,config_additional_dir=/data/1/obdemo/etc3;/data/log1/obdemo/etc2
   

Step 2: Scale out the cluster

1. Log on to the sys tenant of the cluster as the root user.

   [root@xxx /home/admin]# obclient -h10.10.10.1 -P2881 -uroot@sys -p
   Enter password:
   Welcome to the OceanBase.  Commands end with ; or \g.
   Your OceanBase connection id is 3221490218
   Server version: OceanBase 4.1.0.0 (r100000682023020119-c760cbb1023d73a2647f28add283646b1569418f) (Built Feb  1 2023 20:07:36)
   
   Copyright (c) 2000, 2018, OceanBase and/or its affiliates. All rights reserved.
   
   Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
   
   obclient [(none)]>
   

2. Add zones to the cluster.

   Add two zones that are respectively named zone2 and zone3.

   obclient [(none)]> ALTER SYSTEM ADD ZONE zone2;
   Query OK, 0 rows affected
   
   obclient [(none)]> ALTER SYSTEM ADD ZONE zone3;
   Query OK, 0 rows affected
   

3. Start zone2 and zone3.

   Start the new zones.

   obclient [(none)]> ALTER SYSTEM START ZONE zone2;
   Query OK, 0 rows affected
   
   obclient [(none)]> ALTER SYSTEM START ZONE zone3;
   Query OK, 0 rows affected
   

4. Add OBServer nodes to the cluster.

   Add the two OBServer nodes deployed in Step 1 to zone2 and zone3, respectively.

   For example:

   obclient [(none)]> ALTER SYSTEM ADD SERVER '10.10.10.2:2882' ZONE 'zone2';
   Query OK, 0 rows affected
   
   obclient [(none)]> ALTER SYSTEM ADD SERVER '10.10.10.3:2882' ZONE 'zone3';
   Query OK, 0 rows affected
   

5. Check whether the OBServer nodes are added to the cluster.

   Execute the following statement to check whether the OBServer nodes are added. If the nodes appear in the returned list, they are added.

   obclient [(none)]> SELECT SVR_IP,SVR_PORT,ID,ZONE,SQL_PORT,STATUS,START_SERVICE_TIME FROM oceanbase.DBA_OB_SERVERS;
   +------------+----------+----+-------+----------+--------+----------------------------+
   | SVR_IP     | SVR_PORT | ID | ZONE  | SQL_PORT | STATUS | START_SERVICE_TIME         |
   +------------+----------+----+-------+----------+--------+----------------------------+
   | 10.10.10.3 |     2882 |  3 | zone3 |     2881 | ACTIVE | 2023-03-03 17:26:44.207835 |
   | 10.10.10.2 |     2882 |  2 | zone2 |     2881 | ACTIVE | 2023-03-03 17:25:47.772032 |
   | 10.10.10.1 |     2882 |  1 | zone1 |     2881 | ACTIVE | 2023-03-03 10:09:27.040795 |
   +------------+----------+----+-------+----------+--------+----------------------------+
   3 rows in set
   

Step 3: Add replicas for a tenant

1. Log on to the sys tenant of the cluster as the root user.

   [root@xxx /home/admin]# obclient -h10.10.10.1 -P2881 -uroot@sys -p
   Enter password:
   Welcome to the OceanBase.  Commands end with ; or \g.
   Your OceanBase connection id is 3221490218
   Server version: OceanBase 4.1.0.0 (r100000682023020119-c760cbb1023d73a2647f28add283646b1569418f) (Built Feb  1 2023 20:07:36)
   
   Copyright (c) 2000, 2018, OceanBase and/or its affiliates. All rights reserved.
   
   Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
   
   obclient [(none)]>
   

2. Modify the ZONE_LIST parameter of the resource pool.

   Add the new zones to the resource pool of the tenant.

   obclient [(none)]> ALTER RESOURCE POOL pool001 ZONE_LIST=('zone1','zone2','zone3');
   Query OK, 0 rows affected
   

3. Add replicas for the tenant.

   Note

   To add replicas by modifying the locality of a tenant, you must add the replicas one by one.

      obclient [(none)]> ALTER TENANT tenant_oracle001 LOCALITY='F@zone1,F@zone2';
      Query OK, 0 rows affected
   
      obclient [(none)]> ALTER TENANT tenant_oracle001 LOCALITY='F@zone1,F@zone2,F@zone3';
      Query OK, 0 rows affected
   

More information

• Connect to an OceanBase Database tenant by using OBClient
• ALTER SYSTEM ... ZONE
• ALTER SYSTEM ... SERVER
• ALTER RESOURCE POOL
• ALTER TENANT