Scale out

This topic describes how to scale out a single region and a single node.

Expand a single-node cluster to a multi-node cluster

You can expand a single-node cluster to a multi-node cluster in the following two scenarios:

• The initial node is configured with a VIP as the cm_url.

• The initial node is configured with a physical server IP address as the cm_url.

The cm_url of the initial environment is set to the VIP

The specific operation steps are the same as those for the initial single-node deployment. For more information, see Deploy OMS on a single node.

Assume that the initial node has cm_nodes set to xxx.xxx.xxx.xx1 and the expanded node has cm_nodes set to xxx.xxx.xxx.xx2. The config.yaml file is configured as follows.

To synchronize the config.yaml file of the initial node to the latest version, deploy the config.yaml file of the expanded node directly.

# OMS metadata information
oms_meta_host: ${oms_meta_host}
oms_meta_port: ${oms_meta_port}
oms_meta_user: ${oms_meta_user}
oms_meta_password: ${oms_meta_password}

# When you expand a single-node OMS cluster to a multi-node OMS cluster, set the names of the following three databases to the same values as those of the initial node.
drc_rm_db: ${drc_rm_db}
drc_cm_db: ${drc_cm_db}
drc_cm_heartbeat_db: ${drc_cm_heartbeat_db}

# OMS cluster configuration
cm_url: http://VIP:8088
cm_location: ${cm_location}
cm_region: ${cm_region}
cm_region_cn: ${cm_region_cn}
cm_is_default: true
cm_nodes:
 - xxx.xxx.xxx.xx1
 - xxx.xxx.xxx.xx2

# Time series database configuration
# The default value is false. If you want to enable the metric reporting feature, set the value to true and remove the preceding # sign from this parameter.
# tsdb_enabled: false
# If tsdb_enabled is set to true, remove the preceding # sign from the following parameters and set their values based on your business requirements.
# tsdb_service: 'INFLUXDB'
# tsdb_url: '${tsdb_url}'
# tsdb_username: ${tsdb_user}
# tsdb_password: ${tsdb_password}

The cm_url in the initial environment configuration is the IP address of the physical server

In this scenario, you need to correct the database. Perform the following steps:

1. Configure the VIP, SLB, or domain name, and bind all the IP addresses of the physical servers in the current region.

   For example, the VIP is bound to the physical servers with IP addresses xxx.xxx.xxx.xx1 and xxx.xxx.xxx.xx2.

2. Stop the processes in the original container. For example, stop the processes in the oms_330 container.

   sudo docker exec -it oms_330 bash
   supervisorctl stop all
   

3. Take the drc_rm_db database and the cluster_info table as an example. Log in to the database and perform the following operations.

   1. Query the data in the cluster_info table for backup.

      select * from cluster_info;
      

   2. Delete the data in the cluster_info table.

      delete from cluster_info;
      

4. Modify the configuration file in the oms_330 container.

   # /home/admin/conf/config.yaml is a fixed directory in Docker
   sudo vi /home/admin/conf/config.yaml
   
   # Modify the cm_url address and add cm_nodes:
   cm_url: http://VIP:8088
   cm_nodes:
    - xxx.xxx.xxx.xx1
    - xxx.xxx.xxx.xx2
   

5. Perform the initialization operation again.

   sudo sh /root/docker_init.sh
   

   After the initialization is complete, verify that the cm_url address in the cluster_info table is the VIP address.

6. Copy the same configuration file and start the OMS container on another server.

   Notice

   When you start the container, set the value of -e OMS_HOST_IP to the IP address of the current server.

   OMS_HOST_IP=xxx
   CONTAINER_NAME=oms_xxx
   IMAGE_TAG=feature_x.x.x
   
   docker run -dit --net host \
   -v /data/config.yaml:/home/admin/conf/config.yaml \
   -v /data/oms/oms_logs:/home/admin/logs \
   -v /data/oms/oms_store:/home/ds/store \
   -v /data/oms/oms_run:/home/ds/run \
   # Set the following two parameters only when you mount the HTTPS certificate in the OMS container.
   -v /data/oms/https_crt:/etc/pki/nginx/oms_server.crt 
   -v /data/oms/https_key:/etc/pki/nginx/oms_server.key
   -e OMS_HOST_IP=${OMS_HOST_IP} \
   --privileged=true \
   --pids-limit -1 \
   --ulimit nproc=65535:65535 \
   --name ${CONTAINER_NAME} \
   work.oceanbase-dev.com/obartifact-store/oms:${IMAGE_TAG}
   

Expand a single-node single-region environment to a single-node multi-region environment

The operation steps for expanding a single-node single-region environment to a single-node multi-region environment are the same as those for deploying a single-node environment. The difference is that in the config.yaml file, the drc_cm_heartbeat_db parameter must be different from the initial OMS machine's heartbeat database. Additionally, you need to modify the cm_url, cm_location, cm_region, cm_region_cn, cm_is_default, and cm_nodes parameters to the new region's configuration.

1. Log in to the OMS deployment server.

2. (Optional) Deploy the time-series database.

   If you want OMS to collect and display monitoring data, deploy the time-series database. If you do not need to display monitoring data, skip this step. For more information, see Deploy a time-series database.

3. Prepare the configuration file.

   Edit the OMS startup configuration file in the appropriate directory. For example, you can create the configuration file as /root/config.yaml.

   Please replace the required parameters with the actual values of your deployment environment. For example, if you want to expand the machine xxx.xxx.xxx.xx1 to the Jiangsu region, the config.yaml file is configured as follows.

   Notice

   In the config.yaml file, the key and value must be separated by a colon and a space.

   # OMS metadata information
   oms_meta_host: ${oms_meta_host}
   oms_meta_port: ${oms_meta_port}
   oms_meta_user: ${oms_meta_user}
   oms_meta_password: ${oms_meta_password}
   
   # The configurations of drc_rm_db and drc_cm_db are the same as those of the initial node.
   drc_rm_db: ${drc_rm_db}
   drc_cm_db: ${drc_cm_db}
   # drc_cm_heartbeat_db must be specified with a new name to distinguish it from the heartbeat_db database of the initial node.
   drc_cm_heartbeat_db: ${drc_cm_heartbeat_db}
   
   # OMS cluster configuration
   # Fill in the configurations of the expanded region in the following parameters.
   cm_url: http://xxx.xxx.xxx.xx1:8088
   cm_location: ${cm_location}
   cm_region: cn-jiangsu
   cm_region_cn: Jiangsu
   cm_is_default: true
   cm_nodes:
    - xxx.xxx.xxx.xx1
   
   # Time-series database configuration
   # The default value is false. If you want to enable the metric reporting feature, set it to true and remove the comment from the parameter.
   # tsdb_enabled: false
   # If tsdb_enabled is true, remove the comment from the following parameters and fill in the values based on your actual environment.
   # tsdb_service: 'INFLUXDB'
   # tsdb_url: '${tsdb_url}'
   # tsdb_username: ${tsdb_user}
   # tsdb_password: ${tsdb_password}
   

   Parameter	Description	Required
   oms_meta_host	The IP address of the metadata database. Currently, only OceanBase Database in MySQL compatible mode is supported, and the version must be V2.0 or later.	Required
   oms_meta_port	The port number of the metadata database.	Required
   oms_meta_user	The username of the metadata database.	Required
   oms_meta_password	The password of the metadata database.	Required
   drc_rm_db	The name of the database of the management console. It must be the same as that of the initial node.	Required
   drc_cm_db	The name of the metadata database of the cluster management service. It must be the same as that of the initial node.	Required
   drc_cm_heartbeat_db	The name of the heartbeat database of the cluster management service. It must be a new name to distinguish it from the heartbeat_db database of the initial node.	Required
   cm_url	The address of the OMS cluster management service. For example, http://xxx.xxx.xxx.xx1:8088. Note: In a single-node deployment, it is typically set to the IP address of the current OMS server. We recommend that you do not use http://127.0.0.1:8088 because it cannot be expanded to a multi-region multi-node mode.	Required
   cm_location	The region code. The value ranges from 0 to 127. Each region is assigned a unique number, which you can select.	Required
   cm_region	The region, for example, cn-jiangsu. Note: If you use the OMS in a disaster recovery scenario with Alibaba Cloud MSHA, use the region information of Alibaba Cloud as cm_region.	Optional
   cm_region_cn	The Chinese name of the region. For example, Jiangsu.	Optional
   cm_nodes	The IP address list of the OMS cluster management service. In the example, the IP address is xxx.xxx.xxx.xx1.	Required
   cm_is_default	Indicates whether the OMS cluster management service is the default one. Note: When multiple regions are available, only one region can be set to have cm_is_default as true, indicating the default OMS cluster management service.	Optional. Default value: true
   tsdb_enabled	Indicates whether to enable the metric reporting feature (monitoring capability). Valid values: true or false.	Optional. Default value: false
   tsdb_service	The type of the time-series database. Valid values: INFLUXDB and CERESDB.	Optional. Default value: INFLUXDB
   tsdb_url	The IP address of the server where InfluxDB is deployed. If tsdb_enabled is true, modify this parameter based on your actual environment.	Optional
   tsdb_username	The username of the time-series database. If tsdb_enabled is true, modify this parameter based on your actual environment. After you deploy the time-series database, you must manually create a time-series database user and specify the username and password.	Optional
   tsdb_password	The password of the time-series database. If tsdb_enabled is true, modify this parameter based on your actual environment.	Optional

4. Load the downloaded OMS installation package to the local image repository of the Docker container.

   docker load -i <OMS installation package>
   

5. Run the following command to start the container.

   OMS supports accessing the OMS console over HTTP or HTTPS. If you want to securely access OMS, you can provide an HTTPS certificate and mount it to the specified directory in the container. If you access OMS over HTTP, you do not need to configure this.

   OMS_HOST_IP=xxx
   CONTAINER_NAME=oms_xxx
   IMAGE_TAG=feature_x.x.x
   
   docker run -dit --net host \
   -v /data/config.yaml:/home/admin/conf/config.yaml \
   -v /data/oms/oms_logs:/home/admin/logs \
   -v /data/oms/oms_store:/home/ds/store \
   -v /data/oms/oms_run:/home/ds/run \
   # Mount the HTTPS certificate in the OMS container only if you want to access OMS over HTTPS.
   -v /data/oms/https_crt:/etc/pki/nginx/oms_server.crt 
   -v /data/oms/https_key:/etc/pki/nginx/oms_server.key
   -e OMS_HOST_IP=${OMS_HOST_IP} \
   --privileged=true \
   --pids-limit -1 \
   --ulimit nproc=65535:65535 \
   --name ${CONTAINER_NAME} \
   work.oceanbase-dev.com/obartifact-store/oms:${IMAGE_TAG}
   

   Parameter	Description
   OMS_HOST_IP	The IP address of the host.
   CONTAINER_NAME	The name of the container. The format is oms_xxx. Fill in xxx based on the specific version. For example, if you use OMS V3.1.0, fill in oms_310.
   IMAGE_TAG	After you load the OMS installation package with Docker, run the docker images command to obtain the [IMAGE ID] or [REPOSITORY:TAG] of the loaded image, which is the unique identifier of the loaded image.
   /data/oms/oms_logs /data/oms/oms_store /data/oms/oms_run	/data/oms/oms_logs, /data/oms/oms_store, and /data/oms/oms_run can be replaced with the mount directories that you create on the OMS deployment server. These directories store the log files generated during OMS operation, the files generated by the log pull component and the synchronization component, and the persistent data on the local server. Note: In subsequent deployments and upgrades, the mount directories must remain unchanged.
   /home/admin/logs /home/ds/store /home/ds/run	/home/admin/logs, /home/ds/store, and /home/ds/run are fixed directories in the container. The paths cannot be modified.
   /data/oms/https_crt (optional) /data/oms/https_key (optional)	The mount location of the HTTPS certificate in the OMS container. Replace the values based on your actual environment. If you mount the HTTPS certificate, the Nginx service in the OMS container will run in HTTPS mode. You must access OMS in HTTPS mode to use the OMS console service.
   privileged	Grants extended privileges to the container.
   pids-limit	Configures the process limit for the container. -1 indicates no limit.
   ulimit nproc	Configures the upper limit of the number of processes for the user.

6. Enter the new container.

   docker exec -it ${CONTAINER_NAME} bash
   

   Notice

   CONTAINER_NAME is the name of the container. The format is oms_xxx. Fill in xxx based on the specific version. For example, if you use OMS V3.1.0, fill in oms_310.

7. In the root directory, run the metadata initialization command.

   bash /root/docker_init.sh
   

   After you run the command, the initialization process is as follows:

   1. Initialize data in the metadata database.

   2. Generate the configuration files of each component.

   3. Restart all components.

   4. Initialize the OMS resource tags and resource groups.

   During the execution of docker_init.sh, pay attention to the output on the command line. After the execution is successful, the system will prompt you [End] All initialization steps have been completed..

   Notice

   The initialization process takes 2 to 4 minutes. Please wait patiently and do not interrupt the execution.