Deploy OMS Community Edition on multiple nodes in a single region

This topic describes how to deploy OceanBase Migration Service (OMS) Community Edition on multiple nodes in a single region by using deployment tools.

Background

• You can deploy OMS Community Edition on a single node first and scale out to multiple nodes.

• If you choose to deploy OMS Community Edition with the config.yaml configuration file, note that the settings are slightly different from those for the single-node deployment mode. For more information, see the "Template and example of a configuration file" section.

• To deploy OMS Community Edition on multiple nodes, you must apply for a virtual IP address (VIP) and use it as the mount point for the OMS Community Edition console. In addition, you must configure the mapping rules of Ports 8088 and 8089 in the VIP network strategy.

  You can use the VIP to access the OMS Community Edition console even if an OMS Community Edition node fails.

Prerequisites

• The installation environment meets the system and network requirements. For more information, see System and network requirements.

• A MetaDB cluster is in place to serve as the metadata database of OMS Community Edition.

• Make sure that the server to deploy OMS Community Edition can connect to all other servers.

• Make sure that all servers involved in the multi-node deployment can connect to each other and that you can obtain root permissions on a node by using its username and password.

• The OMS Community Edition installation package is obtained. Generally, the package is a tar.gz file whose name starts with oms.

• The downloaded OMS Community Edition installation package has been loaded into the local image repository of the Docker container on each server node.

  docker load -i <OMS Community Edition installation package>
  

• The directory to which the OMS Community Edition container is mounted has been prepared. OMS Community Edition creates three mapping directories in this directory: /home/admin/logs, /home/ds/store, and /home/ds/run. These directories store the runtime components and logs of OMS Community Edition.

• (Optional) A time-series database is installed to store OMS Community Edition performance monitoring data and DDL and DML statistics.

Concepts

You need to replace variable names in some commands and prompts. A variable name is enclosed with angle brackets (<>).

• Directory to which the OMS Community Edition container is mounted: See the description of the mount directory in Prerequisites.

• IP address of the server: the IP address of the host that executes the script. In a single-node deployment scenario, by default, it refers to the IP address in the cluster management (CM) configuration information.

• OMS_IMAGE: the unique identifier of the loaded image. After you load the OMS Community Edition installation package by using Docker, run the docker images command to obtain the [IMAGE ID] or [REPOSITORY:TAG] of the loaded image. The obtained value is the unique identifier of the loaded image. Example:

  $sudo docker images
  REPOSITORY                                          TAG                 IMAGE ID        
  work.oceanbase-dev.com/obartifact-store/oms     feature_3.4.0       2a6a77257d35      
  

  In this example, <OMS_IMAGE> can be work.oceanbase-dev.com/obartifact-store/oms:feature_3.4.0 or 2a6a77257d35. Replace the value of <OMS_IMAGE> in related commands with the preceding value.

• Directory of the config.yaml file: If you want to deploy OMS Community Edition based on the current config.yaml configuration file, this directory is the one where the current configuration file is located.

Multi-node deployment architecture

The following figure shows an architecture of multi-node deployment. Store is the log pulling component, and Incr-Sync is the incremental synchronization component. When OMS Community Edition A fails, the Store ad Incr-Sync processes running on the node are protected by the HA service and switched to OMS Community Edition B or OMS Community Edition C.

Deploy OMS Community Edition without a configuration file

1. Log on to the server where OMS Community Edition is deployed.

2. Optional. Deploy a time-series database.

   If you need to collect and display OMS Community Edition monitoring data, deploy a time-series database. Otherwise, you can skip this step. For more information, see Deploy a time-series database.

3. Run the following command to obtain the deployment script from the loaded image:

   sudo docker run -d --name oms-config-tool <OMS_IMAGE> bash && sudo docker cp oms-config-tool:/root/docker_remote_deploy.sh . && sudo docker rm -f oms-config-tool
   

   Example:

   sudo docker run -d --net host --name oms-config-tool work.oceanbase-dev.com/obartifact-store/oms:feature_4.0-ce bash && sudo docker cp oms-config-tool:/root/docker_remote_deploy.sh . && sudo docker rm -f oms-config-tool
   

4. Use the deployment script to start the deployment tool.

   sh docker_remote_deploy.sh -o <deploy_tool_workdir> -i <IP address of the server> -d <OMS_IMAGE>
   

5. Follow the prompts to complete the deployment. After you set each parameter, press Enter to move on to the next parameter.

   1. Select the deployment mode.

      Select Multiple Nodes in Single Region.

   2. Select the task.

      Select No Configuration File. Deploy OMS Starting from Configuration File Generation.

   3. Enter the following MetaDB configuration information:

      1. Enter the URL, port, username, and password of the MetaDB.

      2. Set the prefix for names of databases in the MetaDB

         For example, when the prefix is set to oms, the databases in the MetaDB are named oms_rm, oms_cm, and oms_cm_hb.

      3. Confirm your settings.

         If the settings are correct, enter y and press Enter to proceed. Otherwise, enter n and press Enter to modify the settings.

      4. If the system displays "The specified database names already exist in the MetaDB. Are you sure that you want to continue?", The message indicates that the database name you entered already exists in the MetaDB. This may be caused by repeated deployment or upgrade of OMS Community Edition. You can enter y and press Enter to proceed, or enter n and press Enter to re-specify the settings.

   4. Perform the following operations to configure the CM service of OMS Community Edition:

      1. Specify the URL of the CM service, which is the VIP or domain name to which all CM servers in the region are mounted. The original parameter is cm-url.

         Enter the VIP or domain name as the URL of the CM service. You can separately specify the IP address and port number in the URL, or use a colon (:) to join the IP address and port number in the : format.

         Note

         The http:// prefix in the URL is optional.

      2. Enter the IP addresses of all servers in the region. Separate them with commas (,).

      3. Confirm the displayed CM settings.

         If the settings are correct, enter y and press Enter to proceed. Otherwise, enter n and press Enter to modify the settings.

   5. Confirm whether to enable OMS Community Edition historical data monitoring.

      • If you have deployed a time-series database, enter y and press Enter to go to the next step to configure the time-series database and enable the monitoring of OMS Community Edition historical data.

      • If you did not deploy a time-series database, enter n and press Enter to go to the step of "determining whether to enable the audit log feature and setting Simple Log Service (SLS) parameters". In this case, OMS Community Edition does not monitor the historical data after deployment.

   6. Configure the time-series database.

      Perform the following operations:

      1. Confirm whether you have deployed a time-series database.

         Enter the value based on the actual situation. If yes, enter y and press Enter. If not, enter n and press Enter to go to the step of "determining whether to enable the audit log feature and setting SLS parameters".

      2. Set the type of the time-series database to INFLUXDB.

         Notice

         At present, only INFLUXDB is supported.

      3. Enter the URL, username, and password of the time-series database.

      4. Confirm whether the displayed settings are correct.

         If the settings are correct, enter y and press Enter to proceed. Otherwise, enter n and press Enter to modify the settings.

   7. Determine whether to enable the audit log feature and write the audit logs to Simple Log Service (SLS).

      To enable the audit log feature, enter y and press Enter to go to the next step to specify the SLS parameters.

      Otherwise, enter n and press Enter to start the deployment. In this case, OMS Community Edition does not audit the logs after deployment.

   8. Specify the following SLS parameters:

      1. Set the SLS parameters as prompted.

      2. Confirm whether the displayed settings are correct.

      If the settings are correct, enter y and press Enter to proceed. Otherwise, enter n and press Enter to modify the settings.

   9. If the configuration file passes the check, all settings are displayed. If the settings are correct, enter n and press Enter to proceed. Otherwise, enter y and press Enter to modify the settings.

      If the configuration file fails the check, modify the configuration information as prompted.

   10. Start the deployment on each node one after another.

       1. Specify the directory to which the OMS Community Edition container is mounted in the host.

          Use a directory with a large capacity.

          For a remote node, the username and password for logging on to the remote node are required. The corresponding user account must have the sudo privileges on the remote node.

       2. Confirm whether the OMS Community Edition image file can be named as OMS_IMAGE.

          If the settings are correct, enter y and press Enter to proceed. Otherwise, enter n and press Enter to modify the settings.

       3. Confirm whether to install a Secure Sockets Layer (SSL) certificate for the OMS Community Edition container.

          If yes, enter y, press Enter, and specify the https_key and https_crt directories as prompted. If not, enter n and press Enter.

   11. Start the deployment on the node.

       If the deployment fails, you can log on to the running OMS container and view the .log file with the docker_init prefix in the /home/admin/logs directory to obtain the log. If the OMS Community Edition container fails to start, the logs cannot be obtained.

Deploy OMS Community Edition with a configuration file

1. Log on to the server where OMS Community Edition is deployed.

2. Optional. Deploy a time-series database.

   If you need to collect and display OMS Community Edition monitoring data, deploy a time-series database. Otherwise, you can skip this step. For more information, see Deploy a time-series database.

3. Run the following command to obtain the deployment script from the loaded image:

   sudo docker run -d --name oms-config-tool <OMS_IMAGE> bash && sudo docker cp oms-config-tool:/root/docker_remote_deploy.sh . && sudo docker rm -f oms-config-tool
   

4. Use the deployment script to start the deployment tool.

   sh docker_remote_deploy.sh -o <deploy_tool_workdir> -c <directory of the config.yaml file> -i <IP address of the server> -d <OMS_IMAGE>
   

   Note

   For more information about settings of the config.yaml file, see the "Template and example of a configuration file" section.

5. Follow the prompts to complete the deployment. After you set each parameter, press Enter to move on to the next parameter.

   1. Select the deployment mode.

      Select Multiple Nodes in Single Region.

   2. Select the task.

      Select Use Configuration File Uploaded with Script Option [-c].

   3. If the system displays "The specified database names already exist in the MetaDB. Are you sure that you want to continue?", The message indicates that the database name you entered already exists in the MetaDB specified in the original configuration file. This may be caused by repeated deployment or upgrade of OMS Community Edition. You can enter y and press Enter to proceed, or enter n and press Enter to re-specify the settings.

   4. If the configuration file passes the check, all settings are displayed. If the settings are correct, enter n and press Enter to proceed. Otherwise, enter y and press Enter to modify the settings.

      If the configuration file fails the check, modify the configuration information as prompted.

   5. Start the deployment on each node one after another.

      1. Specify the directory to which the OMS Community Edition container is mounted in the host.

         Use a directory with a large capacity.

         For a remote node, the username and password for logging on to the remote node are required. The corresponding user account must have the sudo privileges on the remote node.

      2. Confirm whether the OMS Community Edition image file can be named as OMS_IMAGE.

         If the settings are correct, enter y and press Enter to proceed. Otherwise, enter n and press Enter to modify the settings.

      3. Confirm whether to install a Secure Sockets Layer (SSL) certificate for the OMS Community Edition container.

         If yes, enter y, press Enter, and specify the https_key and https_crt directories as prompted. If not, enter n and press Enter.

   6. Start the deployment.

      If the deployment fails, you can log on to the running OMS Community Edition container and view the .log file with the docker_init prefix in the /home/admin/logs directory to obtain the log. If the OMS Community Edition container fails to start, the logs cannot be obtained.

To modify the configuration after deployment, log on to the OMS Community Edition container and perform the following steps:

1. Modify the config.yaml file based on business needs.

2. Run the python -m omsflow.scripts.units.oms_init_manager --init-config-file command.

3. Run the supervisorctl restart oms_console oms_drc_supervisor command.

Template and example of a configuration file

Configuration file template

# Information about the OMS Community Edition MetaDB
oms_meta_host: ${oms_meta_host}
oms_meta_port: ${oms_meta_port}
oms_meta_user: ${oms_meta_user}
oms_meta_password: ${oms_meta_password}

# You can customize the names of the following three databases, which are created in the MetaDB when you deploy OMS Community Edition.
drc_rm_db: ${drc_rm_db}
drc_cm_db: ${drc_cm_db}
drc_cm_heartbeat_db: ${drc_cm_heartbeat_db}

# Configurations of the OMS Community Edition cluster
# To deploy OMS Community Edition on multiple nodes in a single region, you must set the cm_url parameter to a VIP or domain name to which all CM servers in the region are mounted.
cm_url: ${cm_url}
cm_location: ${cm_location}
# The cm_region parameter is not required for single-region deployment.
# cm_region: ${cm_region}
# The cm_region_cn parameter is not required for single-region deployment.
# cm_region_cn: ${cm_region_cn}
cm_is_default: true
cm_nodes:
 - ${host_ip1}
 - ${host_ip2}
     
# Configurations of the time-series database
# Default value: false. To enable metric reporting, set the parameter to `true` and delete the comments for the parameter.
# tsdb_enabled: false 
# If the `tsdb_enabled` parameter is set to `true`, delete comments for the following parameters and specify the values based on your actual configurations.
# tsdb_service: 'INFLUXDB'
# tsdb_url: '${tsdb_url}'
# tsdb_username: ${tsdb_user}
# tsdb_password: ${tsdb_password}

Example

Replace related parameters with the actual values in the target deployment environment.

oms_meta_host: xxx.xxx.xxx.1
oms_meta_port: 2883
oms_meta_user: oms_meta_user
oms_meta_password: ********
drc_rm_db: oms_rm
drc_cm_db: oms_cm
drc_cm_heartbeat_db: oms_cm_heartbeat
cm_url: http://VIP:8088
cm_location: 100
cm_region: cn-anhui
cm_region_cn: cn-anhui
cm_is_default: true
cm_nodes:
 - xxx.xxx.xxx.2
 - xxx.xxx.xxx.3
tsdb_service: 'INFLUXDB'
tsdb_enabled: true
tsdb_url: 'xxx.xxx.xxx.5:8086'
tsdb_username: username
tsdb_password: *************