Scale-out and deployment

2024-09-18 01:56:08  Updated

This topic describes how to scale out OceanBase Migration Service (OMS) Community Edition from a single region or a single node.

Scale out OMS Community Edition from a single node

The following two scenarios are possible:

  • A virtual IP (VIP) address is specified for the cm_url parameter of the initial node.

  • The IP address of a physical server is specified for the cm_url parameter of the initial node.

A VIP is specified for the cm_url parameter in the initial environment

In this scenario, you can replicate the config.yaml configuration file and append one or more rows of values to the cm_nodes parameter. Other parameters and operations are the same as those of the initial node.

Notice:

When you start the container, the value of -e OMS_HOST_IP must be the IP address of the current host.

The following example shows how to change the value of the cm_nodes parameter in the config.yaml configuration file form xxx.xxx.xxx.1 for the initial node to xxx.xxx.xxx.2 for the node after scale-out.

# The information about OMS Community Edition MetaDB.
oms_meta_host: ${oms_meta_host}
oms_meta_port: ${oms_meta_port}
oms_meta_user: ${oms_meta_user}
oms_meta_password: ${oms_meta_password}

# When you scale out OMS Community Edition from one node to multiple nodes, make sure that the names of the following three databases are consistent with those of the initial node.
drc_rm_db: ${drc_rm_db}
drc_cm_db: ${drc_cm_db}
drc_cm_heartbeat_db: ${drc_cm_heartbeat_db}

# The user that consumes the incremental data of OceanBase Database.
# To read the incremental logs of OceanBase Database, create the user in the SYS tenant.
drc_user: ${drc_user}
drc_password: '${drc_password}'

# Configure the OMS Community Edition cluster.
cm_url: http://VIP:8088
cm_location: ${cm_location}
cm_region: ${cm_region}
cm_region_cn: ${cm_region_cn}
cm_is_default: true
cm_nodes:
 - xxx.xxx.xxx.1
 - xxx.xxx.xxx.2

# Configurations of the time series database.
# Default value: false. To enable metric reporting, set the parameter to true and delete comments for the parameter.
# tsdb_enabled: false 
# If the tsdb_enabled parameter is set to true, delete comments for the following parameters and specify the values based on your actual configurations.
# tsdb_service: 'INFLUXDB'
# tsdb_url: '${tsdb_url}'
# tsdb_username: ${tsdb_user}
# tsdb_password: ${tsdb_password}

The IP address of a physical server is specified for the cm_url parameter of the initial environment

In this scenario, you need to perform the following operations to modify the database configurations:

  1. Configure the VIP, SLB, or domain name and bind the IP addresses of all physical servers in the region.

    For example, the VIP is bound to physical servers whose IP addresses are xxx.xxx.xxx.1 and xxx.xxx.xxx.2.

  2. Stop the processes in the original container. Sample statement for the oms_330 container:

    sudo docker exec -it oms_330 bash  
supervisorctl stop all

  3. Use the drc_rm_db database and the cluster_info table as examples. Log on to the database and perform the following operations:

    1. Query and back up the data in the cluster_info table.

      select * from cluster_info;

    2. Delete the data in the cluster_info table.

      delete from cluster_info;

  4. Modify the configuration file in the oms_330 container.

    # The /home/admin/conf/config.yaml directory is fixed in Docker.
sudo vi /home/admin/conf/config.yaml

# Modify cm_url and add cm_nodes.
cm_url: http://VIP:8088
cm_nodes:
 - xxx.xxx.xxx.1
 - xxx.xxx.xxx.2

  5. Reinitialize the data.

    sudo sh /root/docker_init.sh

    After the initialization is completed, check whether the value of the cm_url parameter in the cluster_info table of the database is consistent with the VIP.

  6. Replicate the configuration file and start the OMS Community Edition container on another host.

    Notice:

    When you start the container, the value of -e OMS_HOST_IP must be the IP address of the current host.

    OMS_HOST_IP=xxx
CONTAINER_NAME=oms_xxx
IMAGE_TAG=feature_x.x.x-ce

docker run -dit --net host \
-v /data/config.yaml:/home/admin/conf/config.yaml \
-v /data/oms/oms_logs:/home/admin/logs \
-v /data/oms/oms_store:/home/ds/store \
-v /data/oms/oms_run:/home/ds/run \
# If you mount the SSL certificate in the OMS Community Edition container, you need to set the following two parameters.
-v /data/oms/https_crt:/etc/pki/nginx/oms_server.crt 
-v /data/oms/https_key:/etc/pki/nginx/oms_server.key
-e OMS_HOST_IP=${OMS_HOST_IP} \
--privileged=true \
--pids-limit -1 \
--ulimit nproc=65535:65535 \
--name ${CONTAINER_NAME} \
work.oceanbase-dev.com/obartifact-store/oms:${IMAGE_TAG}

Scale out OMS Community Edition from a single region

The procedure is generally the same as that for deploying OMS Community Edition on a single node. A difference is that the drc_cm_heartbeat_db parameter in the config.yaml file must be different from the database used by the initial OMS Community Edition host. In addition, you need to modify the following parameters for the new region: cm_url, cm_location, cm_region, cm_region_cn, cm_is_default, and cm_nodes.

  1. Log on to the server where OMS Community Edition is deployed.

  2. Optional. Deploy a time series database.

    If you need to collect and display OMS Community Edition monitoring data, deploy a time series database. Otherwise, you can skip this step. For more information, see Deploy a time series database.

  3. Prepare the configuration file.

    Edit the OMS Community Edition configuration file in the appropriate directory. For example, you can create the config.yaml configuration file in the /root/ directory.

    You must replace the sample values of required parameters based on your actual deployment environment. The following example describes how to set the config.yaml file and scale out OMS Community Edition deployed on the host xxx.xxx.xxx.1 in the region Jiangsu.

    Notice:

    In the config.yaml file, you must specify the value in the parameter: value format. A space is required after the colon (:).

    # The information about OMS Community Edition MetaDB.
oms_meta_host: ${oms_meta_host}
oms_meta_port: ${oms_meta_port}
oms_meta_user: ${oms_meta_user}
oms_meta_password: ${oms_meta_password}

# Keep the values of the drc_rm_db and drc_cm_db parameters consistent with those of the initial node.
drc_rm_db: ${drc_rm_db}
drc_cm_db: ${drc_cm_db}
# The name of the drc_cm_heartbeat_db database must be different from that of the heartbeat_db database for the initial node.
drc_cm_heartbeat_db: ${drc_cm_heartbeat_db}

# The user that consumes the incremental data of OceanBase Database.
# To read the incremental logs of OceanBase Database, create the user in the SYS tenant.
drc_user: ${drc_user}
drc_password: '${drc_password}'

# Configure the OMS Community Edition cluster.
# Specify the following parameters for the region to be scaled.
cm_url: http://xxx.xxx.xxx.1:8088
cm_location: ${cm_location}
cm_region: cn-jiangsu
cm_is_default: true
cm_nodes:
 - xxx.xxx.xxx.1

# Configurations of the time series database.
# Default value: false. To enable metric reporting, set the parameter to true and delete comments for the parameter.
# tsdb_enabled: false 
# If the tsdb_enabled parameter is set to true, delete comments for the following parameters and specify the values based on your actual configurations.
# tsdb_service: 'INFLUXDB'
# tsdb_url: '${tsdb_url}'
# tsdb_username: ${tsdb_user}
# tsdb_password: ${tsdb_password}
    Parameter Description Required or optional
    oms_meta_host The IP address of MetaDB. Required
    oms_meta_port The port number of MetaDB. Required
    oms_meta_user The username of MetaDB. Required
    oms_meta_password The password of the MetaDB username. Required
    drc_rm_db The name of the database for the OMS Community Edition console. The name must be the same as that for the initial node. Required
    drc_cm_db The name of the MetaDB for the CM service. The name must be the same as that for the initial node. Required
    drc_cm_heartbeat_db The name of the heartbeat database for the CM service. Specify a name that is different from that of the heartbeat_db database for the initial node. Required
    drc_user The user that reads the incremental logs of OceanBase Database. You need to create the user in the SYS tenant. For more information, see the "User privileges" topic in OMS Community Edition User Guide. Optional
    drc_password The password of the drc user. Optional
    cm_url The URL of the OMS Community Edition CM service. Example: http://xxx.xxx.xxx.1:8088.
    Note:
    In single-node deployment mode, the IP address of the server where OMS Community Edition is deployed is used. We recommend that you do not set it to http://127.0.0.1:8088, which cannot be used for scaling out to multiple nodes in multiple regions.    		 Required
    cm_location The code of the region. Value range: [0,127]. You can select one number for each region. Required
    cm_region The name of the region. Example: cn-jiangsu.
    Notice:
    If you use OMS Community Edition with the Alibaba Cloud Multi-Site High Availability (MSHA) service in an active-active disaster recovery scenario, use the region configured for the Alibaba Cloud service.    		 Optional
    cm_nodes The IP addresses of servers used for OMS Community Edition CM service. In examples in this topic, the value is xxx.xxx.xxx.1. Required
    cm_is_default Indicates whether the OMS Community Edition CM service is enabled by default.
    Notice:
    When multiple regions exist, the cm_is_default parameter of only one region can be set to true, which indicates that the OMS Community Edition CM service is enabled.    		 Optional. Default value: true
    tsdb_enabled Indicates whether metric reporting is enabled for monitoring. Valid values: true and false. Optional. Default value: false
    tsdb_service The type of the time series database. Valid values: INFLUXDB and CERESDB. Optional. Default value: CERESDB
    tsdb_url The IP address of the server where InfluxDB is deployed. You need to modify this parameter based on the actual environment if you set the tsdb_enabled parameter to true. Optional
    tsdb_username The username used to connect to the time series database. You need to modify this parameter based on the actual environment if you set the tsdb_enabled parameter to true. After you deploy the time series database, you need to manually create a user and specify the username and password. Optional
    tsdb_password The password used to connect to the time series database. You need to modify this parameter based on the actual environment if you set the tsdb_enabled parameter to true. Optional

  4. Load the downloaded OMS Community Edition image file to the local image repository of the Docker container.

    docker load -i <The storage path of the OMS Community Edition image>

  5. Run the following command to start the container.

    You can access the OMS Community Edition console by using a HTTP or HTTPS URL. To securely access the OMS Community Edition console, install an SSL certificate and mount it to the specified directory in the container. The certificate is not required for HTTP access.

    OMS_HOST_IP=xxx
CONTAINER_NAME=oms_xxx
IMAGE_TAG=feature_x.x.x-ce

docker run -dit --net host \
-v /data/config.yaml:/home/admin/conf/config.yaml \
-v /data/oms/oms_logs:/home/admin/logs \
-v /data/oms/oms_store:/home/ds/store \
-v /data/oms/oms_run:/home/ds/run \
# If you mount the SSL certificate in the OMS Community Edition container, you need to set the following two parameters.
-v /data/oms/https_crt:/etc/pki/nginx/oms_server.crt 
-v /data/oms/https_key:/etc/pki/nginx/oms_server.key
-e OMS_HOST_IP=${OMS_HOST_IP} \
--privileged=true \
--pids-limit -1 \
--ulimit nproc=65535:65535 \
--name ${CONTAINER_NAME} \
work.oceanbase-dev.com/obartifact-store/oms:${IMAGE_TAG}
    Parameter Description
    OMS_HOST_IP The IP address of the host.
    CONTAINER_NAME The name of the container in the oms_xxx format. Specify xxx based on the actual OMS Community Edition version.
    IMAGE_TAG The tag of the image. To view the image tag, execute the docker images statement after you load the image on the host. The image tag is in the feature_x.x.x-ce format. Specify x.x.x based on the actual OMS Community Edition version.
    /data/oms/oms_logs
    /data/oms/oms_store
    /data/oms/oms_run    		 You can replace /data/oms/oms_logs, /data/oms/oms_store, and /data/oms/oms_run with the mount directories created on the server where OMS Community Edition is deployed. The mount directories store the logs generated during the operating of OMS Community Edition and generated by stores and synchronization components, respectively, to persistently retain the logs on the server.
    Notice:
    The mount directories must remain unchanged during subsequent redeployment or upgrades.
    /home/admin/logs
    /home/ds/store
    /home/ds/run    		 /home/admin/logs, /home/ds/store, and /home/ds/run are default directories in the container and cannot be modified.
    /data/oms/https_crt (optional)
    /data/oms/https_key (optional)    		 The mount directory of the SSL certificate in the OMS container. Specify the directory based on your actual configurations. If you mount an SSL certificate, the NGINX service in the OMS Community Edition container runs in HTTPS mode. In this case, you can access the OMS Community Edition console by using only the HTTPS URL.
    privileged Specifies whether to grant the container scaling privilege.
    pids-limit Specifies whether to limit the number of container processes. The value -1 indicates that the number is unlimited.
    ulimit nproc The maximum number of user processes.

  6. Access the new container.

    docker exec -it  ${CONTAINER_NAME} bash

    Notice:

    CONTAINER_NAME represents the name of the container in the oms_xxx format.

  7. Perform metadata initialization in the root directory.

    bash /root/docker_init.sh

    After you run the command, the following initialization process is executed:

    1. Initialize the data in the MetaDB.

    2. Generate configuration files for respective components.

    3. Restart all components.

    4. Initialize OMS Community Edition resource tags and resource groups.

    In the execution of the docker_init.sh script, pay attention to the output of command lines. When the execution is completed, the following message is displayed: [Completed] All initialization steps are executed.

    Notice:

    The execution takes 2 to 4 minutes to complete. Be patient and do not interrupt the process.

Last
Next

OceanBaseOceanBase

OceanBase, A Highly Scalable Database for Transactional, Analytical, and AI Workloads.

Company

About OceanBaseContact UsPartner

Product

OceanBase CloudOceanBase EnterpriseQuick StartDownloadPricing

Resources

DocsBlogWhite PaperTrust Center

Follow us

LinkedInLinkedInDiscordDiscordTwitterTwitterYoutubeYoutubeForumForumGitHubGitHubStack OverflowStack Overflow

© OceanBase 2024. All rights reserved | Cloud Service Agreement | Privacy Policy | Legal | Security

Contact Us