Scale out

This topic describes how to scale out a single region and a single node.

Scale out a single node to multiple nodes

When you scale out a single node to multiple nodes, the configuration files differ from those for a single-node deployment. This topic uses the example of scaling out a single node to two nodes.

• cm_nodes needs to be scaled out from one node to two nodes.

  "cm_nodes":
      - "oms-0.oms-service.omsregion-1.svc"
      - "oms-1.oms-service.omsregion-1.svc" # The new node
  

• The value of the replicas parameter changes from 1 to 2.

  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
  name: oms
  namespace: oms
  labels:
      app: oms
  spec:
  replicas: 2  # The value changes from 1 to 2.
  

The steps to scale out a single node to multiple nodes are as follows:

1. Enter the k8s environment and prepare the configuration file.

   The following is an example of the configuration file for scaling out a single node to multiple nodes. Replace the parameters with their actual values as needed.

   apiVersion: v1
   kind: Namespace
   metadata:
   name: omsregion-1
   ---
   apiVersion: v1
   kind: ConfigMap
   metadata:
   name: oms-config
   namespace: omsregion-1
   data:
   config.yaml: |
       "apsara_audit_enable": "false"
       "apsara_audit_sls_access_key": ""
       "apsara_audit_sls_access_secret": ""
       "apsara_audit_sls_endpoint": ""
       "apsara_audit_sls_ops_site_topic": ""
       "apsara_audit_sls_user_site_topic": ""
       "cm_is_default": !!bool "true"
       "cm_location": "0"
       # The node address in the format of ${pod_name}.${service_name}.${namespace_name}.svc.
       # The pod name is the name in kind: StatefulSet plus -ID, where the ID starts from 0. For example, in a single-node deployment, if replicas is 1, the pod name is oms-0.
       # The service name is the name in the first kind: Service.
       # The namespace name is the name in kind: Namespace.
       "cm_nodes":
       - "oms-0.oms-service.omsregion-1.svc"
       - "oms-1.oms-service.omsregion-1.svc"
       "cm_region": "hangzhou"
       "cm_region_cn": "Hangzhou"
       "cm_server_port": "8088"
       # The format is http://${service_name}.${namespace_name}.svc:8088.
       "cm_url": "http://oms-service.omsregion-1.svc:8088"
       "drc_cm_db": "cm_db_name"
       "drc_cm_heartbeat_db": "cm_hb_name"
       "drc_rm_db": "rm_db_name"
       "ghana_server_port": "8090"
       "init_db": "true"
       "nginx_server_port": "8089"
       "oms_meta_host": "xxx.xxx.xxx.1"
       "oms_meta_password": "oms_meta_password"
       "oms_meta_port": "2881"
       "oms_meta_user": "xxx@tenant_name"
       "sshd_server_port": "2023"
       "supervisor_server_port": "9000"
       "tsdb_enabled": "true"
       "tsdb_password": "tsdb_password"
       "tsdb_service": "INFLUXDB"
       "tsdb_url": "xxx.xxx.xxx.2:8086"
       "tsdb_username": "tsdb_username"
   ---
   apiVersion: v1
   kind: Service
   metadata:
   name: oms-service
   namespace: omsregion-1
   spec:
   clusterIP: None
   selector:
       app: oms
   ---
   apiVersion: v1
   kind: Service
   metadata:
   name: oms-service-nodeport
   namespace: omsregion-1
   spec:
   type: NodePort
   selector:
       app: oms
   ports:
       - name: nginx
       protocol: TCP
       port: 8089
       targetPort: 8089
       - name: ghana
       protocol: TCP
       port: 8090
       targetPort: 8090
       - name: cm
       protocol: TCP
       port: 8088
       targetPort: 8088
       - name: sshd
       protocol: TCP
       port: 2023
       targetPort: 2023
   ---
   apiVersion: apps/v1
   kind: StatefulSet
   metadata:
   name: oms
   namespace: omsregion-1
   labels:
       app: oms
   spec:
   replicas: 2
   serviceName: "oms-service"
   selector:
       matchLabels:
       app: oms
   template:
       metadata:
       labels:
           app: oms
       spec:
       initContainers:
       - name: copy-config
           image: busybox:1.35
           command: ['sh', '-c', 'cp /config-ro/config.yaml /work/config.yaml']
           volumeMounts:
           - name: config-volume
           mountPath: /config-ro
           readOnly: true
           - name: writable-config
           mountPath: /work
       containers:
       - name: oms
           image: ${OMS_IMAGE}
           env:
           - name: POD_NAME
           valueFrom:
               fieldRef:
               fieldPath: metadata.name
           - name: OMS_HOST_IP
           # The value of POD_NAME is the name of the current Pod, which remains unchanged. oms-service.omsregion-1 should be modified based on the actual ${service_name} and ${namespace_name}.
           value: $(POD_NAME).oms-service.omsregion-1.svc
           - name: HOST_DOCKER_VERSION
           value: "20.10.7"
           ports:
           - containerPort: 8088
           - containerPort: 8089
           - containerPort: 8090
           - containerPort: 9000
           - containerPort: 2023
           volumeMounts:
           - name: writable-config
           mountPath: /home/admin/conf/config.yaml
           subPath: config.yaml
           - name: logs-volume
           mountPath: /home/admin/logs
           - name: store-volume
           mountPath: /home/ds/store
           - name: run-volume
           mountPath: /home/ds/run
       volumes:
       - name: config-volume
           configMap:
           name: oms-config
       - name: writable-config
           emptyDir: {}
   # The disk capacity configuration. You can increase the sizes of the following three mounted disks as needed.
   # We recommend that you set the size of logs-volume to at least 200 GiB.
   # We recommend that you set the size of store-volume to at least 500 GiB. The store-volume stores incremental data, so we recommend that you set a larger size for it.
   # We recommend that you set the size of run-volume to at least 300 GiB.
   volumeClaimTemplates:
   - metadata:
       name: logs-volume
       spec:
       accessModes: [ "ReadWriteOnce" ]
       storageClassName: "local-path"
       resources:
           requests:
           storage: 200Gi
   - metadata:
       name: store-volume
       spec:
       accessModes: [ "ReadWriteOnce" ]
       storageClassName: "local-path"
       resources:
           requests:
           storage: 500Gi
   - metadata:
       name: run-volume
       spec:
       accessModes: [ "ReadWriteOnce" ]
       storageClassName: "local-path"
       resources:
           requests:
           storage: 300Gi
   

2. Install OMS Community Edition.

   For example, if the configuration file is named oms-k8s-statefulset.yaml, the installation command is as follows.

   kubectl apply -f oms-k8s-statefulset.yaml
   

Scale out a single region to multiple regions

When you scale out a single region to multiple regions, you need to create a configuration file for the new region (you can copy the configuration file of the original region and modify it). The main modifications to the new region's configuration file are as follows:

• The namespace name of the new region in k8s is different from that of the original region. For example, if the original region is named omsregion-1, the new region can be named omsregion-2.

  In addition, all instances of omsregion-1 in the original region's configuration file need to be replaced with omsregion-2.

• The cm_is_default parameter in the new region's configuration file needs to be set to false.

  "cm_is_default": !!bool "false"
  

• The ID value of the cm_location parameter in the new region's configuration file needs to be one more than the ID of the existing region. For example, if the ID of the original region is 0, the ID of the new region is 1.

  "cm_location": "1"
  

The following example shows how to scale out the Hangzhou region to the Hangzhou and Beijing regions.

1. Enter the k8s environment and create a configuration file for the new region, Beijing.

   The following example shows the configuration file for scaling out a single region to multiple regions. Please replace the parameters with their actual values as needed.

   • Original Hangzhou region configuration file

     apiVersion: v1
     kind: Namespace
     metadata:
     name: omsregion-1
     ---
     apiVersion: v1
     kind: ConfigMap
     metadata:
     name: oms-config
     namespace: omsregion-1
     data:
     config.yaml: |
         "apsara_audit_enable": "false"
         "apsara_audit_sls_access_key": ""
         "apsara_audit_sls_access_secret": ""
         "apsara_audit_sls_endpoint": ""
         "apsara_audit_sls_ops_site_topic": ""
         "apsara_audit_sls_user_site_topic": ""
         "cm_is_default": !!bool "true"
         # The region code. Each region has a unique number.
         "cm_location": "0"
         # The node address, in the format of ${pod_name}.${service_name}.${namespace_name}.svc.
         # pod_name is the name in kind: StatefulSet plus -ID, where the ID starts from 0. For example, if replicas is 2 in a multi-node deployment, pod_name is oms-0 and oms-1.
         # service_name is the name in the first kind: Service.
         # namespace_name is the name in kind: Namespace.
         "cm_nodes":
         - "oms-0.oms-service.omsregion-1.svc"
         - "oms-1.oms-service.omsregion-1.svc"
         "cm_region": "hangzhou"
         "cm_region_cn": "Hangzhou"
         "cm_server_port": "8088"
         # The format is http://${service_name}.${namespace_name}.svc:8088.
         "cm_url": "http://oms-service.omsregion-1.svc:8088"
         "drc_cm_db": "cm_db_name"
         "drc_cm_heartbeat_db": "cm_hb_name"
         "drc_rm_db": "rm_db_name"
         "ghana_server_port": "8090"
         "init_db": "true"
         "nginx_server_port": "8089"
         "oms_meta_host": "xxx.xxx.xxx.1"
         "oms_meta_password": "oms_meta_password"
         "oms_meta_port": "2881"
         "oms_meta_user": "xxx@tenant_name"
         "sshd_server_port": "2023"
         "supervisor_server_port": "9000"
         "tsdb_enabled": "true"
         "tsdb_password": "tsdb_password"
         "tsdb_service": "INFLUXDB"
         "tsdb_url": "xxx.xxx.xxx.2:8086"
         "tsdb_username": "tsdb_username"
     ---
     apiVersion: v1
     kind: Service
     metadata:
     name: oms-service
     namespace: omsregion-1
     spec:
     clusterIP: None
     selector:
         app: oms
     ---
     apiVersion: v1
     kind: Service
     metadata:
     name: oms-service-nodeport
     namespace: omsregion-1
     spec:
     type: NodePort
     selector:
         app: oms
     ports:
         - name: nginx
         protocol: TCP
         port: 8089
         targetPort: 8089
         - name: ghana
         protocol: TCP
         port: 8090
         targetPort: 8090
         - name: cm
         protocol: TCP
         port: 8088
         targetPort: 8088
         - name: sshd
         protocol: TCP
         port: 2023
         targetPort: 2023
     ---
     apiVersion: apps/v1
     kind: StatefulSet
     metadata:
     name: oms
     namespace: omsregion-1
     labels:
         app: oms
     spec:
     replicas: 2
     serviceName: "oms-service"
     selector:
         matchLabels:
         app: oms
     template:
         metadata:
         labels:
             app: oms
         spec:
         # Add the node selector to specify the IDC where the OMS Community Edition is started.
         # Different regions require different labels on the nodes. For example, the Hangzhou region has zone=zone1.
         nodeSelector:
         zone: "zone1"
         initContainers:
         - name: copy-config
             image: busybox:1.35
             command: ['sh', '-c', 'cp /config-ro/config.yaml /work/config.yaml']
             volumeMounts:
             - name: config-volume
             mountPath: /config-ro
             readOnly: true
             - name: writable-config
             mountPath: /work
         containers:
         - name: oms
             image: ${OMS_IMAGE}
             env:
             - name: POD_NAME
             valueFrom:
                 fieldRef:
                 fieldPath: metadata.name
             - name: OMS_HOST_IP
             # POD_NAME indicates the name of the current Pod. Do not modify it. oms-service.omsregion-1 must be replaced with the actual ${service_name} and ${namespace_name}.
             value: $(POD_NAME).oms-service.omsregion-1.svc
             - name: HOST_DOCKER_VERSION
             value: "20.10.7"
             ports:
             - containerPort: 8088
             - containerPort: 8089
             - containerPort: 8090
             - containerPort: 9000
             - containerPort: 2023
             volumeMounts:
             - name: writable-config
             mountPath: /home/admin/conf/config.yaml
             subPath: config.yaml
             - name: logs-volume
             mountPath: /home/admin/logs
             - name: store-volume
             mountPath: /home/ds/store
             - name: run-volume
             mountPath: /home/ds/run
         volumes:
         - name: config-volume
             configMap:
             name: oms-config
         - name: writable-config
             emptyDir: {}
     # Disk capacity configuration. Adjust the sizes of the following three mount volumes as needed.
     # The minimum recommended size for logs-volume is 200 GiB.
     # The minimum recommended size for store-volume is 500 GiB. Store-volume stores incremental data, so we recommend that you increase the value.
     # The minimum recommended size for run-volume is 300 GiB.
     volumeClaimTemplates:
     - metadata:
         name: logs-volume
         spec:
         accessModes: [ "ReadWriteOnce" ]
         storageClassName: "local-path"
         resources:
             requests:
             storage: 200Gi
     - metadata:
         name: store-volume
         spec:
         accessModes: [ "ReadWriteOnce" ]
         storageClassName: "local-path"
         resources:
             requests:
             storage: 500Gi
     - metadata:
         name: run-volume
         spec:
         accessModes: [ "ReadWriteOnce" ]
         storageClassName: "local-path"
         resources:
             requests:
             storage: 300Gi
     

   • Beijing region configuration file

     apiVersion: v1
     kind: Namespace
     metadata:
     name: omsregion-2
     ---
     apiVersion: v1
     kind: ConfigMap
     metadata:
     name: oms-config
     namespace: omsregion-2
     data:
     config.yaml: |
         "apsara_audit_enable": "false"
         "apsara_audit_sls_access_key": ""
         "apsara_audit_sls_access_secret": ""
         "apsara_audit_sls_endpoint": ""
         "apsara_audit_sls_ops_site_topic": ""
         "apsara_audit_sls_user_site_topic": ""
         "cm_is_default": !!bool "false"
         # The region code. Each region has a unique number.
         "cm_location": "1"
         # The node address, in the format of ${pod_name}.${service_name}.${namespace_name}.svc.
         # pod_name is the name in kind: StatefulSet plus -ID, where the ID starts from 0. For example, if replicas is 2 in a multi-node deployment, pod_name is oms-0 and oms-1.
         # service_name is the name in the first kind: Service.
         # namespace_name is the name in kind: Namespace.
         "cm_nodes":
         - "oms-0.oms-service.omsregion-2.svc"
         - "oms-1.oms-service.omsregion-2.svc"
         "cm_region": "beijing"
         "cm_region_cn": "Beijing"
         "cm_server_port": "8088"
         # The format is http://${service_name}.${namespace_name}.svc:8088.
         "cm_url": "http://oms-service.omsregion-2.svc:8088"
         "drc_cm_db": "cm_db_name"
         "drc_cm_heartbeat_db": "cm_hb_name"
         "drc_rm_db": "rm_db_name"
         "ghana_server_port": "8090"
         "init_db": "true"
         "nginx_server_port": "8089"
         "oms_meta_host": "xxx.xxx.xxx.1"
         "oms_meta_password": "oms_meta_password"
         "oms_meta_port": "2881"
         "oms_meta_user": "xxx@tenant_name"
         "sshd_server_port": "2023"
         "supervisor_server_port": "9000"
         "tsdb_enabled": "true"
         "tsdb_password": "tsdb_password"
         "tsdb_service": "INFLUXDB"
         "tsdb_url": "xxx.xxx.xxx.2:8086"
         "tsdb_username": "tsdb_username"
     ---
     apiVersion: v1
     kind: Service
     metadata:
     name: oms-service
     namespace: omsregion-2
     spec:
     clusterIP: None
     selector:
         app: oms
     ---
     apiVersion: v1
     kind: Service
     metadata:
     name: oms-service-nodeport
     namespace: omsregion-2
     spec:
     type: NodePort
     selector:
         app: oms
     ports:
         - name: nginx
         protocol: TCP
         port: 8089
         targetPort: 8089
         - name: ghana
         protocol: TCP
         port: 8090
         targetPort: 8090
         - name: cm
         protocol: TCP
         port: 8088
         targetPort: 8088
         - name: sshd
         protocol: TCP
         port: 2023
         targetPort: 2023
     ---
     apiVersion: apps/v1
     kind: StatefulSet
     metadata:
     name: oms
     namespace: omsregion-2
     labels:
         app: oms
     spec:
     replicas: 2
     serviceName: "oms-service"
     selector:
         matchLabels:
         app: oms
     template:
         metadata:
         labels:
             app: oms
         spec:
         # Add the node selector to specify the IDC where the OMS Community Edition is started.
         # Different regions require different labels on the nodes. For example, the Beijing region has zone=zone2.
         nodeSelector:
         zone: "zone2"
         initContainers:
         - name: copy-config
             image: busybox:1.35
             command: ['sh', '-c', 'cp /config-ro/config.yaml /work/config.yaml']
             volumeMounts:
             - name: config-volume
             mountPath: /config-ro
             readOnly: true
             - name: writable-config
             mountPath: /work
         containers:
         - name: oms
             image: ${OMS_IMAGE}
             env:
             - name: POD_NAME
             valueFrom:
                 fieldRef:
                 fieldPath: metadata.name
             - name: OMS_HOST_IP
             value: $(POD_NAME).oms-service.omsregion-2.svc
             - name: HOST_DOCKER_VERSION
             value: "20.10.7"
             ports:
             - containerPort: 8088
             - containerPort: 8089
             - containerPort: 8090
             - containerPort: 9000
             - containerPort: 2023
             volumeMounts:
             - name: writable-config
             mountPath: /home/admin/conf/config.yaml
             subPath: config.yaml
             - name: logs-volume
             mountPath: /home/admin/logs
             - name: store-volume
             mountPath: /home/ds/store
             - name: run-volume
             mountPath: /home/ds/run
         volumes:
         - name: config-volume
             configMap:
             name: oms-config
         - name: writable-config
             emptyDir: {}
     # Disk capacity configuration. Adjust the sizes of the following three mount volumes as needed.
     # The minimum recommended size for logs-volume is 200 GiB.
     # The minimum recommended size for store-volume is 500 GiB. Store-volume stores incremental data, so we recommend that you increase the value.
     # The minimum recommended size for run-volume is 300 GiB.
     volumeClaimTemplates:
     - metadata:
         name: logs-volume
         spec:
         accessModes: [ "ReadWriteOnce" ]
         storageClassName: "local-path"
         resources:
             requests:
             storage: 200Gi
     - metadata:
         name: store-volume
         spec:
         accessModes: [ "ReadWriteOnce" ]
         storageClassName: "local-path"
         resources:
             requests:
             storage: 500Gi
     - metadata:
         name: run-volume
         spec:
         accessModes: [ "ReadWriteOnce" ]
         storageClassName: "local-path"
         resources:
             requests:
             storage: 300Gi
     

2. Install the OMS Community Edition in the new region.

   For example, if the configuration file for the new region, Beijing, is named oms-k8s-statefulset-region-2, the installation command is as follows.

   kubectl apply -f oms-k8s-statefulset2.yaml