Single-region multi-node deployment

This topic describes how to deploy OceanBase Migration Service (OMS) Community Edition in a single region and multiple nodes by using k8s.

Terms

OMS_IMAGE: After you load the OMS Community Edition installation package by using Docker, you can run the docker images command to obtain the [IMAGE ID] or [REPOSITORY:TAG] of the loaded image. The [IMAGE ID] or [REPOSITORY:TAG] is the unique identifier of the loaded image, which is represented as <OMS_IMAGE>. Example:

$sudo docker images
REPOSITORY                               TAG                 IMAGE ID          
work.oceanbase-dev.com/oceanbase/oms:feature_4.2.11_ce    2786e8a6eccd        

In this example, <OMS_IMAGE> can be work.oceanbase-dev.com/oceanbase/oms:feature_4.2.11_ce or 2786e8a6eccd. In the subsequent procedures, replace ${OMS_IMAGE} with the value in the preceding example.

Deploy OMS Community Edition

1. Enter the k8s environment and prepare the configuration file.

   Replace the parameters with the actual values based on your deployment environment. The name value in kind: Namespace must be consistent with the namespace value in other types of kind.

   apiVersion: v1
   kind: Namespace
   metadata:
   name: omsregion-1
   ---
   apiVersion: v1
   kind: ConfigMap
   metadata:
   name: oms-config
   namespace: omsregion-1
   data:
   config.yaml: |
       "apsara_audit_enable": "false"
       "apsara_audit_sls_access_key": ""
       "apsara_audit_sls_access_secret": ""
       "apsara_audit_sls_endpoint": ""
       "apsara_audit_sls_ops_site_topic": ""
       "apsara_audit_sls_user_site_topic": ""
       "cm_is_default": !!bool "true"
       "cm_location": "0"
       # The node address is in the format of ${pod_name}.${service_name}.${namespace_name}.svc.
       # The pod_name is the name in kind: StatefulSet plus -ID, where the ID starts from 0. For example, if replicas is 2 in multi-node deployment, the pod_name values are oms-0 and oms-1.
       # The service_name is the name in the first kind: Service.
       # The namespace_name is the name in kind: Namespace.
       "cm_nodes":
       - "oms-0.oms-service.omsregion-1.svc"
       - "oms-1.oms-service.omsregion-1.svc"
       "cm_region": "hangzhou"
       "cm_region_cn": "Hangzhou"
       "cm_server_port": "8088"
       # The format is http://${service_name}.${namespace_name}.svc:8088.
       "cm_url": "http://oms-service.omsregion-1.svc:8088"
       "drc_cm_db": "cm_db_name"
       "drc_cm_heartbeat_db": "cm_hb_name"
       "drc_rm_db": "rm_db_name"
       "ghana_server_port": "8090"
       "init_db": "true"
       "nginx_server_port": "8089"
       "oms_meta_host": "xxx.xxx.xxx.1"
       "oms_meta_password": "oms_meta_password"
       "oms_meta_port": "2881"
       "oms_meta_user": "xxx@tenant_name"
       "sshd_server_port": "2023"
       "supervisor_server_port": "9000"
       "tsdb_enabled": "true"
       "tsdb_password": "tsdb_password"
       "tsdb_service": "INFLUXDB"
       "tsdb_url": "xxx.xxx.xxx.2:8086"
       "tsdb_username": "tsdb_username"
   ---
   apiVersion: v1
   kind: Service
   metadata:
   name: oms-service
   namespace: omsregion-1
   spec:
   clusterIP: None
   selector:
       app: oms
   ---
   apiVersion: v1
   kind: Service
   metadata:
   name: oms-service-nodeport
   namespace: omsregion-1
   spec:
   type: NodePort
   selector:
       app: oms
   ports:
       - name: nginx
       protocol: TCP
       port: 8089
       targetPort: 8089
       - name: ghana
       protocol: TCP
       port: 8090
       targetPort: 8090
       - name: cm
       protocol: TCP
       port: 8088
       targetPort: 8088
       - name: sshd
       protocol: TCP
       port: 2023
       targetPort: 2023
   ---
   apiVersion: apps/v1
   kind: StatefulSet
   metadata:
   name: oms
   namespace: omsregion-1
   labels:
       app: oms
   spec:
   replicas: 2
   serviceName: "oms-service"
   selector:
       matchLabels:
       app: oms
   template:
       metadata:
       labels:
           app: oms
       spec:
       initContainers:
       - name: copy-config
           image: busybox:1.35
           command: ['sh', '-c', 'cp /config-ro/config.yaml /work/config.yaml']
           volumeMounts:
           - name: config-volume
           mountPath: /config-ro
           readOnly: true
           - name: writable-config
           mountPath: /work
       containers:
       - name: oms
           image: ${OMS_IMAGE}
           env:
           - name: POD_NAME
           valueFrom:
               fieldRef:
               fieldPath: metadata.name
           - name: OMS_HOST_IP
           # POD_NAME indicates the name of the current Pod. Do not modify it. oms-service.omsregion-1 must be modified based on the actual ${service_name} and ${namespace_name}.
           value: $(POD_NAME).oms-service.omsregion-1.svc
           - name: HOST_DOCKER_VERSION
           value: "20.10.7"
           ports:
           - containerPort: 8088
           - containerPort: 8089
           - containerPort: 8090
           - containerPort: 9000
           - containerPort: 2023
           volumeMounts:
           - name: writable-config
           mountPath: /home/admin/conf/config.yaml
           subPath: config.yaml
           - name: logs-volume
           mountPath: /home/admin/logs
           - name: store-volume
           mountPath: /home/ds/store
           - name: run-volume
           mountPath: /home/ds/run
       volumes:
       - name: config-volume
           configMap:
           name: oms-config
       - name: writable-config
           emptyDir: {}
   # Configure the disk capacity. Increase the capacity of the following three mount disks as needed.
   # The logs-volume must be at least 200 GiB.
   # The store-volume must be at least 500 GiB. Store incremental data in the store-volume. Increase the value as needed.
   # The run-volume must be at least 300 GiB.
   volumeClaimTemplates:
   - metadata:
       name: logs-volume
       spec:
       accessModes: [ "ReadWriteOnce" ]
       storageClassName: "local-path"
       resources:
           requests:
           storage: 200Gi
   - metadata:
       name: store-volume
       spec:
       accessModes: [ "ReadWriteOnce" ]
       storageClassName: "local-path"
       resources:
           requests:
           storage: 500Gi
   - metadata:
       name: run-volume
       spec:
       accessModes: [ "ReadWriteOnce" ]
       storageClassName: "local-path"
       resources:
           requests:
           storage: 300Gi
   

2. Install OMS Community Edition.

   For example, if the configuration file is named oms-k8s-statefulset.yaml, run the following command:

   kubectl apply -f oms-k8s-statefulset.yaml
   

   To view node details, run the following command:

   kubectl get pods -n ${namespace_name}
   NAME                             READY   STATUS    RESTARTS   AGE
   oms-0                            1/1     Running   0          3m11s
   
   # View the details of the OMS installation logs.
   kubectl logs -f ${pod_name} -n ${namespace_name}
   

3. View the external access port. The port mapped to 8089 is the browser access port for OMS.

   kubectl get svc -n ${namespace_name}
   

Uninstall OMS Community Edition

To uninstall OMS Community Edition, run the following command:

kubectl delete -f ${configuration file name}

Example:

kubectl delete -f oms-k8s-statefulset.yaml