Deploy OceanBase Database in a production environment using the command line

This topic describes how to deploy OceanBase Database in a production environment by using obd in a CLI.

Terms

• Central control server

  The server where obd is installed. The central control server manages the OceanBase cluster and stores the installation packages and configuration information of the OceanBase cluster.

• Target server

  The server where OceanBase Database is installed.

• obd

  OceanBase Deployer is a tool for installing and deploying OceanBase Database. It is called obd for short. For more information, see obd documentation.

• ODP

  OceanBase Database Proxy, abbreviated as ODP (also known as OBProxy), is a dedicated proxy server for OceanBase Database. For more information, see ODP documentation.

• OCP

  OceanBase Cloud Platform, abbreviated as OCP. For more information, see OCP documentation.

• OCP Express

  OCP Express is a web-based management tool for OceanBase Database 4.x. Integrated with an OceanBase cluster, OCP Express allows you to view key performance metrics of the cluster and perform basic database management operations on the cluster. For more information, see OceanBase Cloud Platform Express (OCP Express).

Prerequisites

Before you deploy OceanBase Database, make sure that:

• Your server meets the software and hardware requirements. For more information, see Software and hardware requirements.

• In a production environment, you have performed environment and configuration checks. For more information, see Configure before deployment.

• You have installed obd and the OBClient client on the master control node. We recommend that you install the latest version. For more information, see Install obd and OBClient documentation.

  Note

  When you deploy OceanBase All in One, obd and the OBClient client are automatically installed. If you plan to download and install OceanBase All in One based on the steps provided in Step 1, you can ignore this requirement.

Deployment mode

This topic describes the deployment of three replicas of OceanBase Database. You can choose a suitable deployment solution based on your business requirements. The following table describes the deployment of four servers for OceanBase Database in this example.

Procedure

Step 1: (Optional) Download and install OceanBase All in One

OceanBase Database provides the unified OceanBase All in One package for you to install obd, OceanBase Database, ODP, and OBAgent in one go.

You can also download components of your choice from OceanBase Download Center or specify the versions of the components to download.

Online installation

If your server can connect to the Internet, run the following command to install OceanBase All in One.

[admin@test001 ~]$ bash -c "$(curl -s https://obbusiness-private.oss-cn-shanghai.aliyuncs.com/download-center/opensource/oceanbase-all-in-one/installer.sh)"
[admin@test001 ~]$ source ~/.oceanbase-all-in-one/bin/env.sh

Offline installation

If your server cannot connect to the Internet, follow these steps to install OceanBase All in One offline.

1. Download the latest OceanBase All in One package from OceanBase Download Center and copy it to any directory on the master server.

2. In the directory where the package is located, run the following command to decompress and install the package.

   [admin@test001 ~]$ tar -xzf oceanbase-all-in-one-*.tar.gz
   [admin@test001 ~]$ cd oceanbase-all-in-one/bin/
   [admin@test001 bin]$ ./install.sh
   [admin@test001 bin]$ source ~/.oceanbase-all-in-one/bin/env.sh
   

Step 2: Configure obd

If you deploy the OceanBase cluster offline, you can refer to Step 1 to download and install OceanBase All in One on the master server. The components of OceanBase All in One have been tested for compatibility and are recommended by Ant Group.

If you deploy the OceanBase cluster offline and have specific requirements for the versions of the components, you can download the installation package of the desired version for the corresponding component from OceanBase Download Center. Copy the installation package to any directory on the master server and upload it to the local image repository of obd by following steps 1 to 3.

1. Disable the remote repository.

   [admin@test001 rpm]$ obd mirror disable remote
   

   Note

   By default, the remote repository is disabled after you install OceanBase All in One. You can run the obd mirror list command to confirm this. If the Enabled value of Type=remote is False, the remote image source is disabled.

2. Add the installation package to the local image repository.

   [admin@test001 rpm]$ obd mirror clone *.rpm
   

3. Query the list of installation packages in the local image repository.

   [admin@test001 rpm]$ obd mirror list local
   

4. Select a configuration file.

   If obd was installed by direct download on your server, you can view the sample configuration files provided by obd in the /usr/obd/example directory.

   If obd was installed by decompressing OceanBase All in One on your server, you can view the sample configuration files provided by obd in the ~/.oceanbase-all-in-one/obd/usr/obd/example directory. Select a configuration file based on your resource conditions.

   Note

   Configuration files are divided into two modes: mini and professional. The two modes have basically the same parameters, with slight differences in specifications. You can select a configuration file based on your resource conditions.

   • Mini mode: Suitable for personal devices with at least 8 GB of memory. The name of the configuration file contains the mini or min identifier.

   • Professional mode: Suitable for high-config ECS instances or physical servers with at least 16 CPU cores and 64 GB of memory.

   If you plan to deploy OceanBase on a single server, use the configuration file for single-server deployment.

   • Local single-server deployment sample: mini-local-example.yaml, local-example.yaml

   • Single-server deployment sample: mini-single-example.yaml, single-example.yaml

   • Single-server deployment with ODP sample: mini-single-with-obproxy-example.yaml, single-with-obproxy-example.yaml

   If you plan to deploy OceanBase in a distributed manner across multiple servers, use the configuration file for distributed deployment.

   • Distributed deployment sample: mini-distributed-example.yaml, distributed-example.yaml

   • Distributed deployment with ODP sample: mini-distributed-with-obproxy-example.yaml, distributed-with-obproxy-example.yaml

   • Distributed deployment with ODP and OCP Express sample: default-components.yaml, default-components-min.yaml

   • Complete distributed deployment sample: all-components-min.yaml, all-components.yaml

   Note

   If you plan to deploy OCP Express or OCP, you do not need to deploy Prometheus and Grafana.

5. Modify the configuration file.

   The following example describes how to modify the default-components.yaml file for deploying a distributed OceanBase cluster.

   Note

   You need to modify the following parameters based on your environment.

   1. Modify the username and password.

      ## Only need to configure when remote login is required
      user:
        username: admin
      #   password: your password if need
        key_file: /home/admin/.ssh/id_rsa
      #   port: your ssh port, default 22
      #   timeout: ssh connection timeout (second), default 30
      

      username specifies the username for logging in to a server. Make sure that you have write permission for home_path. password and key_file are used to verify the user. Generally, you only need to specify one of them.

      Notice

      After you configure the key path, if the private key does not require a password, comment out or delete the password parameter. Otherwise, the password specified in the password parameter will be used to log in, which will cause a verification failure.

   2. Modify the IP address, port, and related directories, and configure the memory and password parameters.

      oceanbase-ce:
        # version: 4.3.3.0
        servers:
          # Please don't use hostname, only IP can be supported
          - name: server1
            ip: 10.10.10.1
          - name: server2
            ip: 10.10.10.2
          - name: server3
            ip: 10.10.10.3
        global:
          devname: eth0
          cluster_id: 1
          # please set memory limit to a suitable value which is matching resource. 
          memory_limit: 64G # The maximum running memory for an observer
          system_memory: 30G # The reserved system memory. system_memory is reserved for general tenants.
          datafile_size: 192G # Size of the data file. 
          datafile_next: 200G
          datafile_maxsize: 1T
          log_disk_size: 192G # The size of disk space used by the clog files.
          enable_syslog_wf: false # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true.
          max_syslog_file_count: 4 # The maximum number of reserved log files before enabling auto recycling. The default value is 0.
          # observer cluster name, consistent with obproxy's cluster_name
          appname: obdemo
          ocp_meta_tenant: # The config for ocp express meta tenant
            tenant_name: ocp
            max_cpu: 1
            memory_size: 2G
            log_disk_size: 7680M
          mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started.
          rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started.
          obshell_port: 2886 # Operation and maintenance port for OceanBase Database.
          # The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
          home_path: /home/admin/observer
          # The directory for data storage. The default value is $home_path/store.
          data_dir: /data
          # The directory for clog. The default value is the same as the data_dir value.
          redo_dir: /redo
          root_password: ****** # root user password, can be empty
          proxyro_password: ******** # proxyro user pasword, consistent with obproxy's observer_sys_password, can be empty
        server1:
          zone: zone1
        server2:
          zone: zone2
        server3:
          zone: zone3
      

      You can move inconsistent parameters to the corresponding server and the parameters take precedence over global in the server. Take two servers as an example, where the ports are different:

      server2:
        mysql_port: 3881
        rpc_port: 3882
        zone: zone2
      server3:
        mysql_port: 2881
        rpc_port: 2882
        zone: zone3 
      

      Parameter	Required	Default value	Description
      version	No	The latest version in the image repository	The version of the component to be deployed. Generally, you do not need to specify this parameter.
      servers	Yes	N/A	The IP address of each server is specified in the - name: server identifier (newline)ip: server IP format. Specify the - servers parameter multiple times for multiple servers, and the server identifiers must be unique. If the server IP addresses are unique, you can also use the - <ip> (newline) - <ip> format, where the - <ip> format is equivalent to - name: server IP (newline)ip: server IP.
      devname	No	N/A	The network interface controller (NIC) corresponding to the IP address specified in the - servers parameter. You can run the ip addr command to view the correspondence between the IP address and the NIC.
      memory_limit	No	0	The maximum memory available to the observer process from the environment. If this parameter is not specified, the value of memory_limit_percentage takes effect. For more information, see memory_limit and memory_limit_percentage.
      system_memory	No	0M	The reserved system memory. The value of this parameter reduces the memory specified by memory_limit. If this parameter is not specified, OceanBase Database adaptsively uses the memory.
      datafile_size	No	0	The size of the data file (block_file) for the corresponding node. If this parameter is not specified, the value of datafile_disk_percentage takes effect. For more information, see datafile_size and datafile_disk_percentage.
      datafile_next	No	0	The growth step of disk space, used to set auto expansion. If this parameter is not specified, see Configure dynamic expansion of disk data files for how to enable auto expansion.
      datafile_maxsize	No	0	The maximum available upper limit of disk space, used to set auto expansion. If this parameter is not specified, see Configure dynamic expansion of disk data files for how to enable auto expansion.
      log_disk_size	No	0	The size of the redo log disk. If this parameter is not specified, the value of log_disk_percentage takes effect. For more information, see log_disk_size and log_disk_percentage.
      enable_syslog_wf	No	true	Specifies whether to print system logs of WARN level and above to a separate log file.
      max_syslog_file_count	No	0	The number of log files that can be accommodated before the log files are recycled. A value of 0 indicates that automatic cleanup is not performed.
      appname	No	obcluster	The name of the OceanBase cluster.
      ocp_meta_tenant	No	tenant_name: ocp max_cpu: 1 memory_size: 2147483648	The specifications of the meta tenant for OCP Express. The parameters in the ocp_meta_tenant block are passed as parameters when the tenant is created. Only several important parameters are listed in the preceding example. For more information about configurable parameters, see the parameters supported by the obd cluster tenant create command, which is described in the Cluster commands section.
      mysql_port	Yes	2881	The port number for the SQL service protocol. The default value is 2881.
      rpc_port	Yes	2882	The port number for remote access, which is the RPC communication port between the observer process and other node processes. The default value is 2882.
      obshell_port	Yes	2886	The port number for OceanBase Database maintenance. The default value is 2886.
      home_path	Yes	N/A	The installation path of OceanBase Database. We recommend that you specify this parameter under the admin user.
      data_dir	No	$home_path/store	The directory for storing SSTables and other data. We recommend that you specify an independent disk for this parameter.
      redo_dir	No	Same as data_dir	The directory for clogs. The default value is the same as that of data_dir. We recommend that you specify an independent disk for this parameter.
      root_password	No	obd V2.1.0 before: Empty obd V2.1.0 and later: Random string	The password of the root@sys user of the OceanBase cluster. We recommend that you set a complex password. If you use obd V2.1.0 or later, if this parameter is not specified, a random string is generated.
      proxyro_password	No	obd V2.1.0 before: Empty obd V2.1.0 and later: Random string	The password of the proxyro@sys user, which is used to connect ODP to the OceanBase cluster. If you use obd V2.1.0 or later, if this parameter is not specified, a random string is generated.

   3. Configure the obproxy-ce component and modify the IP address and home_path.

      obproxy-ce:
        # version: 4.3.2.0
        depends:
          - oceanbase-ce
        servers:
          - 10.10.10.4
        global:
          listen_port: 2883 # External port. The default value is 2883.
          prometheus_listen_port: 2884 # The Prometheus port. The default value is 2884.
          rpc_listen_port: 2885
          enable_obproxy_rpc_service: true
          home_path: /home/admin/obproxy
          client_session_id_version: 2 # This parameter is used to specify whether to use the new logic to generate the client session ID. The parameter type is integer. The value range is [1, 2] and the default value is 2 (use the new logic).
          proxy_id: 5 # This parameter is used to set the ID for an ODP. The parameter type is integer. The default value is 0 and the value range is [0, 8191].
          enable_cluster_checkout: false
          skip_proxy_sys_private_check: true
          enable_strict_kernel_release: false
          obproxy_sys_password: ****** # obproxy sys user password, can be empty. When a depends exists, obd gets this value from the oceanbase-ce of the depends.
          observer_sys_password: ***** # proxyro user pasword, consistent with oceanbase-ce's proxyro_password, can be empty. When a depends exists, obd gets this value from the oceanbase-ce of the depends.
      

      Parameter	Required	Default value	Description
      version	No	The latest version in the image repository	The version of the component to be deployed. Generally, you do not need to specify this parameter.
      servers	Yes	N/A	The IP address of each server is specified in the - name: server identifier (newline)ip: server IP format. Specify the - servers parameter multiple times for multiple servers, and the server identifiers must be unique. If the server IP addresses are unique, you can also use the - <ip> (newline) - <ip> format, where the - <ip> format is equivalent to - name: server IP (newline)ip: server IP.
      listen_port	Yes	2883	The listening port of ODP. The default value is 2883.
      prometheus_listen_port	Yes	2884	The listening port of ODP's Prometheus server. The default value is 2884.
      rpc_listen_port	No	2885	The listening port of the RPC service of ODP. The default value is 2885. Note This parameter takes effect only when ODP V4.3.0 or later is deployed and enable_obproxy_rpc_service is set to true.
      enable_obproxy_rpc_service	No	true	Specifies whether to enable the RPC service of ODP. The default is true, which indicates that the RPC service is enabled. If you set this parameter to false, the RPC service is disabled and the configuration of rpc_listen_port becomes invalid. Note This parameter takes effect only when ODP V4.3.0 or later is deployed. If this parameter is not specified, the default value of false will take effect after ODP is upgraded.
      home_path	Yes	N/A	The installation path of ODP. We recommend that you specify this parameter under the admin user.
      client_session_id_version	No	2	Specifies whether to use the new Client Session ID generation logic. The value of this parameter can only be 1 or 2. If this parameter is set to 2, the new Client Session ID generation logic is used.
      proxy_id	No	0	The ID that uniquely identifies ODP to ensure that the Client Session IDs generated by different ODPs do not conflict. The value of this parameter depends on the value of client_session_id_version. The table below describes the value range of this parameter in different cases. If client_session_id_version is set to 1, the value range of proxy_id is [0,255]. If client_session_id_version is set to 2, the value range of proxy_id is [0,8191].
      obproxy_sys_password	No	obd V2.1.0 before: Empty obd V2.1.0 and later: Random string	The password of the root@proxysys user of ODP. If you use obd V2.1.0 or later, if this parameter is not specified, a random string is generated.
      observer_sys_password	No	obd V2.1.0 before: Empty obd V2.1.0 and later: The same random string as proxyro_password	The password of the proxyro@sys user, which is used to connect ODP to the OceanBase cluster. The value of this parameter must be the same as that of proxyro_password. If you use obd V2.1.0 or later, if this parameter is not specified, a random string identical to that of proxyro_password is generated.

   4. Modify the IP address and home_path of the obagent and ocp-express components.

      Notice

      Starting from V2.0.0, obd supports the deployment of OCP Express. obd V2.0.0 only supports the deployment of OCP Express V1.0.0, and OCP Express V1.0.1 and later versions require obd V2.1.0 or later for deployment.

      obagent:
        depends:
          - oceanbase-ce
        servers:
          - name: server1
            # Please don't use hostname, only IP can be supported
            ip: 10.10.10.1
          - name: server2
            ip: 10.10.10.2
          - name: server3
            ip: 10.10.10.3
        global:
          home_path: /home/admin/obagent
      ocp-express:
        depends:
          - oceanbase-ce
          - obproxy-ce
          - obagent
        servers:
          - name: server1
            ip: 10.10.10.4
        global:
          # The working directory for prometheus. prometheus is started under this directory. This is a required field.
          home_path: /home/oceanbase/ocp-server
          memory_size: 1G # The memory size of ocp-express server.  The recommend value is 512MB + (expect node num + expect tenant num) * 60MB.
          admin_passwd: ********
          logging_file_total_size_cap: 10GB # The total log file size of ocp-express server
          # logging_file_max_history: 1 # The maximum of retention days the log archive log files to keep. The default value is unlimited
      

      Parameter	Required	Default value	Description
      servers	Yes	N/A	The IP address of each server is specified in the - name: server identifier (newline)ip: server IP format. Specify the - servers parameter multiple times for multiple servers, and the server identifiers must be unique. If the server IP addresses are unique, you can also use the - <ip> (newline) - <ip> format, where the - <ip> format is equivalent to - name: server IP (newline)ip: server IP.
      home_path	Yes	N/A	The installation path of the component. We recommend that you specify this parameter under the admin user.
      memory_size	Yes	N/A	The memory size of the OCP Express server. We recommend that you use the following formula to calculate the memory size: memory_size = 512 MB + (number of expected nodes × number of expected tenants) × 60 MB, where the number of expected tenants must include the sys tenant and the OCP meta tenant.
      logging_file_total_size_cap	Yes	1GB	The total size of log files. The default value is 1 GB. Notice The unit of this parameter must be GB or MB. If you use G or M as the unit, an error will occur and OCP Express cannot be deployed.
      admin_passwd	Yes	N/A	The password of the admin user for logging in to the OCP Express management console. The password must be 8 to 32 characters in length and must contain at least two digits, two uppercase letters, two lowercase letters, and two special characters (~!@#%^&*_-+=	(){}[]:;,.?/). If you use obd V2.1.0 or later, if this parameter is not specified, a random string is generated.

Step 3: Deploy the OceanBase cluster

1. Deploy the OceanBase cluster.

   [admin@test001 ~]$ obd cluster deploy obtest -c default-components.yaml
   

   If the target server is connected to the Internet, obd will check whether the installation package is deployed on the server when you run the obd cluster deploy command. If not, obd will automatically obtain the installation package from the YUM source.

2. Start the OceanBase cluster.

   [admin@test001 ~]$ obd cluster start obtest
   

   If OCP Express is installed, the access address of OCP Express is displayed. On Alibaba Cloud or in other cloud environments, the installer may output an intranet IP address because it cannot obtain a public IP address. Make sure that you use a correct public IP address.

3. Query the status of the OceanBase cluster.

   # Query the list of clusters managed by obd
   [admin@test001 ~]$ obd cluster list
   
   # Query the status of the obtest cluster
   [admin@test001 ~]$ obd cluster display obtest
   

4. (Optional) Modify the cluster settings.

   OceanBase Database has hundreds of parameters, and some parameters are coupled. We recommend that you do not modify the parameters in the sample configuration file before you become familiar with OceanBase Database. The following example shows how to modify a parameter and make the modification take effect.

   Run the obd cluster edit-config command to go to the parameter editing mode, and modify the cluster parameters.

   [admin@test001 ~]$ obd cluster edit-config obtest
   

   After you modify and save the parameters and exit, obd prompts you on how to make the modification take effect. Enter the following command after you save the modifications:

   Search param plugin and load ok
   Search param plugin and load ok
   Parameter check ok
   Save deploy "obtest" configuration
   Use `obd cluster reload obtest` to make changes take effect.
   

   Copy and execute the command provided by obd.

   [admin@test001 ~]$ obd cluster reload obtest
   

Step 4: Connect to the OceanBase cluster

In this example, OBClient is used to connect to an OceanBase cluster:

obclient -h<IP> -P<PORT> -u<user_name>@<tenant_name>#<cluster_name> -p -c -A
# example
obclient -h10.10.10.4 -P2883 -uroot@sys#obdemo -p -c -A

The parameters are described as follows:

• -h: specifies the IP address for connecting to the OceanBase Database. If you connect to the database directly, use the IP address of an OBServer node. If you connect to the database through an ODP, use the IP address of the ODP.

• -u: specifies the account for connecting to the OceanBase Database. The account format is one of the following: username@tenant name#cluster name, cluster name:tenant name:username, cluster name-tenant name-username, or cluster name.tenant name.username. The default username of the administrator of a MySQL tenant is root.

  Notice

  The cluster name used for connection is the appname value in the configuration file, rather than the deploy name value.

• -P: specifies the port for connecting to the OceanBase Database. If you connect to the database directly, use the value of mysql_port. If you connect to the database through an ODP, use the value of listen_port.

• -p: specifies the password for connecting to the OceanBase Database.

• -c: specifies not to ignore comments in the OBClient runtime environment.

  Note

  Hint, which is a special comment, is not affected by the -c option.

• -A: specifies not to automatically obtain the statistics when you use OBClient to connect to the database.

For more information about how to connect to an OceanBase cluster, see Connect to OceanBase Database.

Step 5: Create a user tenant

We recommend that you create a user tenant for business operations after the OceanBase cluster is deployed. The sys tenant is intended only for cluster management and is unsuitable for business scenarios.

You can create a user tenant by using one of the following methods:

• Method 1: Use obd to create a user tenant.

  obd cluster tenant create <deploy name> [-n <tenant name>] [flags]
  # example
  obd cluster tenant create test -n obmysql --max-cpu=2 --memory-size=2G --log-disk-size=3G --max-iops=10000 --iops-weight=2 --unit-num=1 --charset=utf8
  

  By default, this command creates a tenant based on all the remaining resources of the cluster. You can specify the parameters to control the resources occupied by the tenant. For more information about the command, see obd cluster tenant create in the obd documentation.

• Method 2: Create a user tenant from the command line. For more information, see Create a tenant.

Related operations

Manage clusters

You can run the following commands to manage a cluster deployed by using obd. For more information, see Cluster commands.

# View the list of clusters
obd cluster list

# View the status of a cluster. Here is an example of viewing the status of the obtest cluster.
obd cluster display obtest

# Stop a running cluster. Here is an example of stopping the obtest cluster.
obd cluster stop obtest

# Destroy a deployed cluster. Here is an example of destroying the obtest cluster.
obd cluster destroy obtest

Uninstall OceanBase All in One

You can follow the steps below to uninstall OceanBase All in One.

1. Uninstall the installation package.

   [admin@test001 ~]$ cd oceanbase-all-in-one/bin/
   [admin@test001 bin]$ ./uninstall.sh
   

2. Delete environment variables.

   [admin@test001 bin]$ vim ~/.bash_profile
   # Delete source /home/admin/.oceanbase-all-in-one/bin/env.sh from the file.
   

3. Make the changes take effect.

   After you delete the environment variables, you must log in to the terminal again or run the source command to make the changes take effect. Here is an example:

   [admin@test001 ~]$ source ~/.bash_profile