Deploy OceanBase Database in a production environment by using the CLI

This topic describes how to deploy OceanBase Database in a production environment by running OceanBase Deployer (obd) commands on the CLI.

Terms

• Central control server

  The server on which obd is installed. It stores the installation package and configuration information about OceanBase clusters and is used for cluster management.

• Target server

  The server where you want to install OceanBase Database.

• OceanBase Database

  A self-developed enterprise-grade native distributed database. For more information, see OceanBase Database documentation.

• obd

  OceanBase Deployer (obd) is a tool used to install and deploy OceanBase Database. For more information, see obd Documentation.

• ODP

  OceanBase Database Proxy (ODP), also known as OBProxy, is a dedicated proxy service designed for OceanBase Database. For more information, see ODP Documentation.

• OBAgent

  OBAgent is a monitoring data collection framework for OceanBase Database. It supports both push and pull data collection modes to meet different application scenarios.

• obconfigserver

  OceanBase Configserver (obconfigserver) provides metadata registration, storage, and query services for OceanBase. For more information, see ob-configserver.

• Grafana

  Grafana is an open-source data visualization tool that displays various metrics from data sources to help you intuitively understand the running status and performance metrics of the system. For more information, see the Grafana website.

• Prometheus

  Prometheus is an open-source service monitoring system and time series database. It provides a general data model and fast data collection, storage, and query interfaces. For more information, see the Prometheus website.

Prerequisites

Before you deploy OceanBase Database, make sure that the following requirements are met:

• Your server meets the software and hardware requirements. For more information, see Software and hardware requirements.

• In the production environment, you must check the environment and configurations. For more information, see Configuration before deployment.

• You have installed obd and OBClient on the central control server. We recommend that you install the latest versions. For more information, see Install obd in the OceanBase Installation and Deployment Tool documentation and the OBClient documentation.

  Note

  When you install OceanBase All in One, both obd and OBClient are automatically installed. Therefore, if you intend to download and install OceanBase All in One by following the steps in Step 1, you can ignore this requirement.

Deployment mode

This topic describes a three-replica deployment mode. We recommend that you use four servers for this deployment mode. You can also choose an appropriate deployment solution based on your actual needs. The following table describes how the four servers are used.

Procedure

Step 1: (Optional) Download and install OceanBase All in One

OceanBase provides a unified installation package named OceanBase All in One since V4.0.0. You can use this package to install obd, OceanBase Database, ODP, OBAgent, and other components at a time.

You can also download and install desired components or components of specified versions from OceanBase Download Center based on your actual needs.

Online installation

If your server can connect to the Internet, run the following commands to install the components online:

[admin@test001 ~]$ bash -c "$(curl -s https://obbusiness-private.oss-cn-shanghai.aliyuncs.com/download-center/opensource/oceanbase-all-in-one/installer.sh)"
[admin@test001 ~]$ source ~/.oceanbase-all-in-one/bin/env.sh

Offline installation

If your server cannot connect to the Internet, perform the following steps to install the components offline:

1. Download the latest OceanBase All in One package from OceanBase Download Center and copy it to any directory on the central control server.

2. In the directory where the package is located, run the following commands to decompress and install the package:

   [admin@test001 ~]$ tar -xzf oceanbase-all-in-one-*.tar.gz
   [admin@test001 ~]$ cd oceanbase-all-in-one/bin/
   [admin@test001 bin]$ ./install.sh
   [admin@test001 bin]$ source ~/.oceanbase-all-in-one/bin/env.sh
   

Step 2: Configure obd

If you are deploying an OceanBase cluster offline, you can download and install OceanBase All in One on the central control server by following the steps in Step 1. The components provided in OceanBase All in One have been adapted to each other and are officially recommended.

If you are deploying the cluster offline and have special requirements for the versions of the components, you can download the installation packages of the desired versions from OceanBase Download Center, copy the packages to any directory on the central control server, and perform the following steps 1 to 3 in this directory to upload the packages to the local image repository of obd.

1. Disable remote repositories.

   [admin@test001 rpm]$ obd mirror disable remote
   

   Note

   After you install OceanBase All in One, the remote repositories are automatically disabled. You can run the obd mirror list command for confirmation. If the value of Enabled that corresponds to Type=remote changes to False, the remote image sources are disabled.

2. Add the installation packages to the local image repository.

   [admin@test001 rpm]$ obd mirror clone *.rpm
   

3. View the list of installation packages in the local image repository.

   [admin@test001 rpm]$ obd mirror list local
   

4. Select a configuration file.

   If obd is directly downloaded and installed on your server, you can view the sample configuration files provided by obd in the /usr/obd/example directory.

   If obd is installed by using OceanBase All in One, you can view the sample configuration files provided by obd in the ~/.oceanbase-all-in-one/obd/usr/obd/example directory. Select the corresponding configuration file based on your resource conditions.

   Note

   Two types of configuration files are available: one for the small-scale development mode and one for the professional development mode. The configuration files for the two modes contain basically the same parameters but different parameter values. You can select a file based on your resource conditions.

   • The small-scale development mode applies to individual devices with at least 8 GB of memory. The configuration file name contains mini or min.

   • The professional development mode applies to advanced Elastic Compute Service (ECS) instances or physical servers with at least 16 CPU cores and 64 GB of memory.

   If you select standalone deployment, only one target server is required. In this case, you can use a configuration file for standalone deployment.

   • Sample configuration files for local standalone deployment: mini-local-example.yaml and local-example.yaml

   • Sample configuration files for standalone deployment: mini-single-example.yaml and single-example.yaml

   • Sample configuration files for standalone deployment with ODP: mini-single-with-obproxy-example.yaml and single-with-obproxy-example.yaml

   If you select distributed deployment, multiple target servers are required. In this case, you can use a configuration file for distributed deployment.

   • Sample configuration files for distributed deployment: mini-distributed-example.yaml and distributed-example.yaml

   • Sample configuration files for distributed deployment with ODP: mini-distributed-with-obproxy-example.yaml and distributed-with-obproxy-example.yaml

   • Sample configuration files for distributed deployment with all components: all-components-min.yaml and all-components.yaml

5. Modify the configuration file.

   Note

   You need to modify the following parameters based on the actual environment.

   1. Create a configuration file.

      vim deploy.yaml
      

      deploy.yaml is only an example. You can customize the name of the configuration file based on your actual needs.

   2. Change the username and password.

      ## Only need to configure when remote login is required
      user:
        username: admin
      #   password: your password if need
        key_file: /home/admin/.ssh/id_rsa
      #   port: your ssh port, default 22
      #   timeout: ssh connection timeout (second), default 30
      

      Here, username specifies the username of the account used to log in to the target server. Make sure that this account has the write privilege on home_path. password and key_file are used for user verification. Generally, you only need to specify either of them.

      Notice

      After you specify the path of the key, comment out or delete the password field if your key does not require a password. Otherwise, the value of the password parameter will be taken as the password of the key and used for login, leading to a login verification failure.

   3. Modify the IP address, port, and related directories of each server, and specify memory-related parameters and the password.

      oceanbase-ce:
        # version: 4.3.3.0
        servers:
          # Please don't use hostname, only IP can be supported
          - name: server1
            ip: 10.10.10.1
          - name: server2
            ip: 10.10.10.2
          - name: server3
            ip: 10.10.10.3
        global:
          devname: eth0
          cluster_id: 1
          # please set memory limit to a suitable value which is matching resource. 
          memory_limit: 64G # The maximum running memory for an observer
          system_memory: 30G # The reserved system memory. system_memory is reserved for general tenants.
          datafile_size: 192G # Size of the data file. 
          datafile_next: 200G
          datafile_maxsize: 1T
          log_disk_size: 192G # The size of disk space used by the clog files.
          scenario: htap
          enable_syslog_wf: false # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true.
          max_syslog_file_count: 4 # The maximum number of reserved log files before enabling auto recycling. The default value is 0.
          enable_auto_start: true
          # observer cluster name, consistent with obproxy's cluster_name
          appname: obdemo
          mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started.
          rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started.
          obshell_port: 2886 # Operation and maintenance port for OceanBase Database.
          # The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
          home_path: /home/admin/observer
          # The directory for data storage. The default value is $home_path/store.
          data_dir: /data
          # The directory for clog. The default value is the same as the data_dir value.
          redo_dir: /redo
          root_password: ****** # root user password, can be empty
          proxyro_password: ******** # proxyro user pasword, consistent with obproxy's observer_sys_password, can be empty
        server1:
          zone: zone1
        server2:
          zone: zone2
        server3:
          zone: zone3
      

      If the servers have inconsistent parameters, move the relevant parameters to the sections of target servers and separately set the parameters for the servers. Parameters in server sections have higher priority than those in the global section. For example, you can modify the configuration file as follows to configure different ports for two servers:

      server2:
        mysql_port: 3881
        rpc_port: 3882
        zone: zone2
      server3:
        mysql_port: 2881
        rpc_port: 2882
        zone: zone3
      

      Parameter	Required?	Default value	Description
      version	No	The latest version in the image repository	The version of the component to be deployed. Generally, you do not need to specify this parameter.
      servers	Yes	N/A	Specify each server in the format of - name: server identifier (line feed) ip: server IP address. You can specify multiple servers. The server identifiers must be unique. If the server IP addresses are unique, you can also specify the servers in the - <ip> (line break)- <ip> format. In this case, - <ip> is equivalent to - name: server IP address (line break) ip: server IP address.
      devname	No	N/A	The NIC corresponding to the IP address specified in servers. You can view the mapping between IP addresses and NICs by using the ip addr command.
      memory_limit	No	0	The maximum memory that the observer process can obtain from the environment. If this parameter is not configured, the memory_limit_percentage parameter takes effect. For more information about the two parameters, see memory_limit and memory_limit_percentage.
      system_memory	No	0M	The reserved system memory, which is part of memory_limit. If this parameter is not configured, OceanBase Database reserves memory for the system in adaptive mode.
      datafile_size	No	0	The size of the data file block_file on the corresponding OBServer node. If this parameter is not configured, the datafile_disk_percentage parameter takes effect. For more information, see datafile_size and datafile_disk_percentage.
      datafile_next	No	0	The auto scaling step of disk space. If this parameter is not configured, to enable auto scaling, see Configure auto scaling of disk data files.
      datafile_maxsize	No	0	The maximum disk space allowed for auto scaling. If this parameter is not configured, to enable auto scaling, see Configure auto scaling of disk data files.
      log_disk_size	No	0	The size of the redo log disk. If this parameter is not configured, the log_disk_percentage parameter takes effect. For more information, see log_disk_size and log_disk_percentage.
      scenario	No	htap	Specifies the load type of the cluster. If this parameter is not configured, an interactive option is provided for you to select the load type. Valid values: express_oltp: suitable for workloads such as trading, payment core systems, and high-throughput internet applications. It has no foreign key constraints, stored procedures, long transactions, large transactions, complex joins, or complex subqueries. complex_oltp: suitable for workloads such as banking and insurance systems. They usually involve complex joins, complex correlated subqueries, batch jobs written in PL, and long and large transactions. Parallel execution may be used for short-term queries. olap: suitable for real-time data warehouse analysis scenarios. htap: suitable for mixed OLAP and OLTP workloads. It is usually used to obtain instant insights from active operational data, fraud detection, and personalized recommendations. kv: suitable for key-value workloads and HBase-like wide-column workloads, which usually have very high throughput demands and are latency-sensitive.
      enable_syslog_wf	No	true	Specifies whether to print system logs above the WARN level to a separate log file.
      max_syslog_file_count	No	0	The maximum number of system log files that can be retained. The value 0 indicates to disable automatic recycling.
      enable_auto_start	No	0	Specifies whether to enable the auto-start feature for the observer process. If this parameter is set to true, the observer process is automatically started when the OBServer node restarts. Note When you configure this parameter, make sure that the deployment user has the sudo privilege and the deployment environment is not a container environment.
      appname	No	obcluster	The name of the OceanBase cluster.
      mysql_port	Yes	2881	The SQL port number. Default value: 2881.
      rpc_port	Yes	2882	The Remote Procedure Call (RPC) port number, which is used for communication between the observer process and other node processes. Default value: 2882.
      obshell_port	Yes	2886	The O&M port of OceanBase Database. Default value: 2886.
      home_path	Yes	N/A	The installation path of OceanBase Database. We recommend that you choose a path under the admin user.
      data_dir	No	$home_path/store	The directory for storing SSTables and other data. We recommend that you use a separate disk.
      redo_dir	No	Same as data_dir	The directory for storing clogs. We recommend that you use a separate disk.
      root_password	No	A random string	The password of the super administrator (root@sys) of the OceanBase cluster. We recommend that you set a complex password. If you do not specify this parameter in obd V2.1.0 or later, a random string is automatically generated.
      proxyro_password	No	A random string	The password of the account for connecting the OceanBase cluster (proxyro@sys) through ODP. If you do not specify this parameter in obd V2.1.0 or later, a random string is automatically generated.

   4. Configure the obproxy-ce component and modify the IP address and home_path.

      obproxy-ce:
        # version: 4.3.4.0
        # package_hash: 5c5dca3a355cc7286146ed978ebb26a6342e42e02cd3ca8b9739d300519a449f
        depends:
          - oceanbase-ce
        servers:
          - name: server1
            # Please don't use hostname, only IP can be supported
            ip: 10.10.10.1
          - name: server2
            ip: 10.10.10.2
          - name: server3
            ip: 10.10.10.3
        global:
          prometheus_listen_port: 2884
          listen_port: 2883
          rpc_listen_port: 2885
          enable_obproxy_rpc_service: true
          # vip_address: "10.10.10.5"
          # vip_port: 3306
          home_path: /home/admin/obtest/obproxy
          client_session_id_version: 2
          proxy_id: 822
          obproxy_sys_password: ********
          observer_sys_password: ********
          skip_proxy_sys_private_check: true
          enable_strict_kernel_release: false
          enable_cluster_checkout: false 
      

      Parameter	Required?	Default value	Description
      version	No	The latest version in the image repository	The version of the component to be deployed. Generally, you do not need to specify this parameter.
      depends	No	N/A	Specifies the dependency relationships between components. After this parameter is configured, the current component automatically obtains information from the components it depends on. This parameter is required when a dependency relationship exists between components.
      servers	Yes	N/A	Specify each server in the format of - name: server identifier (line feed) ip: server IP address. You can specify multiple servers. The server identifiers must be unique. If the server IP addresses are unique, you can also specify the servers in the - <ip> (line break)- <ip> format. In this case, - <ip> is equivalent to - name: server IP address (line break) ip: server IP address.
      listen_port	Yes	2883	The ODP listening port. Default value: 2883.
      prometheus_listen_port	Yes	2884	The listening port of ODP Prometheus. Default value: 2884.
      rpc_listen_port	No	2885	The port for RPC communication in ODP. The default value is 2885. Note This parameter takes effect only when ODP V4.3.0 or later is deployed and the value of enable_obproxy_rpc_service is true.
      enable_obproxy_rpc_service	No	true	Specifies whether to enable the RPC service in ODP. The default value is true, which means the RPC service is enabled. The value false means the RPC service is disabled. In this case, the rpc_listen_port parameter will not take effect. Note This parameter takes effect only when ODP V4.3.0 or later is deployed. If this parameter is not configured, the default value false will be used after ODP is upgraded.
      vip_address	No	N/A	When you deploy multiple ODP nodes, you can use this parameter to configure the IP address of the VIP. Notice obd V3.3.0 and later support configuring a VIP for ODP. obd does not provide load balancing deployment. If you need to configure load balancing, deploy and configure the load balancing component in advance.
      vip_port	No	N/A	When you deploy multiple ODP nodes, you can use this parameter to configure the access port of the VIP. Notice obd V3.3.0 and later support configuring a VIP for ODP. obd does not provide load balancing deployment. If you need to configure load balancing, deploy and configure the load balancing component in advance.
      home_path	Yes	N/A	The installation path of ODP. We recommend that you set it to the home directory of the admin user.
      client_session_id_version	No	2	Specifies whether to use the new logic to generate client session IDs. Only the value 1 or 2 can be specified. The value 2 indicates to use the new logic to generate client session IDs.
      proxy_id	No	0	The ID of the ODP, which ensures that client session IDs generated by different ODPs do not conflict. The value range of this parameter is different based on the value of client_session_id_version. If client_session_id_version is set to 1, the value range of proxy_id is [0,255]. If client_session_id_version is set to 2, the value range of proxy_id is [0,8191].
      obproxy_sys_password	No	A random string	The password of the ODP administrator account (root@proxysys). If you do not specify this parameter in obd V2.1.0 or later, a random string is automatically generated.
      observer_sys_password	No	A random string same as proxyro_password	The password of the account (proxyro@sys) for connecting to the OceanBase cluster through ODP. Set this password to the same value as proxyro_password. If you do not specify this parameter in obd V2.1.0 or later, a random string same as proxyro_password is automatically generated.

   5. Modify the monitoring components.

      obagent:
        depends:
          - oceanbase-ce
        # The list of servers to be monitored. This list is consistent with the servers in oceanbase-ce.
        servers:
          - name: server1
            ip: 10.10.10.1
          - name: server2
            ip: 10.10.10.2
          - name: server3
            ip: 10.10.10.3
        global:
          monagent_http_port: 8088
          mgragent_http_port: 8089
          home_path: /home/admin/obagent
          http_basic_auth_password: ********
      prometheus:
        servers:
          - 10.10.10.4
        depends:
          - obagent
        global:
          port: 9090
          # The working directory for prometheus. prometheus is started under this directory. This is a required field.
          home_path: /home/admin/prometheus
          basic_auth_users: #<username>: <password>, the key is the user name and the value is the password.
            admin: ********
      grafana:
        servers:
          - 10.10.10.4
        depends:
          - prometheus
        global:
          port: 3000
          login_password: ********
          home_path: /home/admin/grafana
      

      Parameter	Required?	Default value	Description
      servers	Yes	N/A	Specify each server in the format of - name: server identifier (line feed) ip: server IP address. You can specify multiple servers. The server identifiers must be unique. If the server IP addresses are unique, you can also specify the servers in the - <ip> (line break)- <ip> format. In this case, - <ip> is equivalent to - name: server IP address (line break) ip: server IP address.
      depends	No	N/A	Specifies the dependency relationships between components. After this parameter is configured, the current component automatically obtains information from the components it depends on. This parameter is required when a dependency relationship exists between components.
      home_path	Yes	N/A	The installation path of the component. We recommend that you use a path under the admin user.
      monagent_http_port	Yes	8088	The port of the OBAgent monitoring service.
      mgragent_http_port	Yes	8089	The port of the OBAgent management service.
      http_basic_auth_password	No	A random string	The authentication password of the HTTP service. If you customize the password, it must be at least one character in length and can contain digits (0 to 9), uppercase letters (A to Z), lowercase letters (a to z), and special characters (~^*{}[]_-+). If this parameter is not configured, obd automatically generates a random string. After successful deployment, you can run the obd cluster edit-config command to view the password in the corresponding parameter of the configuration file.
      basic_auth_users	No	admin: *****, where admin is the username and ***** is a randomly generated password	The authentication information of the Prometheus web service. The key is the username and the value is the password.
      login_password	No	A random string	The logon password of Grafana. After successful deployment, you can run the obd cluster edit-config command to view the password in the corresponding parameter of the configuration file.

Step 3: Deploy the OceanBase cluster

1. Deploy an OceanBase cluster.

   [admin@test001 ~]$ obd cluster deploy obtest -c deploy.yaml
   

   After you run the obd cluster deploy command, if your server is connected to the Internet, obd checks whether the desired installation packages exist on the target server. If no, obd automatically obtains the installation packages from the YUM source.

2. Start the OceanBase cluster.

   [admin@test001 ~]$ obd cluster start obtest
   

   After the cluster is started, the access addresses of the obshell Dashboard and the monitoring components are returned. On Alibaba Cloud or other cloud environments, the installation program may fail to obtain a public IP address and thereby return an intranet network address. Make sure that you use the correct address.

3. View the status of the OceanBase cluster.

   # View the list of clusters managed by obd.
   [admin@test001 ~]$ obd cluster list
   
   # View the status of the obtest cluster.
   [admin@test001 ~]$ obd cluster display obtest
   

4. (Optional) Modify the cluster configurations.

   OceanBase Database has hundreds of parameters and some are coupled. We recommend that you do not modify parameters in the sample configuration file before you become familiar with OceanBase Database. The following example shows you how to modify a parameter and make it take effect:

   Run the obd cluster edit-config command to enter the edit mode before you can edit the cluster configurations.

   [admin@test001 ~]$ obd cluster edit-config obtest
   

   After you modify and save the configurations and exit, obd will prompt you on how to validate the modifications. The output after saving the configurations is as follows:

   Search param plugin and load ok
   Search param plugin and load ok
   Parameter check ok
   Save deploy "obtest" configuration
   Use `obd cluster reload obtest` to make changes take effect.
   

   Copy and run the command provided by obd.

   [admin@test001 ~]$ obd cluster reload obtest
   

Step 4: Connect to the OceanBase cluster

Here, we use OBClient as an example to describe how to connect to the OceanBase cluster:

obclient -h<IP> -P<PORT> -u<user_name>@<tenant_name>#<cluster_name> -p -c -A
# example
obclient -h10.10.10.4 -P2883 -uroot@sys#obdemo -p -c -A

where

• -h specifies the IP address for connecting to OceanBase Database. The value is the IP address of an OBServer node in the case of direct connection, and is the IP address of ODP in the case of connection through ODP.

• -u specifies the account for connecting to OceanBase Database. Supported account formats are username@tenant name#cluster name, cluster name:tenant name:username, cluster name-tenant name-username, and cluster name.tenant name.username. The default administrator username for a MySQL tenant is root.

  Notice

  The cluster name used for connection is the one specified by appname in the configuration file, instead of the one specified by deploy name during deployment.

• -P specifies the port for connecting to OceanBase Database. It is the value of the mysql_port parameter in the case of direct connection, and the value of the listen_port parameter in the case of connection through ODP.

• -p specifies the password for connecting to OceanBase Database.

• -c specifies not to ignore comments in the runtime environment of OBClient.

  Note

  Hints are special comments that are not affected by the -c option.

• -A specifies not to automatically retrieve statistics when connecting to OceanBase Database by using OBClient.

For more information about how to connect to an OceanBase cluster, see Connect to OceanBase Database.

Step 5: Create a user tenant

After the OceanBase cluster is deployed, we recommend that you create a user tenant to perform business operations. The sys tenant is intended only for cluster management and is unsuitable for business scenarios.

You can use one of the following methods to create a user tenant:

• Method 1: Create a user tenant by using obd.

  obd cluster tenant create <deploy name> [-n <tenant name>] [flags]
  # example
  obd cluster tenant create test -n obmysql --max-cpu=2 --memory-size=2G --log-disk-size=3G --max-iops=10000 --iops-weight=2 --unit-num=1 --charset=utf8
  

  By default, this command creates a tenant based on all remaining available resources of the cluster. You can configure the parameters to allocate specific amount of resources to the tenant. For more information about this command, see obd cluster tenant create in Cluster commands.

• Method 2: Create a user tenant by using the CLI. For more information, see Create a tenant.

Related operations

Manage clusters

You can run the following commands to manage OceanBase clusters deployed by using obd. For more information, see Cluster commands in obd Documentation.

# View the cluster list.
obd cluster list

# View the status of a cluster. The following takes the `obtest` cluster as an example.
obd cluster display obtest

# Stop a running cluster. The following takes the `obtest` cluster as an example.
obd cluster stop obtest

# Destroy a deployed cluster. The following takes the `obtest` cluster as an example.
obd cluster destroy obtest

Uninstall OceanBase All in One

To uninstall OceanBase All in One, perform the following steps:

1. Uninstall the package.

   [admin@test001 ~]$ cd oceanbase-all-in-one/bin/
   [admin@test001 bin]$ ./uninstall.sh
   

2. Delete environment variables.

   [admin@test001 bin]$ vim ~/.bash_profile
   # Delete the source /home/admin/.oceanbase-all-in-one/bin/env.sh from the file.
   

3. Make the modification take effect.

   After you delete the environment variables, you must log in to the terminal again or run the source command to make the modification take effect. Here is an example:

   [admin@test001 ~]$ source ~/.bash_profile