Deploy OceanBase Database in production environment by using the command line

This topic describes how to deploy OceanBase Database in a production environment by using the command-line interface of OBD.

Terms

• Central control server

  The server where OBD is installed. It is used to manage the OceanBase cluster and store the installation packages and configuration information of the OceanBase cluster.

• Target server

  The server where OceanBase Database is installed.

• OceanBase Database

  A fully self-developed enterprise-level native distributed database. For more information, see OceanBase Database documentation.

• OBD

  OceanBase Deployer, a tool for installing and deploying OceanBase Database, abbreviated as OBD. For more information, see OceanBase Deployer documentation.

• ODP

  OceanBase Database Proxy, a dedicated proxy server for OceanBase Database, abbreviated as ODP (also known as OBProxy). For more information, see OceanBase Database Proxy documentation.

• OBAgent

  OBAgent is a framework for monitoring and data collection in OceanBase Database. It supports both push and pull data collection modes to meet different application requirements.

• obconfigserver

  OceanBase Configserver provides metadata registration, storage, and query services for OceanBase Database. For more information, see ob-configserver.

• Grafana

  Grafana is an open-source data visualization tool that visualizes various metrics from data sources to help you understand system runtime status and performance indicators. For more information, visit the official website of Grafana.

• Prometheus

  Prometheus is an open-source service monitoring system and time series database. It provides a general data model and fast data collection, storage, and query interfaces. For more information, visit the official website of Prometheus.

Prerequisites

Before you deploy OceanBase Database, confirm the following:

• Your machine meets the software and hardware requirements. For details, see Software and hardware requirements.

• In a production environment, you need to perform an environment and configuration check. For specific operations, refer to the Pre-deployment configuration section.

• The OBD and OBClient clients are installed on the central control machine. It is recommended to install the latest versions. For details, see the official website's "OceanBase Installation and Deployment Tools" documentation Quick start/Install OBD and OBClient documentation.

  Note

  The OceanBase All-in-One package automatically installs the OBD and OBClient clients during installation. Therefore, if you plan to follow the steps in point 1 below to download and install OceanBase All-in-One, you can ignore this requirement.

Deployment mode

This topic uses a three-replica deployment mode and recommends using four machines. You can choose an appropriate deployment plan based on your actual situation. The usage of the four machines in this topic is as follows:

Procedure

Step 1: (Optional) Download and install OceanBase All in One

Starting from V4.0.0, OceanBase provides a unified installation package called OceanBase All in One. You can use this package to install components such as OBD, OceanBase Database, ODP, and OBAgent with one click.

You can also download and install specific components or specify component versions from the OceanBase Download Center based on your actual needs.

Online installation

If your machine has an internet connection, you can run the following command to install OceanBase All in One online.

[admin@test001 ~]$ bash -c "$(curl -s https://obbusiness-private.oss-cn-shanghai.aliyuncs.com/download-center/opensource/oceanbase-all-in-one/installer.sh)"
[admin@test001 ~]$ source ~/.oceanbase-all-in-one/bin/env.sh

Offline installation

If your machine cannot connect to the internet, you can follow these steps to install OceanBase All in One offline.

1. Download the latest OceanBase All in One installation package from the OceanBase Download Center and copy it to any directory on your server.

2. In the directory where the package is located, run the following commands to extract and install the package.

   [admin@test001 ~]$ tar -xzf oceanbase-all-in-one-*.tar.gz
   [admin@test001 ~]$ cd oceanbase-all-in-one/bin/
   [admin@test001 bin]$ ./install.sh
   [admin@test001 bin]$ source ~/.oceanbase-all-in-one/bin/env.sh
   

Step 2: Configure OBD

For offline deployment of an OceanBase cluster, you can refer to Step 1 to download and install OceanBase All in One on the central control server. The components provided by OceanBase All in One have undergone mutual adaptation testing and are the officially recommended version.

For offline deployment with specific version requirements for the deployment components, you can download the corresponding version installation packages from the OceanBase Download Center and install them. Copy the downloaded installation packages to any directory on the central control server, and then follow steps 1 to 3 below to upload the packages to the local OBD image repository.

1. Disable remote repositories

   [admin@test001 rpm]$ obd mirror disable remote
   

   Note

   Remote repositories are disabled by default after installing OceanBase All in One. You can confirm this using the obd mirror list command. If the Enabled value for Type=remote is False, the remote image source is disabled.

2. Add the installation package to the local image repository

   [admin@test001 rpm]$ obd mirror clone *.rpm
   

3. View the installation package list in the local image repository

   [admin@test001 rpm]$ obd mirror list local
   

4. Select a configuration file

   If OBD on your machine was installed by direct download, you can view the sample configuration files provided by OBD in the /usr/obd/example directory.

   If OBD on your machine was installed by extracting OceanBase All in One, you can view the sample configuration files provided by OBD in the ~/.oceanbase-all-in-one/obd/usr/obd/example directory. Select the appropriate configuration file based on your resource conditions.

   Note

   The configuration files are divided into two modes: Mini-specification Development and Professional Development. The parameters in the configuration files are essentially the same, but the specifications differ slightly. Choose the mode that matches your actual resource conditions.

   • Mini-specification Development mode: Suitable for personal devices (memory of at least 8 GB). The configuration file name contains the mini or min identifier.

   • Professional Development mode: Suitable for high-specification ECS or physical servers (available resources of at least 16 cores 64 GB).

   If you choose standalone deployment, meaning there is only one target machine, you can refer to the standalone deployment configuration files.

   • Local standalone deployment configuration examples: mini-local-example.yaml, local-example.yaml

   • Standalone deployment configuration examples: mini-single-example.yaml, single-example.yaml

   • Standalone deployment + ODP configuration examples: mini-single-with-obproxy-example.yaml, single-with-obproxy-example.yaml

   If you choose distributed deployment, meaning there are multiple target machines, you can refer to the distributed installation configuration files.

   • Distributed deployment OceanBase Database configuration examples: mini-distributed-example.yaml, distributed-example.yaml

   • Distributed deployment OceanBase Database + ODP configuration examples: mini-distributed-with-obproxy-example.yaml, distributed-with-obproxy-example.yaml

   • Distributed deployment of all components: all-components-min.yaml, all-components.yaml

5. Modify the configuration file.

   Note

   You need to modify the following parameters according to the actual situation of your environment.

   1. Create a configuration file.

      vim deploy.yaml
      

      This is just an example; you can customize the configuration file name based on your actual situation.

   2. Modify the username and password.

      ## Only need to configure when remote login is required
      user:
        username: admin
      #   password: your password if need
        key_file: /home/admin/.ssh/id_rsa
      #   port: your ssh port, default 22
      #   timeout: ssh connection timeout (second), default 30
      

      username is the username for logging into the target machine. Ensure your username has write permissions for home_path. Both password and key_file are used to authenticate the user; usually, only one needs to be filled in.

      Notice

      After configuring the key path, if your key does not require a password, comment out or delete password to prevent it from being mistakenly used as the key password for login, which could cause verification failure.

   3. Modify the machine's IP, port, and related directories, and configure memory-related parameters and passwords.

      oceanbase-ce:
        # version: 4.3.3.0
        servers:
          # Please don't use hostname, only IP can be supported
          - name: server1
            ip: 10.10.10.1
          - name: server2
            ip: 10.10.10.2
          - name: server3
            ip: 10.10.10.3
        global:
          devname: eth0
          cluster_id: 1
          # please set memory limit to a suitable value which is matching resource. 
          memory_limit: 64G # The maximum running memory for an observer
          system_memory: 30G # The reserved system memory. system_memory is reserved for general tenants.
          datafile_size: 192G # Size of the data file. 
          datafile_next: 200G
          datafile_maxsize: 1T
          log_disk_size: 192G # The size of disk space used by the clog files.
          scenario: htap
          enable_syslog_wf: false # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true.
          max_syslog_file_count: 4 # The maximum number of reserved log files before enabling auto recycling. The default value is 0.
          enable_auto_start: true
          # observer cluster name, consistent with obproxy's cluster_name
          appname: obdemo
          mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started.
          rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started.
          obshell_port: 2886 # Operation and maintenance port for OceanBase Database.
          # The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
          home_path: /home/admin/observer
          # The directory for data storage. The default value is $home_path/store.
          data_dir: /data
          # The directory for clog. The default value is the same as the data_dir value.
          redo_dir: /redo
          root_password: ****** # root user password, can be empty
          proxyro_password: ******** # proxyro user pasword, consistent with obproxy's observer_sys_password, can be empty
        server1:
          zone: zone1
        server2:
          zone: zone2
        server3:
          zone: zone3
      

      If there are inconsistent parameters on the machines, you can move the relevant parameters to the corresponding server for configuration. The configuration within the configuration file for a specific server takes precedence over the global setting. For example, if two machines have different port configurations:

      server2:
        mysql_port: 3881
        rpc_port: 3882
        zone: zone2
      server3:
        mysql_port: 2881
        rpc_port: 2882
        zone: zone3 
      

      Parameter	Required	Default value	Description
      version	Optional	Latest version in the image repository	Specifies the version of the component to be deployed. This parameter is usually not required.
      servers	Required	N/A	You must specify each server in the format - name:server identifier (line break)ip:server IP, repeating the line for multiple servers. Server identifiers must be unique. If the server IPs are unique, you can also use the format - <ip> (line break) - <ip>. In this case, - <ip> is equivalent to - name:server IP (line break)ip:server IP.
      devname	Optional	N/A	The network interface card corresponding to the IP specified in servers. You can view the IP-to-NIC mapping using the ip addr command.
      memory_limit	Optional	0	The maximum memory that the observer process can obtain from the environment. If not configured, the memory_limit_percentage parameter takes effect. For more information, see memory_limit and memory_limit_percentage.
      system_memory	Optional	0M	The reserved system memory. This parameter value consumes memory from memory_limit. If not configured, OceanBase Database adapts automatically.
      datafile_size	Optional	0	Specifies the size of the data file (block_file) for the corresponding node. If not configured, the datafile_disk_percentage parameter takes effect. For more information, see datafile_size and datafile_disk_percentage.
      datafile_next	Optional	0	Controls the step size for disk space growth, used to set up automatic expansion. If not configured, to enable automatic expansion, refer to Configure dynamic expansion for disk data files.
      datafile_maxsize	Optional	0	Limits the maximum available disk space, used to set up automatic expansion. If not configured, to enable automatic expansion, refer to Configure dynamic expansion for disk data files.
      log_disk_size	Optional	0	Sets the size of the Redo log disk. If not configured, the log_disk_percentage parameter takes effect. For more information, see log_disk_size and log_disk_percentage.
      scenario	Optional	htap	Sets the cluster workload type. If this parameter is not configured, an interactive option will be provided for the user to select the workload type. Valid values: express_oltp: Suitable for workloads such as trading, core payment systems, and high-throughput internet applications. No foreign keys or other restrictions, no stored procedures, no long or large transactions, no complex joins, no complex subqueries. complex_oltp: Suitable for workloads such as banking and insurance systems. They typically involve complex joins, complex correlated subqueries, batch processing jobs written in PL, as well as long and large transactions. Sometimes use parallel execution for short-running queries. olap: Suitable for real-time data warehouse analytics scenarios. htap: Suitable for hybrid OLAP and OLTP workloads. Typically used to gain instant insights from active operational data, fraud detection, and personalized recommendations. kv: Suitable for key-value workloads and HBase-like wide-column workloads. These workloads typically have very high throughput and are latency-sensitive.
      enable_syslog_wf	Optional	true	Specifies whether to print system logs above WARN level to a separate log file.
      max_syslog_file_count	Optional	0	Sets the number of log files that can be retained before recycling. A value of 0 indicates no automatic cleanup.
      enable_auto_start	Optional	0	Controls whether to enable automatic startup for the observer process. When set to true, the observer process is automatically started if the OBServer node restarts. Note When configuring this parameter, ensure the deployment user has sudo privileges and the deployment environment is not a container environment.
      appname	Optional	obcluster	Sets the OceanBase cluster name.
      mysql_port	Required	2881	Sets the port number for the SQL service protocol. Default is 2881.
      rpc_port	Required	2882	Sets the port number for remote access, which is the RPC communication port between observer processes and processes on other nodes. Default is 2882.
      obshell_port	Required	2886	Sets the OceanBase database maintenance port. Default is 2886.
      home_path	Required	N/A	The installation path of OceanBase Database. It is recommended to place it under the regular user admin directory.
      data_dir	Optional	$home_path/store	Sets the directory for storing data such as SSTables. It is recommended to configure it as a dedicated disk.
      redo_dir	Optional	Same as data_dir	The directory for clogs. By default, it is the same as the value of data_dir. It is recommended to configure it as a dedicated disk.
      root_password	Optional	Random string	The password for the super administrator (root@sys) of the OceanBase cluster. It is recommended to set a complex password. When using obd V2.1.0 or later, a random string is automatically generated if not set.
      proxyro_password	Optional	Random string	The password for the account (proxyro@sys) used by ODP to connect to the OceanBase cluster. When using obd V2.1.0 or later, a random string is automatically generated if not set.

   4. Configure the obproxy-ce component and modify the IP and home_path.

      obproxy-ce:
        # version: 4.3.4.0
        # package_hash: 5c5dca3a355cc7286146ed978ebb26a6342e42e02cd3ca8b9739d300519a449f
        depends:
          - oceanbase-ce
        servers:
          - name: server1
            # Please don't use hostname, only IP can be supported
            ip: 10.10.10.1
          - name: server2
            ip: 10.10.10.2
          - name: server3
            ip: 10.10.10.3
        global:
          prometheus_listen_port: 2884
          listen_port: 2883
          rpc_listen_port: 2885
          enable_obproxy_rpc_service: true
          # vip_address: "10.10.10.5"
          # vip_port: 3306
          home_path: /home/admin/obtest/obproxy
          client_session_id_version: 2
          proxy_id: 822
          obproxy_sys_password: ********
          observer_sys_password: ********
          skip_proxy_sys_private_check: true
          enable_strict_kernel_release: false
          enable_cluster_checkout: false 
      

      Parameter	Required	Default value	Description
      version	Optional	The latest version in the image library.	Specifies the version of the component to be deployed. This parameter is usually not required.
      depends	Optional	N/A	Sets the component dependencies. After configuration, the current component automatically obtains information from its dependent components. This parameter is required if a component has dependencies.
      servers	Required	N/A	Each server must be specified using the format - name: server identifier (line break) ip: server IP. Repeat this for multiple servers. Server identifiers must be unique. If server IPs are unique, you can also use the format - <ip> (line break) - <ip>. In this case, - <ip> is equivalent to - name: server IP (line break) ip: server IP.
      listen_port	Required	2883	The ODP listening port. Default is 2883.
      prometheus.listen_port	Required	2884	The ODP Prometheus listening port. Default is 2884.
      rpc_listen_port	Optional	2885	The RPC service listening port in ODP. Default is 2885. Note This parameter takes effect only when deploying ODP V4.3.0 or later, and when enable_obproxy_rpc_service is set to true.
      enable_obproxy_rpc_service	Optional	true	Controls whether to enable the ODP RPC service. A default value of true indicates the RPC service is enabled, while a value of false stops the RPC service. In this case, the rpc_listen_port configuration will not take effect. Note This parameter takes effect only when deploying ODP V4.3.0 or later. If this parameter is not configured, its default value of false will be used after upgrading ODP.
      vip_address	Optional	N/A	When deploying multi-node ODP, you can configure the VIP IP address using this option. Notice obd supports configuring VIPs for ODP starting from V3.3.0. obd does not provide load balancing deployment. To set up load balancing, please deploy and configure the corresponding load balancer in advance.
      vip_port	Optional	N/A	When deploying multi-node ODP, you can configure the VIP access port using this option. Notice obd supports configuring VIPs for ODP starting from V3.3.0. obd does not provide load balancing deployment. To set up load balancing, please deploy and configure the corresponding load balancer in advance.
      home_path	Required	N/A	The installation path for ODP. It is recommended to place it under the regular user admin.
      client_session_id_version	Optional	2	Used to set whether to use the new Client Session ID generation logic. Can only be configured as 1 or 2. When set to 2, the new Client Session ID generation logic is used.
      proxy_id	Optional	0	Used to set the ID that identifies ODP, ensuring Client Session IDs generated by different ODP instances do not conflict. Depending on the value of the client_session_id_version parameter, the possible values for proxy_id are as follows. When the client_session_id_version parameter is set to 1, the range for proxy_id is [0,255]. When the client_session_id_version parameter is set to 2, the range for proxy_id is [0,8191].
      obproxy_sys_password	Optional	Random string	The password for the ODP administrator account (root@proxysys). When using obd V2.1.0 or later, a random string is automatically generated if this parameter is not set.
      observer_sys_password	Optional	By default, a random string identical to proxyro_password	The password for the account used by ODP to connect to the OceanBase cluster (proxyro@sys). The configured password must match the value of proxyro_password. When using obd V2.1.0 or later, a random string identical to proxyro_password is automatically generated if this parameter is not set.

   5. Modify a monitoring component.

      obagent:
        depends:
          - oceanbase-ce
        # The list of servers to be monitored. This list is consistent with the servers in oceanbase-ce.
        servers:
          - name: server1
            ip: 10.10.10.1
          - name: server2
            ip: 10.10.10.2
          - name: server3
            ip: 10.10.10.3
        global:
          monagent_http_port: 8088
          mgragent_http_port: 8089
          home_path: /home/admin/obagent
          http_basic_auth_password: ********
      prometheus:
        servers:
          - 10.10.10.4
        depends:
          - obagent
        global:
          port: 9090
          # The working directory for prometheus. prometheus is started under this directory. This is a required field.
          home_path: /home/admin/prometheus
          basic_auth_users: #<username>: <password>, the key is the user name and the value is the password.
            admin: ********
      grafana:
        servers:
          - 10.10.10.4
        depends:
          - prometheus
        global:
          port: 3000
          login_password: ********
          home_path: /home/admin/grafana
      

      Parameter	Required	Default value	Description
      servers	Required	N/A	Each server must be specified using the format - name: server identifier (line break) ip: server IP. Repeat this for multiple servers. Server identifiers must be unique. If server IPs are unique, you can also use the format - <ip> (line break) - <ip>. In this case, - <ip> is equivalent to - name: server IP (line break) ip: server IP.
      depends	Optional	N/A	Sets the component dependencies. After configuration, the current component automatically obtains information from its dependent components. This parameter is required if a component has dependencies.
      home_path	Required	N/A	The installation path for the component. It is recommended to place it under the regular user admin.
      monagent_http_port	Required	8088	The OBAgent monitoring service port.
      mgragent_http_port	Required	8089	The OBAgent management service port.
      http_basic_auth_password	Optional	Random string	The HTTP service authentication password. In custom mode, the password must be at least one character long and can contain digits (0~9), uppercase letters (A~Z), lowercase letters (a~z), and special characters (~^*{}[]_-+). If not configured, obd automatically generates a random string. You can view the corresponding parameter in the configuration file for the password using the obd cluster edit-config command after successful deployment.
      basic_auth_users	Optional	admin: ***** where admin is the username and ***** is the randomly generated password	The authentication information for the Prometheus web service. The key is the username, and the value is the password.
      login_password	Optional	Random string	The Grafana login password. You can view the corresponding parameter in the configuration file for the password using the obd cluster edit-config command after successful deployment.

Step 3: Deploy an OceanBase cluster

1. Deploy an OceanBase cluster.

   [admin@test001 ~]$ obd cluster deploy obtest -c deploy.yaml
   

   If you have network access, after you run the obd cluster deploy command, OBD checks whether the target machine has the installation packages required for deployment. If no installation packages are present, OBD automatically retrieves them from the YUM repository.

2. Start an OceanBase cluster.

   [admin@test001 ~]$ obd cluster start obtest
   

   After successful startup, the access addresses for the obshell dashboard and monitoring components are displayed. In Alibaba Cloud or other cloud environments, if the installer cannot obtain a public IP address, it may display an intranet address. This IP is not a public address, and you must use the correct address.

3. Check the status of the OceanBase cluster.

   # View the list of clusters managed by OBD
   [admin@test001 ~]$ obd cluster list
   
   # Check the status of the OBTEST cluster.
   [admin@test001 ~]$ obd cluster display obtest
   

4. (Optional) Modify cluster configurations.

   OceanBase Database has hundreds of parameters, and some are coupled. Before you are familiar with OceanBase Database, we recommend that you do not modify the configurations in the sample configuration file. This example shows how to modify configurations and make them effective.

   Use the obd cluster edit-config command to enter edit mode and modify the cluster configurations.

   [admin@test001 ~]$ obd cluster edit-config obtest
   

   After modifying the configurations, save and exit. OBD will then indicate how to apply the changes. After saving the modifications, enter the following command.

   Search param plugin and load ok
   Search param plugin and load ok
   Parameter check ok
   Save deploy "obtest" configuration
   Use `obd cluster reload obtest` to make changes take effect.
   

   Copy the command output by OBD and execute it.

   [admin@test001 ~]$ obd cluster reload obtest
   

Step 4: Connect to the OceanBase cluster

This example shows how to connect to an OceanBase cluster using the OBClient:

obclient -h<IP> -P<PORT> -u<user_name>@<tenant_name>#<cluster_name> -p -c -A
# example
obclient -h10.10.10.4 -P2883 -uroot@sys#obdemo -p -c -A

Parameter description:

• -h: The IP address for connecting to OceanBase Database. For direct connection, this is the OBServer node address; for connection through ODP, this is the ODP address.

• -u: The account for connecting to OceanBase Database. Supported formats include: username@tenant_name#cluster_name, cluster_name:tenant_name:username, cluster_name-tenant_name-username, or cluster_name.tenant_name.username. The default administrator username for a MySQL-compatible tenant is root.

  Notice

  The cluster name used during connection is the appname configuration name in the configuration file, not the deploy name used during deployment.

• -P: Specifies the port for connecting to OceanBase Database. For direct connections, this is the value of the mysql_port parameter. For connections through ODP, this is the value of the listen_port parameter.

• -p: Specifies the password for connecting to OceanBase Database.

• -c: Indicates that comments should not be ignored in the OBClient runtime environment.

  Note

  Hint is a special comment and is not affected by -c.

• -A: Indicates that statistical information should not be automatically retrieved when OBClient connects to the database.

For more details on connecting to an OceanBase cluster, see Connect to OceanBase Database.

Step 5: Create a user tenant

After deploying an OceanBase cluster, it is recommended to create a user tenant for business operations. The sys tenant is only used for cluster management and is not suitable for business scenarios.

You can use either of the following methods to create a user tenant:

• Method 1: Use obd to create a user tenant.

  obd cluster tenant create <deploy name> [-n <tenant name>] [flags]
  # example
  obd cluster tenant create test -n obmysql --max-cpu=2 --memory-size=2G --log-disk-size=3G --max-iops=10000 --iops-weight=2 --unit-num=1 --charset=utf8
  

  By default, this command creates a tenant using all remaining available resources in the cluster. You can control the amount of resources the tenant occupies by configuring various parameters. For detailed usage, refer to the obd cluster tenant create command in the Cluster commands section of the obd manual.

• Method 2: Create a user tenant via the command line. For detailed operations, refer to Create a tenant.

Related operations

Manage clusters

You can run the following commands to manage clusters deployed by OBD. For detailed commands, see Cluster commands in the OceanBase Installation and Deployment Tool documentation.

# View Cluster List
obd cluster list

# Check the cluster status. Take the deployment named obtest as an example.
obd cluster display obtest

# Stop the running cluster. Take the deployment named obtest as an example.
obd cluster stop obtest

# This topic describes how to terminate a deployed cluster. The following example uses the cluster named obtest.
obd cluster destroy obtest

Uninstall OceanBase All in One

To uninstall installed OceanBase All in One, perform the following steps:

1. Uninstall the installation package.

   [admin@test001 ~]$ cd oceanbase-all-in-one/bin/
   [admin@test001 bin]$ ./uninstall.sh
   

2. Delete environment variables.

   [admin@test001 bin]$ vim ~/.bash_profile
   # Delete the source file /home/admin/.oceanbase-all-in-one/bin/env.sh.
   

3. Apply the changes.

   After deleting the environment variables, you must log in to the terminal again or run the source command to apply the changes. You can refer to the following commands.

   [admin@test001 ~]$ source ~/.bash_profile