Error codes

This topic summarizes the errors that may occur during the use of OBD.

General errors

OBD-1000: Configuration conflict x.x.x.x: xxx port is used for x.x.x.x

Cause: Port conflicts occur in the configuration file.

Solution: You can run the obd cluster edit-config command to open the configuration file, and view and modify the port configuration.

OBD-1001: x.x.x.x:xxx port is already used

Cause: The port has been occupied.

Solution: You can check the configuration and change the port by using one of the following methods:

• Method 1: If you use a configuration file for deployment, run the obd cluster edit-config command to modify the port configuration in the configuration file. Then, run the obd cluster start command to continue to start the cluster.

  Note

  For more information about the commands used in Method 1, see Cluster commands.

• Method 2: If you run the obd demo command for deployment, run the following command to specify the port. In this example, the mysql_port parameter of the oceanbase-ce component is specified:

  obd demo --oceanbase-ce.mysql_port=3881
  

  Note

  For more information about the commands used in Method 2, see Quick deployment command.

• Method 3: If you perform deployment on the GUI of OBD, you can change the port number on the Cluster Configuration page.

OBD-1002: Fail to init x.x.x.x path

Cause:

1. user in the configuration file (the current user by default, if unspecified) does not have the write permission on the corresponding directory.

2. home_path is not empty.

You can determine the cause based on the error information.

Solution:

For case 1, you can resolve the problem in two ways.

• Run the following command to add or modify user information:

  obd cluster edit-config <deploy name>
  

• Log on to the target server and grant the current account the write permission on the corresponding directory.

For case 2, you can also resolve the problem in two ways.

• Select another directory.

• If you are sure that the current directory can be cleared, you can use the -f option. OBD will clear this directory by using the current user.

OBD-1003: fail to clean x.x.x.x:xxx

Cause: user in the configuration file (the current user by default, if unspecified) does not have the write permission on the home_path directory.

Solution: You can resolve the problem in two ways.

• Run the following command to add or modify user information:

  obd cluster edit-config <deploy name>
  

• Log on to the target server and grant the current account the write permission on the corresponding directory.

OBD-1004: Configuration conflict x.x.x.x: xxx is used for x.x.x.x

Cause: Path conflicts occur in the configuration file.

Solution: Check and modify the configuration.

OBD-1005: Some of the servers in the cluster have been stopped

Cause: Some servers in the current configuration have been stopped, but subsequent operations require the services of all servers to be online.

Solution: Run the obd cluster start <deploy_name> --wop command to start all services without loading parameters.

OBD-1006: Failed to connect to xxx

Cause:

1. OceanBase Deployer (OBD) and the specified server are disconnected.

2. The corresponding component process has exited or does not provide service.

3. The account and password do not match.

Solution:

If the error is due to cause 1, you need to solve the network connection issue.

If the error is due to cause 2, you can try restarting the component first. If the startup still fails, please refer to the error of startup failure for troubleshooting, such as OBD-2002.

If the error is due to cause 3, it is likely that you have changed the password by executing SQL statements, and the account password is different from that stored in the configuration file. As a result, OBD cannot connect to the component. In this case, you can use any of the following two solutions:

1. Execute SQL statements to change the password back to that stored in the configuration file of OBD.

2. Run the vi ~/.obd/cluster/<deploy name>/config.yaml command to change the password to the one that is in use for the component.

OBD-1007: (x.x.x.x) xxx must not be less than xxx (Current value: xxx)

Cause: The configuration of the ulimit parameter does not meet the requirements.

Solution: You can modify the corresponding files in the /etc/security/limits.d/ directory and the limits.conf file in the /etc/security/ directory as needed.

OBD-1008: (x.x.x.x) failed to get fs.aio-max-nr and fs.aio-nr

Cause: OBD cannot obtain the asynchronous I/O (AIO) configuration from the server.

Solution: Check whether the current user has the privilege to view fs.aio-max-nr/fs.aio-nr.

cat /proc/sys/fs/aio-max-nr /proc/sys/fs/aio-nr

OBD-1009: x.x.x.x xxx need config: xxx

Cause: A parameter is missing for the related service component.

Solution: Run the following command to open the configuration file. Then, add the missing parameter to the configuration file and run the restart command provided in the output.

obd cluster edit-config <deploy_name>

OBD-1010: x.x.x.x No such net interface: xxx

Cause:

1. The devname parameter cannot be obtained on the CLI.

2. The devname parameter cannot be obtained on the GUI.

Solution:

For the first case, run the following command to open the configuration file. Then, add or modify the devname parameter in the configuration file and run the restart command provided in the output.

obd cluster edit-config <deploy_name>

For the second case, choose Cluster Configuration > More Settings on the GUI. On the page that appears, set the devname parameter.

OBD-1011: (x.x.x.x) Insufficient AIO remaining (Avail: xxx, Need: xxx), The recommended value of fs.aio-max-nr is 1048576

Cause: The number of AIOs available in the system is less than that required by the database.

Solution: Run the following command to modify linux aio-max-nr:

sudo sysctl fs.aio-max-nr=1048576

OBD-1012: xxx

Cause:

1. A type conversion exception occurs. For example, a string is passed in for an int parameter.

2. The value of a parameter is beyond the allowed range. For example, the value range of rpc_port is 1025 to 65535. If the value configured for rpc_port is not within this range, an error is returned.

3. A parameter is missing. For example, a key parameter such as home_path is not configured.

Solution:

For the first case, check the parameter type and modify the value.

For the second case, check the passed-in parameter value and modify it.

For the third case, check the parameter configuration. If any parameter is missing, configure it.

OBD-1013: xxx@x.x.x.x connect failed: xxx

Cause (two most common causes):

1. The username or password was incorrect.

2. The connection timed out.

Solution: If the following error message is returned, run the obd env set OBD_DISABLE_RSA_ALGORITHMS 1 command to disable the rsa-sha2-512 and rsa-sha2-256 algorithms:

Open ssh connection \Exception (client): Unable to agree on a pubkey algorithm for signing a 'ssh-rsa' key!

If the preceding error message is not returned, run an SSH command to attempt to connect to the server.

• If the connection fails, verify the connection information and server configurations.

• If the connection succeeds, modify the connection information in either of the following ways based on the deployment mode:

  • For CLI-based deployment, run the obd cluster edit-config <deploy_name> command to open the configuration file. Then, add or modify the user information in the configuration file and run the command provided in the output to put the modification into effect.

  • For GUI-based deployment, click Previous. On the Node Configuration page (during OceanBase Database deployment) or MetaDB Configuration page (during OCP deployment), modify the information in the User section.

If the problem persists, submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), and designated professionals will help you fix the issue.

OBD-1015: Unable to confirm the primary-standby relationship, rerun with "--ignore-standby" option if you want to proceed despite the risks

Cause: The cluster or tenant involved in the current operation was engaged in a primary/standby relationship. However, an exception occurred in verifying the relationship when a command was being executed.

Solution: Check whether the current cluster or tenant has a standby tenant in another cluster. The check method varies based on whether the cluster is available.

• If the current cluster is available, you can run the following command to check whether a standby tenant exists. Here a cluster named test is used as an example.

  obd cluster tenant show test -g
  

• If the current tenant is unavailable, you can run the following command to view the primary/standby relationships of the current cluster or tenant. Here a cluster named test is used as an example.

  cat ~/.obd/cluster/test/inner_config.yaml
  

  Based on the returned result, run the following command on the corresponding cluster to check whether the primary/standby relationships still exist. Here a cluster named test-standby is used as an example.

  obd cluster tenant show test-standby -g
  

Take the following actions based on the check result. For more information about the commands involved in the following actions, see Cluster commands.

• If the current operation is an upgrade operation and the current cluster or tenant has a standby tenant, you can upgrade the cluster of the standby tenant first and then that of the primary tenant. This is because the cluster version of the standby tenant must not be earlier than that of the primary tenant. If the current cluster contains both primary and standby tenants, you can first run the obd cluster tenant switchover command to switch the primary tenant in the cluster to a standby tenant. After the clusters where the primary and standby tenants reside are upgraded, you can perform a switchover to switch the standby tenant back to the primary tenant.

  Note

  If the current cluster is undergoing an intra-version upgrade, you can run the command again with the check skipped by appending the --ignore-standby option to the command.

• If the current operation is not an upgrade operation, such as destroy, redeploy, or drop, and the current cluster or tenant has a standby tenant, you can remove the primary/standby relationship in one of the following ways:

  • Run the obd cluster tenant decouple command to decouple the standby tenant from the primary tenant. After the decoupling, the standby tenant becomes an independent regular tenant.

  • Run the obd cluster stop command to stop the cluster where the primary tenant resides and then run the obd cluster tenant failover command to perform a failover for the standby tenant. After the failover, the standby tenant becomes the new primary tenant.

  • Run the obd cluster tenant drop command to drop the standby tenant.

• If the current cluster or tenant has no standby tenant or if you can accept the risks caused by the unavailability of the standby tenant, you can run the command again with the check skipped by appending the --ignore-standby option to the command.

OBD-1016: xx.xx.xx.xx failed to get kernel parameter using command "sysctl -a"

Cause: The connection is abnormal or the operating system used does not support the sysctl -a command.

Solution: Retry the operation or submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), where professionals will assist you.

OBD-1017: xx.xx.xx.xx [kernel] xxx must be xxx

Cause: The values of the kernel parameters of the operating system are not within the recommended range.

OceanBase Database has certain requirements for the operating system's kernel parameters. If the values of the parameters do not meet the requirements, unexpected issues may occur in the production environment. Therefore, OBD adds verification for kernel parameters. That is, when the production_mode parameter is set to true or when executing the obd cluster start or obd cluster autodeploy command with the --strict-check option, OBD will enable forcible kernel parameter verification.

Solution: Depending on the environment you are using, there are two possible solutions:

• If you are using a production environment, you can modify the parameter configuration to meet the requirements using the sysctl -w {kernel parameter name}="recommended value" command or the echo "kernel parameter name=recommended value" >> /etc/sysctl.conf; sysctl -p command.

• If you are using a testing environment and do not have permission to modify kernel parameters, you can run the obd cluster edit-config {deployname} command to modify the configuration file and set the production_mode parameter to false to skip the kernel parameter verification.

OBD-1018: could not change xxx

Cause: The configuration file specified by the command contains a parameter that cannot be modified.

Solution: Check the specified configuration file for parameters such user, depends, unuse_lib_repository, and auto_create_tenant.

OBD-1019: component xxx already in cluster

Cause: The component to be added already exists in the cluster. OBD cannot add an existing component to the cluster.

Solution: Run the obd cluster edit-config command to open the configuration file, and view the component information to check whether the component to be added exists in the cluster.

OBD-1020: Update config for component xxx failed

Cause: Failed to update the component configuration.

Solution: Check the configuration file based on the output error code. If you cannot find the root causes, you can submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), and designated professionals will help you fix the issue.

OBD-1021: Component xxx is not in cluster

Cause: The component to be deleted does not exist in the cluster.

Solution: Run the obd cluster edit-config command to open the configuration file, and view the component information to check whether the component to be deleted exists in the cluster.

OBD-1022: Component xxx still depends by xxx, could not remove

Cause: Another component in the cluster depends on the component to be deleted. OBD cannot delete a component that has dependencies.

Solution: Run the obd cluster edit-config command to open the configuration file, and view the dependencies of the component, which are specified by the depends parameter.

OBD-1023: Failed to merge config: xxx

Cause: Failed to merge configuration files.

Solution: Check the configuration file based on the output error code. If you cannot find the root causes, you can submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), and designated professionals will help you fix the issue.

OBD-1024: The cluster will have no remaining components

Cause: The component to be deleted is the only component in the cluster.

Solution: If you want to delete all components of the cluster, run the obd cluster destroy command to destroy the cluster.

OceanBase deployment errors

OBD-2000: x.x.x.x not enough memory

Cause: The memory is insufficient.

Solution: When OBD starts, the memory is strictly calculated based on MemAvailable. If any cached memory can be released, run the following command:

echo 3 > /proc/sys/vm/drop_caches

If the memory is still insufficient, run edit-config and then adjust memory_limt and system_memory. Ensure that the following condition is met: memory_limt/3 ≤ system_memory ≤ memory_limt/2.

OBD-2001: server can not migrate in

Cause: The number of available units is smaller than --unit-num.

Solution: Modify the passed value of --unit-num. Run the following command to view the number of available units:

select count(*) num from oceanbase.__all_server where status = 'active' and start_service_time > 0

OBD-2002: failed to start x.x.x.x observer

Cause: There are multiple causes for this error. Two most common causes are as follows.

• memory_limit is lower than 8 GB.

• system_memory is too large or small. Generally, the following condition must be met: memory_limt/3 ≤ system_memory ≤ memory_limt/2.

Solution:

• If the problem is caused by either of the preceding reasons, take actions accordingly.

• If the problem persists, submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), and designated professionals will help you fix the issue.

OBD-2003: not enough disk space for clog. Use redo_dir to set other disk for clog, or reduce the value of datafile_size

Cause: The disk usage exceeds the limit.

Solution: Adjust the storage of disks.

• For automatic deployment, the disk usage cannot exceed 72%.

• For manual deployment, the disk usage cannot exceed 64%, if the configuration is not modified.

Note

If redo_dir and data_dir are on the same disk, the space to be occupied by data files is included when the disk usage is calculated.

OBD-2004: Invalid: xxx is not a single server configuration item

Cause: The modified parameter is a global one and cannot be separately modified for a single server.

Solution: Place the parameter to modify under global.

OBD-2005: Failed to register cluster. xxx may have been registered in xxx

Cause: The cluster registration failed or the cluster has been registered.

Solution: Check whether the appname parameter is configured for the cluster. If not, the cluster cannot be registered with Config Server. Take the following actions based on whether the cluster has been deployed.

• Case 1: If you want to register a to-be-deployed OceanBase cluster with Config Server, comment out the obconfig_url parameter, start the cluster, and then run the obd cluster edit-config command to configure the obconfig_url parameter. At present, a to-be-deployed cluster cannot be registered with Config Server.

• Case 2: If you want to register a started cluster with Config Server, check whether the obconfig_url parameter is correctly configured.

  • If obconfig_url is incorrectly configured, run the obd cluster edit-config command to open the configuration file and specify the correct Config URL for the obconfig_url parameter.

  • If obconfig_url is correctly configured and you want to forcibly overwrite the registered cluster, you can append the -f option to the obd cluster start command.

OBD-2006: x.x.x.x has more than one network interface. Please set devname for x.x.x.x

Cause:

1. The devname parameter cannot be obtained on the CLI.

2. The devname parameter cannot be obtained on the GUI.

Solution:

For the first case, run the following command to open the configuration file. Then, add or modify the devname parameter in the configuration file and run the restart command provided in the output.

obd cluster edit-config <deploy_name>

For the second case, choose Cluster Configuration > More Settings on the GUI. On the page that appears, set the devname parameter.

OBD-2007: x.x.x.x xxx fail to ping x.x.x.x

Cause: The nodes cannot ping each other.

Solution:

1. Run the obd cluster edit-config command to check whether the value of the devname parameter in the configuration file matches the actual network interface card (NIC) setting. You can use the ip addr command to check the mapping between IP addresses and NICs.

2. Check whether the network connection between the nodes is normal.

OBD-2008: Cluster clocks are out of sync

Cause: The clocks on the servers in the cluster are out of synchronization.

Solution: Synchronize the clocks on the servers.

OBD-2009: x.x.x.x: when production_mode is True, xxx can not be less then xxx

Cause: In the production mode, the values of parameters such as __min_full_resource_pool_mem and memory_limit cannot be smaller than specified values.

Solution:

• If you are deploying a non-production environment, run the following command to open the configuration file. Change the value of the production_mode parameter to False and run the restart command provided in the output.

  obd cluster edit-config <deploy_name>
  

• If you are deploying a production environment, run the following command to open the configuration file. Modify the __min_full_resource_pool_mem and memory_limit parameters to values greater than the specified values. Then, run the restart command provided in the output.

  obd cluster edit-config <deploy_name>
  

OBD-2010: x.x.x.x: system_memory too large. system_memory must be less than memory_limit/memory_limit_percentage

Cause: The value of the system_memory parameter is too large. Its value must be smaller than the result of memory_limit/memory_limit_percentage × total_memory.

Solution:

1. CLI: Run the following command to open the configuration file. Change the value of system_memory and run the restart command provided in the output.

   obd cluster edit-config <deploy_name>
   

2. GUI: Choose Cluster Configuration > More Settings and specify system_memory.

OBD-2011: x.x.x.x: fail to get memory info.\nPlease configure 'memory_limit' manually in configuration file

Cause: The server cannot obtain the memory information.

Solution:

1. CLI: Run the following command to open the configuration file. Specify memory_limit and run the restart command provided in the output.

   obd cluster edit-config <deploy_name>
   

2. GUI: Choose Cluster Configuration > More Settings and specify memory_limit.

Test-related errors

OBD-3000: parse cmd failed

Cause: The mysqltest initialization file is not an .sql file.

Solution: Check the --init-sql-files parameter.

OBD-3001: xxx.sql not found

Cause: The initialization file cannot be found during the initialization of mysqltest.

Solution: Check whether the file declared by --init-sql-files is located under the --init-sql-dir directory.

OBD-3002: Failed to load data

Cause: There are multiple causes for this error. Two most common causes are as follows.

1. The tenant has insufficient resources or is under excessive test stress.

2. An error occurred in the data build script.

Solution:

If the error is due to cause 1, you can use a tenant with larger resource specifications or adjust parameters such as warehouses and load-workers to reduce the test stress.

If the error is due to cause 2, you can rerun the test because the data build script is obtained from the TPC official website. If the issue persists, submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), and designated professionals will help you fix the issue.

OBD-3003: Failed to run TPC-C benchmark

Cause:

1. The test process was stuck and then terminated due to timeout.

2. An error occurred for the TPC-C test command.

Solution:

• You can try to rerun the test directly, or you can adjust parameters such as terminals to reduce the test pressure before you rerun the test.

• If you did not use the obtpcc package provided on the OceanBase Database official website, use obtpcc for testing.

If the issue persists, submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), and designated professionals will help you fix the issue.

OBAgent-related errors

OBD-4000: Fail to reload x.x.x.x

Cause: The http_basic_auth_password of the node is not the same as that stored in OBD, which causes OBD to fail to access OBAgent.

Solution: If the two passwords are the same, check whether an unsupported parameter is included among the modified options, or whether the name of a parameter is incorrect.

OBD-4001: Fail to send config file to x.x.x.x

Cause: (Check whether the error is caused by either of the reasons.)

• The disk space for the home_path directory on OBAgent is insufficient.

• user in the configuration file (the current user by default, if unspecified) does not have the write permission on the home_path directory on OBAgent.

Solution: You can resolve the problem in two ways.

• Run the following command to add or modify user information:

  obd cluster edit-config <deploy name>
  

• Log on to the target server and grant the current account the write permission on the corresponding directory.

ODP-related errors

OBD-4100: x.x.x.x need config "rs_list" or "obproxy_config_server_url"

Cause: The server cannot obtain the rs_list or obproxy_config_server_url information.

Solution: Run the following command to open the configuration file. Add or modify the rs_list or obproxy_config_server_url parameter and run the restart command provided in the output.

obd cluster edit-config <deploy name>

OBD-4101: failed to start x.x.x.x obproxy: xxx

Cause: Failed to start ODP.

Solution: Perform further analysis based on the error message.

Grafana-related errors

OBD-4200: x.x.x.x grafana admin password should not be 'admin'

Cause: The password of the admin user of Grafana cannot be admin.

Solution: Run the following command to open the configuration file. Add or change the password and run the restart command provided in the output.

obd cluster edit-config <deploy name>

OBD-4201: x.x.x.x grafana admin password length should not be less than 5

Cause: The password of the admin user of Grafana cannot be shorter than 5 characters in length.

Solution: Run the following command to open the configuration file. Add or change the password and run the restart command provided in the output.

obd cluster edit-config <deploy name>

OCP Express-related errors

OBD-4300: x.x.x.x: failed to query java version, you may not have java installed

Cause: OBD cannot obtain the Java information from the server.

Solution:

1. Install Java. For more information, see the How do I configure the Java environment before I deploy OCP Express? section in FAQ.

2. If Java has been installed, set the value of the java_bin parameter to the path of the executable file of Java.

OBD-4301: x.x.x.x: ocp-express need java with version xxx

Cause: The Java version on the server is earlier than needed.

Solution: Install Java of the version provided in the error message. If the target Java version has been installed, set the value of the java_bin parameter to the path of the executable file of Java.

OBD-4302: x.x.x.x not enough memory. (Free: xxx, Need: xxx)

Cause: The server has insufficient memory.

Solution:

• If the total memory available for the server is insufficient, run the obd cluster edit-config command to open the configuration file and modify the memory_limit parameter to a smaller value, or replace the server with one that has sufficient memory.

• If the remaining memory on the server is insufficient, run the following command to release cached objects that can be released.

  echo 3 > /proc/sys/vm/drop_caches
  

OBD-4303: x.x.x.x xxx not enough disk space. (Avail: xxx, Need: xxx)

Cause: The server has insufficient disk space.

Solution: Check and clean up the disk.

OBD-4304: OCP express xxx needs to use xxx with version xxx or above

Cause: OCP Express must be used with the corresponding component of the specified version.

Solution: Run the following command to open the configuration file. Modify the version of the corresponding component and run the restart command provided in the output.

obd cluster edit-config <deploy_name>

OBD-4305: There is not enough xxx for ocp meta tenant

Cause: The log disk space and memory are insufficient for creating a meta tenant for OCP Express.

Solution:

• If the OceanBase cluster is deployed in Maximum Utilization mode on the GUI, or deployed by using the obd cluster autodeploy command on the CLI, we recommend that you clean up the disk and memory and then try again.

• If a resource specification is configured for the cluster, increase the values of the resource-related parameters for the oceanbase-ce component, for example, the memory-related parameters such as memory_limit and memory_limit_percentage and the log disk-related parameters such as log_disk_size and log_disk_percentage.

OBD-4306: xxx ocp-express admin_passwd invalid

Cause: The password is incorrectly configured for the administrator account used to log on to OCP Express.

Solution: Run the obd cluster edit-config command to open the configuration file and modify the admin_passwd parameter. The password can contain letters, digits, and special characters. The password must be 8 to 32 characters in length and contain at least two uppercase letters, two lowercase letters, two digits, and two special characters. The supported special characters are ~!@#%^&*_-+=`|(){}[]:;',.?/.

Notice

After modifying the admin_passwd parameter, you must run the obd cluster redeploy command to destroy and redeploy the cluster. The modification takes effect once you redeploy the cluster. The data in the cluster will be lost. Therefore, back up the data before you run the command.

OCP-related errors

OBD-4350: The Server have running task

Cause: When upgrading OCP, there are running tasks in OCP.

Solution: To avoid interrupting tasks due to OCP upgrade, please wait for the tasks to complete before performing the check again.

OBD-4351: The Server have gone

Cause: When upgrading OCP, the host managed by OCP is not online.

Solution: Check the state of the host managed by OCP. For detailed instructions, see Host overview in the OCP documentation.

• If the host is Newly Submitted, it means the host has been newly added. Please wait for the host addition task to complete before performing the check again.

• If the host is Offline, you can try reinstalling OCP Agent. For detailed instructions, see Reinstall OCP Agent.

OBD-4352: Metadb version not fewer than V2.2.50

Cause: The MetaDB version in OCP is earlier than V2.2.50.

Solution: Upgrade the MetaDB in OCP to the latest Long-Term Support (LTS) version.

OBD-4353: {server}: Excessive deviation between machine time and ob time

Cause: The host time and MetaDB time are inconsistent.

Solution: You can follow the steps below to troubleshoot and resolve the issue.

1. Run the following command on the corresponding host to check whether the clock synchronization service (Chrony or NTP) is installed on the server:

   rpm -qa | grep chrony   # Check whether the Chrony service is installed.
   rpm -qa | grep ntp      # Check whether the NTP service is installed.
   

   Based on the returned results, there are two possible approaches:

   • If version information is returned, it means the corresponding clock synchronization service is already installed. Proceed to Step 2.

   • If no version information is returned, the corresponding clock synchronization service is not installed. If neither Chrony nor NTP is installed, install the clock synchronization service first. You can refer to the resources on the Internet for the installation and configuration of the Chrony and NTP services. Here, we provide a brief explanation.

     • Execute the following command to install the clock synchronization service. Choose either Chrony or NTP to install.

       sudo yum install -y chrony    # Install the Chrony service.
       sudo yum install -y ntp       # Install the NTP service.
       

     • Execute the following command to start the clock synchronization service.

       systemctl start chronyd     # Start the Chrony service.
       systemctl start ntpd        # Start the NTP service.
       

     • Retry the deployment operation. If the issue persists, you can submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), where professionals will assist you.

2. Execute the following command to check whether the clock synchronization process (chronyd or ntpd) has exited unexpectedly:

   systemctl status chronyd      # Check the state of the Chrony service.
   systemctl status ntpd         # Check the state of the NTP service.
   

   Based on the returned results, there are two possible approaches:

   • If the value of the Active parameter in the returned results is active (running), submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), where professionals will assist you.

   • If the value of the Active parameter in the returned results is inactive (dead), it means the clock synchronization service is abnormal. Try executing the following command to restart the service:

     systemctl restart chronyd      # Restart the Chrony service.
     systemctl restart ntpd         # Restart the NTP service.
     

     After restarting the service, you can retry the deployment operation. If the issue persists, you can submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), where professionals will assist you.

OBD-4354: {user}@{server}: Not exist

Cause: The startup user for OCP does not exist.

Solution: Specify another startup user or create a startup user on the host where OCP is located. For instructions on creating a user, see User planning in the OCP documentation.

OBD-4355: {user}@{ip}: user xxx not in sudoers or sudoers file not exist

Cause: The user is not allowed to run the sudo command without a password.

Solution: Configure passwordless sudo access for the user or use another user with passwordless sudo permission. For steps on configuring sudo permission, see User planning in the OCP documentation.

OBD-4356: failed to connect meta db

Cause: MetaDB cannot be connected.

Solution: Check whether the connection string for MetaDB is correct.

OBD-4357: database in jdbc_url is not exist

Cause: The database specified in the JDBC connection string does not exist.

Solution: Create the corresponding database in MetaDB.

OBD-4358: unmatched jdbc url, skip meta db connection check

Cause: The format of the JDBC URL is incorrect.

Solution: Check that the value of the jdbc_url parameter is in the "^jdbc:\S+://(\S+?)(|:\d+)/(\S+)" format.

OBD-4359: {server}: ocp-server need java with version xxx and update release must greater than 161

Cause: The Java version does not meet the requirements of OCP.

Solution: Upgrade the Java version to 1.8.0_161 or later.

OBD-4360: {server}: clockdiff not exists. Please install clockdiff manually

Cause: The clockdiff program is not installed on the host.

Solution: Install the clockdiff program.

OBD-4361: tenant(xxx) already exist

Cause: The tenant already exists.

Solution: Log in to MetaDB and delete the tenant with the same name or use a different tenant name.

OBD-4362: {server}:{path} access failed for current user, {server}:{cur_path} access succeed, please run chmod -R 755 {cur_path}

Cause: The user does not have permission to operate the directory involved.

Solution: Execute the provided chmod command to add the necessary permissions for the user to operate the directory.

Config Server-related errors

OBD-4401: Failed to start x.x.x.x ob-configserver

Causes:

1. Cause 1: An internal runtime error occurs when Config Server is started, which leads to service termination.

2. Cause 2: The listening port of Config Server is not enabled in the target deployment server, which leads to access failures.

Solution: Log on to the target deployment server and run the following command:

ps -ef | grep $home_path/bin/ob-configserver

where $home_path specifies the working directory of Config Server. If no running Config Server process is displayed in the returned result, the error is caused by Cause 1. Otherwise, the error is caused by Cause 2.

For Cause 1, you can open the ob-configserver.log file in the $home_path/log directory and view the keyword in the error information. In most cases, the error is caused by incorrect configuration of the connection_url parameter when database_type is set to sqlite3. In this case, correct the configuration error. If the issue persists, submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), and designated professionals will help you fix the issue.

For Cause 2, take the following actions:

• If you use a cloud server instance, log on to the instance and add the listening port of Config Server to the allowlist of the instance.

• If you use a self-managed server, enable the listening port based on the operating system version.

OBD-4402: x.x.x.x ob-configserver config error

Cause: An error is detected in the configurations of Config Server.

Solution: Check whether any parameter is missing or incorrectly configured based on the error description. Perform the check as follows:

• If a virtual IP address (VIP) is used, check whether vip_address and vip_port are specified.

• Check whether database_type and connection_url are specified. When database_type is set to sqlite3, connection_url is optional.

• Check whether the database_type parameter is correctly configured. Valid values of database_type are mysql and sqlite3.

• If database_type is set to sqlite3, check whether the value of connection_url is an absolute path.

OBD-4403: ob-configserver connect to sqlite failed: x.x.x.x: /xxx/xxx/xxx: permission denied

Cause: When database_type is set to sqlite3 for Config Server, the user specified in the configuration file (or the current user if no user is specified in the configuration file) does not have the write privilege on the directory specified by connection_url.

Solution: You can resolve the issue in one of the following ways:

• Run the following command to open the configuration file and add or modify the user information:

  obd cluster edit-config <deploy name>
  

  After saving the modification, you must restart the cluster by using the command provided in the returned result.

• Log on to the target server and grant the current account the write privilege on the corresponding directory.

OBD-4404: ob-configserver connect to mysql failed: xxx: failed url to connect to database: xxx

Cause: When database_type is set to mysql, the database specified by connection_url cannot be connected.

Solution: Verify whether the database specified by connection_url can be connected. If not, specify a connectable database.

OBLogProxy errors

OBD-4501: OBLogProxy xxx needs to use xxx with version xxx or above

Cause: A component of an incorrect version was used during the deployment of OBLogProxy.

Solution: Run the following command to open the configuration file, modify the version parameter of the component indicated in the error message, and then run the restart command based on the output.

obd cluster edit-config <deploy_name>

SQL-related errors

OBD-5000: sql execute failed

Cause: The SQL execution failed.

Solution: Determine the solution based on the actual situation.

obdiag-related errors

OBD-6000: Failed to executable obdiag command, you may not have obdiag installed

Cause: The obdiag component is not installed.

Solution: Run the following command to install the obdiag component:

obd obdiag deploy

OBD-6001: obdiag must contain depend components xxxx

Cause: The component that obdiag depends on is not installed. The obdiag service of OceanBase Deployer (OBD) depends on an OceanBase cluster or OceanBase Database Proxy (ODP) cluster deployed in OBD. If no OceanBase or ODP cluster is deployed in OBD, this error is returned.

Solution: Install the component that the obdiag service depends on. Make sure that at least one OceanBase or ODP cluster is registered with OBD. You can run the obd cluster list command to view all clusters registered with OBD.

OBD-6002: obdiag options xxx format error, please check the value : xxx

Cause: The format of a parameter value in an obdiag command is incorrect.

Solution: Append the -h option to the obdiag command to view the parameter requirements, and specify parameters in correct formats.

# example
obd obdiag gather -h

obd obdiag gather log -h

OBD-6003: Failed to execute obdiag function xxx

Cause: obdiag failed to run the command. It is likely that the obdiag command contains options that do not exist.

Solution: Run the following command to view the obdiag commands supported by OBD.

obd obdiag -h

Unexpected errors

OBD-9999: Unexpected exception: need to be posted on "https://ask.oceanbase.com", and we will help you resolve them

Cause: An unexpected exception occurs during the operation.

Solution: Submit an issue on GitHub (https://github.com/oceanbase/obdeploy/issues), and designated professionals will help you fix the issue.