Do IO Bench error

Problem description

An error is returned at the Do IO Bench step when you create an OceanBase cluster in OceanBase Cloud Platform (OCP). You can find the following error message in the log: rpc call failed. recv response is null. may read timeout.

Possible causes

At the Do IO Bench step, the following command is executed to test the I/O performance of the data disk:

/home/admin/oceanbase/bin/ob_admin io_bench -c /home/admin/oceanbase/etc -d /data/1/obdemo user:root

After the I/O test is completed, the I/O performance report io_resource.conf file is generated in the /home/admin/oceanbase/etc directory. If the file is not generated, the command execution failed, which can be caused by the following issues:

• The data disk is incorrectly mounted.

• The test of the I/O performance of the data disk fails due to issues of the task framework.

Solutions

1. Check whether the data disk is mounted to /data/1 and has sufficient space.

   • If yes, proceed to the next step.

   • Otherwise, perform the following steps to roll back and reinstall the OceanBase cluster.

     1. In the upper-right corner of the Do IO Bench step, click Roll Back .

     2. Delete the hosts on which the OceanBase cluster is installed from the OCP cluster.

     3. Log on to these hosts in OBClient as the root user, mount the disks, and modify the basic configurations. You can delete residual files in the /data/1, /data/log1, and /home/admin directories by running the following commands.

        rm -rf /data/1
        rm -rf /data/log1
        rm -rf /home/admin
        

     4. Restart those hosts.

     5. Run the ntpdate command to ensure that the clock of those hosts is synchronized with the clock source. You must specify the IP address of the clock source in the command in the ntp IP address format, for example, ntp 192.168.0.1.

        If the clock synchronization service is not enabled, run the service ntpd start command to enable it.

     6. Go to the OCP console, add a host and create a cluster again. For more information, see Add a host and Create a cluster.

2. Check whether the io_resource.conf file is generated in the /home/admin/oceanbase/etc directory.

   You can also run the ps -ef|grep ob_admin command to check whether the relevant process is running. This operation is equivalent to checking whether the performance report exists.

   • If yes, the I/O performance test is completed, you can skip the Do IO Bench step and proceed with the installation task.

   • Otherwise, skip the Do IO Bench step and proceed with the installation task. After the OceanBase cluster is installed, run the following command on every OBServer in the cluster that returned error at the Do IO Bench step:

     /home/admin/oceanbase/bin/ob_admin io_bench -c /home/admin/oceanbase/etc -d /data/1/obdemo user:root