Use obdiag to inspect an OceanBase cluster

If OceanBase Diagnostic Tool (obdiag) is independently deployed, you can use obdiag check commands to inspect the status of an OceanBase cluster. The inspection process analyzes the cluster from perspectives of the system kernel parameters and internal tables, identifies the causes of existing or possible exceptions, and provides O&M suggestions.

Prerequisites

You have installed obdiag. For more information, see Install obdiag.

check commands

Syntax

obdiag check [options]

The following table describes the options.

Here is an example:

obdiag check --cases=test

'cat ./check_report/check_report_2023-10-30-16:15:52.table', export type is table
For more details, please run cmd 'cat ./check_report/check_report_2023-10-30-16:15:52.table'
If you want to view detailed obdiag logs, please run:'obdiag display-trace --trace_id a7674ecb-0d99-36fe-b584-3b707b4647bc'

Tutorial for writing a task

A task is a specific inspection scenario. obdiag runs a task based on the dedicated script file in the YAML format.

The script file of a task contains a declaration at the beginning, which helps you implement professional inspections on an OceanBase cluster.

Preparations

Before you write the script file of a task, you must specify the path of the file.

The file must be saved in the directory specified by the CHECK.tasks_base_path parameter in the config.yml configuration file. Check whether the task falls into an existing category in the directory. If not, create a folder to declare the category.

Here is an example:

# Go to ${CHECK.tasks_base_path}/observer, create a folder named `test`, and then create a sample task script file named `test.yaml`.
cd ~/.obdiag/tasks/observer
mkdir test
cd test
touch test.yaml

Write the task script

The task script is a list that declares the steps to be executed in an inspection. You can specify steps for different versions of OceanBase Database, so the script can be used for inspections of OceanBase clusters of the specified versions.

Here is a sample task script:

info: testinfo
task:
  - version: "[3.1.0,3.2.4]"
    steps:
      {steps_object}
  - version: "[4.2.0.0,4.3.0.0]"
    steps:
      {steps_object}

The following table describes the parameters.

steps

steps is a list of multiple execution processes.

An element of steps is a single step that involves the following parameters.

In the following examples, step: serves only as a mark and has no actual meaning.

• get_system_parameter

steps:
- type: get_system_parameter
  parameter_name: parameter
  result:
    set_value: servervm.max_map_count

• ssh

Remotely execute the instruction and obtain the corresponding return value.

steps:
- type: ssh
  ssh: wc -l /proc/${task_OBServer_pid}/maps | awk '{print $1}'
  result:
    set_value: observerMaps

• sql

Execute an SQL statement and obtain the corresponding value.

steps:
- type: sql
  sql: select tenant_name from oceanbase.table_name from where tenant_id=${taskTenantId};
  result:
    set_value: tenant_name

result & verify

This field is the main dependent field of the verify feature and is used to verify the task result.

The format of the result section is as follows:

result:
      set_value: {set_value}
      verify_type: {verify_type}
      report_type: {report_type}
      verify: {verify}
      err_msg: {err_msg}

The following table describes the parameters.

Judgment types supported by verify_type:

At present, the judgment types supported by verify_type apply only the base and int types.

• between: checks whether the value of the parameter specified by the set_value parameter falls within the range specified by the verify parameter.
• max: checks whether the value of the parameter specified by the set_value parameter is less than that specified by the verify parameter.
• min: checks whether the value of the parameter specified by the set_value parameter is greater than that specified by the verify parameter.
• equal: checks whether the value of the parameter specified by the set_value parameter is equal to that specified by the verify parameter.

base

You can replace new_expr with the expression specified by the verify parameter to manually verify the logic in your local environment.

if ${new_expr}; then
    echo "true"
else
    echo "false"
fi

Manually update tasks

We will update these tasks from time to time to provide more effective inspection tasks. To update your inspection package to the latest version, copy the ${code_path}/handler/check/tasks folder and the *check_package.yaml file from the code repository of obdiag to replace the original tasks folder and *check_package.yaml file in the ~/.obdiag/ directory.