If OceanBase Diagnostic Tool (obdiag) is independently deployed, you can run the obdiag gather log command to collect logs of the specified OceanBase cluster.
If your OceanBase cluster is deployed by using OceanBase Deployer (obd), you can run obdiag commands on obd to collect diagnostic information of the cluster. For more information about the commands, see obdiag commands.
Syntax
obdiag gather log [options]
The following table describes the options.
| Option | Required? | Data type | Default value | Description |
|---|---|---|---|---|
| --from | No | string | Empty | The start time of log collection in the yyyy-mm-dd hh:mm:ss format. |
| --to | No | string | Empty | The end time of log collection in the yyyy-mm-dd hh:mm:ss format. |
| --since | No | string | Empty | The most recent period of time for which logs are to be collected, in the <n> <m|h|d> format, where n is a number, m indicates minutes, h indicates hours, and d indicates days. For example, 30m specifies to collect logs of the last 30 minutes. |
| --scope | No | string | all | The type of logs of the OceanBase cluster to be collected. Valid values: observer, election, rootservice, and all. |
| --grep | No | string | Empty | The search keyword. |
| --encrypt | No | string | false | Specifies whether to encrypt the returned files. Valid values: true and false. |
| --store_dir | No | string | The current path where the command is executed | The local path where the results are stored. |
| --temp_dir | No | string | /tmp |
The directory where temporary files generated by the remote node during log collection are stored. |
| --redact | No | string | Empty | The data masking rule, for example, all_sql. If multiple rules are specified, separate them with commas (,). For more information about how to define a data masking rule, see the Define a data masking rule section in this topic.
Note |
| -c | No | string | ~/.obdiag/config.yml |
The path of the configuration file. |
| --inner_config | No | string | Empty | The configurations of obdiag. |
| --config | No | string | Empty | The configurations of the cluster diagnosed by obdiag, in the format of --config key1=value1 --config key2=value2. |
Examples
Method 1: Use obdiag out-of-the-box without a configuration file
Note
For more information about the parameters used in the command in this section, see Configure obdiag.
Collect logs of the specified period
obdiag gather log --scope observer --from "2022-06-30 16:25:00" --to "2022-06-30 18:30:00" --grep STORAGE --encrypt true \ --config obcluster.servers.nodes[0].ip=xx.xx.xx.1 \ --config obcluster.servers.nodes[1].ip=xx.xx.xx.xx.2 \ --config obcluster.servers.global.ssh_username=test \ --config obcluster.servers.global.ssh_password=****** \ --config obcluster.servers.global.home_path=/home/admin/oceanbaseThe output is as follows:
... ZipFileInfo: +-------------------+-----------+ | Node | LogSize | +===================+===========+ | xxx.xxx.xxx.xxx | 36.184M | +-------------------+-----------+ ... ZipFileInfo: +-------------------+-----------+ | Node | LogSize | +===================+===========+ | xxx.xxx.xxx.xxx | 44.176M | +-------------------+-----------+ ... Summary: +-------------------+-----------+----------+------------------+--------+-------------------------------------------------------------------------------+ | Node | Status | Size | Password | Time | PackPath | +===================+===========+==========+==================+========+===============================================================================+ | xxx.xxx.xxx.xxx | Completed | 36.762M | HYmVourcUyRNP8Om | 19 s | obdiag_gather_pack_20220701183246/ob_log_xxx.xxx.xxx.xxx_20220701183247.zip | +-------------------+-----------+----------+------------------+--------+-------------------------------------------------------------------------------+ | xxx.xxx.xxx.xxx | Completed | 638.200M | 1RicMaiLUUNfemnj | 718 s | obdiag_gather_pack_20220701183246/ob_log_xxx.xxx.xxx.xxx_20220701183918.zip | +-------------------+-----------+----------+------------------+--------+-------------------------------------------------------------------------------+Notice
If encryption is enabled, collected logs are encrypted when being compressed. The value in the
Passwordcolumn indicates the password for decrypting the .zip package. Encryption is disabled by default.Collect logs of a recent period
# Collect logs of the last hour with the target nodes accessible to obdiag without a password. obdiag gather log --since 1h \ --config obcluster.servers.nodes[0].ip=xx.xx.xx.1 \ --config obcluster.servers.nodes[1].ip=xx.xx.xx.xx.2 \ --config obcluster.servers.nodes[2].ip=xx.xx.xx.xx.3 \ --config obcluster.servers.global.home_path=/home/admin/oceanbase # Collect logs of the last 30 minutes with the target nodes accessible to obdiag without a password. obdiag gather log --since 30m \ --config obcluster.servers.nodes[0].ip=xx.xx.xx.1 \ --config obcluster.servers.nodes[1].ip=xx.xx.xx.xx.2 \ --config obcluster.servers.nodes[2].ip=xx.xx.xx.xx.3 \ --config obcluster.servers.global.home_path=/home/admin/oceanbase # Collect logs of the last 30 minutes with the information on different OBServer nodes varying. obdiag gather log --since 30m \ --config obcluster.servers.nodes[0].ip=xx.xx.xx.1 \ --config obcluster.servers.nodes[0].ssh_username=test1 \ --config obcluster.servers.nodes[0].ssh_password=****** \ --config obcluster.servers.nodes[0].home_path=/home/test1/oceanbase \ --config obcluster.servers.nodes[1].ip=xx.xx.xx.xx.2 --config obcluster.servers.nodes[1].ssh_username=test2 \ --config obcluster.servers.nodes[1].ssh_password=****** \ --config obcluster.servers.nodes[1].home_path=/home/test2/oceanbase # Collect logs of the last hour with the connection information of the sys tenant specified. In this case, obdiag automatically obtains the IP addresses of all nodes in the cluster. obdiag gather log --since 1h \ --config db_host=xx.xx.xx.xx \ --config db_port=xxxx \ --config tenant_sys.user=root@sys \ --config tenant_sys.password=*** \ --config obcluster.servers.global.ssh_username=test \ --config obcluster.servers.global.ssh_password=****** \ --config obcluster.servers.global.home_path=/home/admin/oceanbaseCollect logs filtered by keyword
# Collect logs of the last 30 minutes with logs filtered by `TRACE_ID`. obdiag gather log --grep "TRACE_ID" \ --config db_host=xx.xx.xx.xx \ --config db_port=xxxx \ --config tenant_sys.user=root@sys \ --config tenant_sys.password=*** \ --config obcluster.servers.global.ssh_username=test \ --config obcluster.servers.global.ssh_password=****** \ --config obcluster.servers.global.home_path=/home/admin/oceanbase # Collect logs of the last 30 minutes with logs filtered by multiple keywords, such as `AAAAA` and `BBBBB`. obdiag gather log --grep "AAAAA" --grep "BBBBB" \ --config db_host=xx.xx.xx.xx \ --config db_port=xxxx \ --config tenant_sys.user=root@sys \ --config tenant_sys.password=*** \ --config obcluster.servers.global.ssh_username=test \ --config obcluster.servers.global.ssh_password=****** \ --config obcluster.servers.global.home_path=/home/admin/oceanbase # Collect logs of the specified period with logs filtered by multiple keywords, such as `AAAAA` and `BBBBB`. obdiag gather log --from "2022-06-30 16:25:00" --to "2022-06-30 18:30:00" --grep "AAAAA" --grep "BBBBB" \ --config db_host=xx.xx.xx.xx \ --config db_port=xxxx \ --config tenant_sys.user=root@sys \ --config tenant_sys.password=*** \ --config obcluster.servers.global.ssh_username=test \ --config obcluster.servers.global.ssh_password=****** \ --config obcluster.servers.global.home_path=/home/admin/oceanbaseNotice
If logs are filtered by multiple keywords, the keywords are evaluated by using the
ANDoperator. In other words, only logs that meet all the specified keywords are collected.
Method 2: Use obdiag with a configuration file
Before you use obdiag, make sure that you have configured the login information of the target nodes in the config.yml configuration file of obdiag in the ~/.obdiag/ directory. For more information, see Configure obdiag.
Collect logs of the specified period
obdiag gather log --scope observer --from "2022-06-30 16:25:00" --to "2022-06-30 18:30:00" --grep STORAGE --encrypt trueThe output is as follows:
... ZipFileInfo: +-------------------+-----------+ | Node | LogSize | +===================+===========+ | xxx.xxx.xxx.xxx | 36.184M | +-------------------+-----------+ ... ZipFileInfo: +-------------------+-----------+ | Node | LogSize | +===================+===========+ | xxx.xxx.xxx.xxx | 44.176M | +-------------------+-----------+ ... Summary: +-------------------+-----------+----------+------------------+--------+-------------------------------------------------------------------------------+ | Node | Status | Size | Password | Time | PackPath | +===================+===========+==========+==================+========+===============================================================================+ | xxx.xxx.xxx.xxx | Completed | 36.762M | HYmVourcUyRNP8Om | 19 s | obdiag_gather_pack_20220701183246/ob_log_xxx.xxx.xxx.xxx_20220701183247.zip | +-------------------+-----------+----------+------------------+--------+-------------------------------------------------------------------------------+ | xxx.xxx.xxx.xxx | Completed | 638.200M | 1RicMaiLUUNfemnj | 718 s | obdiag_gather_pack_20220701183246/ob_log_xxx.xxx.xxx.xxx_20220701183918.zip | +-------------------+-----------+----------+------------------+--------+-------------------------------------------------------------------------------+Notice
If encryption is enabled, collected logs are encrypted when being compressed. The value in the
Passwordcolumn indicates the password for decrypting the .zip package. Encryption is disabled by default.Collect logs of a recent period
# Collect logs of the last hour. obdiag gather log --since 1h # Collect logs of the last 30 minutes. obdiag gather log --since 30m # Collect logs of the last 30 minutes with the configuration file of the target cluster specified. obdiag gather log --since 30m -c /root/config.ymlCollect logs filtered by keyword
# Collect logs of the last 30 minutes with logs filtered by `TRACE_ID`. obdiag gather log --grep "TRACE_ID" # Collect logs of the last 30 minutes with logs filtered by multiple keywords, such as `AAAAA` and `BBBBB`. obdiag gather log --grep "AAAAA" --grep "BBBBB" # Collect logs of the specified period with logs filtered by multiple keywords, such as `AAAAA` and `BBBBB`. obdiag gather log --from "2022-06-30 16:25:00" --to "2022-06-30 18:30:00" --grep "AAAAA" --grep "BBBBB"Notice
If logs are filtered by multiple keywords, the keywords are evaluated by using the
ANDoperator. In other words, only logs that meet all the specified keywords are collected.
Results
The command generates a folder in the specified directory. A sample folder name is obdiag_gather_pack_20240808105846.
Define a data masking rule
In most cases, data masking rules are defined on the business side. In other words, sensitive data is defined by users. obdiag supports custom data masking rules in V2.5.0 and later. Data masking rule files are stored in the ~/.obdiag/gather/redact/ directory. Each .py file corresponds to a data masking rule. The file name is the rule name.
This section describes how to define a data masking rule named a_test.
Create a rule file.
In the
~/.obdiag/gather/redact/directory, create a rule file nameda_test.py, which is also the rule name.Create a rule class.
To facilitate management, you need to create a rule class to inherit the RedactBase class. You can define the class name, which is usually the same as the file name. Here is an example:
from handler.gather.plugins.redact import RedactBase class a_test(RedactBase):Create a rule method.
The rule class needs to implement the redact method.
textis the input file content, which is not split into lines. You can split the content as needed.text_lines = text.split('\n')Return the processed information as a string.
If the file content is split into lines, the processing time increases greatly. We recommend that you do not split the file content unless necessary.
Create a rule instance.
You need to set a variable named after the rule name for being called by the obdiag kernel. In this example, you need to set a
a_testvariable as follows:a_test = a_test()
Complete sample code
from handler.gather.plugins.redact import RedactBase
import re
class a_test(RedactBase):
def redact(self, text):
text_lines = text.split('\n')
redacted_log = ''
for line in text_lines:
if 'test' in line:
redacted_log = redacted_log + re.sub('test', '******', line)
return redacted_log
a_test = a_test()