Logs

An OBServer node generates three types of logs: observer.log, rootservice.log, and election.log. These logs are all stored in the log/ directory under the working directory of the OBServer node.

Each log type, such as observer.log, can be further divided into the following types by the file name:

• observer.log

• observer.log.20210901123456

• observer.log.wf

• observer.log.wf.20210901123456

When the size of the observer.log file reaches 256 MB, it is renamed to the second type. The appended string of digits is a timestamp. For more information about the wf logs, see the description of the enable_syslog_wf parameter.

Log format

The following example shows how a log looks like:

[2021-09-01 11:31:18.605433] INFO  [STORAGE] ob_pg_sstable_garbage_collector.cpp:170 [38715][0][Y0-0000000000000000-0-0] [lt=15] [dc=0] do one gc free sstable by queue(ret=0, free sstable cnt=0)

The log contains the following information:

[time] log level [module] file:row number [thread id][0][trace_id] [lt = time consumed by the previous log (in microseconds)][number of discarded logs]

Log levels

Logs are divided into the following six levels from low to high: DEBUG, TRACE, INFO, WARN, USER_ERR, and ERROR.

The ERROR log is special. It prints the stack where the log is generated. The symbol table is required to parse it.

Notice

The DEBUG log consumes a significant amount of resources. In the latest version of OceanBase Database, the DEBUG log is automatically removed in RELEASE compilation and does not take effect even if it is enabled.

Modules

Logs can be distinguished from one another by their modules.

Thread ID

The ID of the thread that generates the log can be used to track the thread behaviors and is useful for troubleshooting faults such as thread hanging and timeout.

trace_id

The ID of an internal SQL statement of OceanBase Database. The default value is Y0-0000000000000000-0-0. The trace_id can be used to find the execution process of an SQL statement and is an important means of troubleshooting.

Related parameters

This section describes cluster-level parameters that must be used in the sys tenant. You can modify the parameters by using the following syntax:

alter system set enable_syslog_recycle = False;

• enable_syslog_recycle

  Specifies whether to enable log recycling. The default value is False.

  If you set this parameter to True, redundant logs are automatically deleted. For more information, see max_syslog_file_count.

• enable_syslog_wf

  Specifies whether to enable wf logs. The default value is True.

  If you set this parameter to True, logs of the WARN or higher levels in each type of log file are replicated to the wf log file, such as observer.log.wf.

• enable_async_syslog

  Specifies whether to enable asynchronous writing of logs. The default value is True.

  If you set this parameter to False, logs are written in synchronous mode. This ensures that all logs are written before an OBServer node is shut down. However, the OBServer node performance is significantly reduced. We recommend that you set this parameter to True.

• max_syslog_file_count

  The maximum number of log files for each log type. The default value is 0. This parameter takes effect only when the value is greater than 0 and the enable_syslog_recycle parameter is set to True.

  Notice

  Although the value range of this parameter is [0, +∞), the OBServer source code specifies that the value of this parameter cannot exceed the value of the MAX_LOG_FILE_COUNT parameter, which is 10240. You can set a larger value, but the MAX_LOG_FILE_COUNT parameter prevails.

• syslog_io_bandwidth_limit

  The maximum bandwidth for log I/O. The default value is 30 MB.

  Notice

  When the log printing speed exceeds the limit, the following message is printed.

  REACH SYSLOG RATE LIMIT
  

• syslog_level

  The lowest level of the log that is printed. The default value is WDIAG.