Logs

An OBServer node generates three types of logs: observer.log, rootservice.log, and election.log. These logs are all stored in the log/ directory under the installation directory of the OBServer node.

Each log type, such as observer.log, can be further divided into the following types by the file name:

• observer.log

• observer.log.20210901123456

• observer.log.wf

• observer.log.wf.20210901123456

When the size of the observer.log file reaches 256 MB, it is renamed to the second type. The appended string of digits is a timestamp. For more information about the wf logs, see the description of the enable_syslog_wf parameter.

Log format

The following example shows how a log looks like:

[2021-09-01 11:31:18.605433] INFO  [STORAGE] ob_pg_sstable_garbage_collector.cpp:170 [38715][0][Y0-0000000000000000-0-0] [lt=15] [dc=0] do one gc free sstable by queue(ret=0, free sstable cnt=0)

The corresponding log information is as follows:

[time] log level [module] file:row number [thread id][0][trace_id] [lt = time consumed by the previous log (microseconds)][number of discarded logs]

Log levels

The logs are divided into the following six levels from low to high: DEBUG, TRACE, INFO, WARN, USER_ERR, and ERROR.

The ERROR log level is special. It prints the stack where the log is generated. The symbol table is required to parse it.

Notice

The DEBUG log level consumes a significant amount of resources. In the latest version of OceanBase Database, the DEBUG log level is automatically removed in RELEASE compilation and does not take effect even if it is enabled.

Modules

Logs can be distinguished from one another by their modules.

Thread ID

The ID of the thread that generates the log can be used to track the thread behaviors and is useful for troubleshooting faults such as thread hanging and timeout.

trace_id

The ID of an internal SQL statement of OceanBase Database. The default value is Y0-0000000000000000-0-0. The trace_id can be used to find the execution process of an SQL statement and is an important means of troubleshooting.

Related parameters

This section describes cluster-level parameters that must be used in the sys tenant. You can modify the parameters by using the following syntax:

alter system set enable_syslog_recycle = False;

• enable_syslog_recycle

  Specifies whether to enable system log recycling. The default value is False.

  If you enable system log recycling, redundant logs are automatically deleted. For more information, see max_syslog_file_count.

• enable_syslog_wf

  Specifies whether to enable the .wf log. The default value is True.

  If you enable the .wf log, logs of the WARN or higher levels in each type of log files are backed up to the .wf log, for example, observer.log.wf.

• enable_async_syslog

  Specifies whether to enable asynchronous writing of logs. The default value is True.

  If you disable asynchronous writing of logs, logs are written in synchronous mode. This ensures that all logs are written before an OBServer node is shutdown. However, performance of the OBServer node is significantly reduced. We recommend that you set this parameter to True.

• max_syslog_file_count

  The maximum number of log files for each log type. The default value is 0. This parameter takes effect only when the value is greater than 0 and the enable_syslog_recycle parameter is set to True.

  Notice

  Although the value range of this parameter is [0, +∞), the source code of the OBServer node specifies that the value cannot exceed the value of the MAX_LOG_FILE_COUNT parameter, which is 10240. You can set it to a larger value, but only the MAX_LOG_FILE_COUNT parameter takes effect.

• syslog_io_bandwidth_limit

  The maximum bandwidth for log I/O. The default value is 30 MB.

  Notice

  When the log printing speed exceeds the limit, the following message is printed:

  REACH SYSLOG RATE LIMIT
  

• syslog_level

  The lowest level of logs to be printed. The default value is WDIAG.