Overview

This topic describes how a layered minor compaction works.

The storage engine of OceanBase Database uses the log-structured merge-tree (LSM-tree) architecture. In this architecture, data is stored in MemTables and SSTables. When the memory usage of a MemTable reaches a specified threshold, data in the MemTable is flushed to the disk to release the memory space. This process is called a minor compaction. Before a minor compaction, a minor freeze is performed to ensure that no new data is written to the MemTable. The minor freeze prevents new data from being written to the current active MemTable and generates a new active MemTable.

Layered minor compaction

In OceanBase Database V2.1.x and earlier, only one SSTable is maintained at a time during a minor compaction. When data in a MemTable is merged into the SSTable during each minor compaction, the size of the SSTable grows along with the number of minor compactions. In addition, the amount of data to be processed increases along with the number of minor compactions. As a result, the compaction speed decreases, and the memory of the MemTable is fully occupied. OceanBase Database introduces a layered minor compaction strategy since V2.2.x.

The following figure shows how a layered minor compaction works.

The process of a minor compaction in OceanBase Database is divided into the following layers:

• L0 layer

  The L0 layer consists of mini SSTables. However, the L0 layer may contain no mini SSTables based on minor compaction strategies specified by different parameters.

• L1 layer

  The L1 layer consists of minor SSTables that are sequenced by the rowkey. If the number of mini SSTables at the L0 layer reaches the major compaction threshold, they are merged into minor SSTables at the L1 layer. To increase the merge efficiency and reduce write amplification, OceanBase Database allows you to set a write amplification factor. Merge with the minor SSTables at the L1 layer is scheduled only when the ratio of the total size of mini SSTables at the L0 layer to that of minor SSTables at the L1 layer reaches the specified threshold.

• L2 layer

  The L2 layer consists of baseline major SSTables. To ensure that the baseline data is completely consistent across replicas, the major SSTables are read-only in minor compactions.

Minor compaction in queuing table mode

OceanBase Database provides users with a custom table mode called queuing table (also known as buffer table from a business perspective) and implements a special minor compaction strategy for it. You can specify the queuing table mode using the following statement:

obclient> ALTER TABLE table_name TABLE_MODE = 'queuing';

For queuing tables, OceanBase Database introduces an adaptive buffer table minor compaction strategy. The storage layer automatically determines whether to use this strategy for a table based on the statistical information during each minor compaction. When the system detects buffer table-like behavior for a table, it attempts to schedule a buffer minor compaction for that table. This action generates a buffer minor SSTable based on the major SSTable and the latest incremental data at the current read snapshot time for that table. The minor compaction eliminates all delete markers from the incremental data, and subsequent queries based on the newly generated buffer minor SSTable can avoid a significant amount of unnecessary scanning operations.

Trigger methods

A minor compaction can be automatically triggered or manually initiated.