Minor compaction and major compaction

The storage engine of OceanBase Database is built on the log-structured merge-tree (LSM-tree) architecture. In this architecture, data is stored in MemTables and SSTables. When the size of a MemTable exceeds the specified threshold, data in the MemTable is flushed to an SSTable to release the memory space. This process is called a minor compaction. Each minor compaction generates a new SSTable. When the number of minor compactions exceeds the specified threshold or during off-peak hours of a day, the baseline SSTables and the incremental SSTables generated by minor compactions are merged into one SSTable. This process is called a major compaction.

The percentage of memory allowed for MemTables to the total available MemStore memory of a tenant is specified by the memstore_limit_percentage parameter. MemTables are classified into active MemTables and frozen MemTables. When the memory usage of MemTable reaches memstore_limit_percentage × freeze_trigger_percentage, a freeze is automatically triggered to generate a new active MemTable. The existing active MemTable becomes a frozen MemTable. Then, a minor compaction is automatically scheduled. When the minor compaction is completed, the memory occupied by the frozen MemTable is released.

You can increase the frequency of minor compaction by using the following methods:

• Set the freeze_trigger_percentage parameter to a smaller value.

• Execute the alter system minor freeze statement to trigger a minor compaction.

A major compaction is automatically triggered when the number of minor compactions in a tenant reaches the threshold specified by using the major_compact_trigger parameter. In OceanBase Database V4.0, the default value is 0, which indicates that the automatic triggering of a major compaction based on the number of minor compactions is disabled.

You can increase the frequency of major compaction by using the following methods:

• Set the major_compact_trigger parameter to a smaller value.

• Execute the alter system major freeze statement to trigger a major compaction.

Data inconsistencies during a major compaction

The following data inconsistencies may occur during a major compaction:

• The number of rows in the primary table is inconsistent with that in the index table.

• The checksum values of some columns in the major SSTable are inconsistent.

• The checksum values of all rows in the major SSTable are inconsistent.