This topic introduces how layered minor compaction works.
The storage engine of OceanBase Database uses the log-structured merge-tree (LSM-tree) architecture. In this architecture, data is stored in MemTables and SSTables. When the memory usage of a MemTable reaches a specified threshold, data in the MemTable is flushed to the disk to release the memory space. This process is called a minor compaction. Before a minor compaction, a minor freeze is performed to ensure that no new data is written to the MemTable. The minor freeze prevents new data from being written to the current active MemTable and generates a new active MemTable.
Layered minor compaction
Before layered minor compaction is introduced, OceanBase Database maintains only one SSTable at a time. When the MemTable requires a minor compaction, the data in the MemTable is compacted with the data in the SSTable. As the number of minor compactions increases, the size of the SSTable and the amount of data operated in a minor compaction also increase. This slows down minor compactions and may cause insufficient memory space for the MemTable.
Therefore, OceanBase Database introduces the layered minor compaction structure in which the L0 layer is added.
L0 layer (mini SSTable)
The frozen MemTable is directly flushed into a mini SSTable. OceanBase Database can have multiple mini SSTables at the same time.
Multiple mini SSTables with consecutive version numbers in the L0 layer can be compacted into a minor SSTable.
L1 layer (minor SSTable)
Only one minor SSTable exists in most cases.
Each pushdown operation in the L0 layer will generate a new minor SSTable in the L1 layer to replace the original minor SSTable.
L2 layer (major SSTable)
The major SSTable comprises baseline data and is generated during a major compaction. Generally, only one major SSTable exists.
The following figure shows how a layered minor compaction structure works.
Trigger methods
A minor compaction can be automatically triggered or manually initiated.