Compaction

Compaction consists of two processes: mini compaction and minor compaction.

Mini compaction

When the size of the MemTable exceeds a specific threshold, data in the MemTable is flushed to a Mini SSTable to release the memory, which is called mini compaction.

The core of mini compaction is to release memory and data logs. The frozen MemTable in memory is flushed to a disk as a Mini SSTable during mini compaction. The Mini SSTable is a checkpoint of data logs. After mini compaction is completed, the frozen MemTable and logs corresponding to the Mini SSTable can be released.

Minor compaction

As user data is written, the number of Mini SSTables increases. This leads to a higher number of SSTables that need to be accessed during queries, which negatively impacts query performance. Minor compaction combines multiple Mini SSTables into one SSTable. Its primary purpose is to reduce the number of SSTables. When the number of Mini SSTables exceeds the threshold, the system automatically triggers minor compaction in the background.

Based on the SSTable layering strategy, minor compaction can be divided into the following two types:

1. L0 -> L0

   Tiered minor compaction: the system combines several Mini SSTables into one Mini SSTable, which is placed in the L0 layer.

   

2. L0 -> L1

   Level minor compaction: the system combines several Mini SSTables with Minor SSTables into one Minor SSTable, which is placed in the L1 layer.

   

Trigger compaction

Compaction can be triggered manually or automatically.

When the size of the active MemStore memory for a tenant reaches freeze_trigger_percentage * memstore_limit (where memstore_limit = tenant memory * memstore_limit_percentage), the system automatically triggers the freeze (a prerequisite for compaction). After that, the system internally schedules the compaction.

You can also manually trigger compaction using the following O&M command.

Here are some examples:

• Cluster-level compaction

  • Compaction initiated for the sys tenant

    obclient> ALTER SYSTEM MINOR FREEZE TENANT = sys;
    

  • Compaction initiated for all user tenants

    obclient> ALTER SYSTEM MINOR FREEZE TENANT = all_user;
    

  • Compaction initiated for all META tenants

    obclient> ALTER SYSTEM MINOR FREEZE TENANT = all_meta;
    

• Tenant-level compaction

  obclient> ALTER SYSTEM MINOR FREEZE TENANT= prod_tenant;
  

References

For more information about compaction, see Overview of compaction.