Dump consists of two processes: mini compaction and minor compaction. When the size of the MemTable exceeds a certain threshold, the data in the MemTable needs to be dumped into a Mini SSTable to free up memory. This process is called mini compaction. As user data continues to be written, the number of mini SSTables increases. When the number of mini SSTables exceeds a certain threshold, minor compaction is automatically triggered in the background. Mini compaction generates new mini SSTables. When the number of mini compaction operations exceeds a certain threshold, or during the daily business off-peak hours, the system will merge the baseline SSTable (major SSTable) with the incremental SSTables generated by subsequent dumps (Mini/Minor SSTables) into a new major SSTable. This process is called major compaction.
The percentage of memory allowed for MemTables to the total available MemStore memory of a tenant is specified by the memstore_limit_percentage parameter. MemTables are classified into active MemTables and frozen MemTables. When the memory usage of MemTable reaches memstore_limit_percentage × freeze_trigger_percentage, a freeze is automatically triggered to generate a new active MemTable. The existing active MemTable becomes a frozen MemTable. Then, a minor compaction is automatically scheduled. When the minor compaction is completed, the memory occupied by the frozen MemTable is released.
Note
memstore_limit_percentagespecifies the percentage of the total available memory of a tenant that can be used for MemStores. The default value is 0. The range of values is [0, 100). For more information about this parameter, see memstore_limit_percentage.freeze_trigger_percentagespecifies the memory usage threshold for triggering a global freeze for tenants. The default value is 20, and the value range is (0, 100). For more information about this parameter, see freeze_trigger_percentage.
You can speed up triggering dump by using the following methods:
Set the
freeze_trigger_percentageparameter to a smaller value.Manually run
alter system minor freezeto trigger dump.
A major compaction is automatically triggered when the number of freezes in a tenant reaches the threshold specified by using the major_compact_trigger parameter. In OceanBase Database V4.x, the default value is 0, which indicates that the automatic triggering of a major compaction based on the number of freezes is disabled.
Note
The major_compact_trigger parameter specifies the threshold of the number of freezes at which a major compaction is triggered. The value range is [0, 65535]. For more information about this parameter, see major_compact_trigger.
You can speed up triggering merge by using the following methods:
Set the
major_compact_triggerparameter to a smaller value.Manually run
alter system major freezeto trigger merge.
Data inconsistencies during a major compaction
The following data inconsistencies may occur during a major compaction:
Row count inconsistency: The number of rows in the primary table is inconsistent with that in the index table.
Column checksum inconsistency: The checksum values of some columns in the major SSTable are inconsistent.
Data checksum inconsistency: The checksum values of all rows in the major SSTable are inconsistent.