Log writing
To ensure all DML operations remain transactional, redo logs are generated to store the insertion, update, or deletion made to data rows. These logs are called clogs.
Each observer process running on one physical OceanBase Database host has between dozens and hundreds of thousands of partitions sharing a single clog file. A new clog file is opened when the existing one exceeds the specified threshold (which defaults to 64 MB).
The OBServer receives clogs generated by the write request of the partition leader as well as those synchronized from other OBServers (whose partitions belong to the same Paxos group). These are written to the log buffer and then flushed in batches into the clog file through a single I/O thread.
Write requests are synchronized to other OBServers before being flushed on the OBServer where the partition leader is located. A success message is returned to the client after a majority of the OBServers report success.
Data writing
Different from the dirty page flushing mechanism used in conventional database systems, OceanBase Database employs an LSM Tree-based storage engine, where most data block writing takes place during minor compactions and major compactions. During a minor compaction, OceanBase Database compacts data of the frozen MemTable with the SSTable and records a replay checkpoint of the clog in the static data. Technically, when the minor compaction completes, logs generated before the replay checkpoint can be recycled. However, instead of being deleted, these log files are usually reused when the log space is insufficient.
Larger SSTables may be split into multiple threads for minor compaction or major compaction. The table-wide parameter tablet_size is used to adjust the granularity of parallel major compactions. SSTables larger than tablet_size are split based on this parameter for parallel major compaction. Though in general, the number of parallel major compactions will not exceed the number of freeze threads specified.
Query process
Data queries include the single-point query Get, multiple-point query MultiGet, single-range query Scan, multiple-range query MultiScan, and the Exist query which is performed during insertion.
The procedures for Get/MultiGet/Exist are similar, as shown in the following figure.
The procedures for Scan/MultiScan are similar, as shown in the following figure.
