Multi-level caching in shared storage

To provide a more cost-effective database service in a multi-cloud environment, OceanBase Database implements a shared storage (SS) cluster architecture based on general-purpose object storage. In the shared-nothing (SN) mode, the system caches hot data in memory to improve performance. In the SS mode, in addition to memory caching, it also introduces local persistent caching to cache sub-hot data.

In the preceding figure:

• Memory caching: caches the most hot data, including microblocks and index data.

• Persistent caching: the storage medium can be a local physical disk or cloud disk, and is used to cache sub-hot data. Its main functions include:

  1. Caching data read from object storage.

  2. Pre-reading data. For example, during a full table scan, the system pre-caches data to be read locally.

  3. Preheating. Before a major compaction, the system loads hot data into local caching. After a short downtime and restart, the persistent cache remains valid, reducing performance fluctuations in disaster recovery scenarios.

     At the same time, to ensure RTO, the access sequence of the local persistent cache is synchronized in real time to the follower replicas. The leader and follower replicas have nearly identical cached data. During replication and migration, the local cached data is also transferred to the destination, ensuring performance after replication and migration.

  4. Write caching. The local disk's low latency is used as a write cache to record important local states, such as meta files, the latest clog, and mini dumps.

• Object storage: stores all data in the system, including shared clogs and baseline data across the cluster, as well as node-private data written when local physical disks or cloud disks are insufficient, such as incremental data and temporary files. Object storage has nearly unlimited capacity but has high access latency.

Through the above multi-level caching, the access performance of hot data is ensured, and the system can provide a product form that meets both TP and AP business needs based on object storage.