Distributed DBMS Architecture: Components and Design

A distributed database management system (DBMS) relies on a well-designed architecture to distribute computing resources, store large volumes of data, and maintain consistency across multiple nodes. Its architectural model determines how computing and storage resources are organized, how nodes coordinate with one another, and how the system handles failures and growing workloads.

Different distributed DBMS architectures address these requirements in different ways. Some distribute computing and storage across independent nodes, while others separate computing from storage or use middleware to coordinate multiple database instances.

OceanBase adopts a native Shared-Nothing distributed architecture, combining distributed consensus, multi-replica replication, distributed transaction processing, and multi-tenant resource management. This article explores the major architectural models of distributed DBMSs, the core components of OceanBase, its data distribution technologies, and practical hardware and deployment considerations.

Overview of Distributed DBMS Architecture

Three Common Architectural Models

Distributed DBMS architectures can generally be divided into three major categories: Shared-Nothing, Shared-Disk, and middleware-based sharding.

Shared-Nothing Architecture

In a Shared-Nothing architecture, each node has independent computing and storage resources. Nodes coordinate through distributed communication protocols, while consensus algorithms such as Paxos or Raft help maintain data consistency across replicas.

Because computing and storage resources can be distributed across multiple independent nodes, this model offers strong horizontal scalability and high availability. Capacity can be increased by adding servers, making it suitable for workloads that require large-scale data processing and continuous growth.

Shared-Disk Architecture

A Shared-Disk architecture separates the computing layer from the storage layer. Multiple computing nodes access a shared distributed storage pool rather than maintaining completely independent storage resources.

Its main advantage is the ability to scale computing and storage independently. Organizations can add computing capacity without necessarily expanding storage, or increase storage capacity without proportionally increasing computing resources. This approach is particularly useful in cloud-native environments where infrastructure resources need to be provisioned dynamically.

Middleware-Based Sharding Architecture

Middleware-based sharding introduces a proxy or routing layer between applications and conventional database instances. The middleware distributes data across multiple physical databases and directs queries to the appropriate database based on the sharding strategy.

This model is often used to extend traditional relational databases beyond the capacity of a single server. It provides a gradual migration path for organizations seeking to distribute their existing database workloads without completely replacing their original database systems.

OceanBase's Shared-Nothing Architecture

OceanBase adopts a native Shared-Nothing distributed cluster architecture. Each node operates as an independent computing and storage unit, integrating its own SQL engine, transaction engine, and storage engine.

The architecture runs on clusters of general-purpose PC servers without requiring specialized hardware. By distributing data and workloads across independent nodes, OceanBase supports horizontal scalability, high availability, high performance, and cost-efficient infrastructure utilization.

Its compatibility with mainstream database systems also helps organizations integrate distributed database capabilities into existing application environments.

High Availability and Disaster Recovery

High availability is a fundamental requirement for enterprise distributed databases. OceanBase uses distributed consensus and multi-location replica deployment to maintain data consistency and service continuity.

Distributed Leader Election and the Paxos Protocol

OceanBase uses the Paxos consensus protocol to achieve strong consistency across replicas. Transaction logs, known as Clogs, are synchronously persisted through majority-based replication.

When a minority of replicas becomes unavailable, the election mechanism ensures that a unique leader is selected from the available replicas to continue serving requests.

Under the supported majority-replication and failure conditions, this design provides zero-data-loss protection, with a Recovery Point Objective (RPO) of zero.

Multi-Data-Center and Multi-Region Deployment

OceanBase supports distributing replicas across multiple data centers or geographical locations. By deploying replicas across different physical sites within a Paxos group, organizations can build disaster recovery architectures that withstand data center-level failures.

With appropriate infrastructure and replica placement, deployments can also provide city-level disaster recovery capabilities, helping maintain database availability during major infrastructure disruptions.

Core Components of OceanBase Architecture

OceanBase builds its highly available distributed database infrastructure around several key components. OBServer, Zone, and RootService play particularly important roles in cluster operation, resource coordination, and disaster recovery.

OBServer: The Core Database Service Node

OBServer is the primary computing and storage unit in an OceanBase cluster.

Each OBServer runs as a single process that integrates the SQL engine, transaction engine, and storage engine. This full-featured design allows each node to independently handle database operations, manage local data, and participate in replica synchronization.

By distributing these capabilities across multiple nodes, OceanBase can support large-scale data storage and elastic cluster expansion without relying on a dedicated central database server.

Zone: The Logical Availability Zone

A Zone is a fundamental unit for organizing nodes and implementing disaster recovery in OceanBase. It typically corresponds to a physical data center or a cloud availability zone.

OceanBase distributes data replicas across different Zones to improve fault tolerance. For example, a deployment can use a two-site, three-data-center architecture to provide financial-grade disaster recovery capabilities.

This arrangement helps ensure that a failure affecting one physical location does not necessarily interrupt database services across the entire cluster.

RootService: The Cluster Coordination Service

RootService acts as the central coordination service of an OceanBase cluster.

It is responsible for metadata management, resource scheduling, and load balancing. These functions allow the cluster to coordinate changes in its topology and maintain balanced resource utilization as nodes and workloads change.

RootService uses Paxos-based leader election to maintain coordination service availability, reducing the risks associated with traditional architectures that depend on a fixed central controller.

Data Distribution Technologies in OceanBase

OceanBase combines storage management, distributed replication, transaction processing, and resource coordination to support high write throughput while maintaining strong data consistency.

Storage Layer: LSM-Tree Architecture

OceanBase uses an LSM-Tree-based storage architecture that organizes data through a combination of in-memory and disk-based structures.

New data is initially written to an in-memory MemTable, while persistent baseline data is stored in SSTables on disk. During the compaction process, data from different storage structures is merged and reorganized.

This design reduces random disk writes by converting many write operations into more sequential storage operations. Combined with scheduled and incremental compaction mechanisms, it improves write efficiency and supports large-scale data storage workloads.

Replication Layer: Paxos-Based Consensus

At the replication layer, OceanBase uses its Paxos-based consensus mechanism to maintain strong consistency among multiple replicas.

Transaction logs associated with each data partition are persisted through majority confirmation. This ensures that committed log records are protected against failures within the supported replication fault model.

With synchronous majority replication, OceanBase can achieve an RPO of zero, providing strong data protection for enterprise database deployments.

Transaction Layer: Distributed ACID Transactions

OceanBase supports distributed ACID transactions across multiple nodes.

Its transaction architecture relies on a Global Timestamp Service (GTS) and an optimized Two-Phase Commit (2PC) protocol to coordinate distributed transactions and maintain consistency across different data partitions.

Whether processing individual OLTP transactions or more complex HTAP workloads, the system is designed to preserve transaction integrity across the distributed environment.

Application Access and Resource Management

In addition to its storage, replication, and transaction layers, OceanBase provides mechanisms for SQL routing, query execution, and tenant resource isolation.

OBProxy: Intelligent SQL Routing

OBProxy is a stateless proxy service that sits between applications and the OceanBase cluster.

It handles SQL request routing, connection management, and load balancing. It can also support read-write separation according to the configured routing policies.

Applications connect to OBProxy rather than communicating directly with individual database nodes. This hides the complexity of the underlying distributed topology and simplifies application integration.

SQL Layer: Distributed Query Processing

The SQL layer consists of a parser, optimizer, and execution engine. It supports local, remote, and distributed execution plans according to query requirements and data locations.

For queries involving data distributed across multiple nodes, the system can coordinate distributed execution and parallel processing to improve query efficiency.

These capabilities also support HTAP workloads, allowing transactional and analytical operations to run within the same database platform.

Multi-Tenant Layer: Resource Isolation

OceanBase supports multiple tenants within a single cluster. Each tenant can be allocated independent CPU, memory, and I/O resources to isolate workloads and control resource consumption.

This multi-tenant architecture allows organizations to consolidate multiple business databases on shared infrastructure while maintaining logical resource isolation.

Combined with MySQL and Oracle compatibility modes, it also helps enterprises integrate existing database applications and reduce infrastructure and operational costs.

Hardware and Deployment Recommendations

Hardware configuration directly affects the performance, stability, and scalability of a distributed database. Although OceanBase can run on general-purpose servers, infrastructure should be selected according to workload requirements and deployment objectives.

Basic Infrastructure Requirements

For a standard deployment, the recommended starting configuration includes:

• At least three physical or virtual machines.
• 4 vCPUs per machine.
• 16 GB of memory.
• 100 GB of SSD storage or higher-performance disks.

This configuration provides a baseline for deploying and testing an OceanBase cluster. Production environments may require additional resources depending on data volume, transaction throughput, concurrency, and availability requirements.

Hardware for High-Performance Deployments

For compute-intensive workloads and environments with demanding performance requirements, a more powerful server configuration may be appropriate.

A representative configuration includes:

• 32 CPU cores.
• 256 GB of memory.
• 2 TB of SSD storage.
• 10 Gigabit Ethernet network interfaces and compatible switches.

Sufficient CPU and memory capacity helps prevent computing bottlenecks, while high-performance SSDs improve storage access efficiency. Fast network connections are equally important because distributed nodes rely on network communication for replication, transaction coordination, and data exchange.

The final hardware configuration should be validated through workload-specific performance testing.

Considerations for Multi-Data-Center Deployment

Multi-data-center and multi-city deployments require careful consideration of both business continuity requirements and infrastructure costs.

Network latency, bandwidth, and connectivity between locations can directly affect replica synchronization and distributed transaction performance. For cross-city deployments, dedicated network connections may be necessary to provide stable communication between database nodes.

Organizations should therefore evaluate disaster recovery objectives alongside network infrastructure costs and application performance requirements.

With reliable infrastructure and appropriate replica placement, OceanBase can provide highly available database services across multiple physical locations.

Conclusion

A distributed DBMS architecture involves more than distributing data across multiple servers. It requires coordinated mechanisms for computing, storage, replication, transaction processing, and resource management.

OceanBase integrates these capabilities through its Shared-Nothing architecture, OBServer nodes, Zone-based deployment, RootService coordination, LSM-Tree storage, Paxos replication, and distributed transaction processing.

Understanding how these components work together provides a practical foundation for designing, deploying, and managing distributed database environments that need to support growing data volumes, demanding workloads, and high availability requirements.