Overview of data load balancing

OceanBase Database provides load balancing capabilities such as horizontal scaling and dynamic data balancing.

Horizontal scaling refers to the ability to expand or contract service capacity by adjusting the number of service nodes. For example, scaling from a single service node to two service nodes expands service capacity. Horizontal scaling also requires data redistribution. For instance, when scaling from a single service node to two service nodes, data must be evenly distributed across the two service nodes; conversely, when scaling from two service nodes to a single service node, data must be redistributed onto the single service node.

Dynamic data balancing refers to the ability to dynamically balance the load across service nodes by adjusting data distribution while keeping the number of service nodes unchanged. For example, as tables and partitions are dynamically created and deleted, the number of partitions on different service nodes can vary significantly, leading to uneven load distribution. With dynamic partition balancing, partitions can be evenly distributed across service nodes, thereby achieving load balancing.

Horizontal scaling

In OceanBase Database, the storage capacity and read/write service capabilities of a tenant are primarily influenced by the following two factors:

• Unit Number, which refers to the number of service units provided in each zone.

  By increasing or decreasing the Unit Number, you can add or remove service nodes, thereby achieving horizontal scaling for read/write services and storage capacity.

  For detailed operations on scaling a tenant by adjusting the Unit Number, see Scale a tenant by adjusting the Unit Number.

• Primary Zone, which refers to the list of zones that provide read/write services.

  By increasing or decreasing the number of primary zones with the highest priority, you can add or remove zones that provide read/write services, thereby achieving horizontal scaling for read/write services across zones.

  For detailed operations on scaling a tenant by adjusting the Primary Zone, see Scale a tenant by adjusting the Primary Zone.

By dynamically adjusting the Unit Number and Primary Zone, users can achieve horizontal scaling of a tenant's read/write service capabilities within and across zones. The load balancing feature will adaptively adjust log stream and partition distribution based on the configured service capabilities.

Partition balancing

Partition balancing refers to the process of dynamically adjusting partition distribution to achieve balance in the number of partitions, partition weights, table group weights, and storage space across service nodes, even as tables and partitions change dynamically.

OceanBase Database supports multiple table types, including non-partitioned tables, partitioned tables, and subpartitioned tables. Different table types have different balancing strategies. To facilitate the description of balancing effects, OceanBase Database groups table partitions into balancing groups. Within each balancing group, balance in the number of partitions and storage space must be achieved. There is no direct relationship between balancing groups, and the system adaptively adjusts the distribution between them internally. By default, the partition balancing strategy in OceanBase Database is as follows:

• Partitioned tables: Each partitioned table is an independent balancing group. All partitions of the table are scattered across various service nodes.

• Subpartitioned tables: All subpartitions under each partition form an independent balancing group. All subpartitions under each partition are scattered across various service nodes.

• Non-partitioned tables: All non-partitioned tables are considered together as one and only one balancing group. All non-partitioned tables are scattered across various service nodes.

To more flexibly describe the clustering and scattering relationships between data of different tables, OceanBase Database introduces the concept of a table group.

A table group is a logical concept representing a collection of tables. Tables within a table group are physically stored adjacent to each other. Tables with associated relationships often have the same partitioning rules. By clustering and scattering partitions with the same rules together, Partition Wise Join can be achieved, significantly optimizing read and write performance.

In previous versions, OceanBase Database introduced the SHARDING attribute for table groups. Starting from V5.0.1, V5.x also supports the SCOPE attribute. The distribution of partitions within a table group is determined jointly by the SHARDING and SCOPE attributes. The SHARDING attribute determines how partitions corresponding to different tables within the table group are aggregated. A group of aggregated partitions is called a partition group, which is the smallest unit of distribution and scattering. The SCOPE attribute determines the distribution scope of all aggregated partition groups within the table group.

The SHARDING attribute of a table group has four possible values: NONE, PARTITION, SUBPARTITION, and ADAPTIVE. Specifically:

• SHARDING = NONE: No restrictions on the partitioning method of tables within the table group, and partitions are not aggregated across tables.

  Note

  • For versions earlier than V5.0.1 (excluding V4.4.2), all partitions of tables in a SHARDING = NONE table group are on the same log stream and are not scattered. Starting from V5.0.1, whether all partitions of tables in a table group are scattered also depends on the table group's SCOPE attribute.
  • For V4.4.2, before V4.4.2 BP1, all partitions of tables in a SHARDING = NONE table group are on the same log stream and are not scattered. In V4.4.2 BP1 and later BP versions, whether all partitions of tables in a table group are scattered also depends on the table group's SCOPE attribute.

  • For a partitioned table, each partition serves as an independent partition group.

  • For a subpartitioned table, each partition and subpartition serve as an independent partition group.

  • For a non-partitioned table, the entire table serves as one partition group.

  • For a global index table, within a table group with SHARDING = NONE, the table's global indexes are automatically bound to the table group. A global non-partitioned index table serves as one partition group as a whole, while each partition of a global partitioned index table serves as an independent partition group.

• SHARDING = PARTITION: The entire table group forms a balanced group. Partitions under the same partition are aggregated into the same partition group.

• SHARDING = SUBPARTITION: The entire table group forms a balanced group. Subpartitions with the same partitioning key under all partitions are aggregated into the same partition group.

• SHARDING = ADAPTIVE:

  • If all tables are partitioned tables, the entire table group forms a balanced group. Partitions under the same partition are aggregated into the same partition group.

  • If all tables are subpartitioned tables, partitions under the same partition form a balanced group. Within the balanced group, subpartitions under the same subpartition are aggregated into the same partition group.

The SCOPE attribute of a table group has three possible values: SERVER, ZONE, and CLUSTER. Specifically:

• SCOPE = SERVER: The leaders of all partition groups are distributed on the same node.

• SCOPE = ZONE: The leaders of all partition groups are distributed across nodes within the same zone.

• SCOPE = CLUSTER: The leaders of all partition groups are distributed across nodes in the cluster.

The partition distribution for a table group under different combinations of SHARDING and SCOPE attributes is shown in the following table.

For more details and operations on table groups, see Create and manage table groups (MySQL-compatible mode) and Create and manage table groups (Oracle-compatible mode).

Balancing based on table group weights

In OceanBase Database's MySQL-compatible mode, user tables are aggregated by binding a database to a table group with Sharding = 'NONE'. Currently, it supports automatic aggregation of user tables within the same database and aggregation of user tables across multiple databases.

Balancing table group weights is achieved by setting weights for table groups with Sharding = 'NONE', allowing tables within the group to be distributed according to their weights. Note that in V5.x, starting from V5.0.1, for table groups with Sharding = 'NONE', weight setting is supported only for table groups with SCOPE = ZONE or SCOPE = 'SERVER'.

Table group weight balancing is a process within the partition balancing (PARTITION_BALANCE) task. User tenants can trigger it manually or wait for a scheduled partition balancing task to initiate it.

Balancing based on partition weights

To further address issues such as partition hotspots within partitioned tables and between tables, OceanBase Database provides a partition balancing strategy based on weights. Partition weights represent the relative proportion of resources such as CPU, memory, and disk occupied by partitions. Partition weight balancing is a process within the partition balancing (PARTITION_BALANCE) task. User tenants can trigger it manually or wait for a scheduled partition balancing task to initiate it.

The partition weight balancing algorithm minimizes the variance of the sum of partition weights across user log streams through partition movement and exchange.

Only tables with set partition weights participate in partition weight balancing. Partitions without set weights are still balanced based on the number of partitions.

Data load balancing parameters

• enable_rebalance

  The tenant-level parameter enable_rebalance controls whether to perform load balancing between tenants in the sys tenant and within a tenant in the user tenant. The default value is true.

• enable_transfer

  The tenant-level parameter enable_transfer controls whether to perform transfers within a tenant. The default value is true. Specifically:

  • When the value of the enable_rebalance parameter is false, the system does not perform automatic load balancing regardless of the value of the enable_transfer parameter.

  • When the value of the enable_rebalance parameter is true and the value of the enable_transfer parameter is true, the system automatically adjusts the number of log streams within a tenant during tenant scaling. This is done through operations such as log stream splitting, merging, and transfers to achieve leader balance and partition balance within the tenant.

  • When the value of the enable_rebalance parameter is true and the value of the enable_transfer parameter is false, the system does not perform transfers during tenant scaling, nor does it change the number of log streams. Instead, it strives to maintain log stream balance based on the existing log streams.

• partition_balance_schedule_interval

  The tenant-level parameter partition_balance_schedule_interval controls the time interval for generating partition load balancing tasks. When enable_rebalance is set to true, the system uses partition_balance_schedule_interval as the interval to automatically trigger partition load balancing tasks. The default value is 2h, with a range of [0s, +∞). A value of 0s disables partition balancing.

  Notice

  For V4.4.x, the default value of this parameter was changed to 0s starting from V4.4.1. For V4.4.1 and later versions, it is not recommended to use this parameter to control partition balancing. Users can trigger partition balancing periodically or manually by calling the DBMS_BALANCE.TRIGGER_PARTITION_BALANCE subprogram.

• enable_database_sharding_none

  The tenant-level parameter enable_database_sharding_none controls whether to enable automatic aggregation for user tables when creating a database. The default value is False, meaning automatic aggregation for user tables is disabled by default when creating a database.

• zone_disk_balance_tolerance_percentage

  The tenant-level parameter zone_disk_balance_tolerance_percentage controls the tolerance level of the inter-zone table group disk balancing algorithm for imbalance. Disk balancing between zones is triggered only when the difference between the disk usage of the zone with the highest usage and the zone with the lowest usage exceeds the maximum zone disk usage multiplied by this percentage. The default value is 10.