Tenant pre-scaling

OceanBase Database implements a pre-scaling feature to achieve smooth scaling for tenants. This feature works by configuring the scaling factor (ls_scale_out_factor) for the tenant, which determines the number of log streams to be created before scaling occurs. By generating enough log streams in advance, their states and positions will align with those required after scaling, allowing for smooth expansion or contraction operations.

Background information

In the current version, OceanBase Database supports both homogeneous and heterogeneous zone modes for tenants:

• In homogeneous zone mode, all zones under the tenant must have the same UNIT_NUM.

• In heterogeneous zone mode, the UNIT_NUM of each zone can be the same or different, but a tenant can have at most two different UNIT_NUM values across all zones.

This article provides operation guidance for tenant pre-scaling in homogeneous zone mode. For heterogeneous zone mode, it is recommended to use heterogeneous zone maintenance methods for smooth scaling. For an example of smooth scaling using heterogeneous zones, see Smooth scaling application cases.

Introduction to scaling factor

In earlier versions, a tenant could have only one log stream on each node. If a node had enough log streams, log stream position redistribution could be triggered directly during scaling, without needing to use Transfer to split log streams.

To ensure that a tenant has enough log streams before scaling, OceanBase Database introduced the concept of the scaling factor (ls_scale_out_factor). The scaling factor defines how many log streams each log stream should be split into to meet future scaling requirements.

Formula for calculating the scaling factor

In OceanBase Database, the number of regular log streams for a tenant (excluding system and broadcast log streams) is calculated as: number of the tenant's UNIT_NUM (U) × number of the tenant's PRIMARY_ZONE with highest priority (P).

Assume that before scaling, the number of the tenant's UNIT_NUM is U1, and the number of the tenant's PRIMARY_ZONE with highest priority is P1. After scaling, the number of the tenant's UNIT_NUM is U2, and the number of PRIMARY_ZONE with highest priority is P2. The formula for calculating the scaling factor (ls_scale_out_factor) is as follows:

ls_scale_out_factor = LCM (U1 * p1 , U2 * P2)/ (U1 * P1)

In other words, the least common multiple (LCM) of (U1 * p1) and (U2 * P2), divided by (U1 * P1), gives the scaling factor for the tenant.

Pre-scaling

Assume that the current tenant is mq_t1, with locality set to F@zone1, F@zone2, F@zone3. Each zone has one unit, and the PRIMARY_ZONE is distributed across zone1 and zone2. The goal is to increase the tenant's UNIT_NUM to 2, while keeping the number of first-priority PRIMARY_ZONEs unchanged. The steps for pre-scaling are as follows:

1. Log in to the sys tenant of the cluster as the root user.

   For example:

   obclient -h172.30.xxx.xxx -P2883 -uroot@sys#obdemo -pxxxx -A
   

2. Check the TENANT_ID, UNIT_NUM, and PRIMARY_ZONE information for the tenant mq_t1.

   obclient(root@sys)[oceanbase]> SELECT TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, UNIT_NUM FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_NAME = 'mq_t1';
   

   The query result is as follows:

   +-----------+-------------+-------------+-------------------+---------------------------------------------+----------+
   | TENANT_ID | TENANT_NAME | TENANT_TYPE | PRIMARY_ZONE      | LOCALITY                                    | UNIT_NUM |
   +-----------+-------------+-------------+-------------------+---------------------------------------------+----------+
   |      1002 | mq_t1       | USER        | zone1,zone2;zone3 | FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3 |        1 |
   +-----------+-------------+-------------+-------------------+---------------------------------------------+----------+
   1 row in set
   

3. Calculate the expansion factor ls_scale_out_factor and configure it for the tenant.

   In this example, before expansion, tenant mq_t1 has a UNIT_NUM of 1 and two first-priority PRIMARY_ZONE. After expansion, the tenant's UNIT_NUM will be 2, and the number of first-priority PRIMARY_ZONE remains 2. According to the expansion factor formula, the expansion factor is calculated as LCM(2 * 1, 2 * 2) / (2 * 1) = 2.

   Based on this calculation, set the value of the ls_scale_out_factor parameter for the tenant.

   obclient(root@sys)[oceanbase]> ALTER SYSTEM SET ls_scale_out_factor = 2 TENANT = mq_t1;
   

4. Enable the tenant-level parameters enable_rebalance and enable_transfer by setting them to True. This allows enough log streams to be created for the tenant.

   obclient(root@sys)[oceanbase]> ALTER SYSTEM SET enable_rebalance = True TENANT = mq_t1;
   

   obclient(root@sys)[oceanbase]> ALTER SYSTEM SET enable_transfer = True TENANT = mq_t1;
   

5. After waiting for the tenant to generate enough log streams, check whether the number of log streams in the NORMAL state matches expectations.

   According to the expansion factor formula, after expansion, the number of log streams in the NORMAL state (excluding system and broadcast log streams) should be: the original UNIT_NUM (U1) × the original number of first-priority PRIMARY_ZONE (P1) × the expansion factor (ls_scale_out_factor). In this case, the expected number of log streams is 1 * 2 * 2 = 4.

   obclient(root@sys)[oceanbase]> SELECT TENANT_ID, STATUS, COUNT(*) FROM oceanbase.CDB_OB_LS WHERE TENANT_ID = 1002 AND FLAG = '' AND LS_ID != 1 GROUP BY STATUS;
   

   Sample query result:

   +-----------+--------+----------+
   | TENANT_ID | STATUS | COUNT(*) |
   +-----------+--------+----------+
   |      1002 | NORMAL |        4 |
   +-----------+--------+----------+
   1 row in set
   

6. Confirm that there are no remaining load balancing tasks for the tenant being expanded.

   obclient(root@sys)[oceanbase]> SELECT * FROM oceanbase.CDB_OB_BALANCE_JOBS WHERE TENANT_ID = 1002;
   

   If the query result is empty, the pre-scaling operation is complete.

What to do next

After the pre-scaling is completed, you can scale in or out the tenant based on the business needs and the expected scaling plan:

• Scale in or out a tenant by adjusting the unit number

• Scale in or out a tenant by adjusting the primary zone