Resource planning for tenant scale-in and scale-out

Before you perform a scale-in or scale-out operation on a tenant, you can plan resources for the tenant based on its resource usage and scaling plan.

Background information

In the current version, OceanBase Database supports both homogeneous zone mode and heterogeneous zone mode for tenants:

• In homogeneous zone mode, all zones under a tenant must have the same UNIT_NUM.

• In heterogeneous zone mode, zones under a tenant can have the same or different UNIT_NUM, but a tenant can have at most two different UNIT_NUM values across all zones.

This topic provides operational guidance using resource planning for tenant scaling in homogeneous zone mode as an example. The procedures for tenants in heterogeneous zone mode are similar.

Scale out resources for a tenant

You can increase the number of resource units or primary zones to scale out resources for a tenant. During scale-out, you can create log streams to implement load balancing, which requires more resources. Therefore, you must check whether the resources in the resource units meet the requirements before the scale-out.

1. Log in to the sys tenant of the cluster as the root user.

   obclient -h172.30.xxx.xxx -P2883 -uroot@sys#obdemo -pxxxx -A
   

2. Access the database named oceanbase.

   obclient(root@sys)[(none)]> USE oceanbase;
   

3. Query the usage of logical resources in each resource unit of the tenant.

   obclient(root@sys)[oceanbase]> SELECT * FROM oceanbase.GV$OB_TENANT_RESOURCE_LIMIT WHERE TENANT_ID=1002;
   

   The query result is as follows:

   +----------------+----------+-----------+-------+---------------+---------------------+-----------------+----------------+-------------+----------------------+
   | SVR_IP         | SVR_PORT | TENANT_ID | ZONE  | RESOURCE_NAME | CURRENT_UTILIZATION | MAX_UTILIZATION | RESERVED_VALUE | LIMIT_VALUE | EFFECTIVE_LIMIT_TYPE |
   +----------------+----------+-----------+-------+---------------+---------------------+-----------------+----------------+-------------+----------------------+
   | 172.xx.xxx.xxx |     2882 |      1002 | zone1 | ls            |                   2 |               2 |              0 |          13 | memory               |
   | 172.xx.xxx.xxx |     2882 |      1002 | zone1 | tablet        |                 668 |             668 |              0 |      100000 | configuration        |
   +----------------+----------+-----------+-------+---------------+---------------------+-----------------+----------------+-------------+----------------------+
   2 rows in set
   

   The first line of the query result shows information about log streams of user tenant 1002 on OBServer node 172.xx.xxx.xxx:2882. CURRENT_UTILIZATION indicates that the tenant currently has two log streams. MAX_UTILIZATION indicates that the maximum number of concurrent log streams for the tenant has been 2 so far. LIMIT_VALUE indicates that at most 13 log streams can be created for the tenant. memory indicates that this limit value is calculated based on the memory size of the tenant.

   The second line of the query result shows information about tablets of user tenant 1002 on OBServer node 172.xx.xxx.xxx:2882. CURRENT_UTILIZATION indicates that the tenant currently has 668 tablets. MAX_UTILIZATION indicates that the maximum number of concurrent tablets for the tenant has been 668 so far. LIMIT_VALUE indicates that at most 100,000 tablets can be created for the tenant. configuration indicates that this limit value is calculated based on the parameter value.

4. Calculate the amount of logical resources required after the scale-out based on the scenario.

   In OceanBase Database, you can calculate the numbers of log streams and tablets for a tenant by using the following formulas:

   • The number of log streams of a user tenant on all OBServer nodes in the stable state is calculated as follows:

     • When the user tenant has broadcast log streams: U * P + 2

     • When the user tenant has no broadcast log stream: U * P + 1

   • Any log stream can transfer partitions to other log streams. A partition transfer between log streams on the same OBServer node does not create log streams, and a partition transfer between log streams across OBServer nodes creates a log stream. If partitions are transferred, the maximum number of temporary log streams supported for a user tenant can be calculated by using the following formula: (U * P) * ((U - 1) * P).

   • If partitions are transferred, the maximum number of log streams supported on a single OBServer node can be calculated by using the following formula: ((U * P) * ((U - 1) * P)) / U + (P + 2).

   • The maximum number of tablets that you can create for a tenant on each OBServer node is calculated as follows:

     • When the maximum number is subject to the parameter value: (MEMORY_SIZE/1GB) * _max_tablet_cnt_per_gb

     • When the maximum number is subject to the memory size of the tenant: (MEMORY_SIZE * _storage_meta_memory_limit_percentage) / 200MB * 20000

   In the above formula, U represents the tenant's UNIT_NUM value in homogeneous zone mode, or the least common multiple of the two different UNIT_NUM values in heterogeneous zone mode; P represents the number of primary zones with the highest priority in the tenant's PRIMARY_ZONE setting. For detailed explanations of other parameters in the above formula, see View tenant and resource information.

   Based on the above formula, suppose tenant 1002 has 1 PRIMARY_ZONE with the first priority and 2 UNIT_NUM, with broadcast log streams. If the number of first-priority PRIMARY_ZONE needs to be changed to 3, then the number of log streams that reach a stable state on all nodes of the tenant will be 8 (2*3 + 2). If these log streams undergo a transfer, the maximum number of log streams that may exist simultaneously on a single node, MAX_UTILIZATION, could need to reach 14 (((2 * 3) * (2-1) * 3) / 2 + 3 + 2). Therefore, the tenant needs to be expanded to accommodate 14 log streams.

5. Determine the physical resources or parameters to be modified based on the limits on the logical resources.

   The amount of logical resources that you can create for a tenant on each OBServer node is limited by the amount of physical resources or the values of specific parameters. You can query the GV$OB_TENANT_RESOURCE_LIMIT_DETAIL view for the limits on the logical resources of the tenant and the maximum value of each limit.

   obclient(root@sys)[oceanbase]> SELECT * FROM oceanbase.GV$OB_TENANT_RESOURCE_LIMIT_DETAIL WHERE TENANT_ID=1002;
   

   The query result is as follows:

   +----------------+----------+-----------+---------------+---------------+---------------------+
   | SVR_IP         | SVR_PORT | TENANT_ID | RESOURCE_NAME | LIMIT_TYPE    | LIMIT_VALUE         |
   +----------------+----------+-----------+---------------+---------------+---------------------+
   | 172.xx.xxx.xxx |     2882 |      1002 | ls            | configuration |                  90 |
   | 172.xx.xxx.xxx |     2882 |      1002 | ls            | memstore      | 9223372036854775807 |
   | 172.xx.xxx.xxx |     2882 |      1002 | ls            | memory        |                  13 |
   | 172.xx.xxx.xxx |     2882 |      1002 | ls            | data_disk     | 9223372036854775807 |
   | 172.xx.xxx.xxx |     2882 |      1002 | ls            | clog_disk     |                  32 |
   | 172.xx.xxx.xxx |     2882 |      1002 | ls            | cpu           | 9223372036854775807 |
   | 172.xx.xxx.xxx |     2882 |      1002 | tablet        | configuration |              100000 |
   | 172.xx.xxx.xxx |     2882 |      1002 | tablet        | memstore      | 9223372036854775807 |
   | 172.xx.xxx.xxx |     2882 |      1002 | tablet        | memory        |              102400 |
   | 172.xx.xxx.xxx |     2882 |      1002 | tablet        | data_disk     | 9223372036854775807 |
   | 172.xx.xxx.xxx |     2882 |      1002 | tablet        | clog_disk     | 9223372036854775807 |
   | 172.xx.xxx.xxx |     2882 |      1002 | tablet        | cpu           | 9223372036854775807 |
   +----------------+----------+-----------+---------------+---------------+---------------------+
   12 rows in set
   

   The query result shows that the maximum number of log streams that you can create for tenant 1002 on OBServer node 172.xx.xxx.xxx:2882 is subject to the following limits: If the limit type is configuration, you can create at most 90 log streams. If the limit type is memory, you can create at most 13 log streams. If the limit type is clog_disk, you can create at most 32 log streams. The maximum number of tablets that you can create is subject to the following limits: If the limit type is configuration, you can create at most 100,000 tablets. If the limit type is memory, you can create at most 102,400 tablets.

   You can determine the physical resources to be modified based on the preceding limits. For example, in the scenario above, if you want a single OBServer node to support up to 14 log streams, you need to adjust the tenant memory (memory) and log disk (clog_disk).

6. Calculate the amount of physical resources required by a single OBServer node based on the amount of logical resources required after the scale-out.

   Assume that a single OBServer node needs to support up to 14 temporary log streams and up to 40,000 tablets after the scale-out. A sample statement for calculating the physical resources required is as follows:

   obclient(root@sys)[oceanbase]> CALL DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_LOGIC_RES("LS:14, TABLET:40000");
   

   The calculation result is as follows:

   +------------------------+------------+
   | PHYSICAL_RESOURCE_NAME | MIN_VALUE  |
   +------------------------+------------+
   | memstore               |          0 |
   | memory                 | 5553258496 |
   | data_disk              |          0 |
   | clog_disk              | 7516192768 |
   | cpu                    |          0 |
   +------------------------+------------+
   5 rows in set
   

7. Based on the calculation result, refer to Modify the unit config for a tenant to increase the tenant's resource specifications.

Scale in resources for a tenant

If you want to scale in resources for a tenant, you can estimate the amount of logical resources required for business operations the same way as in the preceding scale-out scenario. Then you can call the CALCULATE_MIN_PHY_RES_NEEDED_BY_LOGIC_RES subprogram in the DBMS_OB_LIMIT_CALCULATOR package to calculate the amount of physical resources required for the logical resources and downgrade the unit config for the tenant based on your business needs.

For tenants in homogeneous zone mode, when scaling in by adjusting the unit number, if the tenant has multiple unit groups (in the DBA_OB_UNITS view, units with the same UNIT_GROUP_ID across different Zones belong to the same unit group) and each unit group is relatively idle, and you are unsure which unit group to delete, you can use the following method.

Similarly, for tenants in heterogeneous zone mode, when scaling in by adjusting the unit number of a single resource pool, if each unit is relatively idle and you are unsure which UNIT_LIST to delete, you can also use the following method.

1. Log in to the sys tenant of the cluster as the root user.

   obclient -h172.30.xxx.xxx -P2883 -uroot@sys#obdemo -pxxxx -A
   

2. Access the database named oceanbase.

   obclient(root@sys)[(none)]> USE oceanbase;
   

3. Query the resource units of the tenant to be scaled in.

   obclient(root@sys)[oceanbase]> SELECT UNIT_ID, TENANT_ID, STATUS, RESOURCE_POOL_ID, UNIT_GROUP_ID, ZONE, SVR_IP, SVR_PORT FROM oceanbase.DBA_OB_UNITS WHERE TENANT_ID = 1002; 
   

   The query result is as follows:

   +---------+-----------+--------+------------------+---------------+-------+----------------+----------+
   | UNIT_ID | TENANT_ID | STATUS | RESOURCE_POOL_ID | UNIT_GROUP_ID | ZONE  | SVR_IP         | SVR_PORT |
   +---------+-----------+--------+------------------+---------------+-------+----------------+----------+
   |    1016 |      1002 | ACTIVE |             1003 |          1006 | zone1 | xxx.xx.xxx.194 |     2882 |
   |    1017 |      1002 | ACTIVE |             1003 |          1007 | zone1 | xxx.xx.xxx.198 |     2882 |
   |    1018 |      1002 | ACTIVE |             1003 |          1006 | zone2 | xxx.xx.xxx.192 |     2882 |
   |    1019 |      1002 | ACTIVE |             1003 |          1007 | zone2 | xxx.xx.xxx.196 |     2882 |
   |    1020 |      1002 | ACTIVE |             1003 |          1006 | zone3 | xxx.xx.xxx.204 |     2882 |
   |    1021 |      1002 | ACTIVE |             1003 |          1007 | zone3 | xxx.xx.xxx.197 |     2882 |
   +---------+-----------+--------+------------------+---------------+-------+----------------+----------+
   6 rows in set
   

   According to the query result, the tenant to be scaled in has two units in each zone, with UNIT_ID values 1016, 1017, 1018, 1019, 1020, and 1021.

4. Execute the following command to calculate the minimum amount of physical resources required for each unit of the tenant.

   You need to obtain the minimum physical resources required for each unit separately. For example, in this example, the unit with UNIT_ID 1018 corresponds to OBServer node xxx.xx.xx.192:2882.

   A sample statement for calculating the minimum amount of physical resources required for user tenant 1002 on OBServer node xxx.xx.xx.192:2882 is as follows:

   obclient [oceanbase]> CALL DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_UNIT(TENANT_ID => 1002, SERVER => "xxx.xx.xx.192:2882");
   

   The execution result is as follows:

   +----------------+----------+-----------+------------------------+------------+
   | SVR_IP         | SVR_PORT | TENANT_ID | PHYSICAL_RESOURCE_NAME | MIN_VALUE  |
   +----------------+----------+-----------+------------------------+------------+
   | xxx.xx.xxx.192 |     2882 |      1002 | memstore               |          0 |
   | xxx.xx.xxx.192 |     2882 |      1002 | memory                 | 4294967296 |
   | xxx.xx.xxx.192 |     2882 |      1002 | data_disk              |          0 |
   | xxx.xx.xxx.192 |     2882 |      1002 | clog_disk              | 1073741824 |
   | xxx.xx.xxx.192 |     2882 |      1002 | cpu                    |          0 |
   +----------------+----------+-----------+------------------------+------------+
   5 rows in set
   

   After you obtain the minimum physical resources required for all units, for tenants in homogeneous zone mode, select the unit group that requires relatively fewer resources for deletion; for tenants in heterogeneous zone mode, select the UNIT_LIST in the resource pool that requires relatively fewer resources for deletion.

References

• View tenant and resource information

• Modify the unit config for a tenant

• Modify the number of resource units for a tenant

• Modify the number of primary zones for a tenant