Configure resource isolation within a tenant

This topic describes how to configure resource isolation in Oracle mode.

Prerequisites

• Before you configure resource isolation, we recommend that you learn about basic concepts such as resource group, resource management plan, and plan directive. For more information, see Overview.

• If you need to implement isolation of CPU resources, make sure that the control group (cgroup) directory is configured and the cgroup feature is enabled. This is because the isolation of CPU resources depends on cgroups. For more information, see Configure cgroups.

  If you need to implement only isolation of input/output operations per second (IOPS) resources, you do not need to configure cgroups.

• Before you perform IOPS resource isolation, you must calibrate the disk performance. For more information about how to perform disk performance calibration, see Calibrate the disk performance.

  If you need to only perform CPU resource isolation, you do not need to calibrate the disk performance.

• The user for which resource isolation is to be implemented has been created. For more information, see Create a user.

• If you need to configure SQL statement-level resource isolation, make sure that the database, table, and column for which resource isolation is to be implemented have been created.

Background information

OceanBase Database supports user-level resource isolation, SQL statement-level resource isolation, and function-level resource isolation. For more information, see Overview.

Procedure

(Optional) Step 1: Specify valid values for the MAX_IOPS and MIN_IOPS parameters of the tenant

Before you configure a resource isolation plan after disk calibration, you must make sure that the MAX_IOPS and MIN_IOPS parameters in the unit config of the tenant are set to valid values that are not greater than the baseline IOPS value for 16 KB reads.

1. Log on to the sys tenant of the cluster as the root user.

2. Execute the following statement to query the unit config of the tenant for which resource isolation is to be performed:

   obclient [oceanbase]> SELECT * FROM oceanbase.DBA_OB_UNIT_CONFIGS;
   

   A sample query result is as follows:

   +----------------+-----------------+----------------------------+----------------------------+---------+---------+-------------+---------------+---------------------+---------------------+-------------+
   | UNIT_CONFIG_ID | NAME            | CREATE_TIME                | MODIFY_TIME                | MAX_CPU | MIN_CPU | MEMORY_SIZE | LOG_DISK_SIZE | MAX_IOPS            | MIN_IOPS            | IOPS_WEIGHT |
   +----------------+-----------------+----------------------------+----------------------------+---------+---------+-------------+---------------+---------------------+---------------------+-------------+
   |              1 | sys_unit_config | 2023-12-19 13:55:04.463295 | 2023-12-19 13:56:08.969718 |       3 |       3 |  2147483648 |    3221225472 | 9223372036854775807 | 9223372036854775807 |           3 |
   |           1001 | small_unit      | 2023-12-19 13:56:09.851665 | 2023-12-19 13:56:09.851665 |       1 |       1 |  2147483648 |    6442450944 | 9223372036854775807 | 9223372036854775807 |           1 |
   |           1002 | medium_unit     | 2023-12-19 13:56:10.030914 | 2023-12-19 13:56:10.030914 |       8 |       4 |  8589934592 |   25769803776 | 9223372036854775807 | 9223372036854775807 |           4 |
   |           1003 | large_unit      | 2023-12-19 13:56:10.112115 | 2023-12-19 13:56:10.112115 |      16 |       8 | 21474836480 |   64424509440 | 9223372036854775807 | 9223372036854775807 |           8 |
   +----------------+-----------------+----------------------------+----------------------------+---------+---------+-------------+---------------+---------------------+---------------------+-------------+
   4 rows in set
   

   If the default value INT64_MAX (which is 9223372036854775807) is used for the MAX_IOPS and MIN_IOPS parameters of the tenant, you must replan the IOPS resources for the tenant.

3. Execute the following statement to query the OBServer nodes on which the tenant is deployed:

   obclient [oceanbase]> SELECT DISTINCT SVR_IP, SVR_PORT FROM oceanbase.CDB_OB_LS_LOCATIONS WHERE tenant_id = xxxx;
   

   A sample query result is as follows:

   +----------------+----------+
   | SVR_IP         | SVR_PORT |
   +----------------+----------+
   | xx.xxx.xxx.xx1 |    xxxx1 |
   | xx.xxx.xxx.xx1 |    xxxx2 |
   | xx.xxx.xxx.xx1 |    xxxx3 |
   +----------------+----------+
   3 rows in set
   

4. Execute the following statement to query the baseline IOPS value of the disk on each OBServer node where the tenant is deployed. If the queried baseline IOPS value is less the baseline IOPS value for 16 KB reads, set the queried baseline IOPS value as the upper IOPS limit of the node; otherwise, set the baseline IOPS value for 16 KB reads as the upper IOPS limit of the node.

   obclient [oceanbase]> SELECT * FROM oceanbase.GV$OB_IO_BENCHMARK WHERE MODE='READ' AND SIZE=16384;
   

   A sample query result is as follows:

   +----------------+----------+--------------+------+-------+-------+------+---------+
   | SVR_IP         | SVR_PORT | STORAGE_NAME | MODE | SIZE  | IOPS  | MBPS | LATENCY |
   +----------------+----------+--------------+------+-------+-------+------+---------+
   | xx.xxx.xxx.xx1 |    xxxx1 | DATA         | READ | 16384 | 48162 |  752 |     331 |
   | xx.xxx.xxx.xx1 |    xxxx2 | DATA         | READ | 16384 | 47485 |  741 |     336 |
   | xx.xxx.xxx.xx1 |    xxxx3 | DATA         | READ | 16384 | 48235 |  753 |     331 |
   +----------------+----------+--------------+------+-------+-------+------+---------+
   3 rows in set
   

   Plan the IOPS resources available for the tenant by using the queried disk calibration value of each node as the upper IOPS limit. Multiple tenants in a cluster may be deployed on the same OBServer nodes. You can allocate the IOPS resources based on the actual situation.

   Assume that a cluster has two tenants deployed on the same OBServer nodes, the baseline IOPS value for 16 KB reads is 20000 on each OBServer node, and the loads of the two tenants are similar. You can evenly distribute the IOPS resources to the two tenants based on the actual situation. Specifically, you can set the MAX_IOPS and MIN_IOPS parameters to 10000 for both tenants. You can also set MIN_IOPS to a value smaller than that of MAX_IOPS based on business needs.

5. Execute the following statements to modify the values of MAX_IOPS and MIN_IOPS.

   We recommend that you modify the value of MIN_IOPS first and then that of MAX_IOPS.

   ALTER RESOURCE UNIT unit_name MIN_IOPS = xxx;
   

   ALTER RESOURCE UNIT unit_name MAX_IOPS = xxx;
   

Step 2: Configure a resource isolation plan

Assume that the current tenant contains two users: tp_user and ap_user.

You can configure a resource isolation plan to control the CPU and IOPS resources available for different users or backend tasks.

1. Log on to an Oracle tenant in the cluster as the administrator of the tenant.

2. Call the CREATE_CONSUMER_GROUP subprogram in the DBMS_RESOURCE_MANAGER package to create two resource groups.

   Here is a sample statement:

   BEGIN DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP(
   consumer_group => 'group_name' ,
   COMMENT => 'coments'
   );
   END;
   

   where:

   • CONSUMER_GROUP indicates the name of the resource group.

   • COMMENT indicates the comments on the resource group.

   For example, create two resource groups respectively named big_group and small_group.

   obclient [SYS]> delimiter //
   

   obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP(
   consumer_group => 'big_group' ,
   COMMENT => 'TP'
   );
   END; //
   

   obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP(
   consumer_group => 'small_group' ,
   COMMENT => 'AP'
   );
   END; //
   

   You can query the DBA_RSRC_CONSUMER_GROUPS view to verify whether the resource groups are created. For more information about the DBA_RSRC_CONSUMER_GROUPS view, see DBA_RSRC_CONSUMER_GROUPS.

3. Call the CREATE_PLAN subprogram in the DBMS_RESOURCE_MANAGER package to create a resource management plan.

   Here is a sample statement:

   BEGIN DBMS_RESOURCE_MANAGER.CREATE_PLAN(
   PLAN => 'plan_name',
   COMMENT =>'coments'
   );
   END;
   

   where:

   • PLAN indicates the name of the resource management plan.

   • COMMENT indicates the comments on the resource management plan.

   For example, create a resource management plan named plan_a.

   obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.CREATE_PLAN(
   PLAN => 'plan_a');
   END; //
   

   You can query the DBA_RSRC_PLANS view to verify whether the resource management plan is created. For more information about the DBA_RSRC_PLANS view, see DBA_RSRC_PLANS.

4. Call the CREATE_PLAN_DIRECTIVE subprogram in the DBMS_RESOURCE_MANAGER package to create a plan directive for the resource management plan. When the resource management plan is enabled, the plan directive limits the CPU and IOPS resources available for the resource group.

   Here is a sample statement:

   BEGIN DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE(
   PLAN => 'plan_name',
   GROUP_OR_SUBPLAN => 'group_name',
   COMMENT  => 'comments',
   MGMT_P1 => int_value,
   UTILIZATION_LIMIT => int_value,
   MIN_IOPS => int_value,
   MAX_IOPS => int_value,
   WEIGHT_IOPS => int_value);
   END;
   

   where:

   • PLAN indicates the name of the resource management plan with which the plan directive is associated.

   • GROUP_OR_SUBPLAN indicates the resource group.

   • COMMENT indicates the comments on the plan directive. Default value: NULL.

   • MGMT_P1 indicates the maximum percentage of CPU resources available when the system runs at full load. The default value is 100.

   • UTILIZATION_LIMIT indicates the upper limit on the CPU resources available for the resource group. The default value is 100. The value range is [0, 100]. The value 100 indicates that all CPU resources of the tenant are available for the resource group. The value 40 indicates that at most 40% of the CPU resources of the tenant are available for the resource group.

   • MIN_IOPS indicates the IOPS resources reserved for the resource group in the case of I/O resource contention. The sum of MIN_IOPS values of all resource groups cannot exceed 100. Default value: 0.

   • MAX_IOPS indicates the maximum IOPS resources available for the resource group. The sum of MAX_IOPS values of all resource groups can exceed 100. Default value: 100.

   • WEIGHT_IOPS indicates the weight for IOPS resources. The sum of WEIGHT_IOPS values of all resource groups can exceed 100. Default value: 0.

   Here are two examples:

   • Create a plan directive as follows: Set the resource plan to plan_a, the resource group to big_group, and the maximum CPU resources available to 60% of the total CPU resources of the tenant. Set the minimum IOPS resources available upon I/O contention to 20% of the total IOPS resources, the maximum IOPS resources available to 100% of the total IOPS resources, and the weight of IOPS resources to 20.

     obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE(
     PLAN => 'plan_a',
     GROUP_OR_SUBPLAN => 'big_group',
     COMMENT  => 'TP preferred',
     UTILIZATION_LIMIT =>60,
     MIN_IOPS => 20,
     MAX_IOPS => 100,
     WEIGHT_IOPS => 20);
     END; //
     

   • Create a plan directive as follows: Set the resource plan to plan_a, the resource group to small_group, and the maximum CPU resources available to 40% of the total CPU resources of the tenant. Set the minimum IOPS resources available upon I/O contention to 10% of the total IOPS resources, the maximum IOPS resources available to 90% of the total IOPS resources, and the weight of IOPS resources to 30.

     obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE(
     PLAN => 'plan_a',
     GROUP_OR_SUBPLAN => 'small_group' ,
     COMMENT  => 'AP preferred',
     UTILIZATION_LIMIT =>40,
     MIN_IOPS => 10,
     MAX_IOPS => 90,
     WEIGHT_IOPS => 30);
     END; //
     

   You can query the DBA_RSRC_PLAN_DIRECTIVES or DBA_OB_RSRC_IO_DIRECTIVES view to verify whether the plan directives are created.

   For more information about the DBA_RSRC_PLAN_DIRECTIVES view, see DBA_RSRC_PLAN_DIRECTIVES.

   For more information about the DBA_OB_RSRC_IO_DIRECTIVES view, see DBA_OB_RSRC_IO_DIRECTIVES.

5. Call the SET_CONSUMER_GROUP_MAPPING subprogram in the DBMS_RESOURCE_MANAGER package to create a matching rule for user-level resource isolation based on the actual scenario.

   Here is a sample statement:

   BEGIN DBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING(
      ATTRIBUTE => 'column | user | function',
      VALUE => 'values',
      CONSUMER_GROUP => 'group_name');
      END;
   

   where:

   • ATTRIBUTE indicates the attribute type. The attribute name is case-insensitive.

     • The value column indicates SQL statement-level resource isolation.

     • The value user indicates user-level resource isolation.

     • The value function indicates function-level resource isolation.

   • VALUE indicates the attribute value.

     • If the attribute type is column, specify the database name, table name, column name, constant value, and username.

       • The database name and username are optional. The default database name is the name of the current database, which is the same as the username. If no username is specified, the settings take effect for all users, including those created later in the current tenant.

       • The table name, column name, and constant value are required, and each of them can have only one value. The constant value must be a number or string.

       • When you specify the table name, column name, and username, the specified table, column, and user must exist.

       In the Oracle mode of OceanBase Database, the database name, table name, column name, and username will be automatically converted to uppercase after a statement is executed. You can enclose them with double quotation marks to prevent them from being converted to uppercase. Here is an example:

       obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING(
       ATTRIBUTE        =>  'column',
       VALUE            => '"test"."t1"."c3" = 3 for "user_big"',
       CONSUMER_GROUP   => 'big_group');
       END; //
       

     • If the attribute type is user, specify the username. At present, you can specify only one username.

     • If the attribute type is function, specify a backend task corresponding to the directed acyclic graph (DAG) thread. Eight types of backend tasks are supported: compaction_high, ha_high, compaction_mid, ha_mid, compaction_low, ha_low, ddl, and ddl_high. At present, you can specify only one task.

   • CONSUMER_GROUP indicates the resource group to bind. When an SQL statement hits the matching rule specified by the VALUE parameter, this statement is bound to the specified resource group for execution. At present, an SQL statement can be bound only to one resource group.

     If no resource group is specified, the built-in resource group other group is bound by default. The resources of the built-in resource group other group are as follows:

     • MIN_IOPS = 100 - SUM(Resources of other resource groups in the tenant)

     • MAX_IOPS = 100

     • WEIGHT_IOPS = 100

   Here are several examples:

   • Create a matching rule for SQL statement-level resource isolation.

     • Specify to bind an SQL statement that is initiated by the tp_user user and that has a WHERE clause containing sys.t.c3 = 3 to the batch_group resource group for execution by using the CPU and IOPS resources available for the resource group.

       Notice

       An SQL statement can be bound to the big_group resource group provided that c3 is parsed into sys.t.c3 but the statement does not necessarily need to contain sys.t.. For example, the statement SELECT * FROM sys.t WHERE c3 = 1; will be bound to the big_group resource group.

       obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING(
       ATTRIBUTE => 'column',
       VALUE => 'sys.t.c3=3 for tp_user',
       CONSUMER_GROUP => 'big_group');
       END; //
       

     • Specify to bind an SQL statement that has a WHERE clause containing t.c3=5 to the small_group resource group for execution by using the CPU and IOPS resources available for the resource group.

       obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING(
       ATTRIBUTE => 'column',
       VALUE => 't.c3=5',
       CONSUMER_GROUP => 'small_group');
       END; //
       

   • Create a matching rule for user-level resource isolation.

     • Specify to bind SQL statements initiated by the tp_user user to the big_group resource group for execution by using the CPU and IOPS resources available for the resource group.

       obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING(
       ATTRIBUTE => 'user',
       VALUE => 'tp_user',
       CONSUMER_GROUP => 'big_group');
       END; //
       

     • Specify to bind SQL statements initiated by the ap_user user to the small_group resource group for execution by using the CPU and IOPS resources available for the resource group.

       obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING(
       ATTRIBUTE => 'user',
       VALUE => 'ap_user',
       CONSUMER_GROUP => 'small_group');
       END; //
       

   • Create a matching rule for function-level resource isolation.

     • Specify to bind ha_high tasks to the big_group resource group for execution by using the CPU and IOPS resources available for the resource group.

       obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING(
       ATTRIBUTE => 'function',
       VALUE => 'ha_high',
       CONSUMER_GROUP => 'big_group');
       END; //
       

     • Specify to bind ddl_high tasks to the small_group resource group for execution by using the CPU and IOPS resources available for the resource group.

       obclient [SYS]> BEGIN DBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING(
       ATTRIBUTE => 'function',
       VALUE => 'ddl_high',
       CONSUMER_GROUP => 'small_group');
       END; //
       

   You can query the DBA_RSRC_GROUP_MAPPINGS view to verify whether the matching rule is created. For more information about DBA_RSRC_GROUP_MAPPINGS, see DBA_RSRC_GROUP_MAPPINGS.

6. Enable a proper resource management plan for a resource group.

   The resources available for a resource group vary based on the resource management plan. Therefore, you must enable a proper resource management plan for a resource group.

   obclient [SYS]>delimiter ;
   

   obclient [SYS]> ALTER SYSTEM SET resource_manager_plan = 'plan_a';
   

   Note

   If resource usage does not need to be limited, you can execute the ALTER SYSTEM SET resource_manager_plan = ''; statement to disable all resource management plans.

Considerations

• After a matching rule for resource isolation is added, if you delete the user and create the user again, this matching rule still applies.

• A matching rule for resource isolation does not take effect immediately after it is added, but is expected to take effect within 10 seconds. The time may vary based on the actual environment.

• SQL statement-level resource isolation has a higher priority than user-level resource isolation and function-level resource isolation.

• After a matching rule for resource isolation is added, it takes effect only in the SELECT, INSERT, UPDATE, and DELETE statements, and does not take effect in data definition language (DDL), data control language (DCL), or procedural language (PL) statements. It can take effect in prepared statements.

Impact on performance

• The system performance will not be affected after user-level resource isolation and function-level resource isolation are enabled. This is because the resource group used for executing an SQL statement is determined before the SQL statement is parsed.

• The impact of SQL statement-level resource isolation on performance is caused by retries. In user-level resource isolation, the resource group used for executing an SQL statement is determined before the SQL statement is parsed. However, in SQL statement-level resource isolation, the resource group used for executing an SQL statement is determined when the SQL statement is parsed or hits the plan cache. If the system detects that the resource group being used is not the determined resource group, the system will perform a retry to use the resources in the resource group determined based on the matching rule to execute this SQL statement.

  The impact of SQL statement-level resource isolation falls into three cases:

  1. If an SQL statement does not hit any matching rule, SQL statement-level resource isolation has no impact on the performance.

  2. If an SQL statement hits a matching rule that specifies to use the big_group resource group, this SQL statement is executed by using resources in the big_group resource group. The next SQL statement is also preferentially executed by using resources in this resource group. When the system detects that the matching rule hit by another SQL statement is bound to a different resource group, it will perform a retry to use resources in the new resource group to execute this statement. To continuously execute a batch of SQL statements that are bound to the same resource group, you can use this strategy so that the system needs to retry only the first SQL statement. This reduces the number of retries and causes slight impacts on the performance.

  3. If the expected resource group of each SQL statement is different from that of the previous statement, the system must retry for each SQL statement. This greatly affects the performance.

References

• Overview

• Calibrate the disk performance