Configure cgroup

This topic describes how to configure cgroup for OceanBase Database.

Background information

In the current version of OceanBase Database, tenant worker threads and most background threads are tenant-specific, while network threads are shared. You can configure cgroup to control resources such as CPU or IOPS for tenants. Before you configure cgroup, it is recommended that you understand the concept of cgroup. For more information, see Resource isolation overview.

Limitations and considerations

• Since enabling resource isolation has a significant impact on performance, we recommend that you do not use the cgroup feature for tenant resource isolation in the following scenarios:

  • Single-tenant scenarios, where there is only one tenant in the cluster.

  • Scenarios where tenants have business dependencies. For example, different microservices are placed in different tenants, creating upstream/downstream relationships between them.

  • Small-specification tenant scenarios, where a tenant has 2 or 4 CPU cores.

• To use the cgroup feature, the Linux kernel version must be Linux Kernel 4.19 or later.

• After upgrading OceanBase Database, you need to reconfigure cgroup.

• If an OBServer host restarts, the created cgroup subdirectories (for example, all subdirectories under the /sys/fs/cgroup/ directory) will be cleared, and you need to reconfigure cgroup.

Procedure

Step 1: Configure the cgroup system directory

This section uses configuring the cgroup system directory on one OBServer node as an example. If there are multiple OBServer nodes, configuration is required on each.

1. Log in to the OBServer host as the admin user.

2. Run the following command to mount the /sys/fs/cgroup directory.

   Note

   If the /sys/fs/cgroup directory already exists, skip this step.

   [admin@xxx /]$ sudo mount -t tmpfs cgroups /sys/fs/cgroup
   

   Here, cgroups is a custom name used to identify the mount when viewing mount information.

   The mount result is as follows.

   $df
   Filesystem      1K-blocks       Used Available Use% Mounted on
   /               293601280   28055472 265545808  10% /
   /dev/v01d      2348810240 2113955876 234854364  91% /data/1
   /dev/v02d      1300234240 1170211208 130023032  91% /data/log1
   shm              33554432          0  33554432   0% /dev/shm
   /dev/v04d       293601280   28055472 265545808  10% /home/admin/logs
   cgroups         395752136          0 395752136   0% /sys/fs/cgroup
   

3. Create the /sys/fs/cgroup/cpu directory and change its owner, then mount the cpu subsystem.

   Note

   If the /sys/fs/cgroup/cpu directory already exists and is empty, skip this step.

   [admin@xxx /]$ sudo mkdir /sys/fs/cgroup/cpu
   
   [admin@xxx /]$ sudo chown admin:admin -R /sys/fs/cgroup/cpu
   

   Create a hierarchy named cpu, attach the cpu subsystem to this hierarchy, and mount it to the /sys/fs/cgroup/cpu directory.

   [admin@xxx /]$ sudo mount -t cgroup -o cpu cpu /sys/fs/cgroup/cpu
   

4. Create a subdirectory named oceanbase and change its owner to admin.

   [admin@xxx /]$ sudo mkdir /sys/fs/cgroup/cpu/oceanbase
   
   [admin@xxx /]$ sudo chown admin:admin -R /sys/fs/cgroup/cpu/oceanbase
   

5. Allocate CPU and memory resources for the oceanbase directory.

   1. Run the following command to check the mount status of the cpu, cpuacct, and cpuset subsystems on the machine.

      [admin@xxx /]$ ll /sys/fs/cgroup 
      

   2. Based on the subsystems' mount status, choose the appropriate operation:

      • The cpuset subsystem is mounted together with the cpu and cpuacct subsystems.

        In this scenario, the three subsystems are typically mounted under the same directory. For example, the mount result is as follows.

        drwxr-xr-x 3 root root 0 Jul 24 2020 blkio
        lrwxrwxrwx 1 root root 33 Jul 24 2020 cpu -> /sys/fs/cgroup/cpuset,cpu,cpuacct
        lrwxrwxrwx 1 root root 33 Jul 24 2020 cpuacct -> /sys/fs/cgroup/cpuset,cpu,cpuacct
        lrwxrwxrwx 1 root root 33 Jul 24 2020 cpuset -> /sys/fs/cgroup/cpuset,cpu,cpuacct
        drwxr-xr-x 4 root root 0  Jul 24 2020 cpuset,cpu,cpuacct   
        

        For this scenario, run the following command to allocate CPU and memory resources for the oceanbase directory.

        [admin@xxx /]$ sudo sh -c "echo `cat /sys/fs/cgroup/cpu/cpuset.cpus` > /sys/fs/cgroup/cpu/oceanbase/cpuset.cpus"
        
        [admin@xxx /]$ sudo sh -c "echo `cat /sys/fs/cgroup/cpu/cpuset.mems` > /sys/fs/cgroup/cpu/oceanbase/cpuset.mems"
        

      • The cpuset subsystem is mounted separately from the cpu and cpuacct subsystems.

        In this scenario, the cpuset subsystem is mounted separately from the cpu and cpuacct subsystems (common on ECS instances). The mount result is as follows.

        drwxr-xr-x 2 root root 40 Feb 27 15:27 blkio
        lrwxrwxrwx 1 root root 11 Feb 27 15:27 cpu -> cpu,cpuacct
        lrwxrwxrwx 1 root root 11 Feb 27 15:27 cpuacct -> cpu,cpuacct
        drwxr-xr-x 2 root root 40 Feb 27 15:27 cpu,cpuacct
        drwxr-xr-x 2 root root 40 Feb 27 15:27 cpuset
        

        For this scenario, no additional operations are required. Proceed directly to the next step.

6. Run the following command to set inheritance properties for subdirectories in the oceanbase directory.

   [admin@xxx /]$ sudo sh -c "echo 1 > /sys/fs/cgroup/cpu/oceanbase/cgroup.clone_children"
   

   After the command runs successfully, any cgroup subdirectories created under the oceanbase directory will inherit the properties of the parent directory.

Step 2: Create a soft link between the OceanBase Database installation directory and the cgroup system directory

After configuring the cgroup system directory, you need to create a soft link between the installation directory of OceanBase Database and the cgroup system directory.

1. Log in to the OBServer node as the admin user.

2. Manually create a soft link between the installation directory of OceanBase Database and the cgroup system directory.

   [admin@xxx /home/admin]$ cd /home/admin/oceanbase/
   
   [admin@xxx /home/admin]
   $ ln -sf /sys/fs/cgroup/cpu/oceanbase/ cgroup
   

   Here, /home/admin/oceanbase/ is the installation path of OceanBase Database.

   The execution result is as follows.

   [admin@xxx /home/admin/oceanbase]
   $ll cgroup
   lrwxrwxrwx 1 admin admin 29 Dec  8 11:09 cgroup -> /sys/fs/cgroup/cpu/oceanbase/
   lrwxrwxrwx 1 admin admin 29 Dec  8 11:09 cgroup -> /sys/fs/cgroup/blkio/oceanbase/
   

Step 3: Enable the cgroup feature

OceanBase Database controls whether the cgroup feature is enabled for OBServer through the cluster-level parameter enable_cgroup. Its default value is True, indicating that the cgroup feature is enabled. After creating the soft link and enabling the cgroup feature, the cgroup feature takes effect automatically without restarting the OBServer node.

If the cgroup feature is not enabled, you can refer to the following steps to enable it.

1. Log in to the sys tenant of the cluster as the root user.

2. Run the following command to enable the cgroup feature.

   obclient> ALTER SYSTEM SET enable_cgroup = true;
   

   or:

   obclient> ALTER SYSTEM SET enable_cgroup = 1;
   

What to do next

After you successfully configure the cgroup system directory and enable the cgroup feature, you can control CPU resource usage within a tenant in an emergency by using files such as cpu.cfs_period_us, cpu.cfs_quota_us, and cpu.shares located in the tenant's directory. However, this method is generally not recommended for resource isolation.

It is recommended to use resource groups created by calling the CREATE_CONSUMER_GROUP subprogram in the DBMS_RESOURCE_MANAGER system package through directory files in cgroup for user-level or SQL-level resource isolation. For detailed operations on configuring resource isolation, see Create a resource group and Create a resource isolation plan.

References

Clear cgroup configuration