Modify locality

This topic describes how to modify the locality attribute of a tenant.

Application scenarios for locality

Locality is typically modified to increase, decrease, or adjust the distribution of tenant replicas:

• Increase the number of replicas for a tenant

  You can increase the number of replicas for a tenant in the cluster. Only one replica can be added at a time. For example, if you want to increase the number of replicas for a tenant from 1 to 3, you must change the locality first from F@z1 to F@z1,F@z2, and then from F@z1,F@z2 to F@z1,F@z2,F@z3. For more information, see Add replicas.

  Notice

  You can directly increase the number of replicas for a tenant from 3 to 5, without the need to first change to 4 replicas and then to 5 replicas.

• Reduce the number of replicas for a tenant

  You can reduce the number of replicas for a tenant within the cluster, for example, reduce the number of replicas from 3 to 2, or from 5 to 3. For more information, see Reduce replica.

  Notice

  • In the current version, reducing the number of replicas from 2 to 1 is supported.
  • When reducing the number of replicas from 5 to 3, the operation can be performed directly without the need to first change it to 4 replicas and then to 3 replicas.

• Adjust the distribution of replicas for a tenant

  You can modify the locality of a tenant in the cluster several times to adjust the distribution of replicas across zones. For example, the original locality of a tenant is F@z1,F@z2,F@z3. Assuming that zone z3 expires, the replica must be migrated to z4, and the locality must be changed to F@z1,F@z2,F@z4. Specifically, the locality must be changed first from F@z1,F@z2,F@z3 to F@z1,F@z2,F@z3,F@z4, and then from F@z1,F@z2,F@z3,F@z4 to F@z1,F@z2,F@z4. For more information, see Adjust replica distribution.

Prerequisites

Before modifying locality, verify the resource allocation on each node in the zone to be modified. If the nodes in the zone do not have sufficient resources to accommodate the resource units required by the tenant, the locality modification cannot proceed.

For more information about the resource allocation of each node in a zone, see View tenant and resource information.

Considerations

• The locality of a tenant must be specified.

• Modifying the locality of a tenant also changes the distribution of table replicas for the tenant.

• If a tenant locality modification is not finished, the next tenant locality modification cannot be performed.

• You can perform only one action at a time when you modify the locality. For example, if you want to replace zone3 with zone4, you must add zone4 first and then remove zone3.

• Pay attention to the impact on the primary zone when modifying locality. We recommend that you adjust the primary zone in advance.

  For example:

  1. Assume that zone3 is a primary zone with the highest priority. When you remove zone3, read and write services of the tenant are affected during locality modification.

  2. Assume that the primary zone is set to RANDOM when you add zone4. After zone4 is added, it also provides read and write services.

• Modifying a tenant's locality involves changing Paxos members and copying data. You must evaluate the impact on system stability and performance. Regarding changes of Paxos members, evaluate the impact on the tolerance to OBServer node failures. Regarding changes of replica distribution, evaluate the impact on the service level agreement (SLA) design for IDC and region-wide failures in the deployment architecture. Regarding data copying, observe the impact on other tenants on the same OBServer node. We recommend that you copy data during off-peak hours and prepare a contingency plan in advance.

• Locality modification must not reduce the expected tolerance to OBServer node failures. For example, if you want to replace an IDC in a cluster with three replicas, you must add a replica and then remove a replica to avoid reducing the expected tolerance to OBServer node failures during locality modification. Theoretically, a single-replica failure during locality modification does not affect service continuity. If service continuity is affected, immediately recover the availability of the database. We recommend that you try to rectify hardware faults first.

Procedure

The following example describes how to modify the locality of the mq_t1 tenant from FULL{1}@sa128_obv4_1,FULL{1}@sa128_obv4_2 to F{1}@sa128_obv4_1,F{1}@sa128_obv4_2,F{1}@sa128_obv4_3.

1. Log in to the sys tenant of the cluster as the root user.

   obclient -h172.30.xxx.xxx -P2883 -uroot@sys#obdemo -pxxxx -A
   

2. Access the database named oceanbase.

   obclient>use oceanbase;
   

3. View the locality before modification.

   obclient> SELECT TENANT_ID,TENANT_NAME,TENANT_TYPE,PRIMARY_ZONE,LOCALITY FROM oceanbase.DBA_OB_TENANTS;
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+
   | TENANT_ID | TENANT_NAME | TENANT_TYPE | PRIMARY_ZONE                           | LOCALITY                                                         |
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+
   |         1 | sys         | SYS         | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 |
   |      1001 | META$1002   | META        | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 |
   |      1002 | midas0_3790 | USER        | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 |
   |      1003 | META$1004   | META        | sa128_obv4_1,sa128_obv4_2              | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2                       |
   |      1004 | mq_t1       | USER        | sa128_obv4_1,sa128_obv4_2              | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2                       |
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+
   

4. Modify the locality of the mq_t1 tenant.

   obclient>ALTER TENANT mq_t1 locality="F{1}@sa128_obv4_1,F{1}@sa128_obv4_2,F{1}@sa128_obv4_3";
   

5. View the execution status of the locality change job.

    obclient>SELECT * FROM oceanbase.DBA_OB_TENANT_JOBS WHERE JOB_TYPE = 'ALTER_TENANT_LOCALITY';
    +--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+---------------------------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+
    | JOB_ID | JOB_TYPE              | JOB_STATUS | RESULT_CODE | PROGRESS | START_TIME                 | MODIFY_TIME                | TENANT_ID | SQL_TEXT                                                                                          | EXTRA_INFO                                  | RS_SVR_IP      | RS_SVR_PORT |
    +--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+----------------------------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+
    |      1 | ALTER_TENANT_LOCALITY | SUCCESS    |           0 |      100 | 2023-01-05 19:33:46.730319 | 2023-01-05 19:33:46.920111 |      1002 | ALTER TENANT mq_t1 locality="F{1}@sa128_obv4_1,F{1}@sa128_obv4_2,F{1}@sa128_obv4_3"                | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2 | xx.xx.xx.xx     |        2882 |
    +--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+----------------------------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+
   

   If the value of JOB_STATUS is SUCCESS, the locality change job is successful.

6. View the modified locality.

   obclient>SELECT TENANT_ID,TENANT_NAME,TENANT_TYPE,PRIMARY_ZONE,LOCALITY FROM oceanbase.DBA_OB_TENANTS;
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+--------------------+
   | TENANT_ID | TENANT_NAME | TENANT_TYPE | PRIMARY_ZONE                           | LOCALITY                                                         | PREVIOUS_LOCALITY  |
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+--------------------+
   |         1 | sys         | SYS         | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 | null               |
   |      1001 | META$1002   | META        | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 | null               |
   |      1002 | midas0_3790 | USER        | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 | null               |
   |      1003 | META$1004   | META        | sa128_obv4_1,sa128_obv4_2              | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 | null               |
   |      1004 | mq_t1       | USER        | sa128_obv4_1,sa128_obv4_2              | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 | null               |
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+--------------------+
   

   According to the query result, the locality of mq_t1 has been changed to F{1}@sa128_obv4_1,F{1}@sa128_obv4_2,F{1}@sa128_obv4_3. When PREVIOUS_LOCALITY is empty, the change is successful. When PREVIOUS_LOCALITY is not empty, the change is in progress.

References

• Locality overview

• View locality