Modify locality

This topic describes how to modify the locality of a tenant.

Application scenarios

You can modify the locality of a tenant to increase or decrease the number of replicas for the tenant or adjust the distribution of replicas.

• Increase the number of replicas for a tenant

  You can increase the number of replicas for a tenant in the cluster. Only one replica can be added at a time. For example, if you want to increase the number of replicas for a tenant from 1 to 3, you must change the locality first from F@z1 to F@z1,F@z2, and then from F@z1,F@z2 to F@z1,F@z2,F@z3. For more information, see Add replicas.

  Notice

  You can directly increase the number of replicas for a tenant from 3 to 5, without the need to first change to 4 replicas and then to 5 replicas.

• Reduce the number of replicas for a tenant

  You can reduce the number of replicas for a tenant within the cluster, for example, reduce the number of replicas from 3 to 2, or from 5 to 3. For more information, see Remove replicas.

  Notice

  • The number of replicas for a tenant cannot be reduced from 2 to 1.
  • When reducing the number of replicas from 5 to 3, the operation can be performed directly without the need to first change it to 4 replicas and then to 3 replicas.

• Adjust the distribution of replicas for a tenant

  You can modify the locality of a tenant in the cluster several times to adjust the distribution of replicas across zones. For example, the original locality of a tenant is F@z1,F@z2,F@z3. Assuming that zone z3 expires, the replica must be migrated to z4, and the locality must be changed to F@z1,F@z2,F@z4. Specifically, the locality must be changed first from F@z1,F@z2,F@z3 to F@z1,F@z2,F@z3,F@z4, and then from F@z1,F@z2,F@z3,F@z4 to F@z1,F@z2,F@z4. For more information, see Adjust the distribution of replicas.

Prerequisites

Before you modify the locality of a tenant, verify the resource information of each node in the target zone. If OBServer nodes in the target zone do not have sufficient resources to accommodate the new resource units of the tenant, the locality cannot be changed.

For more information about how to view the resource information of each node in a zone, see View tenant and resource information.

Considerations

• The locality of a tenant must be specified.

• Modifying the locality of a tenant also changes the distribution of table replicas for the tenant.

• If a tenant locality modification is not finished, the next tenant locality modification cannot be performed.

• You can perform only one action at a time when you modify the locality. For example, if you want to replace zone3 with zone4, you must add zone4 first and then remove zone3.

• We recommend that you modify the primary zone before you modify the locality.

  Here are some examples:

  1. Assume that zone3 is a primary zone with the highest priority. When you remove zone3, read and write services of the tenant are affected during locality modification.

  2. Assume that the primary zone is set to RANDOM when you add zone4. After zone4 is added, it also provides read and write services.

• Modifying a tenant's locality involves changing Paxos members and copying data. You must evaluate the impact on system stability and performance. Regarding changes of Paxos members, evaluate the impact on the tolerance to OBServer node failures. Regarding changes of replica distribution, evaluate the impact on the service level agreement (SLA) design for IDC and region-wide failures in the deployment architecture. Regarding data copying, observe the impact on other tenants on the same OBServer node. We recommend that you copy data during off-peak hours and prepare a contingency plan in advance.

• Locality modification must not reduce the expected tolerance to OBServer node failures. For example, if you want to replace an IDC in a cluster with three replicas, you must add a replica and then remove a replica to avoid reducing the expected tolerance to OBServer node failures during locality modification. Theoretically, a single-replica failure during locality modification does not affect service continuity. If service continuity is affected, immediately recover the availability of the database. We recommend that you try to rectify hardware faults first.

Procedure

The following example describes how to modify the locality of the mq_t1 tenant from FULL{1}@sa128_obv4_1,FULL{1}@sa128_obv4_2 to F{1}@sa128_obv4_1,F{1}@sa128_obv4_2,F{1}@sa128_obv4_3.

1. Log on to the sys tenant of the cluster as the root user.

   obclient -h172.30.xxx.xxx -P2883 -uroot@sys#obdemo -pxxxx -A
   

2. Access the database named oceanbase.

   obclient>use oceanbase;
   

3. View the locality.

   obclient> SELECT TENANT_ID,TENANT_NAME,TENANT_TYPE,PRIMARY_ZONE,LOCALITY FROM oceanbase.DBA_OB_TENANTS;
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+
   | TENANT_ID | TENANT_NAME | TENANT_TYPE | PRIMARY_ZONE                           | LOCALITY                                                         |
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+
   |         1 | sys         | SYS         | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 |
   |      1001 | META$1002   | META        | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 |
   |      1002 | midas0_3790 | USER        | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 |
   |      1003 | META$1004   | META        | sa128_obv4_1,sa128_obv4_2              | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2                       |
   |      1004 | mq_t1       | USER        | sa128_obv4_1,sa128_obv4_2              | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2                       |
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+
   

4. Modify the locality of the mq_t1 tenant.

   obclient>ALTER TENANT mq_t1 locality="F{1}@sa128_obv4_1,F{1}@sa128_obv4_2,F{1}@sa128_obv4_3";
   

5. View the execution status of the locality change job.

    obclient>SELECT * FROM oceanbase.DBA_OB_TENANT_JOBS WHERE JOB_TYPE = 'ALTER_TENANT_LOCALITY';
    +--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+
    | JOB_ID | JOB_TYPE              | JOB_STATUS | RESULT_CODE | PROGRESS | START_TIME                 | MODIFY_TIME                | TENANT_ID | SQL_TEXT                                                                     | EXTRA_INFO                                  | RS_SVR_IP      | RS_SVR_PORT |
    +--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+
    |      1 | ALTER_TENANT_LOCALITY | SUCCESS    |           0 |      100 | 2023-01-05 19:33:46.730319 | 2023-01-05 19:33:46.920111 |      1002 | ALTER TENANT mq_t1 locality="F{1}@sa128_obv4_1,F{1}@sa128_obv4_2,F{1}@sa128_obv4_3"                | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2 | xx.xx.xx.xx |        2882 |
    +--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+
   

   If the value of JOB_STATUS is SUCCESS, the locality change job is successful.

6. View the modified locality.

   obclient>SELECT TENANT_ID,TENANT_NAME,TENANT_TYPE,PRIMARY_ZONE,LOCALITY FROM oceanbase.DBA_OB_TENANTS;
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+--------------------+
   | TENANT_ID | TENANT_NAME | TENANT_TYPE | PRIMARY_ZONE                           | LOCALITY                                                         | PREVIOUS_LOCALITY  |
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+--------------------+
   |         1 | sys         | SYS         | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 | null               |
   |      1001 | META$1002   | META        | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 | null               |
   |      1002 | midas0_3790 | USER        | sa128_obv4_2;sa128_obv4_1,sa128_obv4_3 | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 | null               |
   |      1003 | META$1004   | META        | sa128_obv4_1,sa128_obv4_2              | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 | null               |
   |      1004 | mq_t1       | USER        | sa128_obv4_1,sa128_obv4_2              | FULL{1}@sa128_obv4_1, FULL{1}@sa128_obv4_2, FULL{1}@sa128_obv4_3 | null               |
   +-----------+-------------+-------------+----------------------------------------+------------------------------------------------------------------+--------------------+
   

   The query results show that the locality of the mq_t1 tenant is changed to F{1}@sa128_obv4_1,F{1}@sa128_obv4_2,F{1}@sa128_obv4_3. If the PREVIOUS_LOCALITY column is empty, the change is successful. If the PREVIOUS_LOCALITY column is not empty, the change is in progress.

References

• Locality

• View locality