Remove replicas

This topic describes how to remove log stream replicas from a tenant.

Remove replicas by modifying the locality

You can remove log stream replicas from a tenant in a cluster by modifying the locality of the tenant. For more information about how to modify the locality, see Modify locality.

Prerequisites

Before you modify the locality of a tenant, verify the resource information of each node in the target zone. If OBServer nodes in the target zone do not have sufficient resources to accommodate the new resource units of the tenant, the locality cannot be changed.

For more information about how to view the resource information of each node in a zone, see View tenant and resource information.

Considerations

• The locality of the tenant must be specified.

• When the locality of a tenant changes, the distribution of table replicas in the tenant also changes.

• When a tenant locality modification is not finished, the next tenant locality modification cannot be performed.

• After you remove replicas from a tenant, you can remove resource pools from the tenant if you want to release resources.

  Assume that the z1, z2, and z3 zones are configured for a tenant and you want to remove the z3 zone. The following scenarios exist:

  1. The tenant has an independent resource pool in z3. For example, the tenant has two resource pools: resource_pool1 and resource_pool2. resource_pool2 belongs to z3. After you remove the replica in z3, you must remove resource_pool2 from the resource pool list of the tenant. Then you can drop the resource pool to release the resources. For more information, see Drop a resource pool.

  2. The tenant has no independent resource pool in z3. For example, the resource pool resource_pool1 of the tenant belongs to z2 and z3. After you remove the replica in z3, you must remove z3 from the zone list of resource_pool1 to release the resources. For more information about how to modify the zone list of a resource pool, see Modify attributes of a resource pool.

Procedure

The following example describes how to modify the locality of the mysql001 tenant from FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3 to FULL{1}@zone1, FULL{1}@zone2, and reduce the number of replicas for the tenant.

1. Log in to the sys tenant of the cluster as the root user.

   obclient -h172.30.xxx.xxx -P2883 -uroot@sys#obdemo -pxxxx -A
   

2. Access the database named oceanbase.

   obclient>use oceanbase;
   

3. View the locality of the mysql001 tenant.

   obclient> SELECT TENANT_ID,TENANT_NAME,TENANT_TYPE,PRIMARY_ZONE,LOCALITY FROM oceanbase.DBA_OB_TENANTS;
   +-----------+-------------+-------------+-------------------+---------------------------------------------+
   | TENANT_ID | TENANT_NAME | TENANT_TYPE | PRIMARY_ZONE      | LOCALITY                                    |
   +-----------+-------------+-------------+-------------------+---------------------------------------------+
   |         1 | sys         | SYS         | zone1;zone2;zone3 | FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3 |
   |      1001 | META$1002   | META        | zone1;zone2;zone3 | FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3 |
   |      1002 | mysql001    | USER        | zone1;zone2;zone3 | FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3 |
   +-----------+-------------+-------------+-------------------+---------------------------------------------+
   

   The query result shows that the mysql001 tenant has a full-featured replica in each of zone1, zone2, and zone3.

4. Modify the locality of the mysql001 tenant.

   obclient>ALTER TENANT mysql001 locality="FULL{1}@zone1, FULL{1}@zone2";
   

5. View the execution status of the locality modification job.

    obclient>SELECT * FROM oceanbase.DBA_OB_TENANT_JOBS WHERE JOB_TYPE = 'ALTER_TENANT_LOCALITY';
    +--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+
    | JOB_ID | JOB_TYPE              | JOB_STATUS | RESULT_CODE | PROGRESS | START_TIME                 | MODIFY_TIME                | TENANT_ID | SQL_TEXT                                                                     | EXTRA_INFO                                  | RS_SVR_IP      | RS_SVR_PORT |
    +--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+
    |      1 | ALTER_TENANT_LOCALITY | SUCCESS    |           0 |      100 | 2023-01-05 19:33:46.730319 | 2023-01-05 19:33:46.920111 |      1002 | ALTER TENANT mysql001 locality='FULL{1}@zone1, FULL{1}@zone2'                | FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3 | xx.xx.xx.xx |        2882 |
    +--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+
   

   If the value of JOB_STATUS is SUCCESS, the locality modification job is successful.

6. View the modified locality of the mysql001 tenant.

   obclient> SELECT TENANT_ID,TENANT_NAME,TENANT_TYPE,PRIMARY_ZONE,LOCALITY FROM oceanbase.DBA_OB_TENANTS;
   +-----------+-------------+-------------+-------------------+---------------------------------------------+
   | TENANT_ID | TENANT_NAME | TENANT_TYPE | PRIMARY_ZONE      | LOCALITY                                    |
   +-----------+-------------+-------------+-------------------+---------------------------------------------+
   |         1 | sys         | SYS         | zone1;zone2;zone3 | FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3 |
   |      1001 | META$1002   | META        | zone1;zone2       | FULL{1}@zone1, FULL{1}@zone2                |
   |      1002 | mysql001    | USER        | zone1;zone2       | FULL{1}@zone1, FULL{1}@zone2                |
   +-----------+-------------+-------------+-------------------+---------------------------------------------+
   

   The query results in step 3 and step 6 show that the locality of the mysql001 tenant is changed from FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3 to FULL{1}@zone1, FULL{1}@zone2, and the number of replicas for the mysql001 tenant is reduced.

Manually remove a replica

When you remove replicas from a tenant by modifying the locality of the tenant, the system automatically removes the redundant replicas. However, if the system does not properly remove replicas due to exceptions, you can execute the ALTER SYSTEM REMOVE REPLICA statement to remove the redundant log stream replicas.

Limitations

• You can manage log streams of all tenants in the sys tenant but manage only log streams of the current tenant in a user tenant.

• For the same log stream, multiple disaster recovery tasks (such as replica migration) can be performed in parallel, but other tasks (such as adding a replica, deleting a replica, converting the type of a replica, and modifying the quorum size of a log stream) can be performed only one at a time.

  You can query ongoing replica tasks from the CDB_OB_LS_REPLICA_TASKS view in the sys tenant or from the DBA_OB_LS_REPLICA_TASKS view in a user tenant.

Prerequisites

• You have the ALTER SYSTEM privilege.

• You have the SELECT privilege on the following views:

  • DBA_OB_TENANTS

  • DBA_OB_LS/CDB_OB_LS

  • DBA_OB_LS_LOCATIONS/CDB_OB_LS_LOCATIONS

Procedure

Assume that the locality of a tenant named tenant1 is F@zone1,F@zone3, and log stream 1001 of the tenant1 tenant has a full-featured replica in each of zone1, zone2, and zone3. To remove the replica in zone2, perform the following steps:

1. Log in to the target tenant.

   Here is an example:

   obclient -h172.30.xxx.xxx -P2883 -uroot@tenant1#obdemo -pxxxx -A
   

   For more information about how to connect to a database, see Overview (MySQL mode) or Overview (Oracle mode).

2. Query the tenant ID and locality of the target tenant.

   • sys tenant

     obclient [oceanbase]> SELECT TENANT_NAME,TENANT_ID,LOCALITY FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_NAME='tenant1';
     

   • User tenant

     MySQL mode

     Oracle mode

     Execute the following statement in MySQL mode:

     obclient [oceanbase]> SELECT TENANT_NAME,TENANT_ID,LOCALITY FROM oceanbase.DBA_OB_TENANTS;
     

     Execute the following statement in Oracle mode:

     obclient [SYS]> SELECT TENANT_NAME,TENANT_ID,LOCALITY FROM SYS.DBA_OB_TENANTS;
     

     A sample query result is as follows:

     +-------------+-----------+------------------------------+
     | TENANT_NAME | TENANT_ID | LOCALITY                     |
     +-------------+-----------+------------------------------+
     | tenant1     |      1002 | FULL{1}@zone1, FULL{1}@zone3 |
     +-------------+-----------+------------------------------+
     1 row in set
     

     The query result shows that the tenant ID is 1002.

     For more information about the fields in the DBA_OB_TENANTS view, see DBA_OB_TENANTS.

3. Query all log streams of the target tenant.

   • sys tenant

     obclient [oceanbase]> SELECT * FROM oceanbase.CDB_OB_LS WHERE TENANT_ID=1002;
     

     For more information about the fields in the CDB_OB_LS view, see CDB_OB_LS.

   • User tenant

     MySQL mode

     Oracle mode

     Execute the following statement in MySQL mode:

     obclient [oceanbase]> SELECT * FROM oceanbase.DBA_OB_LS;
     

     Execute the following statement in Oracle mode:

     obclient [SYS]> SELECT * FROM SYS.DBA_OB_LS;
     

     A sample query result is as follows:

     +-------+--------+--------------+---------------+-------------+---------------------+----------+---------------------+---------------------+------+
     | LS_ID | STATUS | PRIMARY_ZONE | UNIT_GROUP_ID | LS_GROUP_ID | CREATE_SCN          | DROP_SCN | SYNC_SCN            | READABLE_SCN        | FLAG |
     +-------+--------+--------------+---------------+-------------+---------------------+----------+---------------------+---------------------+------+
     |     1 | NORMAL | zone1;zone3  |             0 |           0 |                NULL |     NULL | 1712136316868815003 | 1712136316868815003 |      |
     |  1001 | NORMAL | zone1;zone3  |          1001 |        1001 | 1711951923021132001 |     NULL | 1712136316868815003 | 1712136316868815003 |      |
     +-------+--------+--------------+---------------+-------------+---------------------+----------+---------------------+---------------------+------+
     2 rows in set
     

     The query result shows that the tenant1 tenant has log streams 1 and 1001.

     For more information about the fields in the DBA_OB_LS view, see DBA_OB_LS.

4. Query the replica distribution of log stream 1001 based on the obtained log stream information.

   • sys tenant

     obclient [oceanbase]> SELECT * FROM oceanbase.CDB_OB_LS_LOCATIONS WHERE LS_ID=1001 AND TENANT_ID=1002;
     

     For more information about the fields in the CDB_OB_LS_LOCATIONS view, see CDB_OB_LS_LOCATIONS.

   • User tenant

     MySQL mode

     Oracle mode

     Execute the following statement in MySQL mode:

     obclient [oceanbase]> SELECT * FROM oceanbase.DBA_OB_LS_LOCATIONS WHERE LS_ID=1001;
     

     Execute the following statement in Oracle mode:

     obclient [SYS]> SELECT * FROM SYS.DBA_OB_LS_LOCATIONS WHERE LS_ID=1001;
     

     A sample query result is as follows:

     +----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+----------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+
     | CREATE_TIME                | MODIFY_TIME                | LS_ID | SVR_IP         | SVR_PORT | SQL_PORT | ZONE  | ROLE     | MEMBER_LIST                                                                      | PAXOS_REPLICA_NUMBER | REPLICA_TYPE | LEARNER_LIST | REBUILD |
     +----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+----------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+
     | 2024-04-03 17:23:06.381521 | 2024-04-03 17:23:15.934957 |  1001 | 172.xx.xxx.192 |     2882 |     2881 | zone3 | FOLLOWER | NULL                                                                             |                 NULL | FULL         |              | FALSE   |
     | 2024-04-01 14:12:08.588944 | 2024-04-03 17:23:14.904070 |  1001 | 172.xx.xxx.212 |     2882 |     2881 | zone2 | FOLLOWER | NULL                                                                             |                 NULL | FULL         |              | FALSE   |
     | 2024-04-01 14:12:08.589084 | 2024-04-03 17:23:14.080074 |  1001 | 172.xx.xxx.226 |     2882 |     2881 | zone1 | LEADER   | 172.xx.xxx.192:2882:1712136186291857,172.xx.xxx.212:2882:1,172.xx.xxx.226:2882:1 |                    3 | FULL         |              | FALSE   |
     +----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+----------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+
     3 rows in set
     

     The query result shows that log stream 1001 has a full-featured replica on each of OBServer nodes 172.xx.xxx.192:2882, 172.xx.xxx.212:2882, and 172.xx.xxx.226:2882. Based on the locality, you must remove the replica of the log stream from OBServer node 172.xx.xxx.212:2882 in zone2.

     For more information about the fields in the DBA_OB_LS_LOCATIONS view, see DBA_OB_LS_LOCATIONS.

5. Execute the ALTER SYSTEM REMOVE REPLICA statement to remove the replica.

   The syntax is as follows:

   ALTER SYSTEM REMOVE REPLICA LS [=] ls_id SERVER [=] 'svr_ip:svr_port' [PAXOS_REPLICA_NUM [=] paxos_replica_num] [TENANT [=] 'tenant_name'];
   

   The parameters in the syntax are described as follows:

   • ls_id: the ID of the log stream for which you want to remove the replica.

   • svr_ip:svr_port: the IP address and port number of the OBServer node from which you want to remove the replica, for example, 172.xx.xxx.212:2882.

   • paxos_replica_num: the modified number of Paxos replicas of the log stream, which is the number of full-featured replicas specified in the locality of the tenant. A log stream supports a maximum of seven full-featured replicas.

     The value of the paxos_replica_num parameter must meet the following conditions:

     1. After the value of the paxos_replica_num parameter is modified, the available replicas are the majority.

     2. The value of the paxos_replica_num parameter must be greater than or equal to the number of members in the MEMBER_LIST of the leader.

     3. The absolute value of the variation of the paxos_replica_num value is not greater than 1. In this example, the value of the paxos_replica_num parameter is 3, and you can change it only to 2.

     If any one of the preceding conditions is not met, the system will return an error. If this parameter is not explicitly specified, the default value is used.

   • tenant_name: the name of the target tenant. You can specify another tenant in the sys tenant and specify only the current tenant in a user tenant. If this parameter is not explicitly specified, the name of the current tenant is used. You cannot use the all, all_user, or all_meta option in this statement to specify all tenants, all user tenants, or all meta tenants.

   • This statement removes only one replica at a time.

   Here is an example: Remove the replica of log stream 1001 on OBServer node 172.xx.xxx.212:2882 and set the PAXOS_REPLICA_NUM parameter to 2.

   obclient> ALTER SYSTEM REMOVE REPLICA LS = 1001 SERVER = '172.xx.xxx.212:2882' PAXOS_REPLICA_NUM = 2;
   

6. Query information about the log stream again to verify that one replica of the log stream is removed.

   • sys tenant

     obclient [oceanbase]> SELECT * FROM oceanbase.CDB_OB_LS_LOCATIONS WHERE LS_ID=1001 AND TENANT_ID=1002;
     

   • User tenant

     MySQL mode

     Oracle mode

     Execute the following statement in MySQL mode:

     obclient [oceanbase]> SELECT * FROM oceanbase.DBA_OB_LS_LOCATIONS WHERE LS_ID=1001;
     

     Execute the following statement in Oracle mode:

     obclient [SYS]> SELECT * FROM SYS.DBA_OB_LS_LOCATIONS WHERE LS_ID=1001;
     

     A sample query result is as follows:

     +----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+------------------------------------------------------------+----------------------+--------------+--------------+---------+
     | CREATE_TIME                | MODIFY_TIME                | LS_ID | SVR_IP         | SVR_PORT | SQL_PORT | ZONE  | ROLE     | MEMBER_LIST                                                | PAXOS_REPLICA_NUMBER | REPLICA_TYPE | LEARNER_LIST | REBUILD |
     +----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+------------------------------------------------------------+----------------------+--------------+--------------+---------+
     | 2024-04-03 17:23:06.381521 | 2024-04-03 17:32:49.079886 |  1001 | 172.xx.xxx.192 |     2882 |     2881 | zone3 | FOLLOWER | NULL                                                       |                 NULL | FULL         |              | FALSE   |
     | 2024-04-01 14:12:08.589084 | 2024-04-03 17:32:49.124409 |  1001 | 172.xx.xxx.226 |     2882 |     2881 | zone1 | LEADER   | 172.xx.xxx.192:2882:1712136186291857,172.xx.xxx.226:2882:1 |                    2 | FULL         |              | FALSE   |
     +----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+------------------------------------------------------------+----------------------+--------------+--------------+---------+
     2 rows in set
     

     The query result shows that log stream 1001 has a full-featured replica on each of OBServer nodes 172.xx.xxx.192:2882 and 172.xx.xxx.226:2882, and the replica on OBServer node 172.xx.xxx.212:2882 has been removed.

References

• Locality

• Modify locality

• View locality

• Add replicas

• Convert the type of a replica

• Modify the number of Paxos replicas of a log stream

• Migrate replicas