Modify the number of Paxos replicas of a log stream

If the number of Paxos replicas of a log stream is different from that of full-featured replicas specified in the locality of the tenant due to exceptions, you can execute the ALTER SYSTEM MODIFY statement to specify the paxos_replica_num parameter to modify the number of Paxos replicas.

Limitations

• You can modify the number of Paxos replicas of log streams of all tenants in the sys tenant but modify only the number of Paxos replicas of log streams of the current tenant in a user tenant.

• For the same log stream in a tenant, multiple replica migration tasks can be performed in parallel, but other disaster recovery tasks (such as adding replicas, deleting replicas, converting replica types, and modifying the quorum size of the log stream) can be performed only one at a time.

  You can query ongoing replica tasks from the CDB_OB_LS_REPLICA_TASKS view in the sys tenant or from the DBA_OB_LS_REPLICA_TASKS view in a user tenant.

• Currently, only manually decreasing paxos_replica_num is supported; manually increasing paxos_replica_num is not supported.

Prerequisites

• Before modifying the quorum size of a log stream leader, confirm that the current user has the ALTER SYSTEM privilege. Otherwise, you cannot execute the ALTER SYSTEM MODIFY statement.

• Before querying the views, confirm that the current user has the SELECT privilege on the following views. Otherwise, you cannot query the related information:

  • DBA_OB_TENANTS

  • DBA_OB_LS/CDB_OB_LS

  • DBA_OB_LS_LOCATIONS/CDB_OB_LS_LOCATIONS

Procedure

Assume that log stream 1002 of a tenant named tenant1 has four full-featured replicas but the value of the paxos_replica_num parameter is 5. In this case, you can change the value of the paxos_replica_num parameter to 4 to make it consistent with the number of full-featured replicas specified in the locality of the tenant.

1. Log in to the target tenant.

   Here is an example:

   obclient -h172.30.xxx.xxx -P2883 -uroot@tenant1#obdemo -pxxxx -A
   

   For more information about how to connect to a database, see Overview (MySQL-compatible mode) and Overview (Oracle-compatible mode).

2. Query the tenant ID of the target tenant.

   • sys tenant

     obclient(root@sys)[oceanbase]> SELECT TENANT_NAME,TENANT_ID FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_NAME='tenant1';
     

   • User tenant

     MySQL-compatible mode

     Oracle-compatible mode

     Execute the following statement in MySQL-compatible mode:

     obclient(root@tenant1)[oceanbase]> SELECT TENANT_NAME,TENANT_ID FROM oceanbase.DBA_OB_TENANTS;
     

     Execute the following statement in Oracle-compatible mode:

     obclient(SYS@tenant1)[SYS]> SELECT TENANT_NAME,TENANT_ID FROM SYS.DBA_OB_TENANTS;
     

     A sample query result is as follows:

     +-------------+-----------+
     | TENANT_NAME | TENANT_ID |
     +-------------+-----------+
     | tenant1     |      1002 |
     +-------------+-----------+
     1 row in set
     

     According to the query result, the tenant ID is 1002.

     For more information about the fields in the DBA_OB_TENANTS view, see DBA_OB_TENANTS.

3. Query all log streams of the target tenant.

   • sys tenant

     obclient(root@sys)[oceanbase]> SELECT * FROM oceanbase.CDB_OB_LS WHERE TENANT_ID=1002;
     

     For more information about the fields in the CDB_OB_LS view, see CDB_OB_LS.

   • User tenant

     MySQL-compatible mode

     Oracle-compatible mode

     Execute the following statement in MySQL-compatible mode:

     obclient(root@tenant1)[oceanbase]> SELECT * FROM oceanbase.DBA_OB_LS;
     

     Execute the following statement in Oracle-compatible mode:

     obclient(SYS@tenant1)[SYS]> SELECT * FROM SYS.DBA_OB_LS;
     

     A sample query result is as follows:

     +-------+--------+--------------+---------------+-------------+---------------------+----------+---------------------+---------------------+-----------+----------------+
     | LS_ID | STATUS | PRIMARY_ZONE | UNIT_GROUP_ID | LS_GROUP_ID | CREATE_SCN          | DROP_SCN | SYNC_SCN            | READABLE_SCN        | FLAG      | UNIT_LIST      |
     +-------+--------+--------------+---------------+-------------+---------------------+----------+---------------------+---------------------+-----------+----------------+
     |     1 | NORMAL | z1;z2        |             0 |           0 |                NULL |     NULL | 1712455113896017202 | 1712455113896017202 |           |                |
     |  1001 | NORMAL | z1;z2        |             0 |        1001 | 1712125812893680165 |     NULL | 1712455113896017202 | 1712455113896017202 |           | 1002,1004,1006 |
     |  1002 | NORMAL | z1;z2        |             0 |        1002 | 1712125812908098857 |     NULL | 1712455114041323052 | 1712455113540551113 |           | 1001,1003,1005 |
     |  1003 | NORMAL | z1;z2        |             0 |           0 | 1712125828880850585 |     NULL | 1712455113896017202 | 1712455113896017202 | DUPLICATE |                |
     +-------+--------+--------------+---------------+-------------+---------------------+----------+---------------------+---------------------+-----------+----------------+
     4 rows in set
     

     According to the query result, the tenant1 tenant has log stream 1, log stream 1001, log stream 1002, and broadcast log stream 1003.

     For more information about the fields in the DBA_OB_LS view, see DBA_OB_LS.

4. Query the replica distribution of each log stream.

   For example, query the replica distribution and the paxos_replica_num value of log stream 1002.

   • sys tenant

     obclient(root@sys)[oceanbase]> SELECT * FROM oceanbase.CDB_OB_LS_LOCATIONS WHERE LS_ID=1002 AND TENANT_ID=1002;
     

     For more information about the fields in the CDB_OB_LS_LOCATIONS view, see CDB_OB_LS_LOCATIONS.

   • User tenant

     MySQL-compatible mode

     Oracle-compatible mode

     Execute the following statement in MySQL-compatible mode:

     obclient(root@tenant1)[oceanbase]> SELECT * FROM oceanbase.DBA_OB_LS_LOCATIONS WHERE LS_ID=1002;
     

     Execute the following statement in Oracle-compatible mode:

     obclient(SYS@tenant1)[SYS]> SELECT * FROM SYS.DBA_OB_LS_LOCATIONS WHERE LS_ID=1002;
     

     A sample query result is as follows:

     +----------------------------+----------------------------+-------+----------------+----------+----------+------+----------+--------------------------------------------------------------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+
     | CREATE_TIME                | MODIFY_TIME                | LS_ID | SVR_IP         | SVR_PORT | SQL_PORT | ZONE | ROLE     | MEMBER_LIST                                                                                                                          | PAXOS_REPLICA_NUMBER | REPLICA_TYPE | LEARNER_LIST | REBUILD |
     +----------------------------+----------------------------+-------+----------------+----------+----------+------+----------+--------------------------------------------------------------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+
     | 2024-01-31 16:50:06.856652 | 2024-01-31 16:52:54.618093 |  1002 | 100.xx.xxx.012 |     5070 |     5105 | z1   | LEADER   | 100.xx.xxx.012:5070:1,100.xx.xxx.001:5071:1706691024376341,100.xx.xxx.003:5073:1706691171162545,100.xx.xxx.004:5074:1706691096303187 |                    5 | FULL         |              | FALSE   |
     | 2024-01-31 16:50:24.450775 | 2024-01-31 16:52:54.760939 |  1002 | 100.xx.xxx.001 |     5071 |     5106 | z1   | FOLLOWER | NULL                                                                                                                                 |                 NULL | FULL         |              | FALSE   |
     | 2024-01-31 16:52:51.216095 | 2024-01-31 16:52:57.135393 |  1002 | 100.xx.xxx.003 |     5073 |     5108 | z2   | FOLLOWER | NULL                                                                                                                                 |                 NULL | FULL         |              | FALSE   |
     | 2024-01-31 16:51:36.356401 | 2024-01-31 16:52:54.686229 |  1002 | 100.xx.xxx.004 |     5074 |     5109 | z3   | FOLLOWER | NULL                                                                                                                                 |                 NULL | FULL         |              | FALSE   |
     +----------------------------+----------------------------+-------+----------------+----------+----------+------+----------+--------------------------------------------------------------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+
     4 rows in set
     

     According to the query result, log stream 1002 has four full-featured (F) replicas but its PAXOS_REPLICA_NUMBER is 5. You need to change the PAXOS_REPLICA_NUMBER of this log stream to 4.

5. Execute the statement to modify paxos_replica_num.

   The syntax is as follows:

   ALTER SYSTEM MODIFY LS [=] ls_id PAXOS_REPLICA_NUM [=] paxos_replica_num [TENANT [=] 'tenant_name'];
   

   Parameter description:

   • ls_id: the ID of the log stream to be modified.

   • paxos_replica_num: the quorum size of the log stream after the change, i.e., the number of full-featured (F) replicas in the tenant locality.

     When you specify this parameter, note the following:

     • After the change, both paxos_replica_num and the number of members in the MEMBER_LIST of the leader must satisfy the majority principle.

     • The value of paxos_replica_num must be greater than or equal to the number of members in the MEMBER_LIST of the current leader. You can query the CDB_OB_LS_LOCATIONS view in the sys tenant or the DBA_OB_LS_LOCATIONS view in a user tenant to confirm.

     • paxos_replica_num can only be decreased, not increased, and the change cannot exceed 1. For example, in this scenario you can only change it to 4.

     If any of the above conditions is not met, the system returns an error.

   • tenant_name: the tenant to operate on. The sys tenant can specify other tenants; a user tenant can only specify itself. If not specified, the current tenant is used. This statement does not support using all, all_user, or all_meta to specify all tenants, all user tenants, or all meta tenants.

   Example:

   obclient> ALTER SYSTEM MODIFY LS = 1002 paxos_replica_num = 4;
   

6. After the statement is executed successfully, query the replica distribution of the log stream again.

   • sys tenant

     obclient(root@sys)[oceanbase]> SELECT * FROM oceanbase.CDB_OB_LS_LOCATIONS WHERE LS_ID=1002 AND TENANT_ID=1002;
     

   • User tenant

     MySQL-compatible mode

     Oracle-compatible mode

     Execute the following statement in MySQL-compatible mode:

     obclient(root@tenant1)[oceanbase]> SELECT * FROM oceanbase.DBA_OB_LS_LOCATIONS WHERE LS_ID=1002;
     

     Execute the following statement in Oracle-compatible mode:

     obclient(SYS@tenant1)[SYS]> SELECT * FROM SYS.DBA_OB_LS_LOCATIONS WHERE LS_ID=1002;
     

     A sample query result is as follows:

     +----------------------------+----------------------------+-------+----------------+----------+----------+------+----------+--------------------------------------------------------------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+
     | CREATE_TIME                | MODIFY_TIME                | LS_ID | SVR_IP         | SVR_PORT | SQL_PORT | ZONE | ROLE     | MEMBER_LIST                                                                                                                          | PAXOS_REPLICA_NUMBER | REPLICA_TYPE | LEARNER_LIST | REBUILD |
     +----------------------------+----------------------------+-------+----------------+----------+----------+------+----------+--------------------------------------------------------------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+
     | 2024-01-31 16:50:06.856652 | 2024-01-31 17:17:23.032797 |  1002 | 100.xx.xxx.012 |     5070 |     5105 | z1   | LEADER   | 100.xx.xxx.012:5070:1,100.xx.xxx.001:5071:1706691024376341,100.xx.xxx.003:5073:1706691171162545,100.xx.xxx.004:5074:1706691096303187 |                    4 | FULL         |              | FALSE   |
     | 2024-01-31 16:50:24.450775 | 2024-01-31 17:17:22.976381 |  1002 | 100.xx.xxx.001 |     5071 |     5106 | z1   | FOLLOWER | NULL                                                                                                                                 |                 NULL | FULL         |              | FALSE   |
     | 2024-01-31 16:52:51.216095 | 2024-01-31 17:17:23.113112 |  1002 | 100.xx.xxx.003 |     5073 |     5108 | z2   | FOLLOWER | NULL                                                                                                                                 |                 NULL | FULL         |              | FALSE   |
     | 2024-01-31 16:51:36.356401 | 2024-01-31 17:17:23.715470 |  1002 | 100.xx.xxx.004 |     5074 |     5109 | z3   | FOLLOWER | NULL                                                                                                                                 |                 NULL | FULL         |              | FALSE   |
     +----------------------------+----------------------------+-------+----------------+----------+----------+------+----------+--------------------------------------------------------------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+
     4 rows in set
     

     From the query result, the PAXOS_REPLICA_NUMBER of log stream 1002 has been changed to 4.

References

• Add replicas

• Reduce replicas

• Change the replica type

• Migrate replicas