Reduce the high availability of OceanBase clusters and tenants

This topic describes how to reduce the high availability of OceanBase clusters and tenants in the OceanBase Cloud Platform (OCP) console.

Scenarios

• Reduce the number of replicas for a tenant and the cluster size. For example, you can switch from the five IDCs across three regions deployment mode to the three IDCs across two regions deployment mode.

• Migrate business to other IDCs. To do so, you need to add a new zone and then delete the old zone.

Prerequisites

• Make sure that you have the following permissions:

  • To delete a zone, make sure that you have the following permissions:
    • Resource Permissions : Cluster Maintenance permission
    • Menu Permissions : Permission on the Overview menu of Clusters
  • To delete a tenant replica from a zone, make sure that you have the following permissions:
    • Resource Permissions : Cluster maintenance or tenant maintenance permission
    • Menu Permissions : Permission on the Overview menu of Tenants

• A tenant and its OceanBase cluster must be in Running state before you delete a replica of the tenant.

• The number of remaining follower and leader replicas of the tenant is greater than half of the total number of follower and leader replicas of the tenant after you delete a replica of the tenant.

• An OceanBase cluster must be in Running state before you delete a zone from the cluster.

• A zone cannot contain any tenant replicas before you delete it.

Considerations

• The arbitration service requires that a tenant must have two or four full-featured replicas. If you have enabled the arbitration service for a tenant with four full-featured replicas, you can only reduce the number of full-featured replicas to 2 for the tenant.
• Before you delete a replica, we recommend that you switch the leader role to another replica by modifying the primary zone of the tenant to ensure that the business is not affected when the replica is deleted.

Procedure

For more information about the procedure, see Delete a zone and Delete a tenant replica from a zone.

A task for deleting a tenant replica is asynchronous. Once triggered, the task is scheduled and gradually executed in the backend by RootService. If a large amount of data is involved, the data deletion takes a long time to complete. We recommend that you periodically check the data deletion status by running the following commands:

# Connect to the oceanbase database of the sys tenant in the OceanBase cluster.
mysql -hxxx.xxx.xxx.xxx -P2881 -uroot@sys -Doceanbase

# Check the task history of RootService. If tenant_balance_finished is displayed, data deletion is completed.
select * from __all_rootservice_event_history order by gmt_create desc limit 10;

# Check server information. The value of start_service_time must be later than that of last_offline_time, and the server status must be active.
select svr_ip, gmt_modified, zone, usec_to_time(start_service_time), usec_to_time(last_offline_time), status from __all_server;

# Check whether data replicas are complete. Replace <number of replicas> with the expected number of replicas. If an empty result is returned, data replicas are complete.
select /*+read_consistency(weak)*/ count(*)c from gv$partition group by table_id, partition_id having c != <number of replicas>;

# Check whether clogs are synchronized. If an empty result is returned, clogs are synchronized.
select svr_ip, svr_port, table_id, partition_idx from __all_virtual_clog_stat where is_in_sync= 0 and is_offline = 0 and replica_type != 16;

On the Resource Management page in the OCP console, you can view the distribution of replicas for a tenant. For more information, see View the unit distribution.

FAQ

What can I do if the following error message is returned when I delete a zone: "Zone {x} contains units, and cannot be deleted."?

Before you delete a zone in OCP, you must delete all replicas in the zone. On the Resource Management page in the OCP console, you can view the distribution of replicas in a zone and delete the replicas.

What can I do if the following error message is returned when I delete a replica for a tenant: "violate locality principal not allowed"?

This error indicates that, after the specified replica is deleted, the remaining follower and leader replicas of the tenant are not in the majority. Therefore, the deletion is denied. You can view the replica distribution of a tenant on the Resource Management page to ensure that the remaining replicas are in the majority.