Delete a node

You can delete a node from an OceanBase cluster. This operation is the reverse of adding a node. It is applicable to scenarios involving elastic scaling and deployment adjustments.

• Elastic scaling scenario: If a cluster has excess resources, you can reduce the resource usage of nodes by adjusting the distribution of Units and reducing the UNIT_NUM of tenants. Finally, you can delete the nodes to decommission the machines.

• Deployment adjustment scenario: When adjusting the deployment architecture of a cluster, if the adjustment of the Locality attribute of a tenant results in no Units of any tenant being distributed in a Zone, you can decommission the Zone by performing a series of operations, such as stopping the Zone, deleting the server, and deleting the Zone.

Considerations

The Delete Server operation involves load balancing, and the resource units (referred to as Units) on the node to be deleted will be migrated to another node within the same zone. This migration process is an automatic load balancing process, primarily controlled by the Root Service. Once the Unit migration is successful, the Delete Server operation can be executed. If you want to specify the target node for the Unit migration, you can manually perform the Unit migration.

During the automatic Unit migration process, the migration task consumes network and I/O resources on both the source and destination nodes. This can lead to an overlap between migration traffic and business traffic, potentially affecting business operations. To avoid this:

• You can first execute the Stop Zone operation before performing the Delete Server operation to ensure that the two types of traffic do not overlap.

  For more information about the Stop Zone operation, see Isolate a node.

• You can also adjust the relevant configuration parameters to control the migration speed.

  The following table lists the configuration parameters related to migration tasks.

  Parameter Name	Description
  balancer_idle_time	A tenant-level parameter that controls the interval between migration tasks. The default value is 10s, indicating that a migration task will be initiated every 10 seconds. The value range is [10s, +∞).
  sys_bkgd_net_percentage	A cluster-level parameter that controls the percentage of network bandwidth occupied by migration tasks. The default value is 60, and the value range is [0, 100].

  For more information about how to view and modify the values of these parameters, see Modify cluster parameters.

Procedure

1. Log in to the sys tenant of the cluster as the root user.

   The following is a sample statement for connecting to the database. You need to use the actual parameters for the connection.

   obclient -h10.xx.xx.xx -P2883 -uroot@sys#obdemo -p***** -A
   

   For more information about how to connect to the database, see Overview of connecting to a database (MySQL mode) and Overview of connecting to a database (Oracle mode).

2. (Optional) Manually migrate units on the server to be deleted.

   Notice

   Before you execute the Delete Server operation, we recommend that you manually migrate units from the server to be deleted.

   1. View the distribution of the current units and obtain the ID of the unit to be migrated.

      obclient [(none)]> SELECT UNIT_ID,TENANT_ID,STATUS,ZONE,SVR_IP FROM oceanbase.DBA_OB_UNITS;
      

      The query result is as follows:

      +---------+-----------+--------+-------+----------------+
      | UNIT_ID | TENANT_ID | STATUS | ZONE  | SVR_IP         |
      +---------+-----------+--------+-------+----------------+
      |       1 |         1 | ACTIVE | zone1 |   xx.xx.xx.237 |
      |       2 |         1 | ACTIVE | zone2 |   xx.xx.xx.238 |
      |       3 |         1 | ACTIVE | zone3 |   xx.xx.xx.218 |
      |    1001 |      1002 | ACTIVE | zone3 |   xx.xx.xx.218 |
      |    1002 |      1002 | ACTIVE | zone1 |   xx.xx.xx.237 |
      |    1003 |      1002 | ACTIVE | zone2 |   xx.xx.xx.238 |
      |    1010 |      1008 | ACTIVE | zone3 |   xx.xx.xx.218 |
      |    1011 |      1008 | ACTIVE | zone2 |   xx.xx.xx.238 |
      |    1012 |      1008 | ACTIVE | zone1 |   xx.xx.xx.237 |
      |    1013 |      1010 | ACTIVE | zone3 |   xx.xx.xx.218 |
      |    1014 |      1010 | ACTIVE | zone1 |   xx.xx.xx.237 |
      |    1015 |      1010 | ACTIVE | zone2 |   xx.xx.xx.238 |
      |    1016 |      NULL | ACTIVE | zone1 |   xx.xx.xx.237 |
      |    1017 |      NULL | ACTIVE | zone2 |   xx.xx.xx.238 |
      +---------+-----------+--------+-------+----------------+
      14 rows in set
      

      In the query result:

      • UNIT_ID: the ID of a unit on the node.

      • TENANT_ID: the ID of the tenant to which the unit belongs. If the value is NULL, the unit does not belong to any tenant.

      • STATUS: the current status of the unit:

        • ACTIVE: the unit is in the normal state.

        • DELETING: the unit is being deleted.

      • ZONE: the zone to which the unit belongs.

      • SVR_IP: the IP address of the node that hosts the unit.

      For more information about the DBA_OB_UNITS view, see DBA_OB_UNITS.

   2. Execute the following statement to manually migrate a unit.

      The statement is as follows:

      obclient [(none)]> ALTER SYSTEM MIGRATE UNIT = unit_id DESTINATION = 'svr_ip:svr_port'; 
      

      The parameters are described as follows:

      • unit_id: the ID of the unit to be migrated.

      • svr_ip: the IP address of the node after migration.

      • svr_port: the RPC port after migration.

      Here is an example that migrates a unit with the ID 1012 to the 172.xx.xx.xx server of the same zone:

      obclient [(none)]> ALTER SYSTEM MIGRATE UNIT = 1012 DESTINATION = '172.xx.xx.xx:2882'; 
      

      The statement can be executed multiple times. If the manual migration of a unit fails, you can fix the errors and execute the statement again.

      For more information about unit migration, see Unit migration.

3. Execute the following statement to delete the server.

   The statement is as follows:

   obclient [(none)]> ALTER SYSTEM DELETE SERVER 'svr_ip:svr_port' [,'svr_ip:svr_port'...] [ZONE [=] 'zone_name']
   

   The parameters are described as follows:

   • svr_ip: the IP address of the server to be deleted.

   • svr_port: the RPC port of the server to be deleted.

   • zone_name: the zone to which the server to be deleted belongs. When you delete multiple servers, these servers must belong to the same zone.

   The following is an example that deletes an OBServer node in the zone1 zone:

   obclient [(none)]> ALTER SYSTEM DELETE SERVER "172.xx.xx.xx:2882" zone='zone1'
   

4. Query the oceanbase.DBA_OB_SERVERS view after all operations are completed to check whether the server is deleted.

   obclient [(none)]> SELECT * FROM oceanbase.DBA_OB_SERVERS;
   

   If the server to be deleted is no longer in the query result, the server is deleted. If the server to be deleted is still in the query result and its status is DELETING, the server is being deleted.

   In the case where the Delete Server operation is executed without manually migrating units, the server may fail to be deleted. If the server fails to be deleted, you can perform the subsequent operations based on What to do next**.

What to do next

If you execute the Delete Server operation without manually migrating units, the units on the server cannot be migrated. This is because the resources of the servers in the same zone as the server are insufficient to accommodate the units. Therefore, the units of the server cannot be migrated. If this occurs, you can view the root service log stored in the /home/admin/oceanbase/log/rootservice.log file on the root service server. If the unit migration fails, you can execute the Cancel Delete Server operation, add a node to the zone to scale out the cluster, and then delete the server. For more information about how to add a node, see Add a node.

The following steps describe how to cancel the delete operation.

1. Log in to the sys tenant of the cluster as the root user.

   The following is a sample statement for connecting to the database. You need to use the actual parameters for the connection.

   obclient -h10.xx.xx.xx -P2883 -uroot@sys#obdemo -p***** -A
   

2. Execute the following statement to cancel the delete operation.

   The statement is as follows:

   obclient [(none)]> ALTER SYSTEM CANCEL DELETE SERVER 'svr_ip:svr_port' [,'svr_ip:svr_port'...] [ZONE [=] 'zone_name']
   

   The parameters are described as follows:

   • svr_ip: the IP address of the server for which you want to cancel the delete operation.

   • svr_port: the RPC port of the server for which you want to cancel the delete operation. The default value is 2882.

   • zone_name: the zone to which the server for which you want to cancel the delete operation belongs.

   Here is an example:

   obclient [(none)]> ALTER SYSTEM CANCEL DELETE SERVER '172.xx.xx.xx:2882' zone='zone1';
   

3. Query the oceanbase.DBA_OB_SERVERS view to check whether the cancel delete operation is successful.

   obclient [(none)]> SELECT * FROM oceanbase.DBA_OB_SERVERS;
   

   If the status of the OBServer node to be canceled is changed from DELETING to ACTIVE, the operation is successful.

References

For more information about node operations, see the following topics:

• View a node

• Restart a node

• Add a node

• Isolate a node

• Replace a node

• Delete a zone

• Unit migration