This topic describes how to locate and troubleshoot switchover failures.
Symptom
A failure occurs when the SWITCHOVER statement is executed to switch a primary tenant to the STANDBY role or switch a standby tenant to the PRIMARY role. Switching a standby tenant to the PRIMARY role has a higher probability of failure.
Troubleshooting procedure
When you execute the SWITCHOVER statement to switch a standby tenant to the PRIMARY role, the standby tenant will verify whether its logs are synchronized with those of the primary tenant. The switchover will fail if a gap exists between the logs of the primary and standby tenants, or if the standby tenant cannot perform the verification because the primary tenant is abnormal.
You can perform the following steps to troubleshoot the issue.
Step 1: Locate the cause of the switchover failure
Determine the cause of the switchover failure based on the returned information or by querying the DBA_OB_ROOTSERVICE_EVENT_HISTORY view.
Log on as the administrator to the sys tenant of the cluster to which the standby tenant belongs.
Execute the following statement to view the cause of the failure:
SELECT * FROM oceanbase.DBA_OB_ROOTSERVICE_EVENT_HISTORY WHERE EVENT LIKE "%switchover to%" ORDER BY TIMESTAMP;In the query result, the error code in the
VALUE2column indicates the cause of the switchover failure.For more information about the
DBA_OB_ROOTSERVICE_EVENT_HISTORYview, see DBA_OB_ROOTSERVICE_EVENT_HISTORY.
Step 2: Determine the solution to the switchover failure based on the error code
The error code for a failure to switch a standby tenant to the PRIMARY role is 4179, which indicates that the logs between the standby and primary tenants are out of synchronization. In this case, confirm whether you have verified the gap between logs of the primary and standby tenants before the switchover. The acceptable gap between the SYNC_SCN values of the primary and standby tenants cannot be greater than 5s. For more information about how to view the SYNC_SCN values of the primary and standby tenants, see View the log synchronization progress.
The following table describes the error codes related to switchover and the corresponding solutions.
| Error code | Description | Solution |
|---|---|---|
| 4179 | The logs between the standby and original primary tenants are out of synchronization. | Check whether the gap between the SYNC_SCN values of the standby tenant and original primary tenant is less than 20s. If yes, perform the switchover again.
NoteAfter you perform a switchover to switch a primary tenant to the STANDBY role or switch a standby tenant to the PRIMARY role, the system automatically advances the SYNC_SCN value of the original primary or standby tenant by 20s. Therefore, it is acceptable if the gap between the |
| 9088 | The role of the original primary tenant is not STANDBY or the original primary tenant is not in the NORMAL state. |
Check whether the role of the original primary tenant is STANDBY and whether the original primary tenant is in the NORMAL state. If yes, the switchover is successful. Retry the switchover operation. |
| 9089 | The log stream status of the original primary tenant does not conform to the actual situation. The reason may be that an exception occurred when the primary tenant switched to the STANDBY role. | Check whether the role of the original primary tenant is STANDBY and whether the original primary tenant is in the NORMAL state. If yes, the switchover is successful. Retry the switchover operation. If the error persists after you retry the switchover operation, contact OceanBase Technical Support. |
| 4043 | The logon to the original primary tenant failed because the username or password of the account for logging on to the original primary tenant is incorrect or the original primary tenant has been dropped. | Check whether the original primary tenant exists. If the original primary tenant has been deleted, only a failover can be performed. For more information about failover, see Failover. |
| 5260 | The original primary tenant has been locked and therefore cannot be queried. | Unlock the original primary tenant and perform the switchover again. For information about how to unlock a tenant, see Lock and unlock a tenant. |