OceanBase Database supports role switching and decoupling for tenants. This topic describes how to use obd to dynamically change the role of a tenant and how to decouple primary and standby tenants.
Concepts
OceanBase Database supports two tenant roles: PRIMARY and STANDBY. A normal tenant is a PRIMARY tenant. A tenant created by using a physical standby database is a STANDBY tenant.
obd allows you to dynamically change the role of a tenant by using commands.
Switchover
Switchover is a planned operation that changes the roles of primary and standby tenants. After you execute the switchover operation, the primary and standby tenants exchange their roles. Data is not lost during the operation, the RPO is 0, and the operation takes only a few seconds.
Failover
Failover is an unplanned operation that changes the role of a standby tenant to PRIMARY when the primary tenant is unavailable. If you execute the failover operation on a standby tenant that has been synchronizing data from the primary tenant, data loss of hundreds of milliseconds occurs during the operation, and the operation takes only a few seconds.
Decoupling
Decoupling is a planned operation that changes the role of a standby tenant to PRIMARY. After you execute the decoupling operation, the standby tenant is decoupled from the primary tenant.
Switchover
After you execute the switchover operation, the topology remains unchanged. The other standby tenants under the original primary tenant are switched to the new primary tenant, and the other standby tenants under the new primary tenant are switched to the new standby tenant. Only the two primary and standby tenants involved in the switchover operation exchange their roles. The other tenants remain in their original positions, as shown in the following figure.
Prerequisites
OceanBase Database is of V4.2.0.0 or later.
The primary and standby tenants are online. You can execute the
obd cluster displaycommand to view the status of the cluster.The status of the primary and standby tenants is
NORMAL, and thesync_statusof the standby tenant isNORMAL. You can execute theobd cluster tenant showcommand to view the status.The role of the primary tenant is
PRIMARY.When you switch a standby tenant to a primary tenant by using network-based data synchronization, all replicas of the standby tenant must be online.
When you switch a standby tenant to a primary tenant by using log archiving-based data synchronization, you must back up and archive the standby tenant before you execute the switchover operation. For more information, see Back up and restore data > Tenant backup.
Procedure
Execute the following command to view the primary and standby relationships of the standby tenant:
obd cluster tenant show clusterB -gThe output is as follows:
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tenant base info | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | sys | SYS | MYSQL | RANDOM | 1.0 | 1.0 | 2.0G | 9223372036854775807 | 9223372036854775807 | 2.0G | 1 | PRIMARY | | B_a | USER | MYSQL | RANDOM | 2.0 | 2.0 | 4.0G | 10000 | 10000 | 5.0G | 2 | STANDBY | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ +-----------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant standby info | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | standby_tenant_name | tenant_status | sync_status | sync_scn_timestamp | err_code | error_comment | switchover_status | switchover_epoch | log_mode | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | B_a | NORMAL | NORMAL | 2023-09-14 09:43:00.199589 | 0 | | NORMAL | 0 | NOARCHIVELOG | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant`s primary info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby_tenant_name | primary_tenant_info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | B_a | IP_LIST=10.10.10.1:2881;10.10.10.2:2881;10.10.10.3:2881,USER=standbyro@A_a,PASSWORD=******,TENANT_ID=1002,CLUSTER_ID=1694680744,COMPATIBILITY_MODE=MYSQL,IS_ENCRYPTED=false | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ query primary-standby relation ok primary-standby relation topology graph clusterA:A_a └── clusterB:B_aExecute the switchover command on the standby tenant:
obd cluster tenant switchover clusterB B_a --tenant-root-password=******View the primary and standby relationships of the original standby tenant again:
obd cluster tenant show clusterB -gThe output is as follows:
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tenant base info | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | sys | SYS | MYSQL | RANDOM | 1.0 | 1.0 | 2.0G | 9223372036854775807 | 9223372036854775807 | 2.0G | 1 | PRIMARY | | B_a | USER | MYSQL | RANDOM | 2.0 | 2.0 | 4.0G | 10000 | 10000 | 5.0G | 2 | PRIMARY | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ query primary-standby relation ok primary-standby relation topology graph clusterB:B_a └── clusterA:A_aIn the output, the role of the original standby tenant has been updated to
PRIMARY, and the topology of the primary and standby relationships has been updated.
Failover
Failover is an unplanned operation that changes the role of a standby tenant to PRIMARY when the primary tenant is unavailable. Failover only modifies log files, not data files.
OceanBase Database has multiple log streams for each tenant. To ensure data consistency after the failover operation, the system selects the smallest SCN among all log streams as the execution point of the failover operation. After the failover operation, all log streams under the tenant are rolled back to this point.
Notice
In a one-primary-and-multi-standby scenario, after you execute the failover operation to change a standby tenant to a primary tenant, the other standby tenants cannot be switched to the new primary tenant. You must choose to decouple the other standby tenants or execute the failover operation on them to generate new primary tenants.
Prerequisites
OceanBase Database is of V4.2.0.0 or later.
The primary tenant is unavailable.
The standby tenant is online, and its status is
NORMAL.All replicas of the standby tenant are online.
Procedure
Execute the following command to view the primary and standby relationships of the standby tenant:
obd cluster tenant show clusterB -gThe output is as follows:
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tenant base info | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | sys | SYS | MYSQL | RANDOM | 1.0 | 1.0 | 2.0G | 9223372036854775807 | 9223372036854775807 | 2.0G | 1 | PRIMARY | | B_a | USER | MYSQL | RANDOM | 2.0 | 2.0 | 4.0G | 10000 | 10000 | 5.0G | 2 | STANDBY | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ +-----------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant standby info | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | standby_tenant_name | tenant_status | sync_status | sync_scn_timestamp | err_code | error_comment | switchover_status | switchover_epoch | log_mode | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | B_a | NORMAL | NORMAL | 2023-09-14 10:44:04.755016 | 0 | | NORMAL | 0 | NOARCHIVELOG | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant`s primary info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby_tenant_name | primary_tenant_info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | B_a | IP_LIST=10.10.10.1:2881;10.10.10.2:2881;10.10.10.3:2881,USER=standbyro@A_a,PASSWORD=******,TENANT_ID=1002,CLUSTER_ID=1694680744,COMPATIBILITY_MODE=MYSQL,IS_ENCRYPTED=false | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ query primary-standby relation ok primary-standby relation topology graph clusterA:A_a └── clusterB:B_aExecute the failover command on the standby tenant:
obd cluster tenant failover clusterB B_a --tenant-root-password=******View the primary and standby relationships of the original standby tenant again:
obd cluster tenant show clusterB -gThe output is as follows:
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tenant base info | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | sys | SYS | MYSQL | RANDOM | 1.0 | 1.0 | 2.0G | 9223372036854775807 | 9223372036854775807 | 2.0G | 1 | PRIMARY | | B_a | USER | MYSQL | RANDOM | 2.0 | 2.0 | 4.0G | 10000 | 10000 | 5.0G | 2 | PRIMARY | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ query primary-standby relation ok
Decouple primary and standby tenants
You can execute the decouple command to decouple a standby tenant from the primary tenant. This topic describes how to execute the decouple command to decouple primary and standby tenants.
Prerequisites
OceanBase Database is of V4.2.0.0 or later.
The standby tenant is online, and its status is
NORMAL.All replicas of the standby tenant are online.
Procedure
Execute the following command to view the primary and standby relationships of the standby tenant:
obd cluster tenant show clusterC -gThe output is as follows:
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tenant base info | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | sys | SYS | MYSQL | RANDOM | 1.0 | 1.0 | 2.0G | 9223372036854775807 | 9223372036854775807 | 2.0G | 1 | PRIMARY | | C_a | USER | MYSQL | RANDOM | 2.0 | 2.0 | 4.0G | 10000 | 10000 | 5.0G | 2 | STANDBY | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ +-----------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant standby info | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | standby_tenant_name | tenant_status | sync_status | sync_scn_timestamp | err_code | error_comment | switchover_status | switchover_epoch | log_mode | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | C_a | NORMAL | NORMAL | 2023-09-14 09:39:32.423752 | 0 | | NORMAL | 0 | NOARCHIVELOG | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant`s primary info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby_tenant_name | primary_tenant_info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | C_a | IP_LIST=10.10.10.1:3881;10.10.10.2:3881;10.10.10.3:3881,USER=standbyro@B_a,PASSWORD=******,TENANT_ID=1002,CLUSTER_ID=1694681525,COMPATIBILITY_MODE=MYSQL,IS_ENCRYPTED=false | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ query primary-standby relation ok primary-standby relation topology graph clusterA:A_a └── clusterB:B_a └── clusterC:C_aExecute the decouple command on the standby tenant:
obd cluster tenant decouple clusterC C_a --tenant-root-password=******View the primary and standby relationships of the original standby tenant again:
obd cluster tenant show clusterC -gThe output is as follows:
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tenant base info | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | sys | SYS | MYSQL | RANDOM | 1.0 | 1.0 | 2.0G | 9223372036854775807 | 9223372036854775807 | 2.0G | 1 | PRIMARY | | C_a | USER | MYSQL | RANDOM | 2.0 | 2.0 | 4.0G | 10000 | 10000 | 5.0G | 2 | PRIMARY | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ query primary-standby relation ok