This topic describes how to use OceanBase Deployer (obd) to quickly deploy primary and standby tenants in three scenarios.
Note
At present, obd only supports network-based primary/standby tenant deployment.
Concepts
A physical standby database is an important part of the high availability solution of OceanBase Database. It can provide high availability, data protection, and disaster recovery for your key applications.
The physical standby database serves as a quasi-real-time hot backup for the production database of OceanBase Database. If the primary database becomes unavailable due to planned or unplanned events (such as the failure of the majority of replicas), a standby database takes over the services and provides both lossless switching (RPO = 0) and lossy switching (RPO > 0), minimizing service downtime and possible data loss. For more information, see Physical standby database disaster recovery in OceanBase Database Documentation.
Prerequisites
The clusters where the primary and standby tenants respectively reside have been created. The clusters use the same OceanBase Database version, which is V4.2.0 or later.
The primary tenant is a user tenant and has full logs.
Run the command
SELECT LS_ID, BEGIN_LSN FROM oceanbase.GV$OB_LOG_STAT WHERE ROLE = 'LEADER' ;to check the value ofBEGIN_LSN. If it is0, the current log stream replica has the complete log since its creation.The cluster where the primary tenant resides is running.
Run the
obd cluster displaycommand to view the cluster status.To try out the feature, allocate a minimum of one CPU core, 2 GB of memory, and 4 GB of log disk space for the primary tenant. For long-term use, it is recommended to allocate at least one CPU core and 4 GB of memory.
Scenario 1: one primary tenant, one standby tenant
In this scenario, there are two clusters: clusterA and clusterB. In clusterA, a user tenant A_a (primary tenant) is created. In clusterB, a standby tenant (B_a) is created for tenant A_a. The steps to perform this operation are as follows:
Create the standby tenant B_a for tenant A_a:
obd cluster tenant create-standby clusterB clusterA A_a -t B_a --tenant-root-password=*****View the primary/standby relationship of the standby tenant:
obd cluster tenant show clusterB -gwhere the
-goption displays the primary/standby relationships related to a cluster in a graph.The returned results are as follows:
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tenant base info | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | sys | SYS | MYSQL | RANDOM | 1.0 | 1.0 | 2.0G | 9223372036854775807 | 9223372036854775807 | 2.0G | 1 | PRIMARY | | B_a | USER | MYSQL | RANDOM | 2.0 | 2.0 | 4.0G | 10000 | 10000 | 5.0G | 2 | STANDBY | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ +-----------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant standby info | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | standby_tenant_name | tenant_status | sync_status | sync_scn_timestamp | err_code | error_comment | switchover_status | switchover_epoch | log_mode | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | B_a | NORMAL | NORMAL | 2023-09-14 08:58:44.173462 | 0 | | NORMAL | 0 | NOARCHIVELOG | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant's primary info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby_tenant_name | primary_tenant_info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | B_a | IP_LIST=10.10.10.1:2881;10.10.10.2:2881;10.10.10.3:2881,USER=standbyro@A_a,PASSWORD=******,TENANT_ID=1002,CLUSTER_ID=1694680744,COMPATIBILITY_MODE=MYSQL,IS_ENCRYPTED=false | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ query primary-standby relation ok primary-standby relation topology graph clusterA:A_a └── clusterB:B_aAs shown in the returned results, the values of both
tenant_statusandsync_statusareNORMAL, indicating that the standby tenant is successfully created and the log transmission is normal.The fields in the returned results are described as follows:
tenant base info: displays information about tenants in the cluster. You need to pay attention to the following fields.
Field Description tenant_name The name of the tenant. tenant_type The type of the tenant. Valid values: SYS (sys tenant) and USER (user tenant). compatibility_mode The compatibility mode of the tenant. Valid values: MySQL and Oracle. OceanBase Database Community Edition only supports MySQL mode. primary_zone The primary zone of the tenant. tenant_role The role of the tenant. Valid values: PRIMARY and STANDBY. standby tenant standby info: displays information about standby tenants in the cluster. You need to pay attention to the following fields.
Field Description standby_tenant_name The name of the standby tenant. tenant_status The state of the standby tenant. The value NORMALindicates that the standby tenant is normal.sync_status The log synchronization status of the standby tenant. The value NORMALindicates that log synchronization is normal.sync_scn_timestamp The log synchronization timestamp. err_code The error code for log synchronization. error_comment The error message for log synchronization. switchover_status The switchover status. The value NORMALindicates that the switchover is normal.switchover_epoch The switchover time. log_mode The log type. The log type is NOARCHIVELOGfor network-based synchronization between the primary and standby tenants.standby tenant's primary info: displays information about the primary tenants of standby tenants in the cluster. You need to pay attention to the following fields.
Field Description standby_tenant_name The name of the standby tenant. primary_tenant_info The information about the primary tenant, including the IP address of the node where the cluster of the primary tenant resides, the name of the primary tenant, the password of the standbyrouser, the ID of the primary tenant, and the ID of the cluster where the primary tenant resides.
Log on to the primary and standby tenants separately. Modify information in the primary tenant (A_a) and check whether the changes are synchronized in the standby tenant (B_a).
Scenario 2: One primary tenant, two standby tenants
In this scenario, there are three clusters: clusterA, clusterB, and clusterC. In clusterA, a user tenant A_a (primary tenant) is created. In clusterB and clusterC, two standby tenants (B_a and C_a) are created for tenant A_a, respectively. The steps to perform this operation are as follows:
Create the standby tenant B_a for tenant A_a:
obd cluster tenant create-standby clusterB clusterA A_a -t B_a --tenant-root-password=*****Create the standby tenant C_a for tenant A_a:
obd cluster tenant create-standby clusterC clusterA A_a -t C_a --tenant-root-password=*****View the primary/standby relationship of the standby tenant, for example, in clusterB:
obd cluster tenant show clusterB -gwhere the
-goption displays the primary/standby relationships related to a cluster in a graph.The returned results are as follows:
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tenant base info | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | sys | SYS | MYSQL | RANDOM | 1.0 | 1.0 | 2.0G | 9223372036854775807 | 9223372036854775807 | 2.0G | 1 | PRIMARY | | B_a | USER | MYSQL | RANDOM | 2.0 | 2.0 | 4.0G | 10000 | 10000 | 5.0G | 2 | STANDBY | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ +-----------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant standby info | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | standby_tenant_name | tenant_status | sync_status | sync_scn_timestamp | err_code | error_comment | switchover_status | switchover_epoch | log_mode | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | B_a | NORMAL | NORMAL | 2023-09-14 09:12:13.819821 | 0 | | NORMAL | 0 | NOARCHIVELOG | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant's primary info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby_tenant_name | primary_tenant_info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | B_a | IP_LIST=10.10.10.1:2881;10.10.10.2:2881;10.10.10.3:2881,USER=standbyro@A_a,PASSWORD=******,TENANT_ID=1002,CLUSTER_ID=1694680744,COMPATIBILITY_MODE=MYSQL,IS_ENCRYPTED=false | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ query primary-standby relation ok primary-standby relation topology graph clusterA:A_a ├── clusterB:B_a └── clusterC:C_aAs shown in the returned results, the values of both
tenant_statusandsync_statusareNORMAL, indicating that the standby tenant is successfully created and the log transmission is normal. For more information about the fields in the returned results, see the field description in Scenario 1.Log on to the primary and standby tenants separately. Modify information in the primary tenant (A_a) and check whether the changes are synchronized in the standby tenants (B_a and C_a).
Scenario 3: One primary tenant with cascading standby tenants
In this scenario, there are three clusters: clusterA, clusterB, and clusterC. In clusterA, a user tenant A_a (primary tenant) is created. In clusterB, a standby tenant B_a is created for tenant A_a. In clusterC, a standby tenant C_a is created for tenant B_a. The steps to perform this operation are as follows:
Create the standby tenant B_a for tenant A_a:
obd cluster tenant create-standby clusterB clusterA A_a -t B_a --tenant-root-password=*****Check whether tenant B_a is successfully created:
obd cluster tenant show clusterB -gwhere the
-goption displays the primary/standby relationships related to a cluster in a graph.The returned results are as follows:
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tenant base info | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | sys | SYS | MYSQL | RANDOM | 1.0 | 1.0 | 2.0G | 9223372036854775807 | 9223372036854775807 | 2.0G | 1 | PRIMARY | | B_a | USER | MYSQL | RANDOM | 2.0 | 2.0 | 4.0G | 10000 | 10000 | 5.0G | 2 | STANDBY | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ +-----------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant standby info | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | standby_tenant_name | tenant_status | sync_status | sync_scn_timestamp | err_code | error_comment | switchover_status | switchover_epoch | log_mode | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | B_a | NORMAL | NORMAL | 2023-09-14 08:58:44.173462 | 0 | | NORMAL | 0 | NOARCHIVELOG | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant's primary info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby_tenant_name | primary_tenant_info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | B_a | IP_LIST=10.10.10.1:2881;10.10.10.2:2881;10.10.10.3:2881,USER=standbyro@A_a,PASSWORD=******,TENANT_ID=1002,CLUSTER_ID=1694680744,COMPATIBILITY_MODE=MYSQL,IS_ENCRYPTED=false | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ query primary-standby relation ok primary-standby relation topology graph clusterA:A_a └── clusterB:B_aAs shown in the returned results, the values of both
tenant_statusandsync_statusareNORMAL, indicating that the standby tenant is successfully created and the log transmission is normal. For more information about the fields in the returned results, see the field description in Scenario 1.Create the standby tenant C_a for tenant B_a:
Note
It is recommended to wait for the synchronization of tenant B_a to be completed before creating tenant C_a.
obd cluster tenant create-standby clusterC clusterB B_a -t C_a --tenant-root-password=*****View the primary/standby relationship of the standby tenant, for example, in clusterB:
obd cluster tenant show clusterB -gwhere the
-goption displays the primary/standby relationships related to a cluster in a graph.The returned results are as follows:
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tenant base info | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ | sys | SYS | MYSQL | RANDOM | 1.0 | 1.0 | 2.0G | 9223372036854775807 | 9223372036854775807 | 2.0G | 1 | PRIMARY | | B_a | USER | MYSQL | RANDOM | 2.0 | 2.0 | 4.0G | 10000 | 10000 | 5.0G | 2 | STANDBY | +-------------+-------------+--------------------+--------------+---------+---------+-------------+---------------------+---------------------+---------------+-------------+-------------+ +-----------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant standby info | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | standby_tenant_name | tenant_status | sync_status | sync_scn_timestamp | err_code | error_comment | switchover_status | switchover_epoch | log_mode | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ | B_a | NORMAL | NORMAL | 2023-09-14 09:24:48.737141 | 0 | | NORMAL | 0 | NOARCHIVELOG | +---------------------+---------------+-------------+----------------------------+----------+---------------+-------------------+------------------+--------------+ +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby tenant's primary info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | standby_tenant_name | primary_tenant_info | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | B_a | IP_LIST=10.10.10.1:2881;10.10.10.2:2881;10.10.10.3:2881,USER=standbyro@A_a,PASSWORD=******,TENANT_ID=1002,CLUSTER_ID=1694680744,COMPATIBILITY_MODE=MYSQL,IS_ENCRYPTED=false | +---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ query primary-standby relation ok primary-standby relation topology graph clusterA:A_a └── clusterB:B_a └── clusterC:C_aLog on to the primary and standby tenants separately. Modify information in the primary tenant (A_a) and check whether the changes are synchronized in the standby tenant (B_a) and in the standby tenant (C_a) of B_a.
FAQ
The error message "primary cluster have not full log xxx" is returned when a standby tenant is being created
Possible Cause: The primary tenant does not have full logs.
Network-based synchronization between the primary and standby tenants requires the primary tenant to have full logs. Before the command is executed to create a standby tenant, obd will first check whether the BEGIN_LSN value of the log stream of the primary tenant is 0. If yes, the replica of the current log stream has full logs since its creation. If no, the replica of the current log stream does not have full logs, indicating that obd cannot create a standby tenant based on network synchronization.
Solution: Log on to the primary tenant and run the following command to check whether the BEGIN_LSN value of the log stream of the primary tenant is 0:
SELECT LS_ID, BEGIN_LSN FROM oceanbase.GV$OB_LOG_STAT WHERE ROLE = 'LEADER' ;
If the BEGIN_LSN value of the log stream of the primary tenant is not 0, the primary tenant does not have full logs, indicating that a standby tenant cannot be created for the primary tenant using obd. Instead, use SQL statements to create a standby tenant for the primary tenant by referring to Create a standby tenant in OceanBase Database Documentation.