Correct the data after a failover

After a failover is executed, the new primary cluster may lose some data compared with the original one regardless of the lossy or lossless nature of the failover. OceanBase Database provides the cluster a read-only mode for you to correct the data.

Background

A cluster in read-only mode is under the DISABLED WITH READ ONLY state. Compared with a cluster under the DISABLED state, they have the following differences and shared features:

• The log transfer feature is disabled for clusters in both states.

• Only the original or new primary cluster can be set to the DISABLED WITH READ ONLY state. A standby cluster cannot be set to the DISABLED WITH READ ONLY state even if it starts in read-only mode.

• A cluster under the DISABLED WITH READ ONLY state supports strong consistency reads for common tenants, in which the logs of different partitions will change. In a cluster under the DISABLED state, logs of common tenants do not change. It can therefore be connected after a lossless failover. For more information, see Connect the original primary cluster after a lossless failover.

The read-only mode has the following features and limits:

• Only the PRIMARY cluster can be set to the read-only mode.

• Strong and weak consistency reads are supported for common tenants in a cluster under the read-only mode.

• Writes and data definition language (DDL) operations are not supported for common tenants in a cluster under the read-only mode.

• You cannot create a tenant in a cluster under the read-only mode

• When VALID primary cluster is unavailable or does not exist, you can run the failover command to switch a read-only cluster to the VALID state.

Procedure

1. Specify the special parameter -m disabled_with_readonly_cluster on all OBservers in the original primary cluster, and start them.

   The read-only mode requires that all OBservers in the original primary cluster are started with the special parameter. After all OBservers are started, the cluster enters the DISABLED WITH READ ONLY state. The state persists in the cluster, so you do not need to specify the special parameter again when you restart an OBserver.

   bin/observer -m disabled_with_readonly_cluster
   

2. After the original primary cluster is started, run the following command to check if it is under the DISABLED WITH READ ONLY state.

   obclient> SELECT CLUSTER_ROLE, CLUSTER_STATUS FROM V$OB_CLUSTER;
   +--------------+-------------------------+
   | CLUSTER_ROLE |         CLUSTER_STATUS          |
   +--------------+-------------------------+
   |     PRIMARY      | DISABLED WITH READ ONLY |
   +--------------+-------------------------+
   1 row in set
   

   After the original primary cluster is started in read-only mode, it isolates itself to disable data synchronization with the new primary cluster or other standby clusters, which ensures data security. The common tenant in the cluster only supports read operations and not writes or DDL operations, so that the data is frozen in the snapshot version prior to the failover.

3. Log on to the common tenant of the cluster and correct the data for the new and original primary clusters.

   OceanBase Database enables strong consistency read by default, so you can correct the data directly. You can also specify strong consistency read by the following methods:

   • Set the following session variable

     obclient> SET ob_read_consistency =STRONG;
     

   • Specify a hint in an SQL statement

     obclient> SELECT /*+read_consistency(strong) */  FROM table_name;