Automatically clear secondary backup data for clusters

After the secondary backup for a cluster succeeds, you can enable the automatic data clearing feature based on your business needs. The system automatically clears expired secondary backup data when this feature is enabled. Only cluster-level secondary backup data can be automatically cleared.

Considerations

• The clearing of archived log data depends on secondary data backup. Before you clear archived log data, make sure that the corresponding secondary data backup file exists. If no corresponding secondary data backup file exists, you cannot clear the archived log data.

• If the secondary backup destination is a network file system (NFS), the system directly deletes the secondary backup files that meet the requirements when the system clears secondary backup data. If the secondary backup destination is Object Storage Service (OSS), the method for clearing secondary backup data depends on the value of the delete_mode parameter in backup_backup_dest. For more information, see Preparations for secondary backup.

• Only data stored in the backup destination specified by backup_backup_dest can be automatically cleared. If the destination of a secondary backup has been changed, it must be manually cleared upon expiration. For more information, see Manually clear secondary backup data.

• At least one copy of secondary backup data is retained after automatic clearing. If only one copy of valid backup data exists, this data copy will never expire.

(Recommended) Enable automatic data clearing by using the backup_backup_dest_option parameter

1. Log on to the sys tenant as the root user.

2. Set a clearing policy for the secondary backup data.

   The clearing policy for secondary backup data is defined by the recovery_window and auto_delete_obsolete_backup parameters in backup_backup_dest_option. For more information about these two parameters, see Preparations for secondary backup.

   For more information about backup_backup_dest_option, see backup_backup_dest_option.

   • auto_delete_obsolete_backup

     This parameter specifies whether to enable automatic clearing for secondary backup data. If auto_delete_obsolete_backup is set to true, the system automatically clears expired secondary backup data.

   • recovery_window

     This parameter specifies the time window for restoring secondary backup data. The system determines whether a copy of secondary backup data has expired based on this time window. For example, the configuration recovery_window='7d' indicates that secondary backup data within the last 7 days can be restored, and the secondary backup data beyond this restore window is considered expired.

     For more information about the recovery_window parameter, see Automatically clear backup data for clusters.

Trigger condition of automatic clearing

After you enable automatic clearing, the system triggers an automatic clearing task when the trigger condition is met.

Automatic clearing interval = min(recovery_window/2, 1d)

When the difference between the current time and time of last successful automatic clearing is greater than automatic clearing interval, the system triggers an automatic clearing task. In general, the interval between two adjacent automatic clearing tasks cannot exceed one day.

For more information about the recovery_window parameter, see Manually clear backup data for clusters. To obtain the time of the last successful automatic clearing, you can query the END_TIME field in the oceanbase.CDB_OB_BACKUP_CLEAN_HISTORY view.`` Perform the following steps:

1. Log on to the sys tenant as the root user.

2. Query the oceanbase.CDB_OB_BACKUP_CLEAN_HISTORY view to obtain the time of last successful automatic clearing.

   For example:

   obclient> SELECT * FROM oceanbase.CDB_OB_BACKUP_CLEAN_HISTORY WHERE tenant_id = 1 AND type = 'DELETE OBSOLETE BACKUP BACKUP' AND ERROR_MSG='' ORDER BY END_TIME DESC LIMIT 1;
   +-----------+--------+---------+----------------------------+----------------------------+-------------+-------------------------------+--------+------------------+-----------+-----------------------------------------------+
   | TENANT_ID | BS_KEY | COPY_ID | START_TIME                 | END_TIME                   | INCARNATION | TYPE                          | STATUS | PARAMETER        | ERROR_MSG | COMMENT                                       |
   +-----------+--------+---------+----------------------------+----------------------------+-------------+-------------------------------+--------+------------------+-----------+-----------------------------------------------+
   |         1 |      4 |       0 | 2022-10-28 11:26:14.932453 | 2022-10-28 11:26:16.546706 |           1 | DELETE OBSOLETE BACKUP BACKUP | STOP   | 1666927274926967 |           | server:xxx, trace_id:xxxx                     |
   +-----------+--------+---------+----------------------------+----------------------------+-------------+-------------------------------+--------+------------------+-----------+-----------------------------------------------+
   1 row in set
   

   Fields in query results:

   • BS_KEY indicates the job ID of the clearing task.

   • COPY_ID indicates the copy_id of the clearing task. The value of this field is not 0 only when you use the backup clearing command to clear the specified secondary backup.

   • END_TIME indicates the time when the clearing task ends.

   • STATUS indicates the status of the clearing task. If the task is recorded in a historical task view, the value of the STATUS field is STOP.

   • ERROR_MSG records the description of a failed clearing task. If the clearing task succeeded, this field is empty.

   Possible causes of an empty query result:

   • A clearing task is being performed. You can check the oceanbase.__all_virtual_backup_clean_info table for ongoing tasks. After all clearing tasks (including manual and automatic tasks) are completed and the automatic clearing interval arrives, the system performs the next automatic clearing task.

   • Secondary backups have never been automatically cleared before. In this case, the system immediately performs an automatic secondary backup clearing task right after you enable automatic clearing.