Introduction to physical backup and restore

As the core component to ensure the high availability of OceanBase Database, the backup and restore module protects data security by preventing misoperations and damages to the storage media. If data is lost due to misoperations or damage to the storage media, you can restore the data.

Overview

The backup and restore module of OceanBase Database provides the backup, restore, and cleanup features.

OceanBase Database supports tenant-level physical backup. Physical backup data includes backup data and archive log data. Therefore, physical backup consists of log archiving and data backup. The tenants mentioned in this topic are user tenants. Physical backup is not supported for sys tenants or meta tenants.

Data backup refers to the backup of data, and includes full backup and incremental backup:

• Full backup refers to the backup of all macroblocks.

• Incremental backup refers to the backup of macroblocks that are added or modified since the last backup.

Data backup covers the following data:

• Tenant information, including the tenant name, cluster name, time zone, locality, and tenant mode (MySQL or Oracle)

• All user table data

  Note

  During data backup, system variables and tenant parameters are backed up, but cluster-level parameters and private system table data are not backed up.

Log archiving refers to the automatic backup of log data. OBServer nodes regularly archive log data to the specified backup path without manual triggering.

The architecture of physical restore is as follows:

1. Physical restore supports tenant-level restore and table-level restore.

   • Tenant-level restore is the process of rebuilding a new tenant based on existing backup data. Tenant-level restore ensures global consistency across tables and partitions.

   • Table-level restore is the process of restoring a specified table from backup data to an existing tenant, which can be the source tenant, another tenant in the same cluster, or a tenant in a different cluster.

2. Tenant-level restore supports full restore and quick restore.

   Notice

   Quick restore does not support data backup or support failover/switchover to a primary tenant. The restored tenant can act only as a standby tenant.

   • Full restore is the process of restoring macroblock data and incremental logs. The restored tenant can provide services after all data is restored from the backup media to your local server. The full restore process consists of the restore and recovery of the system tables and user tables of the tenant. Restore returns the required baseline data to the OBServer node of the destination tenant. Recovery returns the logs corresponding to the baseline data to the OBServer node.

   • Quick restore allows the restored tenant to provide services without restoring macroblock data. This can reduce the restore wait time and your costs.

3. You can specify a point in time for physical restore.

   • Full restore: You do not need to specify a timestamp to restore to.

   • Partial restore to the specified system change number (SCN) or timestamp: An SCN is a precise data version number in OceanBase Database. A timestamp is accurate to microseconds in MySQL mode, and the precision after microseconds will be lost.

For more information, see Restore process.

Backup media

The current version of OceanBase Database supports the following backup media: Alibaba Cloud OSS, NFS, Azure Blob Storage (For OceanBase Database V4.3.5, supported starting from V4.3.5 BP3), AWS S3, and S3-compatible object storage such as Huawei OBS, Google GCS, and Tencent Cloud COS. Some backup media need to meet certain basic requirements before they can be used.

SDK version requirements

The following table describes the relationship between the OceanBase Database version and the object storage SDK version:

API requirements

• OSS:

  • OSS must support the following API operations:

    Operation	Description
    PutObject	Uploads a single object.
    DeleteObject	Deletes a single object.
    DeleteObjects	Deletes multiple objects at a time.
    GetObject	Gets the specified object.
    ListObjects	Lists all objects in the bucket. This operation relies on strong data consistency.
    HeadObject	Gets the metadata of the specified object.
    AppendObject	Appends objects.
    PutObjectTagging (Optional)	Sets and updates the tags of objects.
    GetObjectTagging (Optional)	Gets the tags of objects.
    InitiateMultipartUpload	Initializes a multipart upload task.
    UploadPart	Uploads parts.
    CompleteMultipartUpload	Combines uploaded parts into an object.
    AbortMultipartUpload	Aborts a multipart upload task and deletes uploaded parts.
    ListMultipartUploads	Lists multipart upload tasks that have been initialized but not completed or aborted.
    ListParts	Lists uploaded parts in an upload task.

  • Only the V1 signature algorithm is supported.

• NFS: NFS 3 and later are supported.

• Object storage services compatible with the S3 protocol, such as Huawei OBS, Google GCS, and Tencent Cloud COS.

  1. The object storage services must support the following S3 API operations:

     Operation	Description
     PutObject	Uploads a single object.
     DeleteObject	Deletes a single object.
     DeleteObjects	Deletes multiple objects at a time.
     GetObject	Downloads a single object.
     ListObjects	Lists all objects in the path.
     HeadObject	Gets the metadata of the specified object.
     PutObjectTagging (Optional)	Sets the tags of objects.
     GetObjectTagging (Optional)	Gets the tags of objects.
     CreateMultipartUpload	Initializes a multipart upload task.
     UploadPart	Uploads a single part.
     CompleteMultipartUpload	Combines uploaded parts into an object.
     AbortMultipartUpload	Aborts a multipart upload task and deletes uploaded parts.
     ListMultipartUploads	Lists multipart upload tasks.
     ListParts	Lists uploaded parts in an upload task.

  2. The object storage services must support virtual-hosted–style URLs. For more information, visit the official website of AWS.

Before you select a backup medium, you can run the test_io_device command in ob_admin to check whether the I/O operations and privileges supported by the backup medium meet the backup and restore requirements. You can also run the io_adapter_benchmark command in ob_admin to query the performance of read/write between an OBServer node and the backup medium to estimate the backup performance. For more information about the test_io_device and io_adapter_benchmark commands, see test_io_device and io_adapter_benchmark.

Directory structure

Data backup directories

The directories created by the backup feature in the backup destination and types of files in the directories are as follows:

data_backup_dest
├── format.obbak                                            // The format information of the backup path
├── check_file
│   └── 1002_connect_file_20230111T193020.obbak             // The connectivity check file
├── backup_sets                                             // The summary directory of data backup lists, which contains all data backup sets
│   ├── backup_set_1_full_end_success_20230111T193420.obbak // The placeholder indicating the end of a full backup
│   ├── backup_set_1_full_start.obbak                       // The placeholder indicating the start of a full backup
│   ├── backup_set_2_inc_start.obbak                        // The placeholder indicating the start of an incremental backup
│   └── backup_set_2_inc_end_success_20230111T194420.obbak  // The placeholder indicating the end of an incremental backup
└── backup_set_1_full                                        // The full backup set. The file name suffix of this directory indicates the type of backup: 'full' for a full backup and 'inc' for an incremental backup.
    ├── backup_set_1_full_20230111T193330_20230111T193420.obbak // A placeholder that displays the start and end time of the full backup
    ├── single_backup_set_info.obbak                        // The metadata of the current backup set
    ├── tenant_backup_set_infos.obbak                       // The metadata of all full backup sets of the current tenant
    ├── infos
    │   ├── table_list                                      // The full name file of tables
    │   │   ├── table_list.1702352553000000000.1.obbak      // Table name file 1
    │   │   ├── table_list.1702352553000000000.2.obbak     // Table name file 2
    │   │   └── table_list_meta_info.1702352553000000000.obbak  // The metadata file of table names
    │   ├── major_data_info_turn_1                           // The tenant-level backup files of major turn 1
    │   │   ├── tablet_log_stream_info.obbak                 // The mapping file of tablets and log streams
    │   │   ├── tenant_major_data_macro_range_index.0.obbak  // The major macroblock index
    │   │   ├── tenant_major_data_meta_index.0.obbak         // The major meta index
    │   │   └── tenant_major_data_sec_meta_index.0.obbak     // The major logical ID-physical ID mapping file
    │   ├── minor_data_info_turn_1                           // The tenant-level backup files of minor turn 1
    │   │   ├── tablet_log_stream_info.obbak                 // The mapping file of tablets and log streams
    │   │   ├── tenant_minor_data_macro_range_index.0.obbak  // The minor macroblock index
    │   │   ├── tenant_minor_data_meta_index.0.obbak         // The minor meta index
    │   │   └── tenant_minor_data_sec_meta_index.0.obbak     // The minor logical ID-physical ID mapping file
    │   ├── diagnose_info.obbak                              // The diagnostic information file of the backup set
    |   ├── tenant_parameter.obbak                           // The tenant-level parameters that are not set to default values
    │   ├── locality_info.obbak                              // The locality information of the current backup set, including the resource configuration and replica distribution of the tenant
    │   └── meta_info                                        // The log stream metadata file of the tenant, which contains the metadata of all log streams
    │       ├── ls_attr_info.1.obbak                         // A snapshot of log streams when the backup is performed
    │       └── ls_meta_infos.obbak                          // The meta collection of all log streams
    ├── logstream_1                                          // Log stream 1
    │   ├── major_data_turn_1_retry_0                        // The baseline data of turn 1, retry 0
    │   │   ├── macro_block_data.0.obbak                     // A data file with a size of 512 MB to 4 GB
    │   │   ├── macro_range_index.obbak                      // The macro index
    │   │   ├── meta_index.obbak                             // The meta index
    │   │   └── sec_meta_index.obbak                         // The logical ID-physical ID mapping file
    │   ├── meta_info_turn_1_retry_0                         // The metadata file of log stream 1 of turn 1, retry 0
    │   │   ├── ls_meta_info.obbak                           // The log stream metadata
    │   │   └── tablet_info.1.obbak                          // The tablet meta list of log stream 1
    │   ├── minor_data_turn_1_retry_0                        // The dump data of log stream 1 of turn 1, retry 0
    │   │   ├── macro_block_data.0.obbak
    │   │   ├── macro_range_index.obbak
    │   │   ├── meta_index.obbak
    │   │   └── sec_meta_index.obbak
    │   └── sys_data_turn_1_retry_0                          // The system tablet data of log stream 1 of turn 1, retry 0
    │       ├── macro_block_data.0.obbak
    │       ├── macro_range_index.obbak
    │       ├── meta_index.obbak
    │       └── sec_meta_index.obbak
    └── logstream_1001                                       // Log stream 1001
        ├── major_data_turn_1_retry_0
        │   ├── macro_block_data.0.obbak
        │   ├── macro_range_index.obbak
        │   ├── meta_index.obbak
        │   └── sec_meta_index.obbak
        ├── meta_info_turn_1_retry_0
        │   ├── ls_meta_info.obbak
        │   └── tablet_info.1.obbak
        ├── minor_data_turn_1_retry_0
        │   ├── macro_block_data.0.obbak
        │   ├── macro_range_index.obbak
        │   ├── meta_index.obbak
        │   └── sec_meta_index.obbak
        └── sys_data_turn_1_retry_0
            ├── macro_block_data.0.obbak
            ├── macro_range_index.obbak
            ├── meta_index.obbak
            └── sec_meta_index.obbak

Top-level data backup directories contain the following types of data:

• format.obbak: used for recording the metadata of the backup path.

• check_file: used for checking the connectivity to the user data backup directory.

• backup_sets: the summary directory that contains all data backup sets.

• backup_set_1_full: This directory represents a data backup set, where a directory whose name ends with full is a full backup set, and a directory whose name ends with inc is an incremental backup set. Each data backup generates a corresponding backup set, and the backup set will not be modified after the data backup is completed.

  A data backup set mainly includes the following data:

  • backup_set_1_full_20230111T193330_20230111T193420.obbak: This file records the ID, start time, and end time of the current backup set. This file is used only for display purposes.

  • single_backup_set_info.obbak: This file records the metadata of the current backup set, including the backup timestamp, dependent logs, and other information.

  • tenant_backup_set_infos.obbak: This file records the metadata of all existing backup sets for the current tenant.

  • infos: This directory records the metadata of the data backup set.

  • logstream_1: This directory records all the data of log stream 1, which is the system log stream of an OceanBase Database tenant.

  • logstream_1001: This directory records all the data of log stream 1001, where log streams with IDs greater than 1000 are user log streams of an OceanBase Database tenant.

  In addition, each log stream backup has four types of directories. Directories whose names contain retry record information about log stream-level retries, and directories whose names contain turn record information about tenant-level retries.

  • meta_info_xx: This directory records log stream metadata and tablet metadata.
  • sys_data_xx: This directory records the data of internal system tablets in log streams.
  • minor_data_xx: This directory records the minor-compacted data of regular tablets.
  • major_data_xx: This directory records the baseline data of regular tablets.

Cluster-level configuration backup directory

Each time a cluster-level configuration backup is initiated, the system generates a backup file for the cluster-level configuration in the specified directory. The directory structure is as follows:

cluster_parameters_backup_dest
├── cluster_parameter.20240710T103610.obbak # Information about cluster-level parameters that are not set to default. File naming format:`cluster_parameter.[timestamp]`
└── cluster_parameter.20241018T140609.obbak

Log archive directories

When you use NFS, OSS, or Azure Blob Storage as the backup media, directories at the log archive destination and file types under each directory are as follows:

log_archive_dest
    ├── check_file
    │   └── 1002_connect_file_20230111T193049.obbak // The connectivity check file.
    ├── format.obbak                                // The format information of the backup path.
    ├── rounds                                      // The round placeholder directory.
    │   └── round_d1002r1_start.obarc               // The round start placeholder.
    ├── pieces                                      // The piece placeholder directory.
    │   ├── piece_d1002r1p1_start_20230111T193049.obarc // The piece start placeholder, in the format of piece_DESTID_ROUNDID_PIECEID_start_DATE.
    │   └── piece_d1002r1p1_end_20230111T193249.obarc   // The piece end placeholder, in the format of piece_DESTID_ROUNDID_PIECEID_end_DATE.
    └── piece_d1002r1p1                             // The piece directory, which is named in the format of `piece_DESTID_ROUNDID_PIECEID`.       
        ├── piece_d1002r1p1_20230111T193049_20230111T193249.obarc // The contiguous interval of a piece.
        ├── checkpoint   
        │   └── checkpoint.1673436649723677822.obarc   // The checkpoint SCN. This file is named in the format of `checkpoint.checkpoint_scn`.
        │   └── checkpoint_info.0.obarc             // The checkpoint metadata.
        ├── single_piece_info.obarc                 // The metadata of the piece.
        ├── tenant_archive_piece_infos.obarc        // The metadata of all frozen pieces before this piece.        
        ├── file_info.obarc                         // The list of files in all log streams.
        ├── logstream_1                             // Log stream 1.
        │   ├── file_info.obarc                    // The list of files in log stream 1.
        │   ├── log
        │   │   └── 1.obarc                         // The archive file in log stream 1.  
        │   └── schema_meta                         // The metadata of data dictionaries. This file is generated only in log stream 1.
        │       └── 1677588501408765915.obarc                           
        └── logstream_1001                          // Log stream 1001.
            ├── file_info.obarc                      // The list of files in log stream 1001.
            └── log
                └── 1.obarc                          // The archive file in log stream 1001.      

Top-level log archive directories contain the following types of data:

• format.obbak: used for recording the metadata of the archive path, including the tenant that uses the path.

• check_file: used for checking the connectivity to the user log archive directory.

• rounds: the summary directory that records all rounds of log archiving.

• pieces: the summary directory that records all pieces of log archiving.

• piece_d1002r1p1: the piece directory for log archiving, which is named in the format of piece_DESTID_ROUNDID_PIECEID. Here, DESTID refers to the ID corresponding to log_archive_dest, ROUNDID refers to the ID of the log archiving round, which is a monotonically increasing integer, and PIECEID refers to the ID of the log archiving piece, which is also a monotonically increasing integer.

  A log archiving piece directory includes the following data:

  • piece_d1002r1p1_20230111T193049_20230111T193249.obarc: This file displays the ID, start time, and end time of the current piece and is used only for display purposes.

  • checkpoint: This directory is used to record the archiving checkpoints of active pieces. The ObArchiveScheduler module periodically updates the checkpoint information in this directory. Files in this directory are described as follows:

    • checkpoint.1673436649723677822.obarc: The file name contains the checkpoint SCN of the piece, for example, 1673436649723677822.

    • checkpoint_info.0.obarc: This file records the checkpoint metadata of active pieces, including tenant_id,dest_id,round_id, andpiece_id`. The metadata in a piece remains unchanged.

  • single_piece_info.obarc: This file records the metadata of the current piece.

  • tenant_archive_piece_infos.obarc: This file records the metadata of all frozen pieces in the current tenant.

  • file_info.obarc: This file records the list of log streams within the piece.

  • logstream_1: This directory records the log files of log stream 1, which is the system log stream of an OceanBase tenant.

  • logstream_1001: This directory records the log files of log stream 1001, where log streams with IDs greater than 1000 are user log streams of an OceanBase tenant.

  Additionally, each log stream backup contains three types of data:

  • file_info.obarc: This file records the list of files in the log stream.

  • log: This directory contains all the archive files of the current log stream, with file names consistent with those in the source cluster.

  • schema_meta: This directory records the metadata of data dictionaries. It is only present in the system log stream but not in user log streams.

For AWS S3 and backup media compatible with the S3 protocol, the structure of log archive directories is different from that of OSS, NFS, and Azure Blob Storage. A single archive file consists of multiple small files and corresponding metadata files. The directory structure is as follows.

log_archive_dest
    ├── ......                                   
    └── piece_d1002r1p1                             // The piece directory, which is named in the format of `piece_DESTID_ROUNDID_PIECEID`.       
        ├── ......                         					// The list of files in all log streams.
        ├── logstream_1                             // Log stream 1.
        │   ├── file_info.obarc                    // The list of files in log stream 1.
        │   ├── log
        │   │   └── 1.obarc                         // The archive file in log stream 1, which is identified by a prefix.  
        |		|       └── @APD_PART@0-32472973.obarc  // The actual data in the archive file, including the data from byte 0 to byte 32472973 in the log file. 
        |		|		    └── ......  
        |		|		    └── @APD_PART@FORMAT_META.obarc // The format of the archive file.
        |		|		    └── @APD_PART@SEAL_META.obarc   // The metadata of the archive file.      
        │   └── schema_meta                         // The metadata of the data dictionary. This file is generated only in log stream 1.
        │       └── 1677588501408765915.obarc                           
        └── logstream_1001                          // Log stream 1001.
            ├── file_info.obarc                      // The list of files in log stream 1001.
            └── log
                └── 1.obarc                          // The archive file in log stream 1001.      

In the preceding directory structure, 1.obarc indicates a single archive file that is identified by a prefix 1. The prefix and the name of the archive file are the same. An archive file contains the following three types of data:

• @APD_PART@FORMAT_META.obarc: When data is written to the archive file for the first time, the format_meta file is generated in this directory to record the format of the archive file.

• @APD_PART@0-32472973.obarc: The actual data in the archive file is written to the file named with this prefix, and the start offset and the end offset of each write are recorded in the file name.

• @APD_PART@SEAL_META.obarc: After data is written to the archive file for the last time, the seal_meta file is generated in this directory to record the metadata in the archive file.

Feature differences between OceanBase Database V4.x and V3.x/V2.2x

Log archiving

Data backup

Physical restore

References

For more information about physical backup and restore, see Backup and restore.