Overview of physical backup and restore

Backup and restore is a core component of the high availability feature of OceanBase Database. It is primarily used to ensure data security, including protection against storage medium damage and user errors. If data is lost due to storage medium damage or user errors, the data can be restored through the restore process.

Overview

The backup and restore module of OceanBase Database provides backup, restore, and cleanup features.

OceanBase Database supports tenant-level physical backup. Physical backup consists of data backup and log archiving, so it is composed of these two features. Here, tenant refers to the user tenant, and physical backup is not supported for the sys tenant and Meta tenant.

Data backup refers to the feature of backing up data. This feature is divided into full backup and incremental backup:

• Full backup refers to backing up all macroblocks.

• Incremental backup refers to backing up macroblocks added or modified after the last backup.

The data backed up by data backup includes the following:

• Tenant information, including the tenant name, cluster name, time zone (timezone), replica distribution (locality), and compatibility mode (MySQL-compatible or Oracle-compatible) of the tenant.

• All user table data

  Note

  Data backup backs up system variables and tenant parameters, but does not back up cluster-level parameters and private system table data.

Log archiving refers to the automatic archiving of log data. OBServer nodes periodically archive log data to the specified backup path. This action is fully automatic and does not require external triggering. After the log archiving service is enabled, the data dictionary is also archived.

The overall architecture of physical restore is as follows:

1. Physical restore supports tenant-level restore and table-level restore.

   • Tenant-level restore: Tenant-level restore is the process of rebuilding a new tenant based on existing data backups. Tenant restore ensures global consistency across tables and partitions.

   • Table-level restore: Table-level restore is the process of restoring a user-specified table from backup data to an existing tenant. The existing tenant can be the same as the original tenant, a different tenant in the same cluster, or a tenant in a different cluster.

2. Tenant-level restore supports full restore and fast restore.

   Notice

   Tenants restored by using fast restore cannot be manually initiated for major compactions, cannot be backed up, and cannot be switched to or failed over to primary databases. They can only exist as standby databases.

   • Full restore: Full restore refers to restoring macroblock data and incremental logs. After all data is restored from the backup medium to the local environment, the restored tenant can provide services. The full restore process includes the restore and recover processes for tenant system tables and user tables. Restore refers to restoring the baseline data required for the restore to the OBServer nodes of the target tenant. Recover refers to restoring the logs corresponding to the baseline to the corresponding OBServer nodes.

   • Fast restore: Fast restore refers to providing services to users without restoring macroblock data. This reduces restore wait time and lowers user costs.

3. Select a restore time point for your physical restore.

   • Complete restore: No restore timestamp is specified.

   • Incomplete restore with a specified SCN or timestamp: SCN is the precise version number of OceanBase Database. In Oracle-compatible mode, timestamps are precise to nanoseconds, with no loss of precision. In MySQL-compatible mode, timestamps are precise to microseconds, with loss of precision after microseconds.

For more information about the physical restore process, see Restore process.

Backup media requirements

OceanBase Database supports the following backup media: Alibaba Cloud OSS, NFS, Azure Blob, AWS S3, and object storage services that are compatible with the S3 protocol (such as Huawei OBS, Google GCS, and Tencent Cloud COS). Some backup media require meeting certain requirements before they can be used.

SDK version requirements

The following table describes the correspondence between the object storage SDK version and the observer version.

Interface requirements

• Alibaba Cloud OSS:

  • The following table lists the interfaces supported by Alibaba Cloud OSS.

    Interface name	Description
    PutObject	Uploads a single object.
    DeleteObject	Deletes a single object.
    DeleteObjects	Deletes multiple objects.
    GetObject	Retrieves an object.
    ListObjects	Lists all objects in the bucket (strong consistency required).
    HeadObject	Retrieves the metadata of an object.
    AppendObject	Uploads an object in append mode.
    PutObjectTagging (optional)	Sets or updates the tags of an object.
    GetObjectTagging (optional)	Retrieves the tags of an object.
    InitiateMultipartUpload	Initializes a multipart upload.
    UploadPart	Uploads a part.
    CompleteMultipartUpload	Combines uploaded parts into a single object.
    AbortMultipartUpload	Cancels a multipart upload and deletes the uploaded parts.
    ListMultipartUploads	Lists the information of multipart uploads that have been initialized but not completed or terminated.
    ListParts	Lists the information of uploaded parts in an upload task.

  • Only V1 signature algorithm is supported.

• NFS: The version must be 3 or later.

• Object storage services that are compatible with the S3 protocol (such as Huawei OBS, Google GCS, and Tencent Cloud COS):

  1. The following table lists the S3 APIs that must be supported.

     Interface name	Description
     PutObject	Uploads a single object.
     DeleteObject	Deletes a single object.
     DeleteObjects	Deletes multiple objects.
     GetObject	Downloads a single object.
     ListObjects	Lists all objects in a path.
     HeadObject	Retrieves the metadata of an object.
     PutObjectTagging (optional)	Sets the tags of an object.
     GetObjectTagging (optional)	Retrieves the tags of an object.
     CreateMultipartUpload	Initializes a multipart upload.
     UploadPart	Uploads a single part.
     CompleteMultipartUpload	Combines uploaded parts into a single object.
     AbortMultipartUpload	Cancels a multipart upload and deletes the uploaded parts.
     ListMultipartUploads	Lists the uploaded parts.
     ListParts	Lists the information of uploaded parts in an upload task.

  2. Virtual-hosted–style object access URLs must be supported. For more information about Virtual-hosted–style requests, see AWS S3 documentation.

Before you select a backup media, you can run the test_io_device command in the ob_admin tool to verify whether the I/O interfaces and current I/O permissions provided by the backup media meet the requirements for backup and restore. You can also run the io_adapter_benchmark command in the ob_admin tool to view the read and write performance of OBServer nodes to the backup media, which serves as a reference for backup performance. For more information about the test_io_device and io_adapter_benchmark commands, see test_io_device and io_adapter_benchmark.

Directory structure

Data backup directory

The directory created by the data backup function at the backup destination and the types of files saved in each directory are as follows:

data_backup_dest
├── format.obbak                                            // The format information of the backup path.
├── check_file
│   └── 1002_connect_file_20230111T193020.obbak             // The connectivity check file.
├── backup_sets                                             // The summary directory of the data backup list, recording all the data backup sets.
│   ├── backup_set_1_full_end_success_20230111T193420.obbak // The full backup end placeholder.
│   ├── backup_set_1_full_start.obbak                       // The full backup start placeholder.
│   ├── backup_set_2_inc_start.obbak                        // The incremental backup start placeholder.
│   └── backup_set_2_inc_end_success_20230111T194420.obbak  // The incremental backup end placeholder.
└── backup_set_1_full                                        // The full backup set. Files ending with full indicate full backups, and those ending with inc indicate incremental backups.
    ├── backup_set_1_full_20230111T193330_20230111T193420.obbak // Placeholder showing the start and end times of the full backup.
    ├── single_backup_set_info.obbak                        // The metadata of the current backup set.
    ├── tenant_backup_set_infos.obbak                       // The full backup set information of the current tenant.
    ├── infos
    │   ├── table_list                                      // The list of full tables.
    │   │   ├── table_list.1702352553000000000.1.obbak      // Table name file 1.
    │   │   ├── table_list.1702352553000000000.2.obbak     // Table name file 2.
    │   │   └── table_list_meta_info.1702352553000000000.obbak  // The metadata of the table name files.
    │   ├── major_data_info_turn_1                           // The tenant-level backup files under major turn 1.
    │   │   ├── tablet_log_stream_info.obbak                 // The mapping file between tablets and log streams.
    │   │   ├── tenant_major_data_macro_range_index.0.obbak  // The macro block index of major data.
    │   │   ├── tenant_major_data_meta_index.0.obbak         // The meta index of major data.
    │   │   └── tenant_major_data_sec_meta_index.0.obbak     // The mapping file between the logical ID and physical ID of major data.
    │   ├── minor_data_info_turn_1                           // The tenant-level backup files under minor turn 1.
    │   │   ├── tablet_log_stream_info.obbak                 // The mapping file between tablets and log streams.
    │   │   ├── tenant_minor_data_macro_range_index.0.obbak  // The macro block index of minor data.
    │   │   ├── tenant_minor_data_meta_index.0.obbak         // The meta index of minor data.
    │   │   └── tenant_minor_data_sec_meta_index.0.obbak     // The mapping file between the logical ID and physical ID of minor data.
    │   ├── diagnose_info.obbak                              // The diagnostic information file of the backup set.
    |   ├── tenant_parameter.obbak                           // The tenant-level parameters of the current tenant that are not default settings.
    │   ├── locality_info.obbak                              // The locality information of the current backup set, including the resource configuration information and replica distribution information of the tenant.
    │   └── meta_info                                        // The log stream metadata files of the tenant level, containing the metadata of all log streams.
    │       ├── ls_attr_info.1.obbak                         // The snapshot of the log stream list at the time of backup.
    │       └── ls_meta_infos.obbak                          // The collection of metadata of all log streams.
    ├── logstream_1                                          // The 1st log stream.
    │   ├── major_data_turn_1_retry_0                        // The baseline data under turn 1, retry 0.
    │   │   ├── macro_block_data.0.obbak                     // A data file with a size ranging from 512 MB to 4 GB.
    │   │   ├── macro_range_index.obbak                      // The macro index.
    │   │   ├── meta_index.obbak                             // The meta index.
    │   │   └── sec_meta_index.obbak                         // The mapping file between the logical ID and physical ID.
    │   ├── meta_info_turn_1_retry_0                         // The log stream metadata file under turn 1, retry 0.
    │   │   ├── ls_meta_info.obbak                           // The log stream metadata.
    │   │   └── tablet_info.1.obbak                          // The list of log stream tablet metadata.
    │   ├── minor_data_turn_1_retry_0                        // The minor data under turn 1, retry 0.
    │   │   ├── macro_block_data.0.obbak
    │   │   ├── macro_range_index.obbak
    │   │   ├── meta_index.obbak
    │   │   └── sec_meta_index.obbak
    │   └── sys_data_turn_1_retry_0                          // The system tablet data under turn 1, retry 0.
    │       ├── macro_block_data.0.obbak
    │       ├── macro_range_index.obbak
    │       ├── meta_index.obbak
    │       └── sec_meta_index.obbak
    └── logstream_1001                                       // The 1001st log stream.
        ├── major_data_turn_1_retry_0
        │   ├── macro_block_data.0.obbak
        │   ├── macro_range_index.obbak
        │   ├── meta_index.obbak
        │   └── sec_meta_index.obbak
        ├── meta_info_turn_1_retry_0
        │   ├── ls_meta_info.obbak
        │   └── tablet_info.1.obbak
        ├── minor_data_turn_1_retry_0
        │   ├── macro_block_data.0.obbak
        │   ├── macro_range_index.obbak
        │   ├── meta_index.obbak
        │   └── sec_meta_index.obbak
        └── sys_data_turn_1_retry_0
            ├── macro_block_data.0.obbak
            ├── macro_range_index.obbak
            ├── meta_index.obbak
            └── sec_meta_index.obbak

In the data backup directory, the top-level directory contains the following data:

• format.obbak: used to record the metadata of the backup path.

• check_file: used for connectivity check of the user data backup directory.

• backup_sets: the summary directory of the data backup list, recording the list of all data backup sets.

• backup_set_1_full: this directory represents a data backup set. A directory name ending with full indicates a full backup, and one ending with inc indicates an incremental backup. Each data backup generates a corresponding backup set, which is no longer modified after the data backup is completed.

  In a data backup set, the following data is mainly included:

  • backup_set_1_full_20230111T193330_20230111T193420.obbak: this file displays the ID, start time, and end time of the current backup set. This file is for display only.

  • single_backup_set_info.obbak: this file records the metadata of the current backup set, including the backup point and the dependent logs.

  • tenant_backup_set_infos.obbak: this file records the metadata of all existing backup sets in the current tenant.

  • infos: this directory records the metadata of the data backup set.

  • logstream_1: this directory records all data of log stream 1, which is the system log stream of the OceanBase Database tenant.

  • logstream_1001: this directory records all data of log stream 1001, which is the user log stream of the OceanBase Database tenant.

  In addition, each log stream backup contains four directories. Directories containing retry indicate log stream-level retries, while directories containing turn indicate tenant-level retries:

  • meta_info_xx: this directory records the LS metadata and tablet metadata.
  • sys_data_xx: this directory records the data of the internal system tablets in the LS.
  • minor_data_xx: this directory records the dump data of ordinary tablets.
  • major_data_xx: this directory records the baseline data of ordinary tablets.

Backup directory for cluster-level parameters

Every time a backup of cluster-level parameters is initiated, the system generates a backup file in the specified directory. The specific directory structure is as follows:

cluster_parameters_backup_dest
├── cluster_parameter.20240710T103610.obbak # Information of non-default cluster-level parameters, file naming format: `cluster_parameter.[timestamp]`
└── cluster_parameter.20241018T140609.obbak

Log archive directory

For backup media such as NFS, OSS, and Azure Blob, the log archive directory and the types of files saved in each directory are as follows:

log_archive_dest
    ├── check_file
    │   └── 1002_connect_file_20230111T193049.obbak // Connectivity check file
    ├── format.obbak                                // Formatted information of the backup path
    ├── rounds                                      // Placeholder directory for Rounds
    │   └── round_d1002r1_start.obarc               // Placeholder for the start of a Round
    ├── pieces                                      // Placeholder directory for Pieces
    │   ├── piece_d1002r1p1_start_20230111T193049.obarc // Placeholder for the start of a Piece, named piece_DESTID_ROUNDID_PIECEID_start_DATE
    │   └── piece_d1002r1p1_end_20230111T193249.obarc   // Placeholder for the end of a Piece, named piece_DESTID_ROUNDID_PIECEID_end_DATE
    └── piece_d1002r1p1                             // Directory for a Piece, named piece_DESTID_ROUNDID_PIECEID
        ├── piece_d1002r1p1_20230111T193049_20230111T193249.obarc // Records the continuous interval of the Piece
        ├── checkpoint
        │   └── checkpoint.1673436649723677822.obarc   // Records checkpoint_scn information, with the filename format of checkpoint.checkpoint_scn
        │   └── checkpoint_info.0.obarc             // Records the metadata of the checkpoint
        ├── single_piece_info.obarc                 // Records the metadata of the Piece
        ├── tenant_archive_piece_infos.obarc        // Records the metadata of all frozen Pieces before the current Piece
        ├── file_info.obarc                         // List of all log stream files
        ├── logstream_1                             // Log stream 1
        │   ├── file_info.obarc                    // List of files in log stream 1
        │   ├── log
        │   │   └── 1.obarc                         // Archive files under log stream 1
        │   └── schema_meta                         // Records the metadata of the data dictionary, generated only for log stream 1
        │       └── 1677588501408765915.obarc
        └── logstream_1001                          // Log stream 1001
            ├── file_info.obarc                      // List of files in log stream 1001
            └── log
                └── 1.obarc                          // Archive files under log stream 1001

In the log archive directory above, the top-level directory contains the following data:

• format.obbak: used to record the metadata of the archive path, including information such as the tenant using the path.

• check_file: used for connectivity check of the user log archive directory.

• rounds: the summary list of all Rounds in the log archive, recording the list of all Rounds.

• pieces: the summary list of all Pieces in the log archive, recording the list of all Pieces.

• piece_d1002r1p1: the Piece directory in the log archive, named piece_DESTID_ROUNDID_PIECEID. Here, DESTID refers to the id corresponding to log_archive_dest; ROUNDID refers to the id of the log archive Round, which is a monotonically increasing integer; and PIECEID refers to the id of the log archive Piece, also a monotonically increasing integer.

  In the directory for a log archive Piece, the following data is included:

  • piece_d1002r1p1_20230111T193049_20230111T193249.obarc: this file displays the id, start time, and end time of the current Piece and is for display only.

  • checkpoint: the directory for recording the archive point of an active Piece. The ObArchiveScheduler module periodically updates the archive point information in this directory. Specifically:

    • checkpoint.1673436649723677822.obarc: the filename records the checkpoint_scn information of the Piece, with the naming format checkpoint.checkpoint_scn, where 1673436649723677822 is the corresponding checkpoint_scn.

    • checkpoint_info.0.obarc: this file records the metadata of the active Piece's checkpoint, including tenant_id, dest_id, round_id, piece_id, and other information. The metadata remains unchanged within a Piece.

  • single_piece_info.obarc: this file records the metadata of the current Piece.

  • tenant_archive_piece_infos.obarc: this file records the metadata of all frozen Pieces in the current tenant.

  • file_info.obarc: this file records the list of log streams in the Piece.

  • logstream_1: this directory records the log files of log stream 1, which is the system log stream of the OceanBase Database tenant.

  • logstream_1001: this directory records the log files of log stream 1001, which is the user log stream of the OceanBase Database tenant.

  In addition, each log stream backup has three types of data:

  • file_info.obarc: this file records the list of files in the log stream.

  • log: this directory stores all archive files of the current log stream, with filenames consistent with those of the source cluster's log files.

  • schema_meta: this directory records the metadata of the data dictionary. It exists only for system log streams and is not present in user log streams.

For backup media such as AWS S3 and those accessible via S3-compatible protocols, the directory structure for log archiving differs from that of OSS and NFS. Each archive file is composed of multiple small files and corresponding metadata files. The specific directory structure is as follows:

log_archive_dest
    ├── ......
    └── piece_d1002r1p1                             // Piece directory. The directory name is in the format of piece_DESTID_ROUNDID_PIECEID.
        ├── ......                         					// All log stream files.
        ├── logstream_1                             // Log stream 1.
        │   ├── file_info.obarc                    // Log stream file list for log stream 1.
        │   ├── log
        │   │   └── 1.obarc                         // Archive file for log stream 1. The archive file is identified by a prefix.
        |		|       └── @APD_PART@0-32472973.obarc  // Actual data in the archive file. This file records data from the 0th to 32472973rd byte of the log file.
        |		|		    └── ......
        |		|		    └── @APD_PART@FORMAT_META.obarc // Archive file format.
        |		|		    └── @APD_PART@SEAL_META.obarc   // Archive file metadata.
        │   └── schema_meta                         // Metadata of the data dictionary. This file is generated only for log stream 1.
        │       └── 1677588501408765915.obarc
        └── logstream_1001                          // Log stream 1001.
            ├── file_info.obarc                      // Log stream file list for log stream 1001.
            └── log
                └── 1.obarc                          // Archive file for log stream 1001.

In the log archive directory above, 1.obarc is an archive file. Each archive file is identified by a prefix. The prefix name is the same as the archive file name. Each archive file contains the following three types of data:

• @APD_PART@FORMAT_META.obarc: When you write data to an archive file for the first time, the system writes the format_meta file to the directory. This file records the format of the archive file.

• @APD_PART@0-32472973.obarc: Actual data is written to a file identified by the prefix. The file name records the starting offset and ending offset of the data written to the file.

• @APD_PART@SEAL_META.obarc: After you write data to an archive file for the last time, the system generates the seal_meta file in the directory. This file records the metadata of the archive file.

Differences from V3.x/V2.x features

Log archiving

Data backup

Physical restore

References

For more detailed information about physical backup and restore, see Backup and restore.