The backup and restore module is a core component of OceanBase Database, designed to ensure high availability and protect data security. It effectively prevents misoperations and safeguards against damages to the storage media. In the event of data loss caused by misoperations or storage media damage, you can rely on the backup and restore module to recover and restore the lost data.
Overview
The current version of OceanBase Database supports Alibaba Cloud Object Storage Service (OSS) and Network File System (NFS) as the backup media and provides the backup, restore, and cleanup features.
OceanBase Database supports tenant-level physical backup. Physical backup data includes backup data and archive log data. Therefore, physical backup consists of log archiving and data backup. The tenants mentioned in this topic are user tenants. Physical backup is not supported for sys tenants or meta tenants.
Data backup refers to the backup of data, and includes full backup and incremental backup:
Full backup refers to the backup of all macroblocks.
Incremental backup refers to the backup of macroblocks that are added or modified since the last backup.
Notice
To perform physical backup, you must first enable log archiving.
Data backup covers the following data:
Tenant information, including the tenant name, cluster name, time zone, locality, and tenant mode (MySQL or Oracle)
All user table data
Note
Data backup does not back up tenant parameters, system variables, and private system table data.
The directories created by the backup feature in the backup destination and types of files in the directories are as follows:
data_backup_dest
├── format.obbak // The format information of the backup path.
├── check_file
│ └── 1002_connect_file_20230111T193020.obbak // The connectivity check file.
├── backup_sets // The summary directory of data backup lists, which records all data backup sets lists.
│ ├── backup_set_1_full_end_success_20230111T193420.obbak // The end placeholder for a full backup.
│ ├── backup_set_1_full_start.obbak // The start placeholder for a full backup.
│ ├── backup_set_2_inc_start.obbak // The start placeholder for an incremental backup.
│ └── backup_set_2_inc_end_success_20230111T194420.obbak // The end placeholder for an incremental backup.
└── backup_set_1_full // A full backup set. A directory whose name ends with "full" is a full backup set, and a directory whose name ends with "inc" is an incremental backup set.
├── backup_set_1_full_20230111T193330_20230111T193420.obbak //The placeholder that represents the start and end time of a full backup.
├── single_backup_set_info.obbak // The metadata of the current backup set.
├── tenant_backup_set_infos.obbak // The full backup set information of the current tenant.
├── infos
│ ├── major_data_info_turn_1 // The tenant-level baseline data backup file when the turn ID is 1.
│ │ ├── tablet_log_stream_info.obbak // The file describing the mapping between tablets and log streams.
│ │ ├── tenant_major_data_macro_range_index.0.obbak // The macroblock index in the baseline data backup file.
│ │ ├── tenant_major_data_meta_index.0.obbak // The meta index in the baseline data backup file.
│ │ └── tenant_major_data_sec_meta_index.0.obbak // The mapping between the logic ID and the physical ID of the meta index in the baseline data backup file.
│ ├── minor_data_info_turn_1 // The tenant-level minor-compacted backup file when the turn ID is 1.
│ │ ├── tablet_log_stream_info.obbak // The file describing the mapping between tablets and log streams.
│ │ ├── tenant_minor_data_macro_range_index.0.obbak // The macroblock index in the minor-compacted backup file.
│ │ ├── tenant_minor_data_meta_index.0.obbak // The meta index in the minor-compacted backup file.
│ │ └── tenant_minor_data_sec_meta_index.0.obbak // The mapping between the logic ID and the physical ID of the meta index in the minor-compacted backup file.
│ ├── diagnose_info.obbak // The diagnosis information file of the backup set.
│ ├── locality_info.obbak // The tenant locality information of the current backup set.
│ └── meta_info // The tenant-level log stream metadata file, which contains the metadata of all log streams.
│ ├── ls_attr_info.1.obbak // The snapshot of log streams during backup.
│ └── ls_meta_infos.obbak // The collection of metadata of all log streams.
├── logstream_1 // Log stream 1.
│ ├── major_data_turn_1_retry_0 // The baseline data when the turn ID is 1 and retry ID is 0.
│ │ ├── macro_block_data.0.obbak // A data file sized between 512 MB and 4 GB.
│ │ ├── macro_range_index.obbak // The macro index.
│ │ ├── meta_index.obbak // The meta index.
│ │ └── sec_meta_index.obbak // The file describing the mapping between the logical ID and the physical ID.
│ ├── meta_info_turn_1_retry_0 // The metadata of log streams when the turn ID is 1 and retry ID is 0.
│ │ ├── ls_meta_info.obbak // The log stream metadata.
│ │ └── tablet_info.1.obbak // The metadata of log stream tablets.
│ ├── minor_data_turn_1_retry_0 // The minor-compacted data when the turn ID is 1 and retry ID is 0.
│ │ ├── macro_block_data.0.obbak
│ │ ├── macro_range_index.obbak
│ │ ├── meta_index.obbak
│ │ └── sec_meta_index.obbak
│ └── sys_data_turn_1_retry_0 // The system tablet data when the turn ID is 1 and retry ID is 0.
│ ├── macro_block_data.0.obbak
│ ├── macro_range_index.obbak
│ ├── meta_index.obbak
│ └── sec_meta_index.obbak
└── logstream_1001 // Log stream 1001.
├── major_data_turn_1_retry_0
│ ├── macro_block_data.0.obbak
│ ├── macro_range_index.obbak
│ ├── meta_index.obbak
│ └── sec_meta_index.obbak
├── meta_info_turn_1_retry_0
│ ├── ls_meta_info.obbak
│ └── tablet_info.1.obbak
├── minor_data_turn_1_retry_0
│ ├── macro_block_data.0.obbak
│ ├── macro_range_index.obbak
│ ├── meta_index.obbak
│ └── sec_meta_index.obbak
└── sys_data_turn_1_retry_0
├── macro_block_data.0.obbak
├── macro_range_index.obbak
├── meta_index.obbak
└── sec_meta_index.obbak
In the data backup directory, the top-level directory contains three types of data:
format.obbakandcheck_file: Check the connectivity of the data backup directory.backup_sets: The summary directory that contains all data backup sets.backup_set_1_full: This directory represents a data backup set, where the directory name ending withfullindicates a full backup, andincindicates an incremental backup. Each data backup generates a corresponding backup set, and the backup set will not be modified after the data backup is completed.A data backup set mainly includes the following data:
backup_set_1_full_20230111T193330_20230111T193420.obbak: This file records the ID, start time, and end time of the current backup set. This file is used only for display purposes.single_backup_set_info.obbak: This file records the metadata of the current backup set, including the backup timestamp, dependent logs, and other information.tenant_backup_set_infos.obbak: This file records the metadata of all existing backup sets for the current tenant.infos: This directory records the metadata of the data backup set.logstream_1: This directory records all the data of log stream 1, which is the system log stream of an OceanBase Database tenant.logstream_1001: This directory records all the data of log stream 1001, where log streams with numbers greater than 1000 are user log streams of an OceanBase Database tenant.
In addition, each log stream backup has four types of directories. Directories with
retryindicate log stream-level retries, and directories withturnindicate tenant-level retries.meta_info_xx: This directory records log stream metadata and tablet metadata.sys_data_xx: This directory records the data of internal system tablets in log streams.minor_data_xx: This directory records the minor-compacted data of regular tablets.major_data_xx: This directory records the baseline data of regular tablets.
Log archiving refers to the automatic backup of log data. OBServer nodes regularly archive log data to the specified backup path without manual triggering.
Directories at the backup destination and file types under each directory:
log_archive_dest
├── check_file
│ └── 1002_connect_file_20230111T193049.obbak // The connectivity check file.
├── format.obbak // The format information of the backup path.
├── rounds // The round placeholder directory.
│ └── round_d1002r1_start.obarc // The round start placeholder.
├── pieces // The piece placeholder directory.
│ ├── piece_d1002r1p1_start_20230111T193049.obarc // The piece start placeholder, in the format of piece_DSETID_ROUNDID_PIECEID_DATE_start.
│ └── piece_d1002r1p1_end_20230111T193249.obarc // The piece end placeholder, in the format of piece_DSETID_ROUNDID_PIECEID_DATE_end.
└── piece_d1002r1p1 // The piece directory, named in the format of piece_DSETID_ROUNDID_PIECEID.
├── piece_d1002r1p1_20230111T193049_20230111T193249.obarc // The contiguous interval of a piece.
├── checkpoint
│ └── checkpoint_info.0.obarc // The archiving progress of a piece that is being archived.
├── single_piece_info.obarc // The metadata of the piece.
├── tenant_archive_piece_infos.obarc // The metadata of all frozen pieces before this piece.
├── file_info.obarc // The list of files in all log streams.
├── logstream_1 // Log stream 1.
│ ├── file_info.obarc // The list of files in log stream 1.
│ ├── log
│ │ └── 1.obarc // The archive files in log stream 1.
│ └── schema_meta // The metadata of data dictionaries. This file is generated only in log stream 1.
│ └── 1677588501408765915.obarc
└── logstream_1001 // Log stream 1001.
├── file_info.obarc // The list of files in log stream 1001.
└── log
└── 1.obarc // The archive files in log stream 1001.
In the log archiving directory, the top-level directory contains three types of data:
format.obbakandcheck_file: Check the connectivity of the log archive directory.rounds: The summary directory that records all rounds of log archiving.pieces: The summary directory that records all pieces of log archiving.piece_d1002r1p1: The piece directory for log archiving, which is named in the format ofpiece_DSETID_ROUNDID_PIECEID. Here,DSETIDrefers to the ID corresponding tolog_archive_dest,ROUNDIDrefers to the ID of the log archiving round, which is a monotonically increasing integer, andPIECEIDrefers to the ID of the log archiving piece, which is also a monotonically increasing integer.A log archiving piece directory includes the following data:
piece_d1002r1p1_20230111T193049_20230111T193249.obarc: This file displays the ID, start time, and end time of the current piece and is used only for display purposes.checkpoint: This directory is used to record the archiving timestamps of active pieces. The ObArchiveScheduler module periodically updates the timestamp information in this directory.single_piece_info.obarc: This file records the metadata of the current piece.tenant_archive_piece_infos.obarc: This file records the metadata of all frozen pieces in the current tenant.file_info.obarc: This file records the list of log streams within the piece.logstream_1: This directory records the log files of log stream 1, which is the system log stream of an OceanBase Database tenant.logstream_1001: This directory records the log files of log stream 1001, where log streams with numbers greater than 1000 are user log streams of an OceanBase Database tenant.
Additionally, each log stream backup contains three types of data:
file_info.obarc: This file records the list of files in the log stream.log: This directory contains all the archive files of the current log stream, with file names consistent with those in the source cluster.schema_meta: This directory records the metadata of data dictionaries. It is only present in the system log stream but not in user log streams.
OceanBase Database supports tenant-level and table-level restore. Tenant-level restore is the process of rebuilding a new tenant based on existing backup data. The entire restore process can be completed after you execute the ALTER SYSTEM RESTORE TENANT statement. The tenant-level restore process includes the restore and recovery of tenant system tables and user tables. Restore returns the required baseline data to the OBServer node of the destination tenant. Recovery returns the logs corresponding to the baseline data to the OBServer node.
Table-level restore is the process of restoring a specified table from backup data to an existing tenant, which can be the source tenant, another tenant in the same cluster, or a tenant in a different cluster.
Feature differences between OceanBase Database V4.x and V3.x/V2.2x
Log archiving
| Feature | V3.x/V2.2x | V4.x |
|---|---|---|
| Log archiving level | Cluster level | Tenant level |
| Log archiving granularity | Partition | Log stream |
| Required privileges | Operations such as setting the archive path, enabling archiving, and viewing the archive progress can be performed only in the sys tenant. |
These operations can be performed either in the sys tenant or by an administrator user in a user tenant. |
| Usage |
|
You can use the ALTER SYSTEM SET LOG_ARCHIVE_DEST statement to set the archive path and the piece switching interval for a tenant. The default piece switching interval is 1d, which indicates one day. |
| Log splitting | Log splitting can be disabled and is disabled by default. | Log splitting must be enabled, and the default piece switching interval is one day. |
| Setting the lag of log archiving | You can use the ALTER SYSTEM SET LOG_ARCHIVE_CHECKPOINT_INTERVAL statement. |
You can use the ALTER SYSTEM SET ARCHIVE_LAG_TARGET statement. |
Result returned by the ALTER SYSTEM ARCHIVELOG statement in the sys tenant |
If archiving is enabled for all tenants in the current cluster, archiving is automatically enabled for new tenants created later. | If archiving is enabled for all tenants in the current cluster, archiving is not automatically enabled for new tenants created later. |
| Compression of archive logs | You can use the ALTER SYSTEM SET BACKUP_LOG_ARCHIVE_OPTION statement. |
Not supported |
| Archive-related views | The following three archive-related views are available:
|
The following eight archive-related views are available:
|
| Archive media | SSD only | HDD or SSD |
| Number of archive files | The number of archive files is proportional to the number of partitions. In a scenario with millions of partitions, a large number of small files are generated. | The number of files is small and is irrelevant to the number of partitions. |
| Standby tenant archiving | Not supported | Supported |
Data backup
| Feature | V3.x/V2.2x | V4.x |
|---|---|---|
| Backup level | Cluster level | Tenant level |
| Required privileges | Operations such as setting a backup path, starting backup, and viewing the backup progress can be performed only in the sys tenant. |
These operations can be performed either in the sys tenant or by an administrator user in a user tenant. |
| Setting a backup path | You can use the ALTER SYSTEM SET BACKUP_DEST statement to set a backup path for a cluster. |
You can use the ALTER SYSTEM SET DATA_BACKUP_DEST statement to set a backup path for a tenant. |
| Backing up data of a specified path | You can use the ALTER SYSTEM BACKUP TENANT tenant_name_list TO backup_destination; statement to initiate the backup from the sys tenant. |
Not supported |
| BACKUP PLUS ARCHIVELOG | Not supported | Supported |
| Storage space expansion | Snapshot points are retained during backup, which causes storage space expansion during backup. | Snapshot points are not retained, thereby avoiding the storage space expansion issue. |
| Standby tenant backup | Not supported | Supported |
| Views | The following five backup-related views are available:
|
The following ten backup-related views are available:
|
Physical restore
| Feature | V3.x/V2.2x | V4.x |
|---|---|---|
| Data path | The cluster-level backup path must be specified in the restore command. | Both the data backup path and log archive path must be specified in the restore command. |
| Restore concurrency settings | Execute the ALTER SYSTEM SET RESTORE_CONCURRENCY statement to set the restore concurrency before you run the restore command. |
Specify concurrency in the restore command. |
| Key management |
|
|
| Roles of restored tenants | Primary tenants, namely primary databases | Standby tenants, namely standby databases |
| Upgrade | Tenants are automatically upgraded during the restore process. | You need to manually upgrade the tenants after they are restored. |
| Allowlist-based restore (restore of specified tables in the tenant) | Supported | Not supported |
| Table-level restore | Not supported | Supported in V4.2.1 and later |
Restore by using the ADD RESTORE SOURCE statement |
Supported | Not supported |
References
For more information about physical backup and restore, see Backup and restore.