Backup directory structure

Directory structure of log archiving

Log archiving can be split into directories by the PIECE_SWITCH_INTERVAL parameter, which specifies the time interval for splitting directories. By default, directories are split by day. Each day's log data directory corresponds to one backup piece.

For backup media such as NFS, OSS, and Azure Blob, the directory structure of log archiving is as follows:

log_archive_dest
    ├── check_file
    │   └── 1002_connect_file_20230111T193049.obbak // Connectivity check file
    ├── format.obbak                                // Formatted information of the backup path
    ├── rounds                                      // Placeholder directory for Rounds
    │   └── round_d1002r1_start.obarc               // Placeholder for Round start
    ├── pieces                                      // Placeholder directory for Pieces
    │   ├── piece_d1002r1p1_start_20230111T193049.obarc // Placeholder for Piece start, named as piece_DESTID_ROUNDID_PIECEID_start_DATE
    │   └── piece_d1002r1p1_end_20230111T193249.obarc   // Placeholder for Piece end, named as piece_DESTID_ROUNDID_PIECEID_end_DATE
    └── piece_d1002r1p1                             // Piece directory, named as piece_DESTID_ROUNDID_PIECEID
        ├── file_list.0.obarc
        ├── piece_d1002r1p1_20230111T193049_20230111T193249.obarc // Records the continuous interval of the Piece
        ├── checkpoint
        ├── single_piece_info.obarc                 // Records the metadata of the Piece
        ├── tenant_archive_piece_infos.obarc        // Records the metadata of all frozen Pieces before the current Piece
        ├── file_info.obarc                         // List of all log stream files
        ├── logstream_1                             // Log stream 1
        └── logstream_1001                          // Log stream 1001

In the above directory structure, the top-level directory contains the following data:

• format.obbak: Records the metadata of the archive path, including information about the tenant using the path.

• check_file: Used for connectivity check of the log archive directory.

• rounds: A summary list of Rounds for log archiving, recording all Rounds.

• pieces: A summary list of Pieces for log archiving, recording all Pieces.

• piece_d1002r1p1: The directory for a Piece of log archiving, named as piece_DESTID_ROUNDID_PIECEID. Here, DESTID refers to the id of log_archive_dest; ROUNDID refers to the id of the log archiving Round, which is a monotonically increasing integer; and PIECEID refers to the id of the log archiving Piece, which is also a monotonically increasing integer.

  Within a log archiving Piece directory, the following data is present:

  • file_list.0.obarc: Records the files and directories contained in the current directory.

  • piece_d1002r1p1_20230111T193049_20230111T193249.obarc: Displays the id, start time, and end time of the current Piece. This file is only for informational purposes.

  • checkpoint: The directory for recording the archive point of an active Piece. The ObArchiveScheduler module periodically updates the archive point information in this directory.

  • single_piece_info.obarc: Records the metadata of the current Piece.

  • tenant_archive_piece_infos.obarc: Records the metadata of all frozen Pieces in the current tenant.

  • file_info.obarc: Records the list of log stream files in the Piece.

  • logstream_1: The directory for recording the log files of log stream 1. Log stream 1 is the system log stream of the OceanBase Database tenant.

  • logstream_1001: The directory for recording the log files of log stream 1001. Log streams with ids greater than 1000 are user log streams of the OceanBase Database tenant.

Data backup directory structure

Each data backup corresponds to a backup_set. Every time the user runs ALTER SYSTEM BACKUP DATABASE, a new backup_set directory is generated, containing all the data from this backup.

The directory structure for data backups is as follows:

data_backup_dest
├── format.obbak                                            // Backup path format information
├── check_file
│   └── 1002_connect_file_20230111T193020.obbak             // Connectivity check file
├── backup_sets                                             // Summary directory for data backups, recording all data backup sets
│   ├── backup_set_1_full_end_success_20230111T193420.obbak // Full backup end placeholder
│   ├── backup_set_1_full_start.obbak                       // Full backup start placeholder
│   ├── backup_set_2_inc_start.obbak                        // Incremental backup start placeholder
│   └── backup_set_2_inc_end_success_20230111T194420.obbak  // Incremental backup end placeholder
└── backup_set_1_full                                        // Full backup set, files ending with 'full' indicate a full backup, 'inc' indicate an incremental backup
    ├── file_list.0.obbak
├── backup_set_1_full_20230111T193330_20230111T193420.obbak // Placeholder showing the start and end times of the full backup
    ├── single_backup_set_info.obbak                        // Metadata of the current backup set
    ├── tenant_backup_set_infos.obbak                       // Full backup set information for the current tenant
    ├── infos
    ├── logstream_1                                          // Log stream 1
    └── logstream_1001                                       // Log stream 1001

In the data backup directory, the top-level directory contains the following three types of data:

• format.obbak: Records the metadata of the backup path.

• check_file: Used for connectivity check of the user's data backup directory.

• backup_sets: A summary directory for data backups, recording all data backup sets.

• backup_set_1_full: This directory represents a data backup set. backup_set_1 indicates that the backup_set_id of this backup set is 1. The directory name ending with full indicates a full backup, and inc indicates an incremental backup. Each data backup generates a corresponding backup set, and after the data backup is completed, the backup set will no longer be modified.

  Within a data backup set, the following data is mainly included:

  • file_list.0.obbak: Records the files and directories contained in the current directory.

  • backup_set_1_full_20230111T193330_20230111T193420.obbak: Shows the ID, start, and end times of the current backup set. This file is only for informational purposes.

  • single_backup_set_info.obbak: Records the metadata of the current backup set, including the backup point and dependent logs.

  • tenant_backup_set_infos.obbak: Records the metadata of all existing backup sets for the current tenant.

  • infos: Records the metadata of the data backup set.

  • logstream_1: Records all data of log stream 1, which is the system log stream for the OceanBase Database tenant.

  • logstream_1001: Records all data of log stream 1001, which is the user log stream for the OceanBase Database tenant.

Cluster-level parameter backup directory

Every time a cluster-level parameter backup is initiated, the system generates a backup file in the specified directory. The specific directory structure is as follows:

cluster_parameters_backup_dest
├── cluster_parameter.20240710T103610.obbak # Non-default cluster-level parameter information. File naming format: `cluster_parameter.[timestamp]`
└── cluster_parameter.20241018T140609.obbak