Backup directory structure

Directory structure of log archiving

Log archiving allows users to specify the time interval for splitting directories through the PIECE_SWITCH_INTERVAL parameter. By default, directories are split by day: the log data directory for each day corresponds to one log archive piece.

For backup media such as NFS, OSS, and Azure Blob, the log archive directory structure is as follows:

log_archive_dest
    ├── check_file
    │   └── 1002_connect_file_20230111T193049.obbak // Connectivity check file
    ├── format.obbak                                // Format metadata for the archive path
    ├── rounds                                      // Placeholder directory for rounds
    │   └── round_d1002r1_start.obarc               // Placeholder for the start of a round
    ├── pieces                                      // Placeholder directory for pieces
    │   ├── piece_d1002r1p1_start_20230111T193049.obarc // Placeholder for the start of a piece, named as piece_DESTID_ROUNDID_PIECEID_start_DATE
    │   └── piece_d1002r1p1_end_20230111T193249.obarc   // Placeholder for the end of a piece, named as piece_DESTID_ROUNDID_PIECEID_end_DATE
    └── piece_d1002r1p1                             // Piece directory. Directory name format: piece_DESTID_ROUNDID_PIECEID
        ├── file_list.0.obarc
        ├── piece_d1002r1p1_20230111T193049_20230111T193249.obarc // Records the continuous interval of the Piece
        ├── checkpoint
        ├── single_piece_info.obarc                 // Records the metadata of the current Piece
        ├── tenant_archive_piece_infos.obarc        // Records the metadata of all frozen Pieces before the current Piece
        ├── file_info.obarc                         // List of all log stream files
        ├── logstream_1                             // Log stream 1
        └── logstream_1001                          // Log stream 1001

In the directory structure above, the top-level directory contains the following data:

• format.obbak: Records the metadata of the archive path, including information about the tenants using the path.

• check_file: Used for connectivity checks of the user log archive directory.

• rounds: The summary list of Rounds in log archiving. It records the list of all Rounds.

• pieces: The summary list of Pieces in log archiving. It records the list of all Pieces.

• piece_d1002r1p1: The directory for a log archive Piece. The directory name format is piece_DESTID_ROUNDID_PIECEID. DESTID refers to the ID corresponding to log_archive_dest; ROUNDID refers to the ID of the log archive Round, which is a monotonically increasing integer; PIECEID refers to the piece_id of the log archive Piece, which is also a monotonically increasing integer.

  A log archive Piece directory contains the following data:

  • file_list.0.obarc: This file records the files and directories contained in the current directory.

  • piece_d1002r1p1_20230111T193049_20230111T193249.obarc: This file displays the ID, start time, and end time of the current Piece and is only for information display.

  • checkpoint: This directory records archive positions for active Pieces. The ObArchiveScheduler module periodically updates the position information in this directory.

  • single_piece_info.obarc: This file records the metadata of the current Piece.

  • tenant_archive_piece_infos.obarc: This file records the metadata of all frozen Pieces in the current tenant.

  • file_info.obarc: This file records the list of log streams in the Piece.

  • logstream_1: This directory records the log files of log stream 1. Log stream 1 is the system log stream of the OceanBase Database tenant.

  • logstream_1001: This directory records the log files of log stream 1001. Log streams with IDs greater than 1000 are user log streams of the OceanBase Database tenant.

Directory structure of data backup

Each data backup corresponds to a backup set. Each time a user runs ALTER SYSTEM BACKUP DATABASE, the system generates a new backup set directory that contains all data from that backup.

The data backup directory structure is as follows:

data_backup_dest
├── format.obbak                                            // Format metadata for the backup path
├── check_file
│   └── 1002_connect_file_20230111T193020.obbak             // Connectivity check file
├── backup_sets                                             // Summary directory for data backup lists, recording all data backup sets
│   ├── backup_set_1_full_end_success_20230111T193420.obbak // Full backup end placeholder
│   ├── backup_set_1_full_start.obbak                       // Full backup start placeholder
│   ├── backup_set_2_inc_start.obbak                        // Incremental backup start placeholder
│   └── backup_set_2_inc_end_success_20230111T194420.obbak  // Incremental backup end placeholder
└── backup_set_1_full                                        // Full backup set. The suffix `full` indicates a full backup, and `inc` indicates an incremental backup
    ├── file_list.0.obbak
    ├── backup_set_1_full_20230111T193330_20230111T193420.obbak // Placeholder that displays the start and end times of the full backup
    ├── single_backup_set_info.obbak                        // Metadata of the current backup set
    ├── tenant_backup_set_infos.obbak                       // Full backup set information of the current tenant
    ├── infos
    ├── logstream_1                                          // Log stream 1
    └── logstream_1001                                       // Log stream 1001

In the data backup directory, the top-level directory contains the following data:

• format.obbak: Records the metadata of the backup path.

• check_file: Used for connectivity checks of the user data backup directory.

• backup_sets: The summary directory of data backup sets. It records the list of all data backup sets.

• backup_set_1_full: This directory represents a data backup set. backup_set_1 indicates that the backup_set_id of the backup set is 1. The suffix full indicates a full backup, and inc indicates an incremental backup. Each data backup generates a corresponding backup set. After the data backup is completed, the backup set is no longer modified.

  A data backup set mainly contains the following data:

  • file_list.0.obbak: This file records the files and directories contained in the current directory.

  • backup_set_1_full_20230111T193330_20230111T193420.obbak: This file displays the ID, start time, and end time of the current backup set. This file is only for information display.

  • single_backup_set_info.obbak: This file records the metadata of the current backup set, including the backup position and dependent logs.

  • tenant_backup_set_infos.obbak: This file records the metadata of all backup sets of the current tenant.

  • infos: This directory records the metadata of the data backup set.

  • logstream_1: This directory records all data of log stream 1. Log stream 1 is the system log stream of the OceanBase Database tenant.

  • logstream_1001: This directory records all data of log stream 1001. Log streams with IDs greater than 1000 are user log streams of the OceanBase Database tenant.

Cluster-level configuration item backup directory

Each time a cluster-level configuration item backup is initiated, the system generates a cluster-level configuration item backup file in the specified directory. The directory structure is as follows:

cluster_parameters_backup_dest
├── cluster_parameter.20240710T103610.obbak # Information about cluster-level configuration items that are not set to default values. File naming format: `cluster_parameter.[timestamp]`
└── cluster_parameter.20241018T140609.obbak