Backup directory structure

Directory structure of log archiving

You can specify the time interval for splitting the log archiving directory by using the PIECE_SWITCH_INTERVAL parameter. By default, the log archiving directory is split by day. That is, the log data directory of each day corresponds to a backup piece.

For backup media such as NFS, OSS, and Azure Blob, the directory structure of log archiving is as follows:

log_archive_dest
    ├── check_file
    │   └── 1002_connect_file_20230111T193049.obbak // Connectivity check file
    ├── format.obbak                                // Format metadata for the archive path
    ├── rounds                                      // Placeholder directory for rounds
    │   └── round_d1002r1_start.obarc               // Placeholder for the start of a round
    ├── pieces                                      // Placeholder directory for pieces
    │   ├── piece_d1002r1p1_start_20230111T193049.obarc // Placeholder for the start of a piece, named as piece_DESTID_ROUNDID_PIECEID_start_DATE
    │   └── piece_d1002r1p1_end_20230111T193249.obarc   // Placeholder for the end of a piece, named as piece_DESTID_ROUNDID_PIECEID_end_DATE
    └── piece_d1002r1p1                             // Directory for a piece, named as piece_DESTID_ROUNDID_PIECEID
        ├── piece_d1002r1p1_20230111T193049_20230111T193249.obarc // Records the continuous interval of the piece
        ├── checkpoint
        ├── single_piece_info.obarc                 // Metadata of the piece
        ├── tenant_archive_piece_infos.obarc        // Metadata of all frozen pieces before the current piece
        ├── file_info.obarc                         // List of all log stream files
        ├── logstream_1                             // Log stream 1
        └── logstream_1001                          // Log stream 1001

In the directory structure above, the top-level directory contains the following data:

• format.obbak: Records the metadata of the archive path, including the tenant that uses the path.

• check_file: Used for connectivity check of the user's log archiving directory.

• rounds: A summary list of all rounds in the log archive.

• pieces: A summary list of all pieces in the log archive.

• piece_d1002r1p1: The directory for a piece of log archiving, named as piece_DESTID_ROUNDID_PIECEID. Here, DESTID is the id of log_archive_dest; ROUNDID is the id of the round of log archiving, which is a monotonically increasing integer; and PIECEID is the id of the piece of log archiving, which is also a monotonically increasing integer.

  The directory for a piece of log archiving contains the following data:

  • piece_d1002r1p1_20230111T193049_20230111T193249.obarc: This file shows the id, start time, and end time of the current piece and is only for information display.

  • checkpoint: This directory stores archive checkpoints for the active piece. The ObArchiveScheduler module periodically updates checkpoint information in this directory.

  • single_piece_info.obarc: Records the metadata of the current piece.

  • tenant_archive_piece_infos.obarc: Records the metadata of all frozen pieces in the current tenant.

  • file_info.obarc: Records the list of log stream files in the piece.

  • logstream_1: The directory for recording the log files of log stream 1, which is the system log stream of the OceanBase Database tenant.

  • logstream_1001: This directory stores log files for log stream 1001. Log streams with IDs greater than 1000 are user log streams of the OceanBase Database tenant.

Data backup directory structure

Each data backup corresponds to a backup_set. Every time the user runs ALTER SYSTEM BACKUP DATABASE, a new backup_set directory is generated, containing all the data from that backup.

The directory structure for data backups is as follows:

data_backup_dest
├── format.obbak                                            // Format metadata for the backup path
├── check_file
│   └── 1002_connect_file_20230111T193020.obbak             // Connectivity check file
├── backup_sets                                             // Summary directory for data backup lists, recording all data backup sets
│   ├── backup_set_1_full_end_success_20230111T193420.obbak // Full backup end placeholder
│   ├── backup_set_1_full_start.obbak                       // Full backup start placeholder
│   ├── backup_set_2_inc_start.obbak                        // Incremental backup start placeholder
│   └── backup_set_2_inc_end_success_20230111T194420.obbak  // Incremental backup end placeholder
└── backup_set_1_full                                       // Full backup set. A directory name ending with `full` indicates a full backup; `inc` indicates an incremental backup
    ├── backup_set_1_full_20230111T193330_20230111T193420.obbak // Placeholder showing the start and end times of the full backup
    ├── single_backup_set_info.obbak                        // Metadata of the current backup set
    ├── tenant_backup_set_infos.obbak                       // Full backup set information for the current tenant
    ├── infos
    ├── logstream_1                                          // Log stream 1
    └── logstream_1001                                       // Log stream 1001

At the top level of the data backup directory, the following data is present:

• format.obbak: Records the metadata of the backup path.

• check_file: Used for connectivity checks of the user's data backup directory.

• backup_sets: A summary directory for data backup lists, recording all data backup sets.

• backup_set_1_full: This directory represents a data backup set. If the directory name ends with full, it indicates a full backup; if it ends with inc, it indicates an incremental backup. Each data backup generates a corresponding backup set, which remains unchanged after the backup is completed.

  Within a data backup set, the following data is primarily included:

  • backup_set_1_full_20230111T193330_20230111T193420.obbak: This file displays the ID, start, and end times of the current backup set. It is only for informational purposes.

  • single_backup_set_info.obbak: Records the metadata of the current backup set, including the backup point and dependent logs.

  • tenant_backup_set_infos.obbak: Records the metadata of all existing backup sets for the current tenant.

  • infos: Records the metadata of the data backup set.

  • logstream_1: Records all data for log stream 1, which is the system log stream for the OceanBase Database tenant.

  • logstream_1001: This directory stores all data for log stream 1001. Log streams with IDs greater than 1000 are user log streams of the OceanBase Database tenant.

Cluster-level parameter backup directory

Each time a cluster-level parameter backup is initiated, the system generates a backup file in the specified directory. The specific directory structure is as follows:

cluster_parameters_backup_dest
├── cluster_parameter.20240710T103610.obbak # Backup file for non-default cluster-level parameters. File naming format: `cluster_parameter.[timestamp]`
└── cluster_parameter.20241018T140609.obbak