Prepare for backup

Before performing a data backup, you must complete some preparatory tasks, including configuring the backup destination and backup key.

Configure the backup destination

When configuring the backup destination, note that the sys tenant does not contain user data; it is a management tenant used for cluster administration and does not support backup and recovery. Therefore, the sys tenant does not require the DATA_BACKUP_DEST parameter to be configured.

Considerations

When configuring the backup destination, each tenant's backup path must be set to an independent empty directory. Different tenants cannot have the same backup path configured.

Procedure

1. Log in to the database as a tenant administrator of the sys tenant or a user tenant.

   Note

   The administrator user in MySQL-compatible mode is the root user, and the administrator user in Oracle-compatible mode is the SYS user.

2. Configure the backup destination.

   • Configure the backup destination for a specified tenant in the sys tenant.

     ALTER SYSTEM SET DATA_BACKUP_DEST= 'data_backup_path' TENANT = mysql_tenant;
     

   • Configure the backup destination for a user tenant.

     The statement is as follows:

     ALTER SYSTEM SET DATA_BACKUP_DEST= 'data_backup_path';
     

   Notice

   For scenarios upgrading from OceanBase Database V4.0.x to V4.1.0, the backup path needs to be changed after the version upgrade. For scenarios upgrading from OceanBase Database V4.1.x to V4.2.0, the backup path does not need to be changed after the version upgrade.

   Currently, OceanBase Database supports the following backup media: NFS, Alibaba Cloud OSS, Azure Blob, AWS S3, and object storage compatible with the S3 protocol, such as Huawei OBS, Google GCS, and Tencent Cloud COS. Some backup media may require meeting certain basic requirements before they can be used. For specific requirements of each backup medium, see Backup media requirements in Overview of physical backup and restore.

   Detailed settings are described below.

   Alibaba Cloud OSS

   NFS

   AWS S3

   Object storage compatible with the S3 protocol

   Azure Blob

   When using OSS as the backup medium, in addition to setting the backup path, you can also configure the cleanup mode for backup files using the delete_mode parameter. The delete_mode parameter currently supports delete mode and tagging mode. If the delete_mode parameter is not explicitly configured, delete mode is used by default.

   Furthermore, OSS supports specifying a checksum algorithm to verify the integrity of backup data using the checksum_type parameter. Supported values are md5 and no_checksum. If the checksum_type parameter is not explicitly configured, the default value is md5.

   OceanBase Database also supports using an Alibaba Cloud OSS bucket with a configured compliance retention policy as the backup path. When configuring the backup path, you can use the enable_worm parameter to specify whether OceanBase Database should follow the bucket's retention policy (WORM policy) for write and delete operations on this path. This parameter supports the values true and false. If the enable_worm parameter is not explicitly configured, the default is false. Setting enable_worm=true requires explicitly configuring checksum_type=md5 as well. Once set, this parameter cannot be changed.

   Notice

   OceanBase Database will not automatically set up your compliance retention policy. Before configuring the backup path, ensure the OSS bucket has the correct retention policy (WORM policy) set and is locked.

   For detailed descriptions of each parameter, see SET DATA_BACKUP_DEST.

   Notice

   When using object storage as the backup medium, parameters for the object storage path are separated by the & symbol. Ensure the parameter values you enter contain only uppercase and lowercase letters, numbers, /-_$+=, and wildcard characters. Entering characters other than those listed above may cause the setup to fail.

   An example is as follows:

   • When using OSS as the backup medium, the system tenant sets the backup path for the specified tenant mysql_tenant, configures the cleanup mode for backup files, and specifies the verification algorithm as md5.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='oss://oceanbase-test-bucket/backup/data?host=****.aliyun-inc.com&access_id=****&access_key=****&delete_mode=delete&checksum_type=md5' TENANT = mysql_tenant;
     

     If you do not want to verify the integrity of the backup data, you can set the checksum_type parameter to no_checksum, as shown in the following example:

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='oss://oceanbase-test-bucket/backup/data?host=****.aliyun-inc.com&access_id=****&access_key=****&delete_mode=delete&checksum_type=no_checksum' TENANT = mysql_tenant;
     

   • When using OSS as the backup medium, the system tenant sets the backup path for the specified tenant mysql_tenant, configures the cleanup mode for backup files, specifies the write and delete methods for objects supported by WORM, and sets the verification algorithm to md5.

     Notice

     • OceanBase Database does not currently replace you in setting compliance retention policies. Before configuring the backup path, ensure that the OSS Bucket has a properly set retention policy (WORM policy) and is locked.
     • When using a bucket with a WORM policy configured as the backup path, it is recommended to use the tagging cleanup mode.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='oss://oceanbase-test-bucket/backup/data?host=****.aliyun-inc.com&access_id=****&access_key=****&delete_mode=tagging&checksum_type=md5&enable_worm=true' TENANT = mysql_tenant;
     

   • When using OSS as the backup medium, the user tenant sets a backup path for itself, configures the cleanup mode for backup files, and specifies the verification algorithm as md5.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='oss://oceanbase-test-bucket/backup/data?host=****.aliyun-inc.com&access_id=****&access_key=****&delete_mode=delete&checksum_type=md5';
     

   In the example, oss:// indicates using OSS as the backup media type. The bucket name is oceanbase-test-bucket, and the path within the bucket is /backup/data. The ? character separates the path from the parameters, which are separated by &. host specifies the host address of the bucket. access_id and access_key set the access keys for OSS. The cleanup mode is set to delete, and the MD5 algorithm is used to verify the integrity of archived files. enable_worm=true indicates the system will perform write and delete operations in accordance with WORM specifications. Note that OceanBase Database does not currently replace your compliance retention strategy settings. Before configuring the backup path, ensure the OSS Bucket has a properly configured retention policy (WORM policy) and is locked.

   Notice

   When using NFS as the backup medium, note the following:

   • The value of data_backup_dest cannot be a string containing a question mark (?).
   • The value of data_backup_dest must be an absolute path. Ensure that the OBServer server has write permissions for data_backup_dest.
   • Ensure that all OBServer hosts are mounted with NFS from the same server. To ensure smooth backup, be sure to use the parameters recommended in this document for mounting NFS. For specific NFS mounting operations, see Deploy an NFS client.

   • When NFS is used as the backup medium, the system tenant sets a backup path for the specified tenant mysql_tenant.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST= 'file:///data/nfs/backup/data' TENANT = mysql_tenant;
     

   • When using NFS as the backup medium, the user tenant sets a backup path for itself.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='file:///data/nfs/backup/data';
     

   In the example, file:// indicates that NFS is used as the backup media type, and the backup path is file:///data/nfs/backup/data.

   Like OSS, AWS S3 also supports configuring the cleanup mode for backup files using the delete_mode parameter, with the same configuration method as OSS. Similarly, AWS S3 also supports specifying a checksum algorithm to verify the integrity of backup data using the checksum_type parameter. Supported values are md5 and crc32. If the checksum_type parameter is not explicitly configured, the default value is md5.

   For detailed descriptions of the delete_mode and checksum_type parameters, see SET DATA_BACKUP_DEST.

   Notice

   When using object storage as a backup medium, the parameters for the object storage path are separated by the & symbol. Ensure that the parameter values you enter contain only uppercase and lowercase letters, numbers, /-_$+=, and wildcards. Entering characters other than those listed above may cause setting failure.

   The following examples show how to configure using S3 as a backup medium:

   • When using S3 as a backup medium, the system tenant sets a backup path for the specified tenant mysql_tenant, configures the cleanup mode as delete, and specifies the checksum algorithm as crc32.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='s3://oceanbase-test-bucket/backup/data?host=s3.<region>.amazonaws.com&access_id=****&access_key=****&s3_region=****&delete_mode=delete&checksum_type=crc32' TENANT = mysql_tenant;
     

   • When using S3 as a backup medium, the user tenant sets a backup path for itself, configures the cleanup mode as delete, and specifies the checksum algorithm as crc32.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='s3://oceanbase-test-bucket/backup/data?host=s3.<region>.amazonaws.com&access_id=****&access_key=****&s3_region=****&delete_mode=delete&checksum_type=crc32';
     

   In the examples, s3:// indicates using S3 as the backup medium type, the bucket name oceanbase-test-bucket, and the path within the bucket is /backup/data. The ? is used to separate the path from the parameters, which are separated by &. host is used to set the domain name for the Amazon S3 service, access_id and access_key are used to set the access keys for the AWS service, s3_region is a required parameter when using S3 as a backup medium, indicating the region where the S3 bucket is located, the cleanup mode is delete, and the CRC32 algorithm is used to verify the integrity of backup data.

   For detailed information about the AWS S3 path format, see Appendix: AWS S3 Path Format Description.

   Since most object storages are compatible with the S3 protocol, any object storage that is compatible with S3 and meets the requirements of OceanBase Database (the object storage must be compatible with the behavior of several S3 APIs that implement OceanBase system calls) can be used as a backup medium for OceanBase Database. Access to compatible object storages, such as OBS, GCS, and COS, is provided through S3.

   For object storage services such as OBS, GCS, and COS accessed through the S3 protocol, you can also specify a checksum algorithm for verifying the integrity of backup data using the checksum_type parameter. The only supported value is md5. If the checksum_type parameter is not explicitly configured, its default value is md5. For detailed information about the checksum_type parameter, see SET DATA_BACKUP_DEST.

   Notice

   • When using object storage as the backup medium, parameters in the object storage path are separated by the & symbol. Ensure that the parameter values you enter contain only uppercase and lowercase letters, numbers, /-_$+=, and wildcard characters. Entering characters other than those listed above may cause configuration failure.
   • Some object storage services compatible with the S3 protocol may not support the default URL encoding method of the AWS S3 SDK. When accessing via the s3:// protocol, if object storage-related error codes such as 9129 or 9026 occur, it is recommended to set the cluster-level parameter ob_storage_s3_url_encode_type to specify that AWS S3 uses an RFC 3986-compliant URL encoding method when sending requests. For detailed setup instructions, refer to the Procedure and example: Use COS as the backup medium section below.

   The following example shows how to configure OBS as the backup medium:

   • When using OBS as the backup medium, the system tenant sets the backup path for the specified tenant mysql_tenant.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='s3://oceanbase-test-bucket/backup/data?host=obs.****.myhuaweicloud.com&access_id=****&access_key=****' TENANT = mysql_tenant;
     

   • When using OBS as the backup medium, the user tenant sets the backup path for itself.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='s3://oceanbase-test-bucket/backup/data?host=obs.****.myhuaweicloud.com&access_id=****&access_key=****';
     

   In the preceding examples, s3:// indicates accessing the backup destination via the S3 protocol. The bucket name is oceanbase-test-bucket, and the path within the bucket is /backup/data. The ? symbol separates the path from the parameters, which are separated by &. host sets the domain name of the OBS service, while access_id and access_key set the access keys for the OBS service.

   The following example shows how to configure GCS as the backup medium:

   • When using GCS as the backup medium, the system tenant sets the backup path for the specified tenant mysql_tenant.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='s3://oceanbase-test-bucket/backup/data?host=https://storage.googleapis.com&access_id=****&access_key=****' TENANT = mysql_tenant;
     

   • When using GCS as the backup medium, the user tenant sets the backup path for itself.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='s3://oceanbase-test-bucket/backup/data?host=https://storage.googleapis.com&access_id=****&access_key=****';
     

   In the preceding examples, s3:// indicates accessing the backup destination via the S3 protocol. The bucket name is oceanbase-test-bucket, and the path within the bucket is /backup/data. The ? symbol separates the path from the parameters, which are separated by &. host sets the domain name of the GCS service, while access_id and access_key set the access keys for the GCS service.

   The procedure and example for using COS as the backup medium are as follows:

   1. Check the value of the cluster-level parameter ob_storage_s3_url_encode_type.

      obclient> SHOW PARAMETERS LIKE '%ob_storage_s3_url_encode_type%';
      

      The query result is as follows:

      +-------+----------+----------------+----------+-------------------------------+-----------+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
      | zone  | svr_type | svr_ip         | svr_port | name                          | data_type | value   | info                                                                                                                                                                                        | section  | scope   | source  | edit_level        |
      +-------+----------+----------------+----------+-------------------------------+-----------+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
      | zone1 | observer | 172.xx.xxx.xxx |     2882 | ob_storage_s3_url_encode_type | NULL      | default | Determines the URL encoding method for S3 requests."default": Uses the S3 standard URL encoding method."compliantRfc3986Encoding": Uses URL encoding that adheres to the RFC 3986 standard. | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
      +-------+----------+----------------+----------+-------------------------------+-----------+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
      1 row in set
      

   2. If the value is default, change it to compliantRfc3986Encoding.

      obclient>ALTER SYSTEM SET ob_storage_s3_url_encode_type='compliantRfc3986Encoding';
      

      For specific information about this parameter, see ob_storage_s3_url_encode_type.

   3. Set the backup path and configure the cleanup mode for backup files using the delete_mode parameter.

      The following example shows how to set the backup path for the specified tenant mysql_tenant and configure the cleanup mode to delete when using COS as the backup medium:

      obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='s3://oceanbase-test/backup/data?host=cos.ap-xxxx.myqcloud.com&access_id=***&access_key=***&delete_mode=delete' TENANT = mysql_tenant;
      

      The following example shows how to set the backup path for the current tenant and configure the cleanup mode to delete when using COS as the backup medium:

      obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='s3://oceanbase-test/backup/data?host=cos.ap-xxxx.myqcloud.com&access_id=***&access_key=***&delete_mode=delete';
      

      In the examples, s3:// indicates accessing the backup destination using the S3 protocol. The bucket name is oceanbase-test, and the path within the bucket is /backup/archive. The ? symbol is used to separate the path from parameters, and & is used to separate parameters. host sets the COS host address, which is the endpoint (the address without the bucket name). access_id and access_key are used to set the access keys for COS, and the cleanup mode is set to delete.

   Like OSS, Azure Blob also supports the delete_mode and checksum_type parameters. The delete_mode parameter configures the cleanup mode for backup files. If not explicitly configured, the default delete mode is used. The checksum_type parameter specifies the checksum algorithm for verifying the integrity of backup files. Supported values are md5 and no_checksum. If not explicitly configured, the default value is md5.

   For more details about these parameters, see SET DATA_BACKUP_DEST.

   Examples:

   Notice

   • When using object storage as the backup medium, parameters in the object storage path are separated by the & symbol. Ensure your input parameter values contain only uppercase and lowercase letters, numbers, /-_$+=, and wildcard characters. Including other characters may cause configuration failure.
   • When using Azure Blob as the backup medium, the prefix for the host parameter must be specified as http:// or https://. Additionally, since secure transmission is enabled by default in Azure Blob, access is typically through https://. To access via http://, you need to disable the secure transmission option for Azure Blob. For detailed steps on disabling the secure transmission option, see the Azure Blob official documentation.

   • When using Azure Blob as the backup medium, the system tenant sets the backup path for the specified tenant mysql_tenant, configures the cleanup mode to delete, and specifies the checksum algorithm as md5.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='azblob://oceanbase-test-bucket/backup/data?host=http://****.blob.core.windows.net&access_id=****&access_key=****&delete_mode=delete&checksum_type=md5' TENANT = mysql_tenant;
     

   • When using Azure Blob as the backup medium, the user tenant sets the backup path for itself, configures the cleanup mode to delete, and specifies the checksum algorithm as md5.

     obclient> ALTER SYSTEM SET DATA_BACKUP_DEST='azblob://oceanbase-test-bucket/backup/data?host=http://****.blob.core.windows.net&access_id=****&access_key=****&delete_mode=delete&checksum_type=md5';
     

   In the examples, azblob:// indicates using Azure Blob as the backup medium type. The bucket name is oceanbase-test-bucket, and the path within the bucket is backup/data. The ? symbol is used to separate the path from parameters, and & is used to separate parameters. host is the host address of the bucket, which can be obtained by concatenating the container properties or the connection string of the storage account. access_id is the name of the storage account, access_key is the access key for Azure Blob, the cleanup mode is delete, and the MD5 algorithm is used to verify the integrity of archived files.

3. After successful configuration, you can confirm the detailed configuration information through views.

   The sys tenant can view this information in the CDB_OB_BACKUP_PARAMETER view, and the user tenant can view it in the DBA_OB_BACKUP_PARAMETER view.

   For information about data backup-related parameters and how to view them, see View data backup-related parameters.

Considerations and notes after configuration

1. After the data_backup_dest parameter is successfully configured, the system will create a format file in the directory where the configured destination resides by default. This file is used to verify the validity of the backup destination and ensure the integrity of the data within it. Therefore, when configuring the backup destination data_backup_dest, note the following:

   • If the format file does not exist, the directory where the configured destination resides must be empty for the parameter to be set successfully. Otherwise, the system will report an error -9080 indicating that the format file does not exist.

   • If the format file already exists, its content must pass verification for the parameter to be set successfully. Otherwise, the system will report an error -9081 indicating that the format file does not match. The content verification of the format file mainly checks whether the cluster, tenant, and backup destination type match those of the current operation.

   • During a backup task, if the format file does not exist or fails verification, the task will fail.

2. After the backup destination is successfully set, if you change the object storage key information for security or other reasons, you need to update the access_id and access_key for the existing backup paths. For specific operations, see Update object storage key information.

Backup the tenant key

Before backing up data, you also need to consider the encryption status of the source tenant. If transparent data encryption is configured for the source tenant, you must back up its key.

1. Log in to the database as the tenant administrator of the sys tenant or a user tenant.

2. Back up the key.

   • The system tenant backs up the key for a specified tenant

     ALTER SYSTEM BACKUP KEY TENANT = tenant_name TO 'backup_key_path' ENCRYPTED BY 'password';
     

   • A user tenant backs up the key for itself

     The statement is as follows:

     ALTER SYSTEM BACKUP KEY TO 'backup_key_path' ENCRYPTED BY 'password';
     

   In the statement:

   • backup_key_path: Specifies the path for backing up the key. This path is user-defined and must not be the same as the path used for data backup or log archiving.

   • password: Specifies an encryption password for the key backup path.

   The following example shows the system tenant backing up the key for tenant mysql_tenant to the path file:///data_backup_dest/key:

   obclient [(none)]> ALTER SYSTEM BACKUP KEY TENANT = mysql_tenant TO 'file:///data_backup_dest/key' ENCRYPTED BY '******';
   

3. After configuration, you can view the key backup path through a view.

   • The system tenant views the key backup path through the CDB_OB_BACKUP_STORAGE_INFO view

     SELECT * FROM oceanbase.CDB_OB_BACKUP_STORAGE_INFO;
     

   • A user tenant views the key backup path through the DBA_OB_BACKUP_STORAGE_INFO view

     MySQL-compatible mode

     Oracle-compatible mode

     The statement is as follows:

     SELECT * FROM oceanbase.DBA_OB_BACKUP_STORAGE_INFO;
     

     The statement is as follows:

     SELECT * FROM SYS.DBA_OB_BACKUP_STORAGE_INFO;
     

     For more information about the columns in the CDB_OB_BACKUP_STORAGE_INFO and DBA_OB_BACKUP_STORAGE_INFO views, see CDB_OB_BACKUP_STORAGE_INFO and DBA_OB_BACKUP_STORAGE_INFO.