Prepare for log archiving|V4.3.0| docs|Distributed Database

Prepare for log archiving

Last Updated：2025-12-02 08:08:09 Updated

This topic describes the preparations for log archiving.

Configure the degree of parallelism of log archiving (optional)

Before you enable the log archive mode, you can configure the degree of parallelism of log archiving to speed up log archiving for a tenant.

Log in to the database as the tenant administrator of the sys tenant or a user tenant.
Choose an appropriate statement to set the log_archive_concurrency parameter.

The tenant-level parameter log_archive_concurrency specifies the total number of worker threads for log archiving. The parameter value takes effect immediately without the need to restart the OBServer node. The value range is [0, 100]. The default value is 0, which means that the database uses adaptive parallelism for log archiving. We recommend that you use the default value.

Note

If the tenant has four or fewer CPU cores, we recommend that you use the default value and do not modify the value of this parameter.

You can set the parameter in the following ways:
- Adjust the degree of parallelism of log archiving for a tenant as the sys tenant
```
ALTER SYSTEM SET log_archive_concurrency = 10 TENANT = mysql_tenant;
```
- Adjust the degree of parallelism of log archiving for all user tenants as the sys tenant
```
ALTER SYSTEM SET log_archive_concurrency = 10 TENANT = all_user;
```
  or
```
ALTER SYSTEM SET log_archive_concurrency = 10 TENANT = all;
```
  Note
  
  Starting from OceanBase Database V4.2.1, TENANT = all_user and TENANT = all express the same semantics. If you want an operation to take effect on all user tenants, we recommend that you use TENANT = all_user. TENANT = all will be deprecated.
- Adjust the degree of parallelism of log archiving for a user tenant as the user tenant
```
ALTER SYSTEM SET log_archive_concurrency = 10;
```
For more information about the log_archive_concurrency parameter, see log_archive_concurrency.

Configure the destination for archiving logs

Before you start a log archiving task, you must use the ALTER SYSTEM command to set the LOG_ARCHIVE_DEST parameter. The sys tenant is a cluster management tenant that does not contain user data. It does not support backup and restore, so you do not need to configure a destination for archiving logs in the sys tenant.

The operation of configuring a destination for archiving logs mainly involves setting the LOCATION, BINDING, and PIECE_SWITCH_INTERVAL parameters.

Considerations

Procedure

Log in to the database as the tenant administrator of the sys tenant or a user tenant.

Note

The administrator in MySQL mode is the root user, and that in Oracle mode is the SYS user.
Configure the destination for archiving.
- Configure the destination for the specified tenant in the sys tenant
```
ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=archive_path [BINDING=archive_mode] [PIECE_SWITCH_INTERVAL=piece_switch_interval]' TENANT = tenant_name;
```
- Configure the destination for the current tenant
```
ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=archive_path [BINDING=archive_mode] [PIECE_SWITCH_INTERVAL=piece_switch_interval]';
```
Notice

For scenarios where OceanBase Database V4.0.x is upgraded to V4.1.0, you need to change the archive path after the upgrade. For scenarios where OceanBase Database V4.1.x is upgraded to V4.2.x, you do not need to change the archive path. In addition, when OceanBase Database V4.1.x is upgraded to V4.2.x, you can upgrade the database and archive logs at the same time.

For more information about how to configure the destination, see the following points:
- Configure the LOCATION parameter (mandatory)
  
  The LOCATION parameter specifies the destination for archiving. At present, OceanBase Database supports archiving data to NFS, Alibaba Cloud OSS, Tencent Cloud COS, and Amazon S3. Some backup media may have basic requirements. For more information, see Backup media requirements in Overview of physical backup and restore.
  
  Notice
  
  When using an object storage service as the archive destination, make sure that the access_key parameter does not contain special characters, such as commas.
  Alibaba Cloud OSS
  
  NFS
  
  Tencent Cloud COS
  
  AWS S3
  When you use OSS as the destination for archiving, apart from setting the archive path, host, access_key, and access_id, you can also set the delete_mode parameter to specify the cleanup mode for archive files.
  
  The delete_mode parameter supports two cleanup modes, delete and tagging. If you do not set this parameter, the default value delete takes effect.
  
  delete: specifies to directly delete archive files that meet the cleanup requirements.
  
  If you set this parameter, when you automatically clear archive files, the system will directly delete the files that meet the cleanup requirements.
  
  Notice
  
  This is the default cleanup mode. If you do not set the delete_mode parameter, the delete cleanup mode takes effect.
  
  tagging: specifies to set the Tag for archive files that meet the cleanup requirements and retain the files.
  
  If you set this parameter, when you automatically clear archive files, the system will set a tag for archive files that meet the cleanup requirements. The key of the tag is "delete_mode" and the value is "tagging". This allows you to manage the lifecycle of the files on OSS by using the tag later.
  
  Notice
  
  When you use object storage as the archive medium, separate the parameters in the LOCATION parameter with the & character. Make sure that the parameter values contain only uppercase and lowercase English letters, digits, and the characters /-_$+= and *. If the parameter values contain other characters, the setting may fail.
  
  If you use OSS as the archive medium, the tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination for archiving and set the delete cleanup mode as shown in the following example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=oss://oceanbase-test-bucket/backup/archive?host=***.aliyun-inc.com&access_id=***&access_key=***&delete_mode=delete' TENANT = mysql_tenant;
  
  A user can configure the destination for archiving and set the delete cleanup mode for the current tenant as shown in the following example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=oss://oceanbase-test-bucket/backup/archive?host=***.aliyun-inc.com&access_id=***&access_key=***&delete_mode=delete';
  
  In this example, oss:// indicates to use OSS as the destination for archiving. The storage bucket name is oceanbase-test-bucket, and the path in the storage bucket is /backup/archive. The parameters in the path are separated with ?. The host parameter sets the host address of the storage bucket. The access_id and access_key parameters set the access keys of OSS. The cleanup mode is set to delete.
  
  After you set the delete mode or tagging mode, for more information about how to automatically clear the archive data, see Automatically clear the backed-up data.
  Notice
  
  Consider the following points when you use NFS as the archive medium:
  
  The value of the LOCATION parameter cannot contain the question mark character (?).
  
  The value of the LOCATION parameter must be an absolute path. OBServer node must have read and write permissions for the path specified in the LOCATION parameter.
  
  Make sure that all OBServer nodes mount the same NFS server on the same mount point. To ensure smooth archiving, we recommend that you use the parameters suggested in this topic to mount NFS. For more information, see Deploy an NFS client.
  
  The tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination for archiving in the specified tenant as shown in the following example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive' TENANT = mysql_tenant;
  
  A user can configure the destination for archiving in the current tenant as shown in the following example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive';
  
  In this example, file:// indicates to use NFS as the destination for archiving. The path for archiving is /data/nfs/backup/archive.
  Notice
  
  Consider the following points when you use COS as the archive medium:
  
  Make sure that the List Cache of the bucket is disabled. Otherwise, errors may occur in the consistency of backup data. Contact the COS technical support team to disable the List Cache of the bucket.
  
  If the bucket supports the Multi-AZ feature, the APPEND Object API is not supported. You must disable the Multi-AZ feature. Otherwise, an error will occur when you archive data.
  
  Similar to OSS, you can set the delete_mode parameter to specify the cleanup mode for archive files. The configuration method is the same as for OSS.
  
  Notice
  
  When you use object storage as the archive medium, separate the parameters in the LOCATION parameter with the & character. Make sure that the parameter values contain only uppercase and lowercase English letters, digits, and the characters /-_$+= and *. If the parameter values contain other characters, the setting may fail.
  
  The tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination for archiving and set the delete cleanup mode as shown in the following example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=cos://oceanbase-test-appid/backup/archive?host=cos.ap-***.myqcloud.com&access_id=***&access_key=***&appid=***&delete_mode=delete' TENANT = mysql_tenant;
  
  A user can configure the destination for archiving and set the delete cleanup mode for the current tenant as shown in the following example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=cos://oceanbase-test-appid/backup/archive?host=cos.ap-***.myqcloud.com&access_id=***&access_key=***&appid=***&delete_mode=delete';
  
  In this example, cos:// indicates to use COS as the destination for archiving. The storage bucket name is oceanbase-test-appid, and the path in the storage bucket is /backup/archive. The parameters in the path are separated with ?. The host parameter sets the host address of the storage bucket, which is the endpoint (without the storage bucket name). The access_id and access_key parameters set the access keys of COS. The appid parameter is an optional parameter used for identifying the Tencent cloud account. The cleanup mode is set to delete.
  Similar to OSS and COS, you can set the delete_mode parameter to specify the cleanup mode for archive files. The configuration method is the same as for OSS and COS.
  
  Notice
  
  When you use object storage as the archive medium, separate the parameters in the LOCATION parameter with the & character. Make sure that the parameter values contain only uppercase and lowercase English letters, digits, and the characters /-_$+= and *. If the parameter values contain other characters, the setting may fail.
  
  The tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination for archiving and set the delete cleanup mode as shown in the following example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=s3://oceanbase-test-bucket/backup/archive?host=s3.<region>.amazonaws.com&access_id=******&access_key=******&s3_region=******&delete_mode=delete' TENANT = mysql_tenant;
  
  A user can configure the destination for archiving and set the delete cleanup mode for the current tenant as shown in the following example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=s3://oceanbase-test-bucket/backup/archive?host=s3.<region>.amazonaws.com&access_id=******&access_key=******&s3_region=******&delete_mode=delete';
  
  In this example, s3:// indicates to use S3 as the destination for archiving. The storage bucket name is oceanbase-test-bucket, and the path in the storage bucket is /backup/archive. The parameters in the path are separated with ?. The host parameter sets the domain name of the Amazon S3 service. The access_id and access_key parameters set the access keys of AWS. The s3_region parameter is an optional parameter that specifies the region where the S3 bucket is located. The cleanup mode is set to delete.
- Configure the BINDING parameter (optional)
  
  The BINDING parameter specifies the priority mode for archive and business. At present, the Optional and Mandatory modes are supported. If you do not set this parameter, the default value Optional takes effect.
  - The Optional mode indicates that the business is prioritized. In this mode, if the archiving speed is lower than the log generation speed, the system may recycle logs that have not been archived. Then, the data stream will be interrupted.
  - The Mandatory mode indicates that archive is prioritized. In this mode, if the archiving speed is lower than the data write speed, the system will not allow data write. Then, the data stream will be interrupted.
  The tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination for archiving and set the BINDING parameter to the Mandatory mode as shown in the following example:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional' TENANT = mysql_tenant;
```
  A user can configure the destination for archiving and set the BINDING parameter to the Mandatory mode as shown in the following example:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional';
```
- Configure the PIECE_SWITCH_INTERVAL parameter (optional)
  
  The PIECE_SWITCH_INTERVAL parameter specifies the interval for switching piece, which ranges from [1d, 7d]. If this parameter is not set, the default value 1d takes effect.
  
  The tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination for archiving and set the interval to switch a log piece every other day as shown in the following example:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional PIECE_SWITCH_INTERVAL=1d' TENANT = mysql_tenant;
```
  A user can configure the destination for archiving and set the interval to switch a log piece every other day as shown in the following example:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional PIECE_SWITCH_INTERVAL=1d';
```
After the configuration, you can query the detailed parameter settings in the views.

Considerations and notes

After the parameter LOG_ARCHIVE_DEST is set, the system will create a format file in the specified directory as a validity check for the backup destination and to ensure the integrity of the data in the destination. Therefore, note the following considerations when you configure the log archive destination:

If the format file does not exist, the specified directory must be empty. Otherwise, an error -9080 will occur, indicating that the format file does not exist.
If the format file exists, the content of the file must pass the verification. Otherwise, an error -9081 will occur, indicating that the format file is mismatched.
If the format file does not exist or the verification of the format file fails, the backup task will fail.

After the archive destination is configured, incremental configuration is not supported. For example, assume that the BINDING attribute of the NFS archive path /data/nfs/backup/archive is set to Mandatory, and the PIECE_SWITCH_INTERVAL attribute is set to 1d. If you want to change the value of the PIECE_SWITCH_INTERVAL attribute to 2d but retain the setting of the BINDING attribute, you still need to specify the attribute value of the PIECE_SWITCH_INTERVAL attribute in the command. Otherwise, the default value will be used for the unspecified attribute.

In this case, you need to re-execute the following statements.

Modify the parameters of a specified tenant in the sys tenant.

obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST = 'LOCATION=file:///data/nfs/backup/archive BINDING=Mandatory PIECE_SWITCH_INTERVAL=2d' TENANT = mysql_tenant;

Modify the parameters of the current tenant in the user tenant.

obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST = 'LOCATION=file:///data/nfs/backup/archive BINDING=Mandatory PIECE_SWITCH_INTERVAL=2d';

Configure the archive delay

OceanBase Database uses the tenant-level archive_lag_target parameter to control the latency of log archiving for tenants. The parameter supports the time units of milliseconds (ms), seconds (s), minutes (m), and hours (h). The default value is 2m, which specifies the maximum interval in minutes between two consecutive archive I/Os. This ensures that online logs can be archived in time, thereby reducing the risk of data loss.

In OceanBase Database, log archiving is performed on a log stream basis. If logs are written to a log stream and the time since the last archive of the log stream exceeds the interval specified by the archive_lag_target parameter, the system triggers an archive action to archive the unarchived logs in the log stream.

For example, if the value of the archive_lag_target parameter is set to 120s (2 minutes), each log stream will attempt to archive logs every 2 minutes (unless the log archive buffer is full), ensuring that the interval between two consecutive archive actions for the log stream is less than 2 minutes. Note that when the value of the archive_lag_target parameter is set to 0, the system can achieve quasi-real-time archiving.

For more information about the archive_lag_target parameter, see archive_lag_target.

Prerequisites

Before you modify the value of the archive_lag_target parameter, make sure that the archive destination has been configured. If the archive destination has not been configured, the system will prompt you with the error message log_archive_dest has not been set, set archive_lag_target is not allowed when you attempt to modify the value of the archive_lag_target parameter.

Considerations

If the archive medium is OSS, NFS, or COS, you can set the value of the archive_lag_target parameter to any value within the allowed range; if the archive medium is S3, the value of the archive_lag_target parameter cannot be less than 60s. Otherwise, the system will return an error.
We recommend that you set the value of the archive_lag_target parameter to a reasonable value to avoid excessive I/O frequency, which can compromise system performance. Especially when an object storage service is used, frequent I/O can increase the overheads significantly. A high value cannot meet the requirement on timeliness of data restoration, therefore, the recovery point objective (RPO). You need to find a balance between the RPO and the performance of the archive medium.

Procedure

Log in to the database as the tenant administrator of the sys tenant or a user tenant.
Choose an appropriate statement to configure the archive delay based on your business needs.

The procedure is as follows:
- Configure the archive delay for a specified tenant from the sys tenant
```
obclient> ALTER SYSTEM SET archive_lag_target = '120s' TENANT = mysql_tenant;
```
  Notice
  
  When you configure the archive_lag_target parameter, the TENANT = all_user clause in the sys tenant is not supported.
- Configure the archive delay for the current tenant from a user tenant
```
obclient> ALTER SYSTEM SET archive_lag_target = '120s';
```

Prepare for log archiving

Configure the degree of parallelism of log archiving (optional)

Note

Note

Configure the destination for archiving logs

Considerations

Procedure

Note

Notice

Notice

Notice

Notice

Notice

Notice

Notice

Notice

Considerations and notes

Configure the archive delay

Prerequisites

Considerations

Procedure

Notice

References