Prepare for log archiving|V4.3.1| docs|Distributed Database

Prepare for log archiving

Last Updated：2025-12-04 02:53:14 Updated

This topic describes the preparations for log archiving.

Configure the log archive concurrency level (optional)

If you enable the ARCHIVE mode, you can increase the log archive concurrency level to speed up the log archiving process in the tenant.

Log in to the database as the tenant administrator of the sys tenant or a user tenant.
Choose an appropriate statement to set the log_archive_concurrency parameter.

The log_archive_concurrency parameter set at the tenant level specifies the total number of worker threads for log archiving. The setting takes effect immediately without the need to restart the OBServer node. The value range is [0, 100]. The default value is 0, which means that the database uses adaptive log archive concurrency. We recommend that you use the default value.

Notice

We recommend that you use the default value for small tenants (with less than or equal to four CPU cores) and do not modify the value of this parameter.

You can set the parameter in the following ways:
- Adjust the log archive concurrency level for a user tenant from the sys tenant
```
ALTER SYSTEM SET log_archive_concurrency = 10 TENANT = mysql_tenant;
```
- Adjust the log archive concurrency level for all user tenants from the sys tenant
```
ALTER SYSTEM SET log_archive_concurrency = 10 TENANT = all_user;
```
  or
```
ALTER SYSTEM SET log_archive_concurrency = 10 TENANT = all;
```
  Note
  
  Starting from OceanBase Database V4.2.1, TENANT = all_user and TENANT = all express the same semantics. If you want an operation to take effect on all user tenants, we recommend that you use TENANT = all_user. TENANT = all will be deprecated.
- Adjust the log archive concurrency level for the current user tenant
```
ALTER SYSTEM SET log_archive_concurrency = 10;
```
For more information about the log_archive_concurrency parameter, see log_archive_concurrency.

Configure the destination for archiving logs

Before you start a log archiving task, you must use the ALTER SYSTEM command to set the LOG_ARCHIVE_DEST parameter. The sys tenant does not support backup and restore, and the system tenant does not contain user data. Therefore, the system tenant is suitable for cluster management but does not require a configuration for the destination for archiving logs.

The procedure mainly involves setting the LOCATION, BINDING, and PIECE_SWITCH_INTERVAL parameters.

Considerations

Procedure

Log in to the database as the tenant administrator of the sys tenant or a user tenant.

Note

In MySQL mode, the administrator user is root. In Oracle mode, the administrator user is SYS.
Configure the destination for archiving logs.
- Configure the destination for archiving logs for a specified tenant from the sys tenant
```
ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=archive_path [BINDING=archive_mode] [PIECE_SWITCH_INTERVAL=piece_switch_interval]' TENANT = tenant_name;
```
- Configure the destination for archiving logs for the current tenant
```
ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=archive_path [BINDING=archive_mode] [PIECE_SWITCH_INTERVAL=piece_switch_interval]';
```
Notice

After upgrading from OceanBase Database V4.0.x to V4.1.0, you need to change the archive path after the upgrade. After upgrading from OceanBase Database V4.1.x to V4.2.x, you do not need to change the archive path. In addition, when upgrading from OceanBase Database V4.1.x to V4.2.x, you can archive logs during the upgrade.

For more information about how to configure the destination, see the following points:
- Set the LOCATION parameter (mandatory)
  
  The LOCATION parameter specifies the destination for archiving logs. At present, OceanBase Database supports archiving logs to NFS, Alibaba Cloud OSS, Tencent Cloud COS, AWS S3, and object storage services compatible with the S3 protocol, such as Huawei Cloud OBS and Google Cloud GCS. Some backup media may have basic requirements. For more information, see the Backup media requirements section in Overview of physical backup and restore.
  
  Note
  
  When using an object storage service as the destination for archiving logs, make sure that the access_key parameter does not contain special characters, such as commas.
  Alibaba Cloud OSS
  
  NFS
  
  Tencent Cloud COS
  
  S3
  
  Compatible object storage services
  When you use OSS as the destination for archiving logs, apart from setting the archive path, host, access_key, and access_id, you can also set the delete_mode parameter to configure the cleanup mode for archived logs.
  
  The delete_mode parameter supports two cleanup modes, delete and tagging. If you do not set this parameter, the default value delete is used.
  
  delete: Specifies to directly delete archived logs that meet the cleanup requirements.
  
  If you set this mode, when you automatically clean up archived logs, the system will directly delete the logs that meet the cleanup requirements.
  
  Notice
  
  This is the default cleanup mode. If you do not set the delete_mode parameter, the delete mode is used.
  
  tagging: Specifies to set the Tag for archived logs that meet the cleanup requirements. The logs will still be retained.
  
  If you set this mode, when you automatically clean up archived logs, the system will set a tag for the logs that meet the cleanup requirements. The key of the tag is "delete_mode" and the value is "tagging". This allows you to manage the lifecycle of the logs on OSS based on the tag.
  
  Notice
  
  When you use an object storage service as the destination for archiving logs, separate the parameters in the object storage service path with the & character. Make sure that the parameter values contain only uppercase and lowercase English letters, numbers, /-_$+=, and wildcard characters. If the parameter values contain other characters, the setting may fail.
  
  If you use OSS as the destination for archiving logs, the tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination and set the delete cleanup mode as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=oss://oceanbase-test-bucket/backup/archive?host=***.aliyun-inc.com&access_id=***&access_key=***&delete_mode=delete' TENANT = mysql_tenant;
  
  The user of the current tenant can configure the destination and set the delete cleanup mode as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=oss://oceanbase-test-bucket/backup/archive?host=***.aliyun-inc.com&access_id=***&access_key=***&delete_mode=delete';
  
  In the preceding examples, oss:// indicates that OSS is used as the destination for archiving logs. The name of the bucket is oceanbase-test-bucket, and the path to the archive in the bucket is /backup/archive. The parameters in the path are separated with ?. host specifies the host address of the bucket. access_id and access_key specify the access keys of OSS. The cleanup mode is set to delete.
  
  After you set the delete mode or the tagging mode, for information about how to automatically clean up archived logs, see Automatically cleanup backed-up data.
  Notice
  
  Consider the following points when you use NFS as the destination for archiving logs:
  
  The value of the LOCATION parameter must not contain the question mark character (?).
  
  The value of the LOCATION parameter must be an absolute path, and the OBServer node must have read and write permissions for LOCATION.
  
  Make sure that all OBServer nodes mount the same NFS server. To ensure smooth archiving, it is recommended to use the parameters for mounting NFS specified in this topic. For more information, see Deploy an NFS client.
  
  If you use NFS as the destination for archiving logs, the tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive' TENANT = mysql_tenant;
  
  The user of the current tenant can configure the destination as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive';
  
  In the preceding examples, file:// indicates that NFS is used as the destination for archiving logs. The path to the archive is /data/nfs/backup/archive.
  COS supports the delete_mode parameter to configure the cleanup mode for archived logs, and the configuration method is the same as for OSS.
  
  Notice
  
  When you use an object storage service as the destination for archiving logs, separate the parameters in the object storage service path with the & character. Make sure that the parameter values contain only uppercase and lowercase English letters, numbers, /-_$+=, and wildcard characters. If the parameter values contain other characters, the setting may fail.
  
  If you use COS as the destination for archiving logs, the tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination and set the delete cleanup mode as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=cos://oceanbase-test-appid/backup/archive?host=cos.ap-***.myqcloud.com&access_id=***&access_key=***&appid=***&delete_mode=delete' TENANT = mysql_tenant;
  
  If you use COS as the destination for archiving logs, the user of the current tenant can configure the destination and set the delete cleanup mode as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=cos://oceanbase-test-appid/backup/archive?host=cos.ap-***.myqcloud.com&access_id=***&access_key=***&appid=***&delete_mode=delete';
  
  In the preceding examples, cos:// indicates that COS is used as the destination for archiving logs. The name of the bucket is oceanbase-test-appid, and the path to the archive in the bucket is /backup/archive. The parameters in the path are separated with ?. host specifies the host address of the bucket, which is the endpoint (without the bucket name). access_id and access_key specify the access keys of COS. The cleanup mode is set to delete. The appid parameter, which is the APPID of the Tencent Cloud account, is an optional parameter.
  AWS S3 supports the delete_mode parameter to configure the cleanup mode for archived logs, and the configuration method is the same as for OSS and COS.
  
  Notice
  
  When you use an object storage service as the destination for archiving logs, separate the parameters in the object storage service path with the & character. Make sure that the parameter values contain only uppercase and lowercase English letters, numbers, /-_$+=, and wildcard characters. If the parameter values contain other characters, the setting may fail.
  
  If you use S3 as the destination for archiving logs, the tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination and set the delete cleanup mode as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=s3://oceanbase-test-bucket/backup/archive?host=s3.<region>.amazonaws.com&access_id=******&access_key=******&s3_region=******&delete_mode=delete' TENANT = mysql_tenant;
  
  If you use S3 as the destination for archiving logs, the user of the current tenant can configure the destination and set the delete cleanup mode as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=s3://oceanbase-test-bucket/backup/archive?host=s3.<region>.amazonaws.com&access_id=******&access_key=******&s3_region=******&delete_mode=delete';
  
  In the preceding examples, s3:// indicates that S3 is used as the destination for archiving logs. The name of the bucket is oceanbase-test-bucket, and the path to the archive in the bucket is /backup/archive. The parameters in the path are separated with ?. host specifies the domain name of the Amazon S3 service. access_id and access_key specify the access keys of AWS. s3_region is an optional parameter when you use S3 as the destination for archiving logs. It specifies the region where the S3 bucket is located. The cleanup mode is set to delete.
  Most object storage services are compatible with the S3 protocol. OceanBase Database can archive logs to any object storage service that is compatible with the S3 protocol and meets the requirements of OceanBase Database. You can access compatible object storage services as you access S3, for example, you can access Huawei Cloud OBS and Google Cloud GCS through the S3 protocol.
  
  Notice
  
  When you use an object storage service as the destination for archiving logs, separate the parameters in the object storage service path with the & character. Make sure that the parameter values contain only uppercase and lowercase English letters, numbers, /-_$+=, and wildcard characters. If the parameter values contain other characters, the setting may fail.
  
  If you use OBS as the destination for archiving logs, the tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination as follows:
  
  If you use OBS as the destination for archiving logs, the tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination and set the delete cleanup mode as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=s3://oceanbase-test-bucket/backup/archive?host=obs.****.myhuaweicloud.com&access_id=****&access_key=****' TENANT = mysql_tenant;
  
  If you use OBS as the destination for archiving logs, the user of the current tenant can configure the destination and set the delete cleanup mode as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=s3://oceanbase-test-bucket/backup/archive?host=obs.****.myhuaweicloud.com&access_id=****&access_key=****';
  
  In the preceding examples, s3:// indicates that compatible object storage is used as the destination for archiving logs. The name of the bucket is oceanbase-test-bucket, and the path to the archive in the bucket is /backup/archive. The parameters in the path are separated with ?. host specifies the domain name of the OBS service. access_id and access_key specify the access keys of OBS.
  
  If you use GCS as the destination for archiving logs, the tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination as follows:
  
  If you use GCS as the destination for archiving logs, the tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination and set the delete cleanup mode as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=s3://oceanbase-test-bucket/backup/archive?host=https://storage.googleapis.com&access_id=****&access_key=****' TENANT = mysql_tenant;
  
  If you use GCS as the destination for archiving logs, the user of the current tenant can configure the destination and set the delete cleanup mode as follows:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=s3://oceanbase-test-bucket/backup/archive?host=https://storage.googleapis.com&access_id=****&access_key=****';
  
  In the preceding examples, s3:// indicates that compatible object storage is used as the destination for archiving logs. The name of the bucket is oceanbase-test-bucket, and the path to the archive in the bucket is /backup/archive. The parameters in the path are separated with ?. host specifies the domain name of the GCS service. access_id and access_key specify the access keys of GCS.
- Set the BINDING parameter (optional)
  
  The BINDING parameter specifies the priority mode for archiving and business. At present, the Optional and Mandatory modes are supported. If you do not set this parameter, the default value Optional is used.
  - In the Optional mode, the business takes priority. In this mode, if the archiving of logs (log archive) cannot keep up with the generation of logs, the system may recycle logs that have not been archived, which causes a gap in the log stream.
  - In the Mandatory mode, archiving takes priority. In this mode, if the archiving of logs cannot keep up with the write of user data, the system will prevent you from writing data.
  If you use NFS as the destination for archiving logs, the tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination and set the BINDING parameter as follows:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional' TENANT = mysql_tenant;
```
  The user of the current tenant can configure the destination and set the BINDING parameter as follows:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional';
```
- Set the PIECE_SWITCH_INTERVAL parameter (optional)
  
  The PIECE_SWITCH_INTERVAL parameter specifies the cycle for switching piece, with a value range of [1d, 7d]. If you do not set this parameter, the default value 1d is used.
  
  If you use NFS as the destination for archiving logs, the tenant administrator of the mysql_tenant tenant in the sys tenant can configure the destination and set the system to switch to a new piece every two days as follows:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional PIECE_SWITCH_INTERVAL=1d' TENANT = mysql_tenant;
```
  The user of the current tenant can configure the destination and set the system to switch to a new piece every two days as follows:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional PIECE_SWITCH_INTERVAL=1d';
```
After the configuration, you can query the detailed parameter settings from views.

In the sys tenant, you can query the parameters from the CDB_OB_ARCHIVE_DEST view; in a user tenant, you can query the parameters from the DBA_OB_ARCHIVE_DEST view. For more information, see Query the archive parameters.

Considerations and notes

After the LOG_ARCHIVE_DEST parameter is successfully configured, the system will create a format file in the destination directory specified in the configuration to verify the validity of the backup destination and the integrity of the data in the destination. Therefore, note the following considerations when you configure a log archive destination:

If the format file does not exist, the specified destination directory must be empty. Otherwise, an error -9080 will occur, indicating that the format file does not exist.
If the format file already exists, its content must pass the verification. Otherwise, an error -9081 will occur, indicating that the format file is mismatched. The verification mainly checks whether the cluster, tenant, and backup destination type specified in the format file are consistent with those for the current operation.
If the format file does not exist or its content verification fails during backup task execution, the task will fail.

In addition, after the archive destination is configured, incremental configuration is not supported. For example, assume that the BINDING attribute of the NFS archive path /data/nfs/backup/archive is set to Mandatory, and the PIECE_SWITCH_INTERVAL attribute is set to 1d. If you want to change the value of PIECE_SWITCH_INTERVAL to 2d while retaining the setting of the BINDING attribute, you still need to specify the attribute values in the command. Otherwise, the default values will be used for the unspecified attributes.

You need to re-execute the following statements.

Modify the parameters of a specified tenant in the sys tenant.

obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST = 'LOCATION=file:///data/nfs/backup/archive BINDING=Mandatory PIECE_SWITCH_INTERVAL=2d' TENANT = mysql_tenant;

Modify the parameters of the current tenant in the user tenant.

obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST = 'LOCATION=file:///data/nfs/backup/archive BINDING=Mandatory PIECE_SWITCH_INTERVAL=2d';

Configure the archive delay

OceanBase Database uses the tenant-level parameter archive_lag_target to control the latency of log archiving for a tenant. The parameter supports the time units of milliseconds (ms), seconds (s), minutes (m), and hours (h). The default value is 2m, which specifies the maximum interval in minutes between two consecutive archive I/Os. This ensures that online logs are archived in a timely manner, thereby reducing the risk of data loss.

In OceanBase Database, log archiving is performed on a log stream basis. If logs are written to a log stream and the time since the last archive of the log stream exceeds the time interval specified by the archive_lag_target parameter, the system triggers an archive action to archive the unarchived logs in the log stream.

For example, if the value of the archive_lag_target parameter is set to 120s (2 minutes), each log stream will attempt to archive logs every 2 minutes (unless the log archive buffer is full), ensuring that the interval between two consecutive archive actions for the log stream is less than 2 minutes. Note that when the value of the archive_lag_target parameter is set to 0, the system can achieve quasi-real-time archiving.

For more information about the archive_lag_target parameter, see archive_lag_target.

Prerequisites

Before you modify the value of the archive_lag_target parameter, make sure that the archive destination has been configured. If the archive destination is not configured, the system will prompt you with the error message log_archive_dest has not been set, set archive_lag_target is not allowed when you attempt to modify the value of the archive_lag_target parameter.

Considerations

If the archive medium is OSS, NFS, or COS, you can set the value of the archive_lag_target parameter to any value within the allowed range; if the archive medium is S3, the value of the archive_lag_target parameter cannot be less than 60s. Otherwise, the system will return an error.
We recommend that you set the archive_lag_target parameter to an appropriate value to avoid frequent I/O, which can compromise system performance. Especially when an object storage service is used for archiving, set the value of this parameter to a relatively low value to reduce the latency of log archiving. However, a high value cannot meet the requirement on timeliness for data restoration, so you need to find a proper balance between the recovery point objective (RPO) and the value of this parameter.

Procedure

Log in to the database as the tenant administrator of the sys tenant or a user tenant.
Choose an appropriate statement based on your business situation to configure the archive delay.

The procedure is as follows:
- Configure the archive delay for a specified tenant from the sys tenant
```
obclient> ALTER SYSTEM SET archive_lag_target = '120s' TENANT = mysql_tenant;
```
  Notice
  
  When you configure the archive_lag_target parameter, the TENANT = all_user clause is not supported in the sys tenant.
- Configure the archive delay for the current tenant from a user tenant
```
obclient> ALTER SYSTEM SET archive_lag_target = '120s';
```

Prepare for log archiving

Configure the log archive concurrency level (optional)

Notice

Note

Configure the destination for archiving logs

Considerations

Procedure

Note

Notice

Note

Notice

Notice

Notice

Notice

Notice

Notice

Considerations and notes

Configure the archive delay

Prerequisites

Considerations

Procedure

Notice

References