Prepare for log archiving|V4.2.2|OceanBase Database| docs|Distributed Database

This topic describes the preparations for log archiving.

(Optional) Configure the log archive concurrency

Before you enable the archive mode, you can configure the log archive concurrency to increase the log archiving speed of a tenant.

Log in to the database as the tenant administrator of the sys tenant or a user tenant.
Choose an appropriate statement to set the log_archive_concurrency parameter.

The log_archive_concurrency parameter set at the tenant level configures the total number of worker threads for log archiving. The parameter change takes effect immediately without the need to restart the OBServer node. The value range is [0, 100], and the default value is 0, which means that the database uses adaptive concurrency for log archiving. We recommend that you use the default value.

Notice

If the tenant has four CPU cores or fewer, we recommend that you use the default value and do not modify the value of this parameter.

You can set the parameter in the following ways:
- Adjust the log archive concurrency for a user tenant from the sys tenant
```
ALTER SYSTEM SET log_archive_concurrency = 10 TENANT = mysql_tenant;
```
- Adjust the log archive concurrency for all user tenants from the sys tenant
```
ALTER SYSTEM SET log_archive_concurrency = 10 TENANT = all_user;
```
  or
```
ALTER SYSTEM SET log_archive_concurrency = 10 TENANT = all;
```
  Note
  
  Starting from OceanBase Database V4.2.1, TENANT = all_user and TENANT = all express the same semantics. If you want an operation to take effect on all user tenants, we recommend that you use TENANT = all_user. TENANT = all will be deprecated.
- Adjust the log archive concurrency for a user tenant from a user tenant
```
ALTER SYSTEM SET log_archive_concurrency = 10;
```
For more information about the log_archive_concurrency parameter, see log_archive_concurrency.

Configure the archive destination

Before you start a log archiving task, you must use the ALTER SYSTEM command to set the LOG_ARCHIVE_DEST parameter. The sys tenant does not support backup and restore, so it does not require an archive destination.

The operation of configuring an archive destination mainly involves setting the LOCATION, BINDING, and PIECE_SWITCH_INTERVAL attributes.

Considerations

Procedure

Log in to the database as the tenant administrator of the sys tenant or a user tenant.

Note

In MySQL mode, the administrator user is the root user. In Oracle mode, the administrator user is the SYS user.
Configure the destination for archiving.
- Configure the destination for archiving for a specified tenant as the tenant administrator of the sys tenant
```
ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=archive_path [BINDING=archive_mode] [PIECE_SWITCH_INTERVAL=piece_switch_interval]' TENANT = tenant_name;
```
- Configure the destination for archiving for the current tenant as the user tenant
```
ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=archive_path [BINDING=archive_mode] [PIECE_SWITCH_INTERVAL=piece_switch_interval]';
```
Notice

After upgrading from OceanBase Database V4.0.x to V4.1.0, you must change the archive path. After upgrading from OceanBase Database V4.1.x to V4.2.x, you do not need to change the archive path. When upgrading from OceanBase Database V4.1.x to V4.2.x, log archive can be performed during the upgrade.

The following table describes the configuration attributes.
- Configure the LOCATION attribute (mandatory)
  
  The LOCATION attribute specifies the destination for archiving. Currently, OceanBase Database supports archiving to NFS, Alibaba Cloud OSS, and Tencent Cloud COS.
  OSS
  
  NFS
  
  COS
  When using OSS as the destination for archiving, in addition to the archive path, host, access_key, and access_id, you can also use the delete_mode attribute to specify the cleanup mode for archive files.
  
  The delete_mode attribute supports two values, delete and tagging. If you do not specify a value, the default value delete takes effect.
  
  delete: specifies to directly delete archive files that meet the cleanup requirements.
  
  If you set the attribute to this value, when you automatically clean up archive files, the system will directly delete the files that meet the cleanup requirements.
  
  Note
  
  This value is the default value. If you do not specify the delete_mode attribute, the delete value takes effect.
  
  tagging: specifies to set the Tag attribute for archive files that meet the cleanup requirements. The files will remain on OSS after the attribute is set.
  
  If you set the attribute to this value, when you automatically clean up archive files, the system will set the Tag attribute for the files that meet the cleanup requirements. The key of the Tag attribute is "delete_mode" and the value is "tagging". This allows you to manage the lifecycle of the files on OSS later by using the specified tags.
  
  Notice
  
  When using object storage as the destination for archiving, separate the parameters in the object storage path with the & character. Make sure that the parameters contain only uppercase and lowercase English letters, numbers, /-_$+=, and wildcard characters. If the parameters contain other characters, the setting may fail.
  
  When using OSS as the destination for archiving, the tenant administrator of the sys tenant can configure the destination for archiving for a specified tenant and set the tagging cleanup mode. Here is an example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=oss://oceanbase-test-bucket/backup/archive?host=xxx.aliyun-inc.com&access_id=xxx&access_key=xxx&delete_mode=tagging' TENANT = mysql_tenant;
  
  The user tenant can configure the destination for archiving for the current tenant and set the tagging cleanup mode. Here is an example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=oss://oceanbase-test-bucket/backup/archive?host=xxx.aliyun-inc.com&access_id=xxx&access_key=xxx&delete_mode=tagging';
  
  In the preceding examples, oss:// indicates that OSS is used as the destination for archiving. The bucket name is oceanbase-test-bucket, and the path in the bucket is /backup/archive. The parameters after the ? are other parameters in the path. host is the host address of the bucket. access_id and access_key are the access keys of OSS. The cleanup mode is set to tagging.
  
  After you set the delete mode or the tagging mode, for information about how to automatically clean up archive data, see Automatically clean up backed-up data.
  Notice
  
  When using NFS as the destination for archiving, note that:
  
  The value of the LOCATION attribute cannot contain a question mark (?).
  
  The value of the LOCATION attribute must be an absolute path, and the OBServer node must have read and write permissions for the path specified by LOCATION.
  
  All OBServer nodes must mount the same NFS server. To ensure smooth archiving, it is recommended to use the parameters suggested in this topic to mount NFS. For more information, see Deploy an NFS client.
  
  When using NFS as the destination for archiving, the tenant administrator of the sys tenant can configure the destination for archiving for a specified tenant. Here is an example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive' TENANT = mysql_tenant;
  
  The user tenant can configure the destination for archiving for the current tenant. Here is an example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive';
  
  In the preceding examples, file:// indicates that NFS is used as the destination for archiving. The path for archiving is /data/nfs/backup/archive.
  Notice
  
  When using COS as the destination for archiving, note that:
  
  You must disable the List Cache feature of the bucket. Otherwise, backup data consistency errors may occur. Contact the COS technical support team to help you disable the List Cache feature.
  
  A bucket with the Multi-AZ feature enabled does not support the APPEND Object API. You must disable the Multi-AZ feature. Otherwise, an error will occur when you archive logs.
  
  Do not include commas or other special characters in the access_key parameter.
  
  Similar to OSS, you can use the delete_mode attribute to specify the cleanup mode for archive files in COS. The supported values and meanings are the same as those for OSS.
  
  Notice
  
  When using object storage as the destination for archiving, separate the parameters in the object storage path with the & character. Make sure that the parameters contain only uppercase and lowercase English letters, numbers, /-_$+=, and wildcard characters. If the parameters contain other characters, the setting may fail.
  
  When using COS as the destination for archiving, the tenant administrator of the sys tenant can configure the destination for archiving for a specified tenant and set the tagging cleanup mode. Here is an example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=cos://oceanbase-test-appid/backup/archive?host=cos.ap-xxxx.myqcloud.com&access_id=xxx&access_key=xxx&appid=xxx&delete_mode=tagging' TENANT = mysql_tenant;
  
  The user tenant can configure the destination for archiving for the current tenant and set the tagging cleanup mode. Here is an example:
  
  obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=cos://oceanbase-test-appid/backup/archive?host=cos.ap-xxxx.myqcloud.com&access_id=xxx&access_key=xxx&appid=xxx&delete_mode=tagging';
  
  In the preceding examples, cos:// indicates that COS is used as the destination for archiving. The bucket name is oceanbase-test-appid, and the path in the bucket is /backup/archive. The parameters after the ? are other parameters in the path. host is the host address of the bucket, which is also the endpoint (without the bucket name) of COS. access_id and access_key are the access keys of COS. appid is a required parameter that specifies the APPID of the Tencent cloud account. The cleanup mode is set to tagging.
- Configure the BINDING attribute (optional)
  
  The BINDING attribute specifies the priority mode for archiving and business. Currently, the Optional and Mandatory modes are supported. If you do not configure this attribute, the default value Optional takes effect.
  - The Optional mode indicates that the business process takes priority. In this mode, if the log archiving process cannot keep up with the log generation process, the system may recycle logs that have not been archived, which causes a gap in the log stream.
  - The Mandatory mode indicates that the archiving process takes priority. In this mode, if the log archiving process cannot keep up with the data write process, the system will block data writes.
  When using NFS as the destination for archiving, the tenant administrator of the sys tenant can configure the destination for archiving for a specified tenant and set the BINDING attribute. Here is an example:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional' TENANT = mysql_tenant;
```
  The user tenant can configure the destination for archiving for the current tenant and set the BINDING attribute. Here is an example:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional';
```
- Configure the PIECE_SWITCH_INTERVAL attribute (optional)
  
  The PIECE_SWITCH_INTERVAL attribute specifies the interval for switching pieces of logs, in days. The value range is [1d, 7d]. If you do not configure this attribute, the default value 1d takes effect.
  
  When using NFS as the destination for archiving, the tenant administrator of the sys tenant can configure the destination for archiving and set the attribute to switch to a new piece of logs every two days. Here is an example:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional PIECE_SWITCH_INTERVAL=1d' TENANT = mysql_tenant;
```
  The user tenant can configure the destination for archiving and set the attribute to switch to a new piece of logs every two days. Here is an example:
```
obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST='LOCATION=file:///data/nfs/backup/archive BINDING=Optional PIECE_SWITCH_INTERVAL=1d';
```
After the configuration, you can query the detailed parameter settings through views.

In the sys tenant, you can query the CDB_OB_ARCHIVE_DEST view for the parameter settings; in a user tenant, you can query the DBA_OB_ARCHIVE_DEST view for the parameter settings. For more information, see Query the archive parameters.

Considerations and notes

After the parameter LOG_ARCHIVE_DEST is set, the system will create a format file in the destination directory specified in the configuration to verify the validity of the backup destination and the integrity of the data in the destination. Therefore, note the following considerations when you configure the log archive destination:

If the format file does not exist, the specified destination directory must be empty. Otherwise, the system returns an error with the error code -9080, indicating that the format file does not exist.
If the format file exists, the content of the file must pass the verification. Otherwise, the system returns an error with the error code -9081, indicating that the format file is mismatched. The content verification of the format file checks whether the cluster, tenant, and backup destination type are consistent with those for the current operation.
If the format file does not exist or fails the verification during backup, the task fails.

After the archive destination is configured, incremental configuration is not supported. For example, assume that the BINDING attribute of the NFS archive path /data/nfs/backup/archive is set to Mandatory, and the PIECE_SWITCH_INTERVAL attribute is set to 1d. If you want to change the value of the PIECE_SWITCH_INTERVAL attribute to 2d while retaining the value of the BINDING attribute, you still need to specify the attribute values in the command. Otherwise, the unspecified attributes will use the default values.

To modify the attribute values, execute the following statements again.

Modify the attribute values of a specified tenant in the sys tenant.

obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST = 'LOCATION=file:///data/nfs/backup/archive BINDING=Mandatory PIECE_SWITCH_INTERVAL=2d' TENANT = mysql_tenant;

Modify the attribute values of the current tenant.

obclient> ALTER SYSTEM SET LOG_ARCHIVE_DEST = 'LOCATION=file:///data/nfs/backup/archive BINDING=Mandatory PIECE_SWITCH_INTERVAL=2d';

Configure the archive lag target

OceanBase Database uses the tenant-level parameter archive_lag_target to control the latency of log archiving for a tenant. The parameter supports the ms (millisecond), s (second), m (minute), and h (hour) time units. The default value is 2m (2 minutes). This parameter specifies the maximum interval between two consecutive archive I/Os, ensuring that online logs are archived in a timely manner to reduce the risk of data loss.

OceanBase Database archives logs by log stream. If a log stream has logs written and the time interval since the last archive of the log stream exceeds the interval specified by the archive_lag_target parameter, the system initiates an archive action to archive the unarchived logs in the log stream.

For example, if the value of the archive_lag_target parameter is set to 120s (2 minutes), each log stream will attempt to initiate an archive action every 2 minutes (unless other conditions are met, such as the buffer for log archive caching being filled up), ensuring that the interval between the current archive action and the last archive action for the log stream is less than 2 minutes. Note that when the value of the archive_lag_target parameter is set to 0, the system can achieve quasi-real-time archiving.

For more information about the archive_lag_target parameter, see archive_lag_target.

Considerations

We recommend that you set the archive_lag_target parameter to an appropriate value to avoid frequent I/O, which can compromise system performance, especially when object storage is used. A large value may fail to meet the timeliness requirement for data recovery, therefore increasing the RPO. You need to strike a balance between the two based on your business needs and the performance of the archive medium.

Procedure

Log in to the database as the tenant administrator of the sys tenant or a user tenant.
Choose an appropriate statement based on your situation to configure the archive lag target.

The procedure is as follows:
- Configure the archive lag target for a specified tenant from the sys tenant
```
obclient> ALTER SYSTEM SET archive_lag_target = '120s' TENANT = mysql_tenant;
```
- Configure the archive lag target for all user tenants from the sys tenant
```
obclient> ALTER SYSTEM SET archive_lag_target = '120s' TENANT = all_user;
```
- Configure the archive lag target for the current user tenant
```
obclient> ALTER SYSTEM SET archive_lag_target = '120s';
```

OceanBase

Customer Stories

Documentation

Prepare for log archiving

(Optional) Configure the log archive concurrency

Notice

Note

Configure the archive destination

Considerations

Procedure

Note

Notice

Note

Notice

Notice

Notice

Notice

Considerations and notes

Configure the archive lag target

Considerations

Procedure

References