Standard Operating Procedure (SOP) is a standardized fault recovery and data reconstruction feature provided by OCP. For common complex operational scenarios in OceanBase Database O&M, SOP orchestrates multi-step O&M processes into standardized task templates, supporting a hybrid mode that combines automatic execution with manual triggering, helping you complete fault recovery operations safely and efficiently.
Applicability
This content applies only to OCP Enterprise Edition. OCP Community Edition does not provide this feature.
Permission Description
SOP permissions are not configured by selecting items in role definitions; instead, the system automatically determines them based on the logged-in user's identity. Users with different identities can perform the following operations:
Identity |
Available Operations |
|---|---|
| System Administrator (ADMIN) | View and operate all SOP tasks. |
| Organization Administrator (ORG_ADMIN) | View and operate all SOP tasks within the organization. |
| Read-only Administrator (ADMIN_VIEWER) | You can view tasks, but cannot create, edit, start, or execute them. The read-only administrator of the default organization can view all tasks, while the read-only administrators of other organizations can only view tasks within their own organizations. |
| Other Log-in Users | You can view and operate only the SOP tasks you created, regardless of whether SOP is configured for the role. |
Unlogged-in users cannot access SOP.
Note
SOP does not appear as an independent permission item in role configurations.
Relationship with Cluster/Tenant Permissions
After the SOP interface is enabled, resource permissions for the operation target are still verified during creation, startup, and execution. For example, if you do not have the Update permission for a cluster, you cannot create or execute an SOP task for that cluster even if you can access the SOP page.
Prerequisites
- The currently logged-in OCP user has either Cluster Management, Tenant Management, or Host Management resource permission (for the target resource).
- The Update permission for the cluster/tenant/host (for the creation, startup, or execution step).
Feature Introduction
SOP provides various pre-configured standard O&M process templates, each containing a set of standard steps to be executed in sequence. You can select an appropriate process type based on your business scenario, configure the automatic execution properties for the steps, and advance the process via manual confirmation when needed.
The main features of SOP include:
- Process Templating: Each SOP type provides a pre-configured standard step template. The execution order, type, and identifier of the steps are fixed, ensuring operational standardization.
- Combination of Automatic and Manual: You can configure whether each step is to be executed automatically. Automatically executing steps are triggered automatically after the previous step completes. Non-automatically executing steps are not triggered automatically and require manual progression to the next step.
- Process Tracing: The execution status, start/end time, associated tasks, retry count, and error information for each step are fully recorded, facilitating issue troubleshooting and auditing.
- Flexible Fault Handling: If a step fails, you can retry a single failed step or all failed steps. For non-automatically executing steps, you can modify the auto-execution switch or step parameters before execution.
Task List
On the Standard Process page, you can view all SOP tasks, including information such as the task name, task type, execution object, execution progress, creation time, creator, start time, end time, and status.
- You can enter a task name in the search box to perform a fuzzy search for tasks.
- Click a task name or Execution Details to view the execution details of the task, including step configuration, execution status, and step execution information. For more information, see View execution details of a standard O&M process.
- You can create a new standard O&M process task. For more information, see Create a standard O&M process.
Task status description
Status |
Description |
|---|---|
| To be Executed (NOT_STARTED) | The task has been created but has not yet started. |
| WAITING_CONFIRM | The task is in progress. You must manually proceed to the next step. |
| RUNNING | The task is in progress, and the current step is being automatically executed. |
| FAILED | A step failed during task execution. |
| Completed (COMPLETED) | All enabled steps for the task have been successfully completed. |
SOP type description
OCP currently supports the following three standard O&M process types:
Enable arbitration service for abnormal replicas
This process is used to ensure tenant availability by enabling the arbitration service when a replica in the cluster becomes abnormal. The process includes steps such as confirming the operation object, adding an arbitration service, deleting tenant replicas, deleting OBSERVERs, enabling tenant arbitration service, and verifying the availability of the tenant arbitration service. You must specify the target cluster and the arbitration service or arbitration service group.
Data reconstruction after OBSERVER failure
This process is used to reconstruct data after an OBSERVER node fails by deleting the failed node and adding a new node. The process includes steps such as confirming the operation object, deleting tenant replicas, deleting OBSERVERs, forcibly deleting the host (optional), adding OBSERVERs, adding tenant replicas, and verifying the successful reconstruction of tenant replicas. You can pre-select an alternative host.
Reconstruct a standby tenant
This process is used to restore data when a standby tenant's data is abnormal or corrupted by backing up the primary tenant's data and recreating the standby tenant. The process includes steps such as confirming the operation object, backing up the primary tenant, decoupling the primary and standby tenants, locking the original primary and standby tenants, deleting the original primary and standby tenants, creating a new standby tenant, and verifying primary-standby synchronization. Primary-standby synchronization supports two modes: network-based (NETWORK) and log-backup-based (LOG_BACKUP).