This topic describes the long-running background tasks in OceanBase Cloud Platform (OCP). Background tasks refer to tasks triggered and executed by the OCP server to maintain OCP, check the status of managed objects, and clean up data.
To view the scheduling and running history of background tasks, you can consult the dag_instance and task_instance tables in OCP MetaDB. For example, you can specify dag_instance.id = task_instance.dag_id in a JOIN clause. The dag_instance table records the execution details of background tasks. The task_instance table records the execution details of each child task of the active background tasks.
To view the logs of background tasks, you can search by keywords in the OCP log file specified for the logging.file parameter. By default, the OCP log file is ${user.home}/logs/ocp/ocp.log.
The following table lists the long-running background tasks in OCP, including the task name, description, cron expression, and information for troubleshooting.
OCP background tasks
| Task name | Description | Cron expression | Troubleshooting - database | Troubleshooting - OCP log |
|---|---|---|---|---|
| Archive alarm history data | Archives the expired alert blocking configurations and notifications. | 0 0 3 * * * | ocp2_alarm_filter (Alert blocking configurations) ocp2_alarm_notification (Alert notifications) ocp2_alarm_filter_history (Configuration history for alert blocking) ocp2_alarm_notification_history (Notification history) | "Archive expired alarm history data: result=archive [filter|notification] success, affectRows=xxx" |
| Sync all cluster info | Regularly synchronizes the information about the OceanBase clusters managed by OCP to the ob_cluster table. This task only runs when the values of the operateStatus and clusterStatus parameters of the ob_cluster table are respectively NORMAL and RUNNING. | 0 * * * * * | ob_cluster (The OceanBase cluster managed by OCP) | "cluster sync success, cluster= [id=xx,name=xx,..]" -- Cluster synchronized. "cluster sync failed, cluster=xxx, exceptionType=xxx, message=xxx" -- Cluster synchronization failed. -- Cluster synchronization log " syncForCluster start, clusterId=xx init clusterSyncContext, context=xx create clusterOperator, clusterId=xx get current from ob cluster, clusterId=xx updateZones done, clusterId=xx syncForCluster done, clusterId=xx " |
| Refresh config properties | Refreshes OCP configurations. | 0 */3 * * * * | config_properties (Configuration information of OCP system parameters) | "Refresh config properties on [host:port] at [date & time] " |
| Partitioning daemon | Monitors tenant partitions, creates new partitions each day based on specific policies, and deletes expired partitions. | 4 */10 * * * * | MonitorDB: ob_cluster_system_event (OceanBase Root Service events collected), ob_metric_data_1 (Monitoring data collected every second), ob_metric_data_60 (Monitoring data collected every 60 seconds) | -- Partition maintained. "Finish watching scope" -- Partition maintenance failed. "Watching scope error" or "Watching table error". |
| Sync system event | Regularly synchronizes RootService events from OceanBase clusters to the ob_cluster_system_event table of MonitorDB. | 0 */10 * * * * | MonitorDB: ob_cluster_system_event (OceanBase RootService events collected) OceanBase internal table: oceanbase.__all_rootservice_event_history | -- Synchronized: "Batch saved [number] events to database for cluster [cluster id] Sync-ed [number] system events for cluster [cluster id], startTime [start time], endTime [end time], limit [limit], offset [offset] Finished sync system events: cluster [cluster id], startTime [start time], endTime [end time] " -- Synchronization failed: "An exception occurred when saving events [exception log]" |
| Check all cluster status | Regularly checks the status of OceanBase clusters in the ob_cluster table by creating test database connections. Prints an error log and sends an alert if connection creation failed. | 0 * * * * * | ob_cluster (The OceanBase cluster managed by OCP) | -- Connected to the cluster: "check cluster status okay, clusterId=[cluster id], obVersion=[ob version]" -- Failed to connect to the cluster: "check cluster status unavailable, clusterId=[cluster id], failedReason=[exception msg]" -- Alert type and keyword "ob_cluster_status_check_failed" + "cluster connect check" |
| Check host status | Tests if pos_proxy on the host can execute remote commands by running the whoami command. Sends an alert and changes the host status to OFFLINE in the case of failure. Changes the status back to AVAILABLE or ONLINE if the execution succeeds again. Checks the amount of node_exporters on the host. Prints an exception and sends an alert if the amount is abnormal. | 0 * * * * * | compute_host.status compute_host_agent.last_available_time ocp_exporter_address | "Host agent check result: current OS user is root" -- Agent checked. "Check exporter status: host [id] desired exporter count: [number], actual exporter count: [number]" -- Exporter checked. "Host agent check failed: host [host id] reason [exception msg]" -- Agent check failed. "Alarm: host_unavailable for host ip [ip address]" -- Exporter check failed. "Alarm: no_enough_exporter for host [id]" -- Exporter check failed. "host_unavailable" -- Agent failure alert type "no_enough_exporter" -- Exporter failure alert type |
| Clean ocp task log files | Clean up the task logs on an OCP server. | 0 0 1 * * * | None | Python script that cleans OCP logs: 'user.home' -> '/ocp-server/python-task/src/task/ob_task_clean_log.py' The task log directory to clean: 'user.home' -> '/logs/task/' The file to which the cleaning Python process prints its log: 'user.home' -> '/logs/task/task.2.xxx.log' Exception log of an OCP cleaning task: "failed to clean task log files: [exception msg]" |
| Collect all cluster compaction info | Collect the compaction information of all clusters. | 0 * * * * * | ob_cluster_compaction (Compaction record of OceanBase clusters) ob_zone_compaction (Compaction record of OceanBase zones) ob_server_compaction (Compaction record of OBServers) ob_tenant_compaction_stats (Compaction record in the OceanBase tenant) | -- Collected: "Update cluster [cluster id] compaction status: from [xx] to [xx]" "Compaction post collect done, clusterId= [cluster id], version=[version]" -- Collection failed: "Skip collect for not RUNNING cluster, clusterId=[cluster id], status=[xx]" "Skip collect for version 1, the first compaction will be version 2" "Validate compaction version failed: [reason]" |
| Sync tenant information | Synchronizes tenant information. | 30 0/1 * * * ? | ob_tenant (OceanBase tenant information table) | -- Synchronized: "[SyncAllTenantInfoTask] begin to sync tenant." "[SyncAllTenantInfoTask] sync tenants of specified cluster, clusterId=[cluster id]" "[SyncAllTenantInfoTask] sync tenant finished." -- Synchronization failed: "Zero tenants found for cluster [cluster id]" |
| Detect alarms on schedule | Detector of alerts related to backup and recovery. | 0 */1 * * * * | dag_schedule (Task scheduling details) dag_instance (Background task execution details) | -- Executed generate tasks, templateName=Detect alarms, nodeCount=1, taskCount=1 set state running for task: [task id] set state successful for task: [task id] -- Execution failed Print exception call stack Keyword: BackupAlarmService |
| Clean expired data and log files on schedule | Scheduled cleaner for backup and recovery. Cleans expired data and logs. | 0 0 0 * * * | dag_schedule (Task scheduling details) dag_instance (Background task execution details) | -- Executed generate tasks, templateName=Clean expired data and log files, nodeCount=1, taskCount=1 set state running for task: [task id] set state successful for task: [task id] -- Execution failed Print exception call stack Keyword: BackupCleanService |
| Check system obproxy | Check the health status of the system obproxy that OCP uses. | 0 * * * * * | OCP system parameters: ocp.system.obproxy.address ocp.system.obproxy.port | -- System OBProxy is running properly Prints the following log: system obproxy is running -- System OBProxy is not running properly Prints the following log: system obproxy is down Sends an alert at the same time. -- Failed to execute the check method Prints the following record to the exception call stack: check connection of system obproxy failed. exception: xxx |
| Collect cluster daily metric | Collects the daily cluster monitoring statistics. | 0 0 5 * * * | None | |
| Check obproxy | Checks whether business obproxy is running properly. | 0 * * * * * | obproxy_server (The obproxy_server metadata table) | |
| Sync root service list | Synchronizes the rs_list address. | 0 */2 * * * ? | ob_cluster (The OceanBase cluster managed by OCP) | |
| SQL stats partitioning | The scheduled cron job of the SQL performance table. | 0 0 */1 * * * | dag_schedule (Task scheduling details) dag_instance (Background task execution details) OCP system parameter to disable the task logic: ocp.perf.sql.partition-worker-disabled | |
| Refresh agent config | OCP regularly pushes agent configuration information to each host. | 0 */10 * * * * | None | OCP log: Refreshing OCP agent configuration. Storage path of configuration information on hosts: /home/admin/ocp_agent/conf/config.conf. Content of a configuration information file: [OCP_API] api_url =http://xxx.xxx.xxx.xxx:8080/services HOST ip = xxx.xxx.xxx.xxx:8080/services CLUSTER name = xxx OBPROXY id = x name = obproxy port = xxxx exporter_port = XXXX (different from obproxy__ port) |
Cron expression
A cron expression is used to schedule the tasks and consists of the following six fields:
second, minute, hour, day of month, month, day of week
Fields and values in a cron expression
| Field | Value range | Supported special characters |
|---|---|---|
| second | 0-59 | ,-*/ |
| minute | 0-59 | ,-*/ |
| hour | 0-23 | ,-*/ |
| day of month | 1-31 | ,-*/? |
| month | 1-12 or JAN-DEC | ,-*/ |
| day of week | 1-7 or MON-SUN | ,-*/? |
Cron expression examples
| Expression | Description |
|---|---|
| 0 0 * * * * | "0 second 0 minute": runs once every hour, starting at 0 minute 0 second. |
| */10 * * * * * | Runs once every 10 seconds. |
| 0 0 8-10 * * * | Runs once at 8:00, 9:00, and 10:00 every day, starting at 0 minute 0 second. |
| 0 0/30 8-10 * * * | Runs once at 8:00, 8:30, 9:00, 9:30,10:00, and 10:30 every day. |
| 0 0 9-17 * * MON-FRI | Runs from 9:00 to 17:00 Monday through Friday. |
| 0 0 9-17 * * 1-5 | Runs from 9:00 to 17:00 Monday through Friday. For the day of the week field, Monday is represented by 1, and Sunday is represented by 7. This is slightly different from the crontab expression. |
| 0 0 0 25 12 ? | Runs on December 25 every year, starting at 00:00:00. |