This topic describes how to troubleshoot clock synchronization issues.
Applicable versions
The solution provided in this topic is applicable to all versions of OceanBase Database.
Troubleshooting logic
The clock synchronization latency between OBServer nodes must be within 100 ms for an OceanBase cluster. If clock synchronization is unstable or the clock offset exceeds the threshold, the stability and availability of the OceanBase cluster are directly affected. Therefore, in case of cluster unavailability or frequent primary/standby switchovers, check the clock synchronization status.
Generally, in the commercial edition of OceanBase Database delivered by OceanBase, clock synchronization is configured and checked before a cluster is created. In case of clock out-of-synchronization, check the clock status and offset.
Troubleshooting procedure
In OceanBase Database deployment, two services are usually used for clock synchronization: Network Time Protocol (NTP) and chrony. To check the clock synchronization status and offset, perform the following operations.
Clock synchronization based on NTP
The NTP service must be started in -x mode (slew mode) so that the clock offset (if any) can be adjusted slowly. The -x option ensures that the clock offset is to be adjusted by 1s every 2000s by default.
Check the NTP status.
Run the
ntpstatcommand to check the status of the NTP server. If the result issynchronised to NTP server, the NTP configurations are synchronized.[root@hostname /]# ntpstat synchronised to NTP server (xx.xxx.xx.xxx) at stratum 4 time correct to within 4 ms polling server every 32 sCheck the NTP offset.
Run the following command multiple times to ensure that the returned value of
offsetis stable. If the value ofoffsetis less than 50, in milliseconds , clock synchronization is normal.[root@hostname /]# ntpq -p|grep -E "\*|\=|remote" remote refid st t when poll reach delay offset jitter ========================================================================== *xxxxxxxxxxx xxxxx 2 u 16 64 377 1.333 0.046 0.033
Clock synchronization based on chrony
Check the chrony status.
Run the following command to check the synchronization status of chrony. If the returned status is Normal, the clock is synchronized.
$chronyc -n tracking |grep Leap Leap status : NormalCheck the chronyc offset.
Run the following command multiple times. The last column in the output indicates the clock offset. A clock offset less than 50 ms is acceptable.
[admin@hostname ~]$ chronyc sources -v MS Name/IP address strarum Poll Reach LastRX Last sample ================================================================================== ^- <hostname> 6 4 377 15 -251us [ -251us] -/- 12ms ^* <hostname> 6 4 377 16 -322us [ -340us] -/- 26ms
If you cannot identify the cause of the clock synchronization issue by following one of the preceding procedures, contact OceanBase Technical Support.