OMS Oracle full data migration design and impact

When OMS does a full migration for Oracle, it uses an efficient technique that minimally impacts the source system. This document will detail the design principles and discuss the performance impact on the source system.

Technical implementation

1. Data slicing

   OMS uses the block information of tables in the Oracle database to slice the data tables accordingly. Each slice represents a data query task.

2. Assembling Rowid

   The block information in the slices can eventually be assembled into Rowid. Rowid is a unique identifier in the Oracle database. It is used to locate and access rows in the table.

3. Rowid range query

   When OMS queries the source data, it used the range of Rowid in the slices as the query condition for efficient data retrieval.

4. Concurrent queries

   Different slices query the source database concurrently to increase the data migration speed.

Performance impact

1. No locks

   OMS does not apply locks to the source tables during migration. This does not affect the normal operation of the source system.

2. Low resource consumption

   Since Rowid is used as the query condition, the resource consumption during data retrieval is low. The use of Rowid can reduce the performance consumption caused by full table scans.

Incremental migration

OMS does not need to locate the incremental timestamp. The detailed procedure is as follows:

1. Incremental log fetching

   OMS fetches incremental log before starting the full migration. OMS saves the fetched logs generated by the upstream database to disk.

2. Incremental log playback

   • After the full migration is completed, the incremental log playback, that is, incremental synchronization, is started. Incremental synchronization will consume the data saved on disk into the downstream database.
   • You need to find the timestamp before the full migration to make sure that the accuracy of the incremental data.

3. Multiple write guarantees

   • OMS supports multiple writes during the process from full migration to incremental synchronization. Based on the at least once and idempotency principles, data consistency is guaranteed through PK (primary key) or UK (unique key).
   • Oracle uses Rowid to guarantee idempotency, thereby ensuring data correctness during multiple writes.