Parameters of the Full-Verification component

2025-11-03 03:26:08  Updated
Parameter Default value Description Applicable version
limitator.platform.threads.number The default value is subject to resource configurations for full verification. The number of data source read threads to be verified. The number of read threads at the source and that at the target are subject to the limitator.platform.threads.number and limitator.source.max.thread.size parameters. The calculation logic is as follows:
  1. The default value of limitator.source.max.thread.size is 6.
  2. The initial number of read threads for the source/target is limitator.platform.threads.number/2.
  3. The smaller one of the preceding two values is used as the number of read threads.
For example, if the value of limitator.platform.threads.number is 10, each of the source and target has five read threads. If the value of limitator.platform.threads.number is 14, each of the source and target has six read threads.
All versions
limitator.platform.split.threads.number 16 The number of splitting threads to be verified. All versions
limitator.select.batch.max 600 The maximum number of records of each split part. In the case of table splitting based on the primary key or unique key, the value is the accurate number of records processed in each split part.

Note

We recommend that you do not modify this parameter unless a splitting performance bottleneck occurs.

All versions
limitator.reviewer.rounds.max 20 The number of reverification rounds.

Note

If you want to quickly obtain data inconsistencies or in a confirmed data inconsistency scenario, you can set the value of this parameter to 0 to directly output the data comparison results without reverification.

All versions
limitator.reviewer.review.batch.max 100 The number of inconsistent unique keys to be verified by each reverification task. All versions
limitator.review.query.batch.limit 100 The number of unique keys queried at the source/target at a time in reverification scenarios. This parameter is used in combination with limitator.reviewer.review.batch.max. For example, if limitator.reviewer.review.batch.max is set to 100 and limitator.review.query.batch.limit is set to 50, one reverification task needs to reverify 100 unique keys in total and can query 50 unique keys at the source/target each time. Therefore, the reverification task will access the database twice. Versions later than oboms-verifier-2.0.0-xxx
filter.verify.rectify.type No Indicates whether to correct inconsistent data. Valid values:
  • no: specifies not to correct inconsistent data detected.
  • now: specifies to immediately correct inconsistent data detected.

Notice

  • It is prohibited to set this parameter to now unless in emergency scenarios. If you need to set this parameter to now, contact OMS Community Edition Technical Support for confirmation.
  • Automatic correction is supported in versions later than oboms-verifier-2.0.0-xxx but not supported in oboms-verifier-1.xxx for migration between MySQL-compatible tenants of OceanBase Database or migration from a MySQL database to a MySQL-compatible tenant of OceanBase Database.
All versions
filter.verify.inmod.tables N/A Specifies to verify tables by using the inmode mechanism. OMS Community Edition supports general verification and inmode verification. The two verification modes differ in the method used by the target to read data. In inmode verification, the target does not read data based on the split range. Instead, the target calculates primary key values based on the data read by the source from a split part, and then reads data based on the primary key values.

Notice

The inmode verification is inapplicable if the target has data that does exist at the source, and will downgrade the verification performance.

Here are some examples of general scenarios:
  • Specify to use inmode verification for all tables of the current task: filter.verify.inmod.tables=.*;.*;.*
  • Specify to use inmode verification for the T1 table in the D1 database: filter.verify.inmod.tables = D1;T1;.*
  • Specify to use inmode verification for the T1 and T2 tables in the D1 database: filter.verify.inmod.tables = D1;T1;.*\|D1;T2;.*
  • Specify to use inmode verification for the T1 and T2 tables in the D1 database and the T3 and T4 tables in the D2 database: filter.verify.inmod.tables = D1;T1;.*\|D1;T2;.*\|D2;T3;.*\|D2;T4;.*
All versions
limitator.table.diff.max 10000 The maximum number of data inconsistencies allowed for each table. If the number of data inconsistencies in a table exceeds this value, Full-Verification marks the table as an abnormal one and will not verify the remaining data.

Notice

We recommend that you do not set this parameter to a value greater than 10 million.

All versions
limitator.ready.verify.record.queue.size 64 The size of data to be verified. Versions later than oboms-verifier-2.0.0-xxx
limitator.ready.rectify.record.queue.size 64 The size of data to be corrected. Versions later than oboms-verifier-2.0.0-xxx
limitator.sink.slice.queue.size 64 The size of the splitting queue at the target. Versions later than oboms-verifier-2.0.0-xxx
task.coordinatorSection.reviewTaskQueueSize 64 The size of the reverification task queue. Versions later than oboms-verifier-2.0.0-xxx
limitator.verify.thread.size 8 The number of verification threads. Versions later than oboms-verifier-2.0.0-xxx
limitator.review.thread.size 16 The number of reverification threads. Versions later than oboms-verifier-2.0.0-xxx
limitator.rectify.thread.size 32 The number of correction threads. Versions later than oboms-verifier-2.0.0-xxx
limitator.slice.split.batch.size N/A The number of unique keys concatenated in a single IN query at the target. Versions later than oboms-verifier-1.0.23-20230904200300
task.sourceMasterSection.matchAllColumn true Specifies whether to perform verification only if all fields of source tables are mapped to those of target tables. Valid values: true and false.
  • true: specifies that verification is performed only if all fields are mapped.
  • false: specifies that verification is performed only on mapped fields.
OMS Community Edition V4.2.4 and later
task.sourceMasterSection.excludeColumn N/A The fields of the source tables that are not to be verified. Note that you cannot exclude fields in primary keys or unique keys. Otherwise, errors will occur during the verification. You can use a regular expression to specify fields to be excluded in the format of {"regular expression":"c1,c2,.."}. The table name in the regular expression must be in the format of schema.tablename. OMS Community Edition V4.2.4 and later
task.sourceMasterSection.joinField None The fields used for table join. The format is {"database name.table name": "source field name:target field name,...."}. Multiple fields are separated by commas (,).
If the source field name and target field name are the same, you can configure it as {"database name.table name": "source field name,....."}.
OMS Community Edition V4.2.11 and later

The applicable versions in the preceding table are versions of Full-Verification. You can perform the following steps to query the current Full-Verification version of OMS Community Edition:

  1. Log in to the server where OMS Community Edition is deployed.

  2. Run the following command to enter the Docker container:

    docker exec -it ${CONTAINER_NAME} bash
    

    Notice

    CONTAINER_NAME specifies the name of the container.

  3. Run the following command to query the version of Full-Verification:

    rpm -qa | grep "verifier"
    

Contact Us