Modify system parameters

2023-06-28 11:59:36  Updated

OceanBase Migration Service (OMS) Community Edition allows the admin user to modify system parameters and general users to view system parameters.

Procedure

  1. Log on to the OMS Community Edition console.

  2. In the left-side navigation pane, choose System Management > System Parameters.

    The table on the System Parameters page contains the following columns: Parameter Name, Value, Module, Description, and Modified At.

    Parameter Description Default value
    oms.captcha.enable Specifies whether to enable the verification code feature. After you change the value to true, an image verification code appears on the logon page. The image verification code will time out in 10 minutes. You must enter the image verification code to log on to OMS. A timeout or input error will cause a logon failure. false
    mysql.store.metabuilder.filter Specifies whether the MySQL store filters metadata based on the whitelist. Valid values: true and false.
    • true: indicates that metadata is filtered based on the whitelist.
    • false: indicates that all metadata is pulled without filtering.
    In scenarios without online DDL statements, which need to be implemented by using the RENAME TABLE statement, we recommend that you set this parameter to true to reduce the time for obtaining metadata. If online DDL statements are used, set this parameter to false. Otherwise, subsequent data cannot be consumed after an online DDL statement is executed.
    false
    mysql_to_obmysql.charset.mapping The conversion rule for character sets that are not supported in a project of migrating data from a MySQL database to a MySQL tenant of OceanBase Database. []
    Example: [{"charset":"utf16le","mappedCharset":"utf16"},{"charset":"*","mappedCharset":"utf8mb4"}]
    mysql_to_obmysql.collation.mapping The conversion rule for collations that are not supported in a project for migrating data from a MySQL database to a MySQL tenant of OceanBase Database. []
    Example: [{"collation":"utf16le_general_ci","mappedCollation":"utf16_general_ci"},{"collation":"*","mappedCollation":"utf8mb4_general_ci"}]
    alarm.thresholds The alert thresholds.
    • failedLengthOfTimeThreshold: the failure alert threshold for a project.
    • syncDelayThreshold: the delay alert threshold for a synchronization project.
    • syncFailedLengthOfTimeThreshold: the failure time alert threshold for a synchronization project.
    • migrateDelayThreshold: the delay alert threshold for a migration project.
    • migrateFailedLengthOfTimeThreshold: the failure time alert threshold for a migration project.
    • alarmRestrainTimeOfMin: the alert suppression time by alert level.
    • HIGH: the high protection level.
    • MEDIUM: the medium protection level.
    • LOW: the low protection level.
    • IGNORE: the No Protection level.
    {"delayThreshold":{"HIGH":30,"MEDIUM":300,"LOW":900},"failedLengthOfTimeThreshold":{"HIGH":30,"MEDIUM":300,"LOW":900},"alarmRestrainTimeOfMin":{"HIGH":3,"MEDIUM":3,"LOW":3,"IGNORE":100},"rule":"OMS_CONFIG_RULE_ALARM_THRESHOLDS"}
    ha.config Specifies whether to enable high availability (HA). For more information, see Modify HA configurations. {"enable":false,"enableHost":false,"enableStore":true,"enableConnector":true,"enableJdbcWriter":true,"subtopicStoreNumberThreshold":5,"checkRequestIntervalSec":600,"checkHostDownIntervalSec":540,"checkModuleExceptionIntervalSec":240,"clearAbnormalResourceHours":72}
    migration.checker.params.fast The parameters that need to be specified when the concurrency of the checker is Fast.
    • limitator.platform.threads.number: the number of threads.
    • limitator.select.batch.max: the batch query size in full data migration or verification.
    • limitator.image.insert.batch.max: the batch INSERT size in full data migration.
    • limitator.datasource.connections.max: the number of connections. If the number of concurrent threads exceeds the number of connections, the concurrent data is invalid.
    • limitator.java.opt: the Java virtual machine (JVM) parameters.
    {"limitator.platform.threads.number": 32, "limitator.select.batch.max": 1200, "limitator.image.insert.batch.max": 400,"limitator.datasource.connections.max":50, "limitator.java.opt":"-server -Xms16g -Xmx16g -Xmn8g -Xss256k"}
    migration.checker.params.normal The parameters that need to be specified when the concurrency of the checker is Normal.
    • limitator.platform.threads.number: the number of threads.
    • limitator.select.batch.max: the batch query size in full data migration or verification.
    • limitator.image.insert.batch.max: the batch INSERT size in full data migration.
    • limitator.datasource.connections.max: the number of connections. If the number of concurrent threads exceeds the number of connections, the concurrent data is invalid.
    • limitator.java.opt: the Java virtual machine (JVM) parameters.
    {"limitator.platform.threads.number": 8, "limitator.select.batch.max": 600, "limitator.image.insert.batch.max": 200,"limitator.datasource.connections.max":50, "limitator.java.opt":"-server -Xms8g -Xmx8g -Xmn4g -Xss256k"}
    migration.checker.params.steady The parameters that need to be specified when the concurrency of the checker is set to Smooth.
    • limitator.platform.threads.number: the number of threads.
    • limitator.select.batch.max: the batch query size in full data migration or verification.
    • limitator.image.insert.batch.max: the batch INSERT size in full data migration.
    • limitator.datasource.connections.max: the number of connections. If the number of concurrent threads exceeds the number of connections, the concurrent data is invalid.
    • limitator.java.opt: the Java virtual machine (JVM) parameters.
    {"limitator.platform.threads.number": 4, "limitator.select.batch.max": 200, "limitator.image.insert.batch.max": 100,"limitator.datasource.connections.max":50, "limitator.java.opt":"-server -Xms4g -Xmx4g -Xmn2g -Xss256k"}
    migration.db.support_versions The source database versions supported in data migration. The key is the database type, and the value is a regular expression containing supported database versions.
    • "MYSQL": "(5.5|5.6|5.7|8.0).*": indicates that OMS supports MySQL V5.5, V5.6, V5.7, and V8.0.
    • "MARIADB": "10.[12345].*": indicates that OMS supports MariaDB 10.1.0 to 10.5.9.
    • "POSTGRESQL": "(10).\*": indicates that OMS supports POSTGRESQL 10.x.
    { "MYSQL": "(5.5|5.6|5.7|8.0).*", "MARIADB": "10.[12345].*", "POSTGRESQL": "(10).*"}
    migration.mysql.support_charsets The whitelist of character sets supported by the source MySQL database in data migration. The value is an array of character sets supported by MySQL. Each element is one MySQL character set. ["binary","gbk","gb18030","utf8mb4","utf16","utf8"]
    migration.mysql.support_datatypes The whitelist of data types supported by the source MySQL database in data migration. The value is an array of data types supported by MySQL. Each element is one MySQL data type. []
    ops.dms.logic_name.suffix.pattern The prefix of the DMS-based logical table synchronization task. Empty
    ops.store.max_count_per_subtopic The maximum number of active store processes allowed under a subtopic. The value indicates the maximum number of active store processes allowed. 6
    precheck.skippable_flags Specifies whether to skip the precheck. In the case of failed precheck items, if you confirm that they have no impact on the database service, you can set their values to true in the precheck.skippable_flags parameter. The value of this parameter is of the JSON type. Example:
    { "DB_ACCOUNT_FULL_READ_PRIVILEGE": true, "DB_ACCOUNT_INCR_READ_PRIVILEGE":true, "DB_SERVICE_STATUS":true }.
    For more information about the values of different precheck items, see the "Precheck items" section in this topic.
    {}
    sync.unified.config The general parameter for an OMS Community Edition synchronization project. It has the following three fields:
    • enableHadoopVendorsKafkaServer: specifies whether the Kafka server supports Hadoop.
    • disableIdentificationAlgorithm: specifies whether to disable hostname (domain name) verification for the address of the created Kafka data source that requires SSL authentication. If the SSL root certificate provided does not contain the address of this Kafka data source, you can set this parameter to true to disable hostname verification.
    • checkStoreStartedMinSyncProcess: the minimum synchronization progress for determining whether the store is properly started. Default value: 3s. You can change the value and the change takes effect in real time.
      The full migration starts only when the store is running and the synchronization progress exceeds the specified minimum value.
    • fullJvmMem: the initial memory for the connector during full migration. Default value: 4096 MB.
    • incrJvmMem: the initial memory for the connector during incremental migration. Default value: 2048 MB.
    {"enableHadoopVendorsKafkaServer":false,"checkStoreStartedMinSyncProcess":3,"fullJvmMem":4096,"incrJvmMem":2048}
    store.topic.mode.config Specifies the rule that is used to build a whitelist of store subtopics in a data synchronization project in OMS.
      OceanBase Database supports the subscription of store subtopics based on tenants or clusters.
      In the oceanbase field, you can specify OCEANBASE_TENANT or OCEANBASE_CLUSTER for mode. The mode_num indicates the maximum subscription granularity for the specified mode.
    • MySQL supports the subscription of store subtopics based only on service instances. In the mysql field, you can specify only MYSQL_INSTANCE for mode.
    {"oceanbase":{"mode":"OCEANBASE_TENANT","modeNum":1},"mysql":{"mode":"MYSQL_INSTANCE","modeNum":1}
    sync.connnector.max.size The maximum number of concurrent data synchronization projects. 2
    sync.ddl.supported The DDL operations supported for data synchronization projects. {"supportConfigs":{"ADB_SINK":["ALTER_TABLE","ALTER_TABLE_ADD_COLUMN","ALTER_TABLE_MODIFY_COLUMN"],"DATAFLOW_SINK":["ALTER_TABLE","ALTER_TABLE_ADD_COLUMN","ALTER_TABLE_MODIFY_COLUMN"]}}
    migration.timeout The timeout period for executing migration objects. {"ddl.timeout.in.private.cloud": 172800000, "ddl.timeout.in.public.cloud": 172800000}
  3. Click the edit icon in the Value column for a specified parameter.

  4. In the Modify Value dialog box, set Value or click Reset to Default.

  5. Click OK.

Precheck items

The following table describes precheck items that are controlled by the precheck.skippable_flags parameter. The value true indicates that the key can be skipped, and the value false indicates that the key cannot be skipped. For example, if the precheck of the unique key and foreign key can be skipped, you can specify the following statements to configure the precheck.skippable_flags parameter:

{
  "DB_UK_INDEX": true,
  "DB_FOREIGN_REFERENCE":true,
}

You can log on to the OMS Community Edition console, go to the details page of a data migration project, and view the names of the precheck items on the Precheck tab, which are prefixed with "Source-" or "Destination-".

Precheck item Enumeration name
Unique key check DB_UK_INDEX
Incremental log check DB_INCR_LOG
Foreign key check DB_FOREIGN_REFERENCE
Account full read permission check DB_ACCOUNT_FULL_READ_PRIVILEGE
Account write permission check DB_ACCOUNT_WRITER_PRIVILEGE
Account Incremental Read Permission Check DB_ACCOUNT_INCR_READ_PRIVILEGE
OB cluster node connectivity check DB_NODE_CONNECT
White List 64K Length Limit Check DB_WHITE_LIST_LENGTH
Account read privilege on oceanbase.gv$memstore check DB_MEMSTORE_READ_PRIVILEGE
Table name uniqueness check DB_TABLE_NAME_UNIQUE
LOB field 48m limit check DB_TABLE_LOB_SIZE
Database ROW_MOVEMENT check DB_ROW_MOVEMENT
Check for the migration of no more than 508/4092 fields in table migration DB_COLUMN_COUNT
Database data type check DB_DATA_TYPE
Database engine check DB_ENGINE
Inner account full read permission check DB_INNER_ACCOUNT_FULL_READ_PRIVILEGE
Database running status check DB_SERVICE_STATUS
Account permission on table creation check DB_ACCOUNT_CREATE_PRIVILEGE
Partition table check DB_PARTITION_TABLE
Kafka topic existence check KAFKA_TOPIC
RocketMQ topic existence check ROCKETMQ_TOPIC
Check whether the source and target ends are the same LOGIC_TABLE_SAME_SOURCE_AND_DEST
Check the clock synchronization of the database DB_TIME_SYNC

Contact Us