OceanBase Migration Service (OMS) Community Edition allows an admin user to modify system parameters and a general user to only view system parameters.
Procedure
Log on to the console of OMS Community Edition.
In the left-side navigation pane, choose System Management > System Parameters.
The table on the System Parameters page contains the following columns: Parameter Name, Value, Module, Description, and Modified At.
Parameter Description Default value struct.obkv.createtable When you migrate data from an HBase database to OBKV, you can modify the value of this parameter to specify the default CREATE TABLE statement in OBKV for schema migration in all tasks. create table if not exists {TABLE_NAME} (Kvarbinary(1024) not null,Qvarbinary(256) not null,Tbigint not null,Vvarbinary(1048576),primary key(K,Q,T)) partition by key(K) partitions 64;mysql_to_obmysql.charset.mapping The rules for converting unsupported character sets in a task that migrates data from a MySQL database to OceanBase Database Community Edition. []
[{"charset":"utf16le","mappedCharset":"utf16"},{"charset":"*","mappedCharset":"utf8mb4"}]sync.unified.config The parameters that you must specify for all synchronization tasks of OMS Community Edition: enableHadoopVendorsKafkaServer: specifies whether the Kafka server supports Hadoop.disableIdentificationAlgorithm: specifies whether to disable hostname (domain name) verification for the address of the created Kafka data source that requires SSL authentication. If the SSL root certificate provided does not contain the address of this Kafka data source, you can set this parameter totrueto disable hostname verification.checkStoreStartedMinSyncProcess: the minimum synchronization progress for determining whether the store is properly started. Default value: 3s. You can change the value and the change takes effect in real time.
The full migration starts only when the store is running and the synchronization progress exceeds the specified minimum value.fullJvmMem: the initial memory for the connector during full migration. Default value: 4096 MB.incrJvmMem: the initial memory for the connector during incremental migration. Default value: 2048 MB.
{"enableHadoopVendorsKafkaServer":false,"checkStoreStartedMinSyncProcess":3,"fullJvmMem":4096,"incrJvmMem":2048} sync.ddl.supported The DDL operations supported for data synchronization tasks. {"supportConfigs":{"ADB_SINK":["ALTER_TABLE","ALTER_TABLE_ADD_COLUMN","ALTER_TABLE_MODIFY_COLUMN"],"DATAFLOW_SINK":["ALTER_TABLE","ALTER_TABLE_ADD_COLUMN","ALTER_TABLE_MODIFY_COLUMN"]}} sync.connnector.max.size The maximum number of concurrent data synchronization tasks. 2 struct.transfer.config The parameters related to the execution of a schema migration task. {"dbcat.ob.query.timeout": 15, "ob.parallel": 2, "independent.obj.convert.batch.size": 50, "independent.obj.convert.partition.size": 10, "independent.obj.execute.batch.size": 50, "independent.obj.execute.partition.size": 2, "independent.core.pool.size": 256, "independent.max.pool.size": 256, "independent.queue.capacity": 16} store.topic.mode.config The rules for building an allowlist of store subtopics in a data synchronization task in OMS Community Edition. - OceanBase Database supports store subtopics in unshared and shared modes. Store subtopics in shared mode can be shared within a cluster and among tenants.
In theoceanbasefield, you can specifyUN_SHARE,DATABASE,OCEANBASE_TENANT, orOCEANBASE_CLUSTERformode.mode_numspecifies the maximum subscription granularity for the specified mode.- Sharing within a cluster: A store is shared within a cluster. The store configurations in tenants are invalid. The first created store is reused. A new store is created only when the current store does not meet the timestamp requirements.
- Sharing among tenants:
When the value ofmode_numis 1, different stores are created for different tenants.
When the value ofmode_numis greater than 1, multiple tenants share the same store. The number of affected tenants is the value ofmode_numminus 1, and the first created store is reused. A new store is created only when the current store does not meet the timestamp requirements.
- OceanBase Database in MySQL mode supports the subscription of store subtopics based only on service instances.
In themysqlfield, you can specify onlyINSTANCEformode.
{"oceanbase":{"mode":"OCEANBASE_TENANT","modeNum":1},"mysql":{"mode":"INSTANCE","modeNum":1}} precheck.skippable_flags Specifies whether to skip the precheck. In the case of failed precheck items, if you confirm that they have no impact on the database service, you can set their values to true in the precheck.skippable_flagsparameter. The value of this parameter is of the JSON type. Example:{ "DB_ACCOUNT_FULL_READ_PRIVILEGE": true, "DB_ACCOUNT_INCR_READ_PRIVILEGE":true, "DB_SERVICE_STATUS":true }.
For more information about the values of different precheck items, see the "Precheck items" section in this topic.{} ops.store.max_count_per_subtopic The maximum number of active store processes allowed under a subtopic. The value indicates the maximum number of active store processes allowed. 6 oms.user.password.expiration.date.config The expiration strategy for different user passwords. {"rootRolePasswordValidityDays":90,"rootViewerRolePasswordValidityDays":90,"adminRolePasswordValidityDays":90,"adminViewerRolePasswordValidityDays":90,"userRolePasswordValidityDays":90,"userViewerRolePasswordValidityDays":90,"userPasswordValidityDaysTipsThreshold":30} oms.captcha.enable Specifies whether to enable the verification code feature. After you change the value to true, an image verification code appears on the logon page. The image verification code will time out in 10 minutes. You must enter the verification code to log on to OMS Community Edition. A timeout or input error will cause a logon failure. false obmysql41_to_obmysql40_and_earlier.collation.mapping The rules for converting unsupported collations in a task that migrates data from OceanBase Database Community Edition V4.1.0 to OceanBase Database Community Edition of an earlier version. [{"collation":"latin1_swedish_ci","mappedCollation":"utf8mb4_general_ci"}] obmysql41_to_obmysql40_and_earlier.charset.mapping The rules for converting unsupported character sets in a task that migrates data from OceanBase Database Community Edition V4.1.0 to OceanBase Database Community Edition of an earlier version. [{"charset":"latin1","mappedCharset":"utf8mb4"}] mysql_to_obmysql.collation.mapping The rules for converting unsupported collations in a task that migrates data from a MySQL database to a MySQL tenant of OceanBase Database. []
Example: [{"collation":"utf16le_general_ci","mappedCollation":"utf16_general_ci"},{"collation":"*","mappedCollation":"utf8mb4_general_ci"}]alarm.thresholds The alert thresholds. failedLengthOfTimeThreshold: the failure alert threshold for a task.syncDelayThreshold: the delay alert threshold for a synchronization task.syncFailedLengthOfTimeThreshold: the failure time alert threshold for a synchronization task.migrateDelayThreshold: the delay alert threshold for a migration task.migrateFailedLengthOfTimeThreshold: the failure time alert threshold for a migration task.alarmRestrainTimeOfMin: the alert suppression time by alert level.HIGH: the high protection level.MEDIUM: the medium protection level.LOW: the low protection level.IGNORE: the No Protection level.
{"delayThreshold":{"HIGH":30,"MEDIUM":300,"LOW":900},"failedLengthOfTimeThreshold":{"HIGH":30,"MEDIUM":300,"LOW":900},"alarmRestrainTimeOfMin":{"HIGH":3,"MEDIUM":3,"LOW":3,"IGNORE":100},"rule":"OMS_CONFIG_RULE_ALARM_THRESHOLDS"} mysql.store.metabuilder.filter Specifies whether the MySQL store filters metadata based on the allowlist. Valid values: true and false. - true: indicates that metadata is filtered based on the allowlist.
- false: indicates that all metadata is pulled without filtering.
false migration.timeout The timeout configuration for the migration object. {"ddl.timeout.in.private.cloud": 172800000, "ddl.timeout.in.public.cloud": 172800000} migration.record.init.batch_size The initial batch size of schema migration objects. 100 migration.mysql.support_datatypes The allowlist of data types supported by the source MySQL database in data migration. The value is an array of data types supported by MySQL. Each element is one MySQL data type. [] migration.mysql.support_collations The allowlist of collations supported by the source MySQL database in data migration. ["binary","gbk","gb18030","utf8mb4","utf16","utf8"] migration.mysql.support_charsets The allowlist of character sets supported by the source MySQL database in data migration. The value is an array of character sets supported by MySQL. Each element is one MySQL character set. ["binary","utf8mb4","utf8"] migration.db.support_versions The source database versions supported in data migration. The key is the database type, and the value is a regular expression containing supported database versions. "MYSQL": "(5.5\|5.6\|5.7\|8.0).*"indicates that OMS Community Edition supports MySQL 5.5, 5.6, 5.7, and 8.0."MARIADB": "10.\[12345\].\*"indicates that OMS Community Edition supports MariaDB 10.1.0 to 10.5.9."POSTGRESQL": "(10).\*"indicates that OMS Community Edition supports PostgreSQL 10.x.
{ "MYSQL": "(5.5|5.6|5.7|8.0).*", "MARIADB": "10.[12345].*", "POSTGRESQL": "(10).*"} migration.db.dest.support_versions The destination database versions supported in data migration. {"POLARDB_X_1": {"OB_MYSQL": "(1 2 3).*"}} migration.checker.params.steady The parameters that need to be specified when the concurrency of the Full-Import component is set to Stable. limitator.platform.threads.number: the number of threads.limitator.select.batch.max: the batch query size in full data migration or verification.limitator.image.insert.batch.max: the batch INSERT size in full data migration.limitator.datasource.connections.max: the number of connections. If the number of concurrent threads exceeds the number of connections, the concurrent data is invalid.limitator.java.opt: the Java virtual machine (JVM) parameters.
{"limitator.platform.threads.number": 4, "limitator.select.batch.max": 200, "limitator.image.insert.batch.max": 100,"limitator.datasource.connections.max":50, "limitator.java.opt":"-server -Xms4g -Xmx4g -Xmn2g -Xss256k"} migration.checker.params.normal The parameters that need to be specified when the concurrency of the Full-Import component is set to Normal. limitator.platform.threads.number: the number of threads.limitator.select.batch.max: the batch query size in full data migration or verification.limitator.image.insert.batch.max: the batch INSERT size in full data migration.limitator.datasource.connections.max: the number of connections. If the number of concurrent threads exceeds the number of connections, the concurrent data is invalid.limitator.java.opt: the JVM parameters.
{"limitator.platform.threads.number": 8, "limitator.select.batch.max": 600, "limitator.image.insert.batch.max": 200,"limitator.datasource.connections.max":50, "limitator.java.opt":"-server -Xms8g -Xmx8g -Xmn4g -Xss256k"} migration.checker.params.fast The parameters that need to be specified when the concurrency of the Full-Import component is set to Fast. limitator.platform.threads.number: the number of threads.limitator.select.batch.max: the batch query size in full data migration or verification.limitator.image.insert.batch.max: the batch INSERT size in full data migration.limitator.datasource.connections.max: the number of connections. If the number of concurrent threads exceeds the number of connections, the concurrent data is invalid.limitator.java.opt: the JVM parameters.
{"limitator.platform.threads.number": 32, "limitator.select.batch.max": 1200, "limitator.image.insert.batch.max": 400,"limitator.datasource.connections.max":50, "limitator.java.opt":"-server -Xms16g -Xmx16g -Xmn8g -Xss256k"} ha.config Specifies whether to enable high availability (HA). For more information, see Modify high availability configurations. {"enable":false,"enableHost":false,"enableStore":true,"enableConnector":true,"enableJdbcWriter":true,"subtopicStoreNumberThreshold":5,"checkRequestIntervalSec":600,"checkHostDownIntervalSec":540,"checkModuleExceptionIntervalSec":240,"clearAbnormalResourceHours":72} oms.telemetry OMS supports the telemetry feature. If the value of enabledistrue, the telemetry feature is enabled. For more information, see Telemetry parameters{"url":"<https://openwebapi.oceanbase.com","enabled":true} oms.ce.plugin.log.clean OMS Community Edition V4.2.3 and later support the log cleanup feature. If you enable the log cleanup feature, it runs on an hourly basis to clear the specified log files. This prevents log files from occupying excessive disk space. logCleanSwitch: Specifies whether to enable log cleanup. Default value:false.fuzzyLogNames: the name of the log file to clean up. You can specify the full file name or use a wildcard. An asterisk (*) represents any characters. A question mark (?) represents a single character.logRetentionMinutes: the log retention period. Default value:-60, which specifies to clear log files generated one hour ago.
{"logCleanSwitch":false,"fuzzyLogNames":["congo_20*","liboblog.log.20*","libobcdc.log.20*","connector.20*","common-default.log.20*","check_query.log.20*","common-error.log.20*","oms-ha.log.20*","oms-scheduler.log20*","oms-web.log.20*","oms-step.log.20*","oms-alarm.log.20*","connection-error.log.20*","database.log.20*","oms-api.log.20*","oms-integration.20*","cm-web.log.20*","dao-digest.log.20*","service.log.20*"],"logRetentionMinutes":-60} Click the edit icon in the Value column for a specified parameter.
In the Modify Value dialog box, set Value or click Reset to Default.
Click OK.
Precheck items
The following table describes precheck items that are controlled by the precheck.skippable_flags parameter. The value true indicates that the key can be skipped, and the value false indicates that the key cannot be skipped. For example, if the precheck of the unique key and foreign key can be skipped, you can specify the following statements to configure the precheck.skippable_flags parameter:
{
"DB_UK_INDEX": true,
"DB_FOREIGN_REFERENCE":true,
}
You can log on to the console of OMS Community Edition, go to the details page of a data migration task, and view the names of the precheck items on the Precheck tab, which are prefixed with "Source-" or "Destination-".
| Precheck item | Enumeration name |
|---|---|
| Check whether the LOB field exceeds 48 MB in length | DB_TABLE_LOB_SIZE |
| Check the ROW_MOVEMENT parameter | ROW_MOVEMENT |
| Check the time zone of the database | DB_TIMEZONE |
| Check the privilege of the account to create a table | DB_ACCOUNT_CREATE_PRIVILEGE |
| Minimum privileges for an Oracle account | DB_ORACLE_MIN_PRIVILEGE |
| Check the table type | DB_TABLE_TYPE |
| Check the connectivity of the database | RDB_CONNECT |
| Check the connectivity of the message queue | MQ_CONNECT |
| Check the connectivity of the logical table | LOGIC_DB_CONNECT |
| Check the existence of logical tables | LOGIC_TABLE_EXIST |
| Check the privilege to obtain configUrl | LOGIC_DB_ACCOUNT_INCR_DRC_READ_PRIVILEGE |
| Check the existence of Message Queue topics | MQ_TOPIC_EXIST |
| TiCDC Kafka topic existence check | TIDB_KAFKA_TOPIC_EXIST |
| Check the existence of Datahub topics for schema synchronization | DATAHUB_TOPIC_NOT_EXIST |
| Check the existence of databases | RDB_SCHEMA_EXIST |
| Check the case-sensitivity for database names | DB_CASE_SENSITIVE |
| Check the database version | DB_VERSION |
| Check the wal_level parameter of the database | DB_WAL_LEVEL |
| Database sql mode check | DB_SQL_MODE |
| Check the incremental logs | DB_INCR_LOG |
| Check database clock synchronization | DB_TIME_SYNC |
| Check primary/standby database | DB_MASTER_SLAVE |
| Check the maximum packet size allowed | DB_MAX_ALLOWED_PACKET |
| Check the read privilege of the account on oceanbase.memstore | DB_MEMSTORE_READ_PRIVILEGE |
| Check the CREATE privilege of the MySQL account | DB_MYSQL_CREATE_PRIVILEGE |
| Check whether the MySQL account has authorized OMS to maintain heartbeat data | DB_MYSQL_UPDATE_HEARTBEAT_PRIVILEGE |
| Check the privilege of the drc_user user to read the OceanBase database in the sys tenant | DB_STRUCT_OB_SYSTEM_VIEW_READ_PRIVILEGE |
| Check the connectivity of OceanBase cluster nodes | DB_NODE_CONNECT |
| Check the uniqueness of the table name | DB_TABLE_NAME_UNIQUE |
| Check the existence of the table | RDB_TABLE_EXIST |
| Disable the check for the existence of LOB fields | LOB_FIELD_NOT_EXIST |
| Check the schema consistency of logical tables | LOGIC_TABLE_SCHEMA_CONSISTENCY |
| Same destination databases and source databases do not constitute a circular replication check | LOGIC_TABLE_SAME_SOURCE_AND_DEST |
| Check the schema migration privilege | DB_STRUCT_PRIVILEGE |
| Check the write privilege of the account | DB_ACCOUNT_WRITER_PRIVILEGE |
| Check the full read privileges of the account | DB_ACCOUNT_FULL_READ_PRIVILEGE |
| Check the incremental read privilege of the account | DB_ACCOUNT_INCR_READ_PRIVILEGE |
| Database character set check | DB_CHARSET |
| Check the maximum number of fields in table migration | DB_COLUMN_COUNT |
| Check the database CONSTRAINT | DB_CONSTRAINT |
| Check the data type of the primary key | DB_DATA_TYPE_INDEX |
| Check the database engine | DB_ENGINE |
| Check foreign key dependency integrity between objects | DB_FOREIGN_REFERENCE |
| Check the functional unique index table | DB_FUNCTION_BASED |
| Check the full read privileges of the internal accounts | DB_INNER_ACCOUNT_FULL_READ_PRIVILEGE |
| Check the existence of the foreign key table | DB_NO_FOREIGN_KEY |
| Check whether the Oracle foreign key constraints are supported | DB_ORACLE_FK_SUPPORT_CHECK |
| Disable Pseudo-Column Existence Check | DB_PSEUDO_COLUMN_CHECK |
| Check the data type of the database | DB_DATA_TYPE |
| Allowlist 64K length limit check | DB_WHITE_LIST_LENGTH |
| Check the consistency of case-sensitivity configurations for database and table names | DB_LOWER_CASE_TABLE_NAMES |
| Reading privilege precheck for OceanBase system view gv$sysstat | OB_SYS_STAT_VIEW_READ_PRIV |
| Check the object dependency integrity | RDB_OBJECT_DEPENDENCY_INTEGRITY |
| Check the limits on Oracle reverse incremental migration | DB_ORACLE_REVERSE_LIMIT |
| Check the limits on reverse incremental migration | DB_REVERSE_LIMIT |
| Check whether the same table is used as the source and the destination | DB_TABLE_CYCLICALLY |
| Check the unique key | DB_UK_INDEX |
| Check the ROW_MOVEMENT parameter of the database | DB_ROW_MOVEMENT |
| Check the unique key of the logical table | LOGIC_TABLE_UK_INDEX |
| Check the full read privileges of the account on logical tables | LOGIC_DB_ACCOUNT_FULL_READ_PRIVILEGE |
| Check the incremental read privilege of the account on logical tables | LOGIC_DB_ACCOUNT_INCR_READ_PRIVILEGE |
| Check the schema consistency | SYNC_SCHEMA_CONSISTENCY |
| Check whether the column-level supplemental logging is enabled | DB_COL_LEVEL_SUPPLEMENTAL_LOG |
| Check the read privilege on the OceanBase system views for full migration or verification | OB_MYSQL_SYS_VIEW_READ_PRIV |
| Check the read privilege on system views | STRUCT_SYS_VIEW_READ_PRIV |
| Check the SHOW VIEW privilege of MySQL accounts | DB_MYSQL_SHOW_VIEW_PRIVILEGE |