OceanBase Migration Service (OMS) Community Edition allows the admin user to modify system parameters and a general user to only view system parameters.
Procedure
Log in to the console of OMS Community Edition.
In the left-side navigation pane, choose System Management > System Parameters.
The table on the System Parameters page contains the following columns: Parameter Name, Value, Module, Description, and Modified At.
Parameter Description Default value struct.obkv.createtable When you migrate data from an HBase database to OBKV, you can modify the value of this parameter to specify the default CREATE TABLE statement in OBKV for schema migration in all tasks. OMS Community Edition V4.2.10-CE supports TTL. create table if not exists {TABLE_NAME} (Kvarbinary(1024) not null,Qvarbinary(256) not null,Tbigint not null,Vvarbinary(1048576),primary key(K,Q,T)) partition by key(K) partitions 64;mysql_to_obmysql.charset.mapping The rules for converting unsupported character sets in a task that migrates data from a MySQL database to OceanBase Database Community Edition. []
Example: [{"charset":"utf16le","mappedCharset":"utf16"},{"charset":"*","mappedCharset":"utf8mb4"}]sync.unified.config The general parameter for a synchronization task in OMS Community Edition. It has the following three fields: enableHadoopVendorsKafkaServer: specifies whether the Kafka server supports Hadoop.disableIdentificationAlgorithm: specifies whether to disable host name (domain name) verification for the address of the created Kafka data source that requires SSL authentication. If the SSL root certificate provided does not contain the address of this Kafka data source, you can set this parameter totrueto disable host name verification.checkStoreStartedMinSyncProcess: the minimum synchronization progress to verify whether the store starts. The default value is 3 seconds. You can change the value and the change takes effect immediately.
The full migration starts only when the store is running and the synchronization progress exceeds the specified minimum value.fullJvmMem: the initial memory of the Connector component for full migration. The default value is4096 MB.incrJvmMem: the initial memory of the Connector component for incremental synchronization. The default value is2048 MB.
{"enableHadoopVendorsKafkaServer":false,"checkStoreStartedMinSyncProcess":3,"fullJvmMem":4096,"incrJvmMem":2048} sync.ddl.supported The DDL operations supported for data synchronization tasks. {"supportConfigs":{"ADB_SINK":["ALTER_TABLE","ALTER_TABLE_ADD_COLUMN","ALTER_TABLE_MODIFY_COLUMN"],"DATAFLOW_SINK":["ALTER_TABLE","ALTER_TABLE_ADD_COLUMN","ALTER_TABLE_MODIFY_COLUMN"]}} sync.connnector.max.size The maximum number of concurrent data synchronization tasks. 2 struct.transfer.config The parameters related to the execution of a schema migration task. {"dbcat.ob.query.timeout":15,"ob.parallel":4,"independent.obj.convert.batch.size":50,"independent.obj.convert.partition.size":10,"independent.obj.execute.batch.size":50,"independent.obj.execute.partition.size":2,"independent.core.pool.size":256,"independent.max.pool.size":256,"independent.queue.capacity":16,"checkenable":true} store.topic.mode.config The rule that is used to build an allowlist of store subtopics in a data synchronization task in OMS Community Edition. - OceanBase Database supports store subtopics in unshared and shared modes. Store subtopics in shared mode can be shared within a cluster and among tenants.
In theoceanbasefield, you can specifyUN_SHARE,OCEANBASE_TENANT, orOCEANBASE_CLUSTERformode.mode_numspecifies the maximum subscription granularity for the specified mode.- Sharing within a cluster: A store is shared within a cluster. The store configurations in tenants are invalid. The first created store is reused. A new store is created only when the current store does not meet the timestamp requirements.
- Sharing among tenants:
When the value ofmode_numis1, different stores are created for different tenants.
When the value ofmode_numis greater than 1, multiple tenants share the same store. The number of affected tenants is the value ofmode_numminus 1, and the first created store is reused. A new store is created only when the current store does not meet the timestamp requirements.
- OceanBase Database in the MySQL compatible mode supports only the subscription to store subtopics based on service instances.
In themysqlfield, you can specifyINSTANCEorUN_SHAREformode.
{"oceanbase":{"mode":"OCEANBASE_TENANT","modeNum":1},"mysql":{"mode":"INSTANCE","modeNum":1}} precheck.skippable_flags Specifies whether to skip the precheck. In the case of failed precheck items, if you confirm that they have no impact on the database service, you can set their values to truein theprecheck.skippable_flagsparameter. The value of this parameter is of the JSON type. Example:{ "DB_ACCOUNT_FULL_READ_PRIVILEGE": true, "DB_ACCOUNT_INCR_READ_PRIVILEGE":true, "DB_SERVICE_STATUS":true }
For more information about the values of different precheck items, see the Precheck items section in this topic.{} ops.store.max_count_per_subtopic The maximum number of active store processes allowed under a subtopic. The value indicates the maximum number of active store processes allowed. 6 oms.user.password.expiration.date.config The expiration strategy for different user passwords. {"rootRolePasswordValidityDays":90,"rootViewerRolePasswordValidityDays":90,"adminRolePasswordValidityDays":90,"adminViewerRolePasswordValidityDays":90,"userRolePasswordValidityDays":90,"userViewerRolePasswordValidityDays":90,"userPasswordValidityDaysTipsThreshold":30} oms.captcha.enable Specifies whether to enable the verification code feature. After you change the value to true, an image verification code appears on the login page. The image verification code will time out in 10 minutes. You must enter the verification code to log in to OMS Community Edition. A timeout or input error will cause a login failure.false obmysql41_to_obmysql40_and_earlier.collation.mapping The rules for converting unsupported collations in a task that migrates data from OceanBase Database Community Edition V4.1.0 to OceanBase Database Community Edition of an earlier version. [{"collation":"latin1_swedish_ci","mappedCollation":"utf8mb4_general_ci"}] obmysql41_to_obmysql40_and_earlier.charset.mapping The rules for converting unsupported character sets in a task that migrates data from OceanBase Database Community Edition V4.1.0 to OceanBase Database Community Edition of an earlier version. [{"charset":"latin1","mappedCharset":"utf8mb4"}] mysql_to_obmysql.collation.mapping The rules for converting unsupported collations in a task that migrates data from a MySQL database to a MySQL-compatible tenant of OceanBase Database. []
Example: [{"collation":"utf16le_general_ci","mappedCollation":"utf16_general_ci"},{"collation":"*","mappedCollation":"utf8mb4_general_ci"}]alarm.thresholds The alert thresholds. failedLengthOfTimeThreshold: the threshold that triggers a task failure alert when the task execution time exceeds it.syncDelayThreshold: the threshold that triggers an alert when a synchronization task remains in the delayed state longer than this period.syncFailedLengthOfTimeThreshold: the threshold that triggers an alert when a synchronization task remains in the failed state longer than this period.migrateDelayThreshold: the threshold that triggers an alert when a migration task remains in the delayed state longer than this period.migrateFailedLengthOfTimeThreshold: the threshold that triggers an alert when a migration task remains in the failed state longer than this period.alarmRestrainTimeOfMin: the alert suppression time by alert level.HIGH: the high protection level.MEDIUM: the medium protection level.LOW: the low protection level.IGNORE: the no protection level.
{"delayThreshold":{"HIGH":30,"MEDIUM":300,"LOW":900},"failedLengthOfTimeThreshold":{"HIGH":30,"MEDIUM":300,"LOW":900},"alarmRestrainTimeOfMin":{"HIGH":3,"MEDIUM":3,"LOW":3,"IGNORE":100},"rule":"OMS_CONFIG_RULE_ALARM_THRESHOLDS"} mysql.store.metabuilder.filter Specifies whether the MySQL store filters metadata based on the allowlist. Valid values: true: indicates that metadata is filtered based on the allowlist.false: indicates that all metadata is pulled without filtering.
RENAME TABLEstatement, we recommend that you set this parameter totrueto save the time for obtaining metadata. If online DDL statements are used, set this parameter tofalse. Otherwise, subsequent data cannot be consumed after an online DDL statement is executed.false migration.timeout The timeout configuration for migration objects. {"ddl.timeout.in.private.cloud": 172800000, "ddl.timeout.in.public.cloud": 172800000} migration.record.init.batch_size The initial batch size of schema migration objects. 100 migration.mysql.support_datatypes The allowlist of data types supported by the source MySQL database in data migration. The value is an array of data types supported by MySQL. Each element is one MySQL data type. [] migration.mysql.support_collations The allowlist of collations supported by the source MySQL database in data migration. ["binary","gbk","gb18030","utf8mb4","utf16","utf8"] migration.mysql.support_charsets The allowlist of character sets supported by the source MySQL database in data migration. The value is an array of character sets supported by MySQL. Each element is one MySQL character set. ["binary","utf8mb4","utf8"] migration.db.support_versions The source database versions supported in data migration. The key is the database type, and the value is a regular expression containing supported database versions. "MYSQL": "(5.5\|5.6\|5.7\|8.0).*"indicates that OMS Community Edition supports MySQL 5.5, 5.6, 5.7, and 8.0."MARIADB": "10.\[123456\].\*"indicates that OMS Community Edition supports MariaDB 10.1 to 10.6."POSTGRESQL": "(10).\*"indicates that OMS Community Edition supports PostgreSQL 10.x.
{ "MYSQL": "(5.5|5.6|5.7|8.0).*", "MARIADB": "10.[123456].*", "POSTGRESQL": "(10).*"} migration.db.dest.support_versions The target database versions supported in data migration. {"POLARDB_X_1": {"OB_MYSQL": "(1|2|3).*"}} migration.checker.params.steady The parameters that need to be specified when the concurrency of the Full-Import or Full-Verification component is set to Stable. limitator.platform.threads.number: the number of threads.limitator.select.batch.max: the batch query size in full migration or verification.limitator.image.insert.batch.max: the batch insert size in full migration.limitator.datasource.connections.max: the number of connections. If the number of concurrent threads exceeds the number of connections, the excess concurrent threads are invalid.limitator.java.opt: the Java virtual machine (JVM) parameters.
{"limitator.platform.threads.number": 4, "limitator.select.batch.max": 200, "limitator.image.insert.batch.max": 100,"limitator.datasource.connections.max":50, "limitator.java.opt":"-server -Xms4g -Xmx4g -Xmn2g -Xss256k"} migration.checker.params.normal The parameters that need to be specified when the concurrency of the Full-Import or Full-Verification component is set to Normal. limitator.platform.threads.number: the number of threads.limitator.select.batch.max: the batch query size in full migration or verification.limitator.image.insert.batch.max: the batch insert size in full migration.limitator.datasource.connections.max: the number of connections. If the number of concurrent threads exceeds the number of connections, the excess concurrent threads are invalid.limitator.java.opt: the JVM parameters.
{"limitator.platform.threads.number": 8, "limitator.select.batch.max": 600, "limitator.image.insert.batch.max": 200,"limitator.datasource.connections.max":50, "limitator.java.opt":"-server -Xms8g -Xmx8g -Xmn4g -Xss256k"} migration.checker.params.fast The parameters that need to be specified when the concurrency of the Full-Import or Full-Verification component is set to Fast. limitator.platform.threads.number: the number of threads.limitator.select.batch.max: the batch query size in full migration or verification.limitator.image.insert.batch.max: the batch insert size in full migration.limitator.datasource.connections.max: the number of connections. If the number of concurrent threads exceeds the number of connections, the excess concurrent threads are invalid.limitator.java.opt: the JVM parameters.
{"limitator.platform.threads.number": 32, "limitator.select.batch.max": 1200, "limitator.image.insert.batch.max": 400,"limitator.datasource.connections.max":50, "limitator.java.opt":"-server -Xms16g -Xmx16g -Xmn8g -Xss256k"} ha.config Specifies whether to enable high availability (HA). For more information, see Modify HA configurations. {"enable":false,"enableHost":false,"enableStore":true,"enableConnector":true,"enableJdbcWriter":true,"subtopicStoreNumberThreshold":5,"checkRequestIntervalSec":600,"checkHostDownIntervalSec":540,"checkModuleExceptionIntervalSec":240,"clearAbnormalResourceHours":72} oms.telemetry OMS Community Edition supports the telemetry feature. If the value of enabledistrue, the telemetry feature is enabled. For more information, see Telemetry parameters.{"url":"https://openwebapi.oceanbase.com","enabled":true} oms.ce.plugin.log.clean OMS Community Edition V4.2.3 and later support the log cleanup feature. If you enable the log cleanup feature, it runs on an hourly basis to clear the specified log files. This prevents log files from occupying excessive disk space. logCleanSwitch: Specifies whether to enable log cleanup. Default value:false.fuzzyLogNames: the name of the log file to clean up. You can specify the full file name or use a wildcard. An asterisk (*) represents any characters. A question mark (?) represents a single character.logRetentionMinutes: the log retention period. Default value:-60, which specifies to clear log files generated one hour ago.
{"logCleanSwitch":false,"fuzzyLogNames":["congo_20*","liboblog.log.20*","libobcdc.log.20*","connector.20*","common-default.log.20*","check_query.log.20*","common-error.log.20*","oms-ha.log.20*","oms-scheduler.log20*","oms-web.log.20*","oms-step.log.20*","oms-alarm.log.20*","connection-error.log.20*","database.log.20*","oms-api.log.20*","oms-integration.20*","cm-web.log.20*","dao-digest.log.20*","service.log.20*"],"logRetentionMinutes":-60} cloud.container.threshold.config OMS Community Edition supports the alert threshold configuration for the CPU, memory, and disk usage of a container. - cpuRateThreshold: the CPU alert threshold
- memRateThreshold: the memory alert threshold
- diskRateThreshold: the disk usage alert threshold
{"cpuRateThreshold":85,"memRateThreshold":85,"diskRateThreshold":80} Click the edit icon in the Value column of the parameter that you want to modify.
In the Modify Value dialog box, set Value or click Restore Default Value.
Click OK.
Precheck items
The following table describes precheck items that are controlled by the precheck.skippable_flags parameter. The value true indicates that the key can be skipped, and the value false indicates that the key cannot be skipped. For example, if the precheck of the unique key and foreign key can be skipped, you can specify the following statements to configure the precheck.skippable_flags parameter:
{
"DB_UK_INDEX": true,
"DB_FOREIGN_REFERENCE":true,
}
You can log in to the console of OMS Community Edition, go to the details page of a data migration task, and view the names of the precheck items on the Precheck tab, which are prefixed with "Source-" or "Target-".
| Precheck item | Enumeration name |
|---|---|
| Check whether the LOB field exceeds 48 MB in length | DB_TABLE_LOB_SIZE |
Check the ROW_MOVEMENT parameter |
ROW_MOVEMENT |
| Check the time zone of the database | DB_TIMEZONE |
| Check the privilege of the account to create a table | DB_ACCOUNT_CREATE_PRIVILEGE |
| Check the minimum privileges for an Oracle account | DB_ORACLE_MIN_PRIVILEGE |
| Check the table type | DB_TABLE_TYPE |
| Check the connectivity of the database | RDB_CONNECT |
| Check the connectivity of the message queue | MQ_CONNECT |
| Check the connectivity of the logical table | LOGIC_DB_CONNECT |
| Check the existence of logical tables | LOGIC_TABLE_EXIST |
| Check the privilege to obtain ConfigUrl | LOGIC_DB_ACCOUNT_INCR_DRC_READ_PRIVILEGE |
| Check the existence of message queue topics | MQ_TOPIC_EXIST |
| Check the existence of TiCDC Kafka topics | TIDB_KAFKA_TOPIC_EXIST |
| Check the existence of DataHub topics for schema synchronization | DATAHUB_TOPIC_NOT_EXIST |
| Check the existence of databases | RDB_SCHEMA_EXIST |
| Check the case-sensitivity for database names | DB_CASE_SENSITIVE |
| Check the database version | DB_VERSION |
Check the wal_level parameter of the database |
DB_WAL_LEVEL |
| Check the SQL mode of the database | DB_SQL_MODE |
| Check the incremental logs | DB_INCR_LOG |
| Check the clock synchronization of the database | DB_TIME_SYNC |
| Check the primary/standby database | DB_MASTER_SLAVE |
| Check the maximum packet size allowed | DB_MAX_ALLOWED_PACKET |
Check the read privilege of the account on oceanbase.memstore |
DB_MEMSTORE_READ_PRIVILEGE |
Check the CREATE privilege of the MySQL account |
DB_MYSQL_CREATE_PRIVILEGE |
| Check whether the MySQL account has authorized OMS Community Edition to maintain heartbeat data | DB_MYSQL_UPDATE_HEARTBEAT_PRIVILEGE |
Check the privilege of the drc_user user to read the oceanbase database in the sys tenant |
DB_STRUCT_OB_SYSTEM_VIEW_READ_PRIVILEGE |
| Check the connectivity of OceanBase cluster nodes | DB_NODE_CONNECT |
| Check the uniqueness of the table name | DB_TABLE_NAME_UNIQUE |
| Check the existence of tables | RDB_TABLE_EXIST |
| Verify that no LOB field exists | LOB_FIELD_NOT_EXIST |
| Check the schema consistency of logical tables | LOGIC_TABLE_SCHEMA_CONSISTENCY |
| Verify that same target databases and source databases do not constitute a circular replication | LOGIC_TABLE_SAME_SOURCE_AND_DEST |
| Check the schema migration privilege | DB_STRUCT_PRIVILEGE |
| Check the write privilege of the account | DB_ACCOUNT_WRITER_PRIVILEGE |
| Check the full read privilege of the account | DB_ACCOUNT_FULL_READ_PRIVILEGE |
| Check the incremental read privilege of the account | DB_ACCOUNT_INCR_READ_PRIVILEGE |
| Check the character set of the database | DB_CHARSET |
| Check the maximum number of fields in table migration | DB_COLUMN_COUNT |
| Check the database constraints | DB_CONSTRAINT |
| Check the data type of the primary key | DB_DATA_TYPE_INDEX |
| Check the database engine | DB_ENGINE |
| Check the integrity of foreign key dependencies between objects | DB_FOREIGN_REFERENCE |
| Check the function-based unique index table | DB_FUNCTION_BASED |
| Check the full read privilege of the internal accounts | DB_INNER_ACCOUNT_FULL_READ_PRIVILEGE |
| Check the existence of foreign key tables | DB_NO_FOREIGN_KEY |
| Check whether the foreign key constraints of Oracle databases are supported | DB_ORACLE_FK_SUPPORT_CHECK |
| Verify that no pseudocolumn exists | DB_PSEUDO_COLUMN_CHECK |
| Check the data type of the database | DB_DATA_TYPE |
| Check whether the allowlist exceeds 64 KB in length | DB_WHITE_LIST_LENGTH |
| Check the consistency of case-sensitivity configurations for database and table names | DB_LOWER_CASE_TABLE_NAMES |
Check the read privilege on the OceanBase Database system view gv$sysstat |
OB_SYS_STAT_VIEW_READ_PRIV |
| Check the integrity of dependencies between objects | RDB_OBJECT_DEPENDENCY_INTEGRITY |
| Check the limitations on reverse increment from Oracle databases | DB_ORACLE_REVERSE_LIMIT |
| Check the limitations on reverse increment | DB_REVERSE_LIMIT |
| Check whether the same table is used as the source and target | DB_TABLE_CYCLICALLY |
| Check the unique key | DB_UK_INDEX |
Check the ROW_MOVEMENT parameter of the database |
DB_ROW_MOVEMENT |
| Check the unique key of the logical table | LOGIC_TABLE_UK_INDEX |
| Check the full read privilege of the account on logical tables | LOGIC_DB_ACCOUNT_FULL_READ_PRIVILEGE |
| Check the incremental read privilege of the account on logical tables | LOGIC_DB_ACCOUNT_INCR_READ_PRIVILEGE |
| Check the schema consistency | SYNC_SCHEMA_CONSISTENCY |
| Check whether column-level supplemental logging is enabled | DB_COL_LEVEL_SUPPLEMENTAL_LOG |
| Check the read privilege on OceanBase Database system views for full migration or verification | OB_MYSQL_SYS_VIEW_READ_PRIV |
| Check the read privilege on system views | STRUCT_SYS_VIEW_READ_PRIV |
Check the SHOW VIEW privilege of MySQL accounts |
DB_MYSQL_SHOW_VIEW_PRIVILEGE |