CreateProject|V4.3.1|OceanBase Migration Service| docs|Distributed Database

This topic describes the CreateProject OpenAPI provided by OceanBase Migration Service (OMS): how to create a task.

Purpose

You can call this API to create a data migration, synchronization, or validation task.

Call description

Limitations

This API is supported in OMS V4.1.0 and later.

Request path

POST /api/v2?Action=CreateProject

Request parameters

Parameter	Type	Description
name	string	The task name, which cannot contain spaces.
type	string	The task type. `MIGRATION` indicates a data migration task, `SYNC` indicates a data synchronization task, and `VERIFY` indicates a data validation task.
labelIds	array	The IDs of the labels. At most five IDs are supported.
sourceEndpointId	string	The ID of the source data source. To obtain the ID, call the `ListDataSource` interface. The ID is in the e_**** format.
sinkEndpointId	string	The ID of the target data source. To obtain the ID, call the `ListDataSource` interface. The ID is in the e_**** format.
transferMapping	object	The mapping information of the migration objects.
├─ mode	string	The object mapping mode. Valid values: `SPECIFIC` (specified object), `WILDCARD` (matching rule), and `COMPLEX` (contains both matching rules and specific objects).
├─ databases	object []	The list of database objects. For more information, see "Parameters in databases/databasesBlack".
├─ databasesBlack	object []	The list of blacklisted database objects in the matching rule mode. For more information, see "Parameters in databases/databasesBlack".
├─ tableAndViewWhiteList	array	The wildcard whitelist of tables and views in the matching rule mode.
├─ tableAndViewBlackList	array	The wildcard blacklist of tables and views in the matching rule mode.
commonTransferConfig	object	The general migration configuration.
├─ tableCategory	string	The type of supported tables. Valid values: ALL (all tables), WITH_UNIQUE_ROW_ID (tables with unique row identifiers), and WITHOUT_UNIQUE_ROW_ID (tables without unique row identifiers). Default value: ALL.
├─ activeActive	boolean	Specifies whether the task is a disaster recovery active-active scenario (deprecated in OMS V4.3.1).
├─ mqSerializerType	string	The JSON serialization type when data is delivered to a message queue. Valid values: DEFAULT, CANAL, DATAWORKS_V2, SHAREPLEX, DEFAULT_WITH_SCHEMA, DEBEZIUM, DEBEZIUMFLATTEN, and DEBEZIUMSMT.
├─ mqPartitionMode	string	The partition routing mode when data is delivered to a message queue. Valid values: ONE (specify the partition for delivery), HASH (hash partition the data based on the primary key or the sharding column value), and TABLE (hash partition the data based on the database and table name).
├─ mqPartition	integer	When `partitionMode` is `ONE`, this parameter specifies the position of the partition in the current topic.
├─ datahubTopicType	string	The type of the topic to which data is delivered in the DataHub format. Valid values: `TUPLE` and `BLOB`.
├─ rocketMqProducerGroup	string	The name of the producer group. This parameter is effective only when the target is a RocketMQ database. Default value: OMS.
├─ rocketMqMsgTags	string	The message tag. This parameter is effective only for the RocketMQ database. Default value: default_tag. When the value of this parameter is `default`, _tag is automatically added as a suffix.
├─ rocketMqEnableMsgTrace	boolean	Specifies whether to enable message tracing. This parameter is effective only when the target is a RocketMQ database.
├─ rocketMqSendMsgTimeout	long	The timeout period for message delivery. This parameter is effective only when the target is a RocketMQ database.
├─ dataWorksBusinessName	string	The identifier of the business system. This parameter identifies the business system that originates the data so that you can perform custom processing on the data later. This parameter is supported only when the JSON serialization type is DataWorks. The value of this parameter must be 1 to 20 characters long.
├─ sinkStoreFormat	string	The storage type of the target table. Valid values: Default, Row Storage, Column Storage, and Hybrid Row-Column Storage. This parameter is effective only when the target OceanBase Database is V4.3.0 or later, and the migration type is Schema Migration or Incremental Synchronization > DDL synchronization.
├─ sourceStoreFormat	string	The storage type of the source table. Valid values: Default, Row Storage, Column Storage, and Hybrid Row-Column Storage. This parameter is effective only when the source OceanBase Database is V4.3.0 or later, and the migration type is Reverse Increment > DDL synchronization.
├─ enableHiddenColumnsAndIndex	boolean	Specifies whether to add hidden columns and indexes for tables without non-null unique keys. This parameter applies to the following five types of data migration tasks: Oracle to OB_MySQL, Oracle to OB_Oracle, OB_MySQL to OB_MySQL, OB_Oracle to OB_Oracle, and OB_Oracle to Oracle.
├─ interactiveMode	boolean	Specifies whether to manually advance the task through the interface.
enableStructTransfer	boolean	Specifies whether to enable schema migration.
structTransferConfig	object	The schema migration configuration.
├─ byteCharConvertStrategy	string	The byte/char conversion strategy. Valid values: `IGNORE_BYTE_IF_BYTE_USED`, `FORCE_CHAR_IF_BYTE_USED`, `EXPAND_LEN_IF_BYTE_USED`, and `DO_NOTHING_IF_BYTE_USED` (default).
├─ deferIndexCreation	boolean	Specifies whether to enable index creation deferral. Default value: false. This parameter is deprecated in OMS V4.3.0 and later. We recommend that you use the `indexMigrationType` parameter instead.
├─ indexMigrationType	string	The ordinary index migration mode. Valid values: `NOT_MIGRATION` (do not migrate), `WITH_STRUCT_TRANSFER` (migrate with schema migration), and `AFTER_FULL_TRANSFER` (migrate after full migration). Full migration cannot be configured as the migration mode.
├─ autoProcessedToNextStep	boolean	Specifies whether to automatically go to the next step after schema migration is complete. Default value: true.
├─ triggerMigrationType	string	The trigger migration mode. Valid values: `NOT_MIGRATION` (do not migrate), `WITH_STRUCT_TRANSFER` (migrate with schema migration), and `WITH_TRANSFER_APP_SWITCH` (migrate during forward switchover). This parameter is not supported for data migration tasks other than those from an Oracle database to an Oracle compatible mode of OceanBase Database. If you set this parameter for a data migration task from an Oracle database to an Oracle compatible mode of OceanBase Database, you must set Forward switchover as the trigger migration mode.
enableFullTransfer	boolean	Specifies whether to enable full migration.
enableFullVerify	boolean	Specifies whether to enable full validation.
enableCountVerify	boolean	Specifies whether to enable row count validation.
enableIncrVerify	boolean	Specifies whether to enable incremental validation.
fullTransferConfig	object	The full migration configuration.
├─ nonePkUkTruncateDstTable	boolean	Specifies whether to truncate the target table when you perform full migration from a source table without a unique index.
├─ allowDestTableNotEmpty	boolean	The processing strategy when the target table contains records. Valid values: Ignore and Stop Migration. If this parameter is set to true, Ignore, OMS writes data to the target table. If a conflict occurs between the original data and the written data, OMS logs the conflicting data and keeps the original data unchanged. Notice If you set this parameter to Ignore, full validation will use the IN mode to pull data. OMS cannot verify whether the target table contains data that does not exist in the source table, and the validation performance will be degraded to some extent. If this parameter is set to false, Stop Migration, an error is returned when migration is attempted for the target table that contains data. Make sure that the data in the target table is properly processed before you continue the migration. Notice If you click Resume after an error occurs, OMS ignores this parameter and continues to migrate table data. Proceed with caution.
├─ fullTransferSpeedMode	string	The full migration speed. Valid values: `STEADY`, `NORMAL`, `FAST`, and `CUSTOM`.
├─ fullTransferSpeedConfig	string	The full migration speed configuration. This parameter is effective only when `fullTransferSpeedMode` is set to `CUSTOM`.
├─├─ memory	integer	The memory limit, in GB.
├─├─ readerWorkerNum	integer	The number of concurrent readers.
├─├─ writerWorkerNum	integer	The number of concurrent writers.
├─ fullVerifySpeedMode	string	The full validation speed. Valid values: `STEADY`, `NORMAL`, `FAST`, and `CUSTOM`.
├─ fullVerifySpeedConfig	string	The full validation speed configuration. This parameter is effective only when `fullVerifySpeedMode` is set to `CUSTOM`.
├─├─ memory	integer	The memory limit, in GB.
├─├─ readerWorkerNum	integer	The number of concurrent readers.
├─├─ writerWorkerNum	integer	The number of concurrent writers.
├─ throttleRps	integer	The maximum allowed transfer rate in records per second during full migration. Set this parameter to null to indicate no restriction on RPS.
├─ throttleIOPS	integer	The maximum allowed transfer traffic in bytes per second during full migration. Set this parameter to null to indicate no restriction on IOPS.
enableIncrTransfer	boolean	Specifies whether to enable incremental synchronization.
enableReverseIncrTransfer	boolean	Specifies whether to enable reverse incremental synchronization.
incrTransferConfig	object	The incremental synchronization configuration.
├─ startTimestamp	string	The start timestamp of incremental synchronization. This parameter is supported only when incremental synchronization is enabled but full migration is not enabled.
├─ recordTypeWhiteList	array	The types of incremental data to be synchronized in the incremental synchronization phase. Valid values: HEARTBEAT, INSERT, UPDATE, DELETE, BEGIN, COMMIT, ROLLBACK, DDL, and ROW.
├─ storeLogKeptHour	integer	The retention period of incremental logs when the incremental log pull component is a store, in hours. The default value is 7, and the maximum value is 365.
├─ enableSequencingWithinTxn	boolean	Specifies whether to enable sequence number arrangement within a transaction when the incremental log pull component is a store.
├─ enableIncrSyncStatistics	boolean	Specifies whether to enable statistics collection for incremental records (DML and DDL operations).
├─ incrOnlineDdlConfig	array	The value of this parameter indicates the name of an online DDL tool. Valid values: GH_OST, PT_OSC, and ALIYUN_DMS, which respectively indicate the gh-ost, pt-osc, and Alibaba Cloud DMS tools. This parameter takes effect on forward incremental synchronization from a MySQL database to a MySQL-compatible tenant of OceanBase Database. This parameter takes effect on reverse incremental synchronization from a MySQL-compatible tenant of OceanBase Database to a MySQL database.
├─ incrSyncConcurrency	integer	The number of threads for incremental synchronization. Default value: 64.
├─ incrLogPullSpeedMode	string	The incremental log pull speed. Valid values: STEADY (low), NORMAL (medium), FAST (high), and CUSTOM (custom).
├─ incrLogPullSpeedConfig	string	The incremental log pull resource configuration. This parameter is effective only when `incrLogPullSpeedMode` is set to `CUSTOM`.
├─├─ memory	integer	The memory limit for incremental log pull.
├─ incrLogSyncSpeedMode	string	The incremental data write speed. Valid values: STEADY (low), NORMAL (medium), FAST (high), and CUSTOM (custom).
├─ incrLogSyncSpeedConfig	string	The incremental data write resource configuration. This parameter is effective only when `incrLogSyncSpeedMode` is set to `CUSTOM`.
├─├─ memory	integer	The memory limit for incremental log write.
├─├─ readerWorkerNum	integer	The number of concurrent readers for Kafka. This parameter is supported only when the source is an Oracle database and incremental data is obtained from Kafka.
├─├─ writerWorkerNum	integer	The number of concurrent writers for incremental logs.
├─ throttleRps	integer	The maximum allowed transfer rate in records per second during incremental synchronization. Set this parameter to null to indicate no restriction on RPS.
├─ throttleIOPS	integer	The maximum allowed transfer traffic in bytes per second during incremental synchronization. Set this parameter to null to indicate no restriction on IOPS.
├─ supportDDLTypes	array	The supported DDL operations. CREATE_TABLE ALTER_TABLE DROP_TABLE RENAME_TABLE TRUNCATE_TABLE CREATE_INDEX DROP_INDEX
├─ conflictPolicy	Enum	The data conflict strategy. Valid values: `IGNORE` (ignores errors and keeps the original data in the target database) and `OVERWRITE` (ignores conflicts and overwrites the data in the target database with the data in the source database). The default value is `IGNORE`. This parameter is required only when you create a bidirectional synchronization task.
reverseIncrTransferConfig	object	The reverse incremental synchronization configuration.
├─ startTimestamp	string	The start timestamp of reverse incremental synchronization. This parameter is supported only when incremental synchronization is enabled but full migration is not enabled.
├─ recordTypeWhiteList	array	The types of incremental data to be synchronized in the reverse incremental synchronization phase. Valid values: HEARTBEAT, INSERT, UPDATE, DELETE, BEGIN, COMMIT, ROLLBACK, DDL, and ROW.
├─ storeLogKeptHour	integer	The retention period of reverse incremental logs when the incremental log pull component is a store, in hours. The default value is 7, and the maximum value is 365.
├─ enableSequencingWithinTxn	boolean	Specifies whether to enable sequence number arrangement within a transaction when the incremental log pull component is a store.
├─ enableIncrSyncStatistics	boolean	Specifies whether to enable statistics collection for reverse incremental records (DML and DDL operations).
├─ incrOnlineDdlConfig	array	The value of this parameter indicates the name of an online DDL tool. Valid values: GH_OST, PT_OSC, and ALIYUN_DMS, which respectively indicate the gh-ost, pt-osc, and Alibaba Cloud DMS tools. This parameter takes effect on forward incremental synchronization from a MySQL database to a MySQL-compatible tenant of OceanBase Database. This parameter takes effect on reverse incremental synchronization from a MySQL-compatible tenant of OceanBase Database to a MySQL database.
├─ incrSyncConcurrency	integer	The number of threads for reverse incremental synchronization. Default value: 64.
├─ incrLogPullSpeedMode	string	The reverse incremental log pull speed. Valid values: STEADY (low), NORMAL (medium), FAST (high), and CUSTOM (custom).
├─ incrLogPullSpeedConfig	string	The reverse incremental log pull resource configuration. This parameter is effective only when `incrLogPullSpeedMode` is set to `CUSTOM`.
├─├─ memory	integer	The memory limit for reverse incremental log pull.
├─ incrLogSyncSpeedMode	string	The reverse incremental data write speed. Valid values: STEADY (low), NORMAL (medium), FAST (high), and CUSTOM (custom).
├─ incrLogSyncSpeedConfig	string	The reverse incremental data write resource configuration. This parameter is effective only when `incrLogSyncSpeedMode` is set to `CUSTOM`.
├─├─ memory	integer	The memory limit for reverse incremental log write.
├─├─ readerWorkerNum	integer	The number of concurrent readers for Kafka. This parameter is supported only when the target is an Oracle database and incremental data is obtained from Kafka.
├─├─ writerWorkerNum	integer	The number of concurrent writers for reverse incremental logs.
├─ throttleRps	integer	The maximum allowed transfer rate in records per second during reverse incremental synchronization. Set this parameter to null to indicate no restriction on RPS.
├─ throttleIOPS	integer	The maximum allowed transfer traffic in bytes per second during reverse incremental synchronization. Set this parameter to null to indicate no restriction on IOPS.
├─ supportDDLTypes	array	The supported DDL operations. CREATE_TABLE ALTER_TABLE DROP_TABLE RENAME_TABLE TRUNCATE_TABLE CREATE_INDEX DROP_INDEX
fullVerifyConfig	object	The configuration for full validation. When `enableFullVerify` is `false`, `fullVerifyConfig` is null. When `enableFullVerify` is `true`, you must configure the following subparameters.
├─ sourceThreads	integer	The number of concurrent threads on the source side, which ranges from 1 to 512.
├─ sinkThreads	integer	The number of concurrent threads on the destination side, which ranges from 1 to 512.
├─ mode	string	The validation mode, which can be `AUTO` (adaptive mode) or `IN` (IN mode).
├─ sourceThrottleBps	integer	The BPS limit on the source side, which ranges from 1 to 1024. The unit is MiB/s. If this parameter is left empty, no limit is imposed.
├─ sinkThrottleBps	integer	The BPS limit on the destination side, which ranges from 1 to 1024. The unit is MiB/s. If this parameter is left empty, no limit is imposed.
├─ enableHiddenColumnsAndIndex	boolean	Whether to add hidden columns and hidden indexes to tables without non-null unique keys. This parameter applies to the following five types of data validation tasks: Oracle -> OB_MySQL, Oracle -> OB_Oracle, OB_MySQL -> OB_MySQL, OB_Oracle -> OB_Oracle, and OB_Oracle -> Oracle.
countVerifyConfig	object	The configuration for row count validation. When `enableCountVerify` is `false`, `countVerifyConfig` is null. When `enableCountVerify` is `true`, you must configure the following subparameters.
├─ sourceThreads	integer	The number of concurrent threads on the source side, which ranges from 1 to 512.
├─ sinkThreads	integer	The number of concurrent threads on the destination side, which ranges from 1 to 512.
├─ concurrencyTablesNumber	integer	The number of tables to be verified concurrently. This parameter applies to both the source and destination sides. The value ranges from 1 to 100.
├─ queryTimeout	integer	The query timeout period, which applies to both the source and destination sides. The value ranges from 1 to 1440. The unit is seconds.
incrVerifyConfig	object	The configuration for incremental validation. When `enableIncrVerify` is `false`, `incrVerifyConfig` is null. When `enableIncrVerify` is `true`, you must configure the following subparameters.
├─ startCheckpoint	string	The start point of incremental validation, which ranges from Now-7d to Now+7d. The format is a timestamp.
├─ endCheckpoint	string	The end point of incremental validation, which ranges from startCheckpoint to +00. The format is a timestamp. If this parameter is not specified, incremental validation is performed continuously.
├─ initialInspectionCooldownTimeout	integer	The cooldown period for hot data during initial inspection.
├─ initialInspectionDelayTimeout	integer	The period for detecting inconsistent data during initial inspection.
├─ reInspectionThreads	integer	The number of concurrent threads for reinspection, which ranges from 1 to 512.
├─ reInspectionthrottleBps	integer	The BPS limit for reinspection, which ranges from 1 to 1024. If this parameter is left empty, no limit is imposed. The unit is MiB/s.
├─ reInspectiontolerateTimeout	integer	The maximum time for reinspection, in minutes.
├─ reInspectionRounds	integer	The maximum number of reinspection rounds.
├─ reInspectionDiffWaitMaxIntervalMs	long	The maximum waiting time for reinspection.
direction	Enum	Specifies the migration direction of the data migration task. Valid values: `NORMAL` (unidirectional data migration task), `FORWARD` (forward task in bidirectional synchronization), and `REVERSE` (reverse task in bidirectional synchronization). Default value: `NORMAL`.
forwardProjectId	string	The task ID of the forward task in a bidirectional synchronization task. This parameter takes effect only when `direction` is `REVERSE`.

Parameters in databases/databasesBlack

Parameter	Type	Description
mappedName	string	The name of the database in the target.
name	string	The name of the database in the source.
id	string	The ID of the database.
tables	array	The configurations of table objects to be migrated in the current database. When `mode` is `WILDCARD`, this parameter specifies the configurations for renaming and row filtering of specific table objects.
├─ mappedName	string	The mapped name of the table.
├─ name	string	The name of the table.
├─ id	string	The ID of the table.
├─ whereClause	string	The `WHERE` clause in the standard SQL statement for configuring row filtering. Notice When `mode` is `WILDCARD`, this parameter is not supported.
├─ filterColumns	array	The filtered column set. Notice When `mode` is `WILDCARD`, this parameter is not supported.
├─ shardColumns	array	The sharding column set. Notice When `mode` is `WILDCARD`, this parameter is not supported.
views	array	The configurations of view objects to be migrated in the current database.
├─ mappedName	string	The mapped name of the view.
├─ name	string	The name of the view.
├─ id	string	The ID of the view.
├─ whereClause	string	The `WHERE` clause in the standard SQL statement for configuring row filtering. Notice When `mode` is `WILDCARD`, this parameter is not supported.
├─ filterColumns	array	The filtered column set. Notice <When `mode` is `WILDCARD`, this parameter is not supported.
├─ shardColumns	array	The sharding column set. Notice When `mode` is `WILDCARD`, this parameter is not supported.
functions	array	The configurations of function objects to be migrated in the current database.
├─ name	string	The name of the function.
├─ mappedName	string	The mapped name of the function. This parameter cannot be modified.
procedures	array	The configurations of stored procedure objects to be migrated in the current database.
├─ name	string	The name of the stored procedure.
├─ mappedName	string	The mapped name of the stored procedure. This parameter cannot be modified.
packages	array	The configurations of package objects to be migrated in the current database.
├─ name	string	The name of the package.
├─ mappedName	string	The mapped name of the package. This parameter cannot be modified.
synonyms	array	The configurations of synonym objects to be migrated in the current database.
├─ name	string	The name of the synonym.
├─ mappedName	string	The mapped name of the synonym. This parameter cannot be modified.
sequences	array	The configurations of sequence objects to be migrated in the current database.
├─ name	string	The name of the sequence.
├─ mappedName	string	The mapped name of the sequence. This parameter cannot be modified.
specificTables	array	The configurations of specific table objects. When `mode` is `WILDCARD`, this parameter specifies the renaming and row filtering configurations of specific table objects.
├─ mappedName	string	The mapped name of the table.
├─ name	string	The name of the table.
├─ id	string	The ID of the table.
├─ whereClause	string	The `WHERE` clause in the standard SQL statement for configuring row filtering. Notice When `mode` is `WILDCARD`, this parameter is not supported.
├─ filterColumns	array	The filtered column set. Notice When `mode` is `WILDCARD`, this parameter is not supported.
├─ shardColumns	array	The sharding column set. Notice When `mode` is `WILDCARD`, this parameter is not supported.
specificViews	array	The configurations of specific view objects. When `mode` is `WILDCARD`, this parameter specifies the renaming configurations of specific view objects.
├─ mappedName	string	The mapped name of the view.
├─ name	string	The name of the view.
├─ id	string	The ID of the view.
├─ whereClause	string	The `WHERE` clause in the standard SQL statement for configuring row filtering. Notice When `mode` is `WILDCARD`, this parameter is not supported.
├─ filterColumns	array	The filtered column set. Notice When `mode` is `WILDCARD`, this parameter is not supported.
├─ shardColumns	array	The sharding column set. Notice When `mode` is `WILDCARD`, this parameter is not supported.
specificFunctions	array	The configurations of specific function objects. When `mode` is `WILDCARD`, this parameter specifies the renaming configurations of specific function objects.
├─ name	string	The name of the function.
├─ mappedName	string	The mapped name of the function. This parameter cannot be modified.
specificProcedures	array	The configurations of specific stored procedure objects. When `mode` is `WILDCARD`, this parameter specifies the renaming configurations of specific stored procedure objects.
├─ name	string	The name of the stored procedure.
├─ mappedName	string	The mapped name of the stored procedure. This parameter cannot be modified.
specificPackages	array	The configurations of specific package objects. When `mode` is `WILDCARD`, this parameter specifies the renaming configurations of specific package objects.
├─ name	string	The name of the package.
├─ mappedName	string	The mapped name of the package. This parameter cannot be modified.
specificSynonyms	array	The configurations of specific synonym objects. When `mode` is `WILDCARD`, this parameter specifies the renaming configurations of specific synonym objects.
├─ name	string	The name of the synonym.
├─ mappedName	string	The mapped name of the synonym. This parameter cannot be modified.
specificSequences	array	The configurations of specific sequence objects. When `mode` is `WILDCARD`, this parameter specifies the renaming configurations of specific sequence objects.
├─ name	string	The name of the sequence.
├─ mappedName	string	The mapped name of the sequence. This parameter cannot be modified.

Response parameters

Parameter	Type	Description
success	boolean	Specifies whether the call is successful.
errorDetail	object	Details of the error.
code	string	The error code.
message	string	The description.
advice	string	The suggestion.
requestId	string	The request ID.
pageNumber	integer[int32]	The page number. This parameter is effective only when you query data in pages.
pageSize	integer[int32]	The page size. This parameter is effective only when you query data in pages.
totalCount	integer[int64]	The total number of records. This parameter is effective only when you query data in pages.
cost	string	The time spent in processing the request, in seconds.
data	string	The ID of the task.

Examples

Request example

mode is set to SPECIFIC

{
    "name":"test",
    "type":"MIGRATION",
    "labelIds":[
        "l_4w****wygjk"
    ],
    "sourceEndpointId":"e_4wg07f****",
    "sinkEndpointId":"e_4wfy2****",
    "transferMapping":{
        "mode":"SPECIFIC",
        "databases":[
            {
                "mappedName":"gez3***",
                "name":"ion1****bmysql",
                "id":null,
                "tables":[
                    {
                        "name":"a",
                        "mappedName":"a_new",
                        "id":null,
                        "whereClause":"`id` > 3",
                        "filterColumns":null,
                        "shardColumns":null,
                    }
                ],
                "views":null,
                "specificTables":null,
                "specificViews":null
            }
        ],
        "databasesBlack":null,
        "tableAndViewWhiteList":null,
        "tableAndViewBlackList":null
    },
    "commonTransferConfig":{
        "tableCategory":null,
        "activeActive":false,
        "mqSerializerType":null,
        "mqPartitionMode":null,
        "mqPartition":0,
        "datahubTopicType":null,
        "rocketMqProducerGroup":null,
        "rocketMqMsgTags":null,
        "rocketMqEnableMsgTrace":null,
        "rocketMqSendMsgTimeout":null,
        "dataWorksBusinessName":null,
        "interactiveMode":true
    },
    "enableStructTransfer":true,
    "structTransferConfig":{
        "byteCharConvertStrategy":"DO_NOTHING_IF_BYTE_USED",
        "autoProcessedToNextStep": true,
        "indexMigrationType": "AFTER_FULL_TRANSFER",
        "triggerMigrationType":"NOT_MIGRATION"
    },
    "enableFullTransfer":true,
    "fullTransferConfig":{
        "nonePkUkTruncateDstTable":null,
        "allowDestTableNotEmpty":true,
        "fullTransferSpeedMode":"CUSTOM",
        "fullTransferSpeedConfig":{
            "memory":4,
            "readerWorkerNum":1,
            "writerWorkerNum":2
            },
        "fullVerifySpeedMode":"CUSTOM",
        "fullVerifySpeedConfig":{
            "memory":4,
            "readerWorkerNum":1,
            "writerWorkerNum":2
            },    
        "throttleRps": 1000,
        "throttleIOPS": 1000         
    },
    "enableIncrTransfer":true,
    "enableReverseIncrTransfer":true,
    "incrTransferConfig":{
        "startTimestamp":1689201369,
        "storeLogKeptHour":120,
        "recordTypeWhiteList":[
            "DELETE",
            "INSERT",
            "UPDATE"
        ],
        "enableSequencingWithinTxn":true,
        "incrSyncConcurrency":64,
        "incrLogPullSpeedMode":"NORMAL",
        "incrLogPullSpeedConfig":{
            "memory":4
            },
        "incrLogSyncSpeedMode":"CUSTOM",
        "incrLogSyncSpeedConfig":{
            "memory":4,
            "readerWorkerNum":1,
            "writerWorkerNum":2
            },
        "throttleRps": 1000,
        "throttleIOPS": 1000,
        "enableIncrSyncStatistics":true,
        "supportDDLTypes":[
            "CREATE_TABLE",
            "ALTER_TABLE"
        ],
        "incrOnlineDdlConfig":[
        "GH_OST",
        "PT_OSC",
        "ALIYUN_DMS"
        ],
        "conflictPolicy":"IGNORE"
    },
    "ReverseIncrTransferConfig":{
        "startTimestamp":1689201369,
        "recordTypeWhiteList":[
            "DELETE",
            "INSERT",
            "UPDATE"
        ],
        "storeLogKeptHour":120,
        "incrSyncConcurrency":64,
        "incrLogPullSpeedMode":"NORMAL",
        "incrLogPullSpeedConfig":{
            "memory":4
            },
        "incrLogSyncSpeedMode":"CUSTOM",
        "incrLogSyncSpeedConfig":{
            "memory":4,
            "readerWorkerNum":1,
            "writerWorkerNum":2
            },
        "throttleRps": 1000,
        "throttleIOPS": 1000,
        "enableIncrSyncStatistics":true,
        "supportDDLTypes":[
            "CREATE_TABLE",
            "ALTER_TABLE"
        ]
    },
    "direction":"REVERSE",
    "forwardProjectId":"np_6xxxx"
}

mode is set to WILDCARD

{
"name": "test-wildcard",
"type": "MIGRATION",
"sourceEndpointId": "e_70w****",
"sinkEndpointId": "e_70wr****",
"transferMapping": {
    "mode": "WILDCARD",
        "tableAndViewWhiteList": [
    "db01.*=test.*"
    ],
    "databases": [
            {
                "name":"db01",
                "mappedName":"test",
                "tables":[
                    {
                        "name":"*"
                    }
                ]
            }

    ]
},
"commonTransferConfig": {
    "tableCategory": "ALL",
    "activeActive": false,
    "enableHiddenColumnsAndIndex": true,
    "interactiveMode": true
},
"enableStructTransfer": true,
"enableIncrTransfer": true,
"enableReverseIncrTransfer": true,
"structTransferConfig": {
    "autoProcessedToNextStep": false,
    "indexMigrationType": "WITH_STRUCT_TRANSFER"
},
"incrTransferConfig": {
    "startTimestamp": "1756192557",
    "recordTypeWhiteList": [
    "INSERT",
    "UPDATE",
    "DELETE"
    ],
    "storeLogKeptHour": 120,
    "enableSequencingWithinTxn": true,
    "enableIncrSyncStatistics": true,
    "incrLogPullSpeedMode": "CUSTOM",
    "incrLogPullSpeedConfig": {
    "memory": 7
    },
    "incrLogSyncSpeedMode": "CUSTOM",
    "incrLogSyncSpeedConfig": {
    "writerWorkerNum": 7,
    "memory": 7
    },
    "throttleRps": null,
    "throttleIOPS": null,
    "supportDDLTypes": [

    ],
    "conflictPolicy":"IGNORE"
},
"reverseIncrTransferConfig": {
    "recordTypeWhiteList": [
    "INSERT",
    "UPDATE",
    "DELETE"
    ],
    "throttleRps": null,
    "throttleIOPS": null,
    "supportDDLTypes": [

    ],
    "reverseReuseIncrConfigFlag": true,
    "storeLogKeptHour": 120,
    "incrLogPullSpeedMode": "CUSTOM",
    "incrLogPullSpeedConfig": {
    "memory": 7
    },
    "incrLogSyncSpeedMode": "CUSTOM",
    "incrLogSyncSpeedConfig": {
    "writerWorkerNum": 7,
    "memory": 7
    }
},
"direction":"REVERSE",
"forwardProjectId":"np_6xxxx"
}

For scenarios involving renaming and matching rules, here are some examples.

Migrate all tables in the database

Example 1: Select the migration objects and do not rename.

    {
    "mode":"WILDCARD",
    "databases":[
        {
            "mappedName":"qiu_test",
            "name":"qiu_test",
            "specificTables":[

            ],
            "specificViews":[

            ],
            "tables":[
                {
                    "name":"*"
                }
            ],
            "views":[
                {
                    "name":"*"
                }
            ]
        }
    ],
    "databasesBlack":[
        {
            "name":"qiu_test",
            "mappedName":"qiu_test",
            "tables":[
                {
                    "name":"test_lxquan16*"
                }
            ],
            "views":[
                {
                    "name":"test_lxquan16*"
                }
            ]
        }
    ],
    "tableAndViewWhiteList":[
        "qiu_test.*"
    ],
    "tableAndViewBlackList":[
        "qiu_test.test_lxquan16*"
    ]
  }

Example 2: Select the migration objects and rename.

{
    "mode":"WILDCARD",
    "databases":[
        {
            "mappedName":"qiu_test",
            "name":"qiu_test",
            "specificTables":[
                {
                    "mappedName":"test_620",
                    "name":"test_lxquan620",
                    "whereClause":"`c01` > 10"
                },
                {
                    "mappedName":"test_3",
                    "name":"test_lxquan3"
                }
            ],
            "specificViews":[

            ],
            "tables":[
                {
                    "name":"*"
                }
            ],
            "views":[
                {
                    "name":"*"
                }
            ]
        }
    ],
    "databasesBlack":[
        {
            "name":"qiu_test",
            "mappedName":"qiu_test",
            "tables":[
                {
                    "name":"test_lxquan1?*"
                }
            ],
            "views":[
                {
                    "name":"test_lxquan1?*"
                }
            ]
        }
    ],
    "tableAndViewWhiteList":[
        "qiu_test.*"
    ],
    "tableAndViewBlackList":[
        "qiu_test.test_lxquan1?*"
    ]
}

Migrate specific tables in the database

Example 1: Select the migration objects and do not rename.

{
    "mode":"WILDCARD",
    "databases":[
        {
            "mappedName":"qiu_test",
            "name":"qiu_test",
            "specificTables":[

            ],
            "specificViews":[

            ],
            "tables":[
                {
                    "name":"test_lxquan167*"
                }
            ],
            "views":[
                {
                    "name":"test_lxquan167*"
                }
            ]
        }
    ],
    "databasesBlack":[
        {
            "name":"qiu_test",
            "mappedName":"qiu_test",
            "tables":[
                {
                    "name":"test_lxquan1671"
                },
                {
                    "name":"test_lxquan1672"
                }
            ],
            "views":[
                {
                    "name":"test_lxquan1671"
                },
                {
                    "name":"test_lxquan1672"
                }
            ]
        }
    ],
    "tableAndViewWhiteList":[
        "qiu_test.test_lxquan167*"
    ],
    "tableAndViewBlackList":[
        "qiu_test.test_lxquan1671",
        "qiu_test.test_lxquan1672"
    ]
}

Example 2: Select the migration objects and rename.

{
    "mode":"WILDCARD",
    "databases":[
        {
            "mappedName":"qiu_test",
            "name":"qiu_test",
            "specificTables":[
                {
                    "mappedName":"test_1670",
                    "name":"test_lxquan1670"
                }
            ],
            "specificViews":[

            ],
            "tables":[
                {
                    "name":"test_lxquan167*"
                }
            ],
            "views":[
                {
                    "name":"test_lxquan167*"
                }
            ]
        }
    ],
    "databasesBlack":[
        {
            "name":"qiu_test",
            "mappedName":"qiu_test",
            "tables":[
                {
                    "name":"test_lxquan1671"
                },
                {
                    "name":"test_lxquan1672"
                },
                {
                    "name":"test_lxquan1673"
                },
                {
                    "name":"test_lxquan1674"
                },
                {
                    "name":"test_lxquan1675"
                },
                {
                    "name":"test_lxquan1676"
                },
                {
                    "name":"test_lxquan1677"
                },
                {
                    "name":"test_lxquan1678"
                },
                {
                    "name":"test_lxquan1671"
                },
                {
                    "name":"test_lxquan167"
                }
            ],
            "views":[
                {
                    "name":"test_lxquan1671"
                },
                {
                    "name":"test_lxquan1672"
                },
                {
                    "name":"test_lxquan1673"
                },
                {
                    "name":"test_lxquan1674"
                },
                {
                    "name":"test_lxquan1675"
                },
                {
                    "name":"test_lxquan1676"
                },
                {
                    "name":"test_lxquan1677"
                },
                {
                    "name":"test_lxquan1678"
                },
                {
                    "name":"test_lxquan1671"
                },
                {
                    "name":"test_lxquan167"
                }
            ]
        }
    ],
    "tableAndViewWhiteList":[
        "qiu_test.test_lxquan167*"
    ],
    "tableAndViewBlackList":[
        "qiu_test.test_lxquan1671",
        "qiu_test.test_lxquan1672",
        "qiu_test.test_lxquan1673",
        "qiu_test.test_lxquan1674",
        "qiu_test.test_lxquan1675",
        "qiu_test.test_lxquan1676",
        "qiu_test.test_lxquan1677",
        "qiu_test.test_lxquan1678",
        "qiu_test.test_lxquan1671",
        "qiu_test.test_lxquan167"
    ]
}

mode is set to COMPLEX

{
"name": "test-complex",
"type": "MIGRATION",
"sourceEndpointId": "e_70wq****",
"sinkEndpointId": "e_70wr2****",
"transferMapping": {
    "mode": "COMPLEX",
    "tableAndViewWhiteList": [
    "db01.*=test.*",
    "db02.*"
    ],
    "tableAndViewBlackList": [

    ],
    "schemas": [

    ],
    "whiteTableRules": [
    {
        "schemaMapping": {
        "name": "db01",
        "mappedName": "test"
        },
        "objectMapping": {
        "name": "*",
        "mappedName": "*"
        }
    },
    {
        "schemaMapping": {
        "name": "db02"
        },
        "objectMapping": {
        "name": "*"
        }
    }
    ],
    "blackTableRules": [

    ],
    "whiteViewRules": [
    {
        "schemaMapping": {
        "name": "db01",
        "mappedName": "test"
        },
        "objectMapping": {
        "name": "*",
        "mappedName": "*"
        }
    },
    {
        "schemaMapping": {
        "name": "db02"
        },
        "objectMapping": {
        "name": "*"
        }
    }
    ],
    "blackViewRules": [

    ]
},
"commonTransferConfig": {
    "tableCategory": "ALL",
    "activeActive": false,
    "enableHiddenColumnsAndIndex": true,
    "interActiveMode": true
},
"enableStructTransfer": true,
"enableIncrTransfer": true,
"enableReverseIncrTransfer": true,
"structTransferConfig": {
    "autoProcessedToNextStep": false,
    "indexMigrationType": "WITH_STRUCT_TRANSFER"
},
"incrTransferConfig": {
    "startTimestamp": "1756192557",
    "recordTypeWhiteList": [
    "INSERT",
    "UPDATE",
    "DELETE"
    ],
    "storeLogKeptHour": 120,
    "enableSequencingWithinTxn": true,
    "enableIncrSyncStatistics": true,
    "incrLogPullSpeedMode": "CUSTOM",
    "incrLogPullSpeedConfig": {
    "memory": 7
    },
    "incrLogSyncSpeedMode": "CUSTOM",
    "incrLogSyncSpeedConfig": {
    "writerWorkerNum": 7,
    "memory": 7
    },
    "throttleRps": null,
    "throttleIOPS": null,
    "supportDDLTypes": [

    ],
    "conflictPolicy":"IGNORE"
},
"reverseIncrTransferConfig": {
    "recordTypeWhiteList": [
    "INSERT",
    "UPDATE",
    "DELETE"
    ],
    "throttleRps": null,
    "throttleIOPS": null,
    "supportDDLTypes": [

    ],
    "reverseReuseIncrConfigFlag": true,
    "storeLogKeptHour": 120,
    "incrLogPullSpeedMode": "CUSTOM",
    "incrLogPullSpeedConfig": {
    "memory": 7
    },
    "incrLogSyncSpeedMode": "CUSTOM",
    "incrLogSyncSpeedConfig": {
    "writerWorkerNum": 7,
    "memory": 7
    }
},
"direction":"REVERSE",
"forwardProjectId":"np_6xxxx"
}

type is VERIFY

{
    "id": "np_6****",
    "name": "test****",
    "labelIds": [],
    "sourceEndpointType": "ORACLE",
    "sinkEndpointType": "OB_ORACLE",
    "sourceEndpointId": "e_6****",
    "sinkEndpointId": "e_6****",
    "transferMapping": {
        "mode": "SPECIFIC",
        "databases": [
            {
                "name": "CHENJIAN",
                "mappedName": "ORAAUDITOR",
                "tables": [
                    {
                        "name": "TABLE2",
                        "mappedName": "TABLE2",
                        "whereClause": "Y > 10",
                        "filterColumns": [
                            "X",
                            "Y"
                        ],
                        "shardColumns": null,
                        "columns": [
                            {
                                "columnName": "Y",
                                "mappedName": "Y1",
                                "position": null,
                                "columnType": null,
                                "recordFieldType": null,
                                "rawColumnType": null,
                                "columnKey": null,
                                "nullable": null,
                                "defaultValue": null,
                                "dataLength": null,
                                "dataPrecision": null,
                                "dataScale": null,
                                "encoding": null,
                                "columnComment": null,
                                "notNullMappedName": "Y1",
                                "generateField": false
                            }
                        ],
                        "partitionNames": []
                    },
                    {
                        "name": "TABLE3",
                        "mappedName": "TABLE3",
                        "whereClause": null,
                        "filterColumns": null,
                        "shardColumns": null,
                        "columns": null,
                        "partitionNames": []
                    },
                    {
                        "name": "TABLE4",
                        "mappedName": "TABLE4",
                        "whereClause": "",
                        "filterColumns": [
                            "ID"
                        ],
                        "shardColumns": null,
                        "columns": [],
                        "partitionNames": []
                    },
                    {
                        "name": "TABLE5",
                        "mappedName": "TABLE5",
                        "whereClause": "",
                        "filterColumns": [
                            "ID"
                        ],
                        "shardColumns": null,
                        "columns": [],
                        "partitionNames": []
                    },
                    {
                        "name": "TABLE6",
                        "mappedName": "TABLE6",
                        "whereClause": "",
                        "filterColumns": [
                            "VAL_MAPPING"
                        ],
                        "shardColumns": null,
                        "columns": [],
                        "partitionNames": []
                    },
                    {
                        "name": "TABLE7",
                        "mappedName": "TABLE7",
                        "whereClause": "",
                        "filterColumns": [
                            "ID",
                            "NAME"
                        ],
                        "shardColumns": null,
                        "columns": [
                            {
                                "columnName": "ID",
                                "mappedName": "ID2",
                                "position": null,
                                "columnType": null,
                                "recordFieldType": null,
                                "rawColumnType": null,
                                "columnKey": null,
                                "nullable": null,
                                "defaultValue": null,
                                "dataLength": null,
                                "dataPrecision": null,
                                "dataScale": null,
                                "encoding": null,
                                "columnComment": null,
                                "notNullMappedName": "ID2",
                                "generateField": false
                            }
                        ],
                        "partitionNames": []
                    },
                    {
                        "name": "TABLE8",
                        "mappedName": "TABLE8",
                        "whereClause": "",
                        "filterColumns": [
                            "X"
                        ],
                        "shardColumns": null,
                        "columns": [
                            {
                                "columnName": "X",
                                "mappedName": "Y",
                                "position": null,
                                "columnType": null,
                                "recordFieldType": null,
                                "rawColumnType": null,
                                "columnKey": null,
                                "nullable": null,
                                "defaultValue": null,
                                "dataLength": null,
                                "dataPrecision": null,
                                "dataScale": null,
                                "encoding": null,
                                "columnComment": null,
                                "notNullMappedName": "Y",
                                "generateField": false
                            }
                        ],
                        "partitionNames": []
                    }
                ],
                "procedures": [
                    {
                        "name": "PL1",
                        "mappedName": "PL1"
                    }
                ],
                "functions": [
                    {
                        "name": "FUNCTION1",
                        "mappedName": "FUNCTION1"
                    }
                ]
            }
        ]
    },
    "commonTransferConfig": {
        "enableHiddenColumnsAndIndex": true
    },
    "enableFullVerify": true,
    "enableCountVerify": false,
    "enableIncrVerify": false,
    "fullVerifyConfig": {
        "sourceThreads": 4,
        "sinkThreads": 4,
        "mode": "AUTO",
        "sourceThrottleBps": null,
        "sinkThrottleBps": null,
        "enableHiddenColumnsAndIndex": true
    },
    "countVerifyConfig": null,
    "incrVerifyConfig": null,
    "type": "VERIFY",
    "incrTransferConfig": {
        "storeTemplate": []
    },
    "reverseIncrTransferConfig": {
        "storeTemplate": []
    },
    "precheckId": "t_6****"
}

Response example

{
    "Success":true,
    "ErrorDetail":{
        "Code":"CM-RES***11",
        "Level":"WARN",
        "Message":"A system error occurred.",
        "Proposal":"Contact the administrator."
    },
    "Code":"INNER_ERROR",
    "Message":"A system error occurred.",
    "Advice":"Contact the administrator.",
    "RequestId":"XCVSADG****DSGDS",
    "PageNumber":1,
    "PageSize":1,
    "TotalCount":1,
    "Cost":"27 ms",
    "Data":"np_****"
}

Enterprise Edition

Community Edition

CreateProject

Purpose

Call description

Limitations

Request path

Request parameters

Notice

Notice

Notice

Notice

Notice

Notice

Notice

Notice

Notice

Notice

Notice

Notice

Notice

Notice

Response parameters

Examples

Request example

Response example