This topic describes the CreateProject OpenAPI provided by OceanBase Migration Service (OMS): how to create a task.
Purpose
This interface is used to create a data migration or synchronization task.
Call description
Limitations
This interface applies to OMS V4.1.0 and later.
Request path
POST /api/v2?Action=CreateProject
Request parameters
| Parameter | Type | Description |
|---|---|---|
| name | string | The task name, which cannot contain spaces. |
| type | string | The task type. MIGRATION indicates a data migration task, and SYNC indicates a data synchronization task. |
| labelIds | array | The IDs of the labels. At most five IDs are supported. |
| sourceEndpointId | string | The ID of the source data source. To obtain the ID, call the ListDataSource interface. The ID is in the e_**** format. |
| sinkEndpointId | string | The ID of the target data source. To obtain the ID, call the ListDataSource interface. The ID is in the e_**** format. |
| transferMapping | object | The mapping information of the migration objects. |
| ├─ mode | string | The object mapping mode. Valid values: SPECIFIC (specified object) and WILDCARD (matching rule). |
| ├─ databases | object [] | The list of database objects. For more information, see "Parameters in databases/databasesBlack". |
| ├─ databasesBlack | object [] | The list of blacklisted database objects in the matching rule mode. For more information, see "Parameters in databases/databasesBlack". |
| ├─ tableAndViewWhiteList | array | The wildcard whitelist of tables and views in the matching rule mode. |
| ├─ tableAndViewBlackList | array | The wildcard blacklist of tables and views in the matching rule mode. |
| commonTransferConfig | object | The general migration configuration. |
| ├─ tableCategory | string | The type of supported tables. Valid values: ALL (all tables), WITH_UNIQUE_ROW_ID (tables with unique row identifiers), and WITHOUT_UNIQUE_ROW_ID (tables without unique row identifiers). Default value: ALL. |
| ├─ activeActive | boolean | Specifies whether the task is a disaster recovery active-active scenario. |
| ├─ mqSerializerType | string | The JSON serialization type when data is delivered to a message queue. Valid values: DEFAULT, CANAL, DATAWORKS_V2, SHAREPLEX, DEFAULT_WITH_SCHEMA, DEBEZIUM, DEBEZIUMFLATTEN, and DEBEZIUMSMT. |
| ├─ mqPartitionMode | string | The partition routing mode when data is delivered to a message queue. Valid values: ONE (specify the partition for delivery), HASH (hash partition the data based on the primary key or the sharding column value), and TABLE (hash partition the data based on the database and table name). |
| ├─ mqPartition | integer | When partitionMode is ONE, this parameter specifies the position of the partition in the current topic. |
| ├─ datahubTopicType | string | The type of the topic to which data is delivered in the DataHub format. Valid values: TUPLE and BLOB. |
| ├─ rocketMqProducerGroup | string | The name of the producer group. This parameter is effective only when the target is a RocketMQ database. Default value: OMS. |
| ├─ rocketMqMsgTags | string | The message tag. This parameter is effective only for the RocketMQ database. Default value: default_tag. When the value of this parameter is default, _tag is automatically added as a suffix. |
| ├─ rocketMqEnableMsgTrace | boolean | Specifies whether to enable message tracing. This parameter is effective only when the target is a RocketMQ database. |
| ├─ rocketMqSendMsgTimeout | long | The timeout period for message delivery. This parameter is effective only when the target is a RocketMQ database. |
| ├─ dataWorksBusinessName | string | The identifier of the business system. This parameter identifies the business system that originates the data so that you can perform custom processing on the data later. This parameter is supported only when the JSON serialization type is DataWorks. The value of this parameter must be 1 to 20 characters long. |
| ├─ sinkStoreFormat | string | The storage type of the destination table. Valid values: Default, Row storage, Column storage, and Hybrid columnar storage. This parameter is effective only when the destination OceanBase Database is V4.3.0 or later, and the migration type is Schema Migration or Incremental Synchronization > DDL synchronization. |
| ├─ sourceStoreFormat | string | The storage type of the source table. Valid values: Default, Row storage, Column storage, and Hybrid columnar storage. This parameter is effective only when the source OceanBase Database is V4.3.0 or later, and the migration type is Reverse Increment > DDL synchronization. |
| ├─ enableHiddenColumnsAndIndex | boolean | Specifies whether to add hidden columns and indexes for tables without non-null unique keys. This parameter applies to the following five types of data migration tasks: Oracle to OB_MySQL, Oracle to OB_Oracle, OB_MySQL to OB_MySQL, OB_Oracle to OB_Oracle, and OB_Oracle to Oracle. |
| enableStructTransfer | boolean | Specifies whether to enable schema migration. |
| structTransferConfig | object | The schema migration configuration. |
| ├─ byteCharConvertStrategy | string | The byte/char conversion strategy. Valid values: IGNORE_BYTE_IF_BYTE_USED, FORCE_CHAR_IF_BYTE_USED, EXPAND_LEN_IF_BYTE_USED, and DO_NOTHING_IF_BYTE_USED (default). |
| ├─ deferIndexCreation | boolean | Specifies whether to enable index creation deferral. Default value: false. This parameter is deprecated in OMS V4.3.0 and later. We recommend that you use the indexMigrationType parameter instead. |
| ├─ indexMigrationType | string | The ordinary index migration mode. Valid values: NOT_MIGRATION (do not migrate), WITH_STRUCT_TRANSFER (migrate with schema migration), and AFTER_FULL_TRANSFER (migrate after full migration).Full migration cannot be configured as the migration mode. |
| ├─ autoProcessedToNextStep | boolean | Specifies whether to automatically go to the next step after schema migration is complete. Default value: true. |
| ├─ triggerMigrationType | string | The trigger migration mode. Valid values: NOT_MIGRATION (do not migrate), WITH_STRUCT_TRANSFER (migrate with schema migration), and WITH_TRANSFER_APP_SWITCH (migrate during forward switchover).This parameter is not supported for data migration tasks other than those from an Oracle database to an Oracle compatible mode of OceanBase Database. If you set this parameter for a data migration task from an Oracle database to an Oracle compatible mode of OceanBase Database, you must set Forward switchover as the trigger migration mode. |
| enableFullTransfer | boolean | Specifies whether to enable full migration. |
| enableFullVerify | boolean | Specifies whether to enable full verification. |
| fullTransferConfig | object | The full migration configuration. |
| ├─ nonePkUkTruncateDstTable | boolean | Specifies whether to truncate the destination table when you perform full migration from a source table without a unique index. |
| ├─ allowDestTableNotEmpty | boolean | The processing strategy when the destination table contains records. Valid values: Ignore and Stop Migration.
|
| ├─ fullTransferSpeedMode | string | The full migration speed. Valid values: STEADY, NORMAL, FAST, and CUSTOM. |
| ├─ fullTransferSpeedConfig | string | The full migration speed configuration. This parameter is effective only when fullTransferSpeedMode is set to CUSTOM. |
| ├─├─ memory | integer | The memory limit, in GB. |
| ├─├─ readerWorkerNum | integer | The number of concurrent readers. |
| ├─├─ writerWorkerNum | integer | The number of concurrent writers. |
| ├─ fullVerifySpeedMode | string | The full verification speed. Valid values: STEADY, NORMAL, FAST, and CUSTOM. |
| ├─ fullVerifySpeedConfig | string | The full verification speed configuration. This parameter is effective only when fullVerifySpeedMode is set to CUSTOM. |
| ├─├─ memory | integer | The memory limit, in GB. |
| ├─├─ readerWorkerNum | integer | The number of concurrent readers. |
| ├─├─ writerWorkerNum | integer | The number of concurrent writers. |
| ├─ throttleRps | integer | The maximum allowed transfer rate in records per second during full migration. Set this parameter to null to indicate no restriction on RPS. |
| ├─ throttleIOPS | integer | The maximum allowed transfer traffic in bytes per second during full migration. Set this parameter to null to indicate no restriction on IOPS. |
| enableIncrTransfer | boolean | Specifies whether to enable incremental synchronization. |
| enableReverseIncrTransfer | boolean | Specifies whether to enable reverse incremental synchronization. |
| incrTransferConfig | object | The incremental synchronization configuration. |
| ├─ startTimestamp | string | The starting timestamp of incremental synchronization. This parameter is supported only when incremental synchronization is enabled but full migration is not enabled. |
| ├─ recordTypeWhiteList | array | The types of incremental data to be synchronized in the incremental synchronization phase. Valid values: HEARTBEAT, INSERT, UPDATE, DELETE, BEGIN, COMMIT, ROLLBACK, DDL, and ROW. |
| ├─ storeLogKeptHour | integer | The retention period of incremental logs when the incremental log pull component is a store, in hours. The default value is 7, and the maximum value is 365. |
| ├─ enableSequencingWithinTxn | boolean | Specifies whether to enable sequence number arrangement within a transaction when the incremental log pull component is a store. |
| ├─ enableIncrSyncStatistics | boolean | Specifies whether to enable statistics collection for incremental records (DML and DDL operations). |
| ├─ incrOnlineDdlConfig | array | The value of this parameter indicates the name of an online DDL tool. Valid values: GH_OST, PT_OSC, and ALIYUN_DMS, which respectively indicate the gh-ost, pt-osc, and Alibaba Cloud DMS tools.
|
| ├─ incrSyncConcurrency | integer | The number of threads for incremental synchronization. Default value: 64. |
| ├─ incrLogPullSpeedMode | string | The incremental log pull speed. Valid values: STEADY (low), NORMAL (medium), FAST (high), and CUSTOM (custom). |
| ├─ incrLogPullSpeedConfig | string | The incremental log pull resource configuration. This parameter is effective only when incrLogPullSpeedMode is set to CUSTOM. |
| ├─├─ memory | integer | The memory limit for incremental log pull. |
| ├─ incrLogSyncSpeedMode | string | The incremental data write speed. Valid values: STEADY (low), NORMAL (medium), FAST (high), and CUSTOM (custom). |
| ├─ incrLogSyncSpeedConfig | string | The incremental data write resource configuration. This parameter is effective only when incrLogSyncSpeedMode is set to CUSTOM. |
| ├─├─ memory | integer | The memory limit for incremental log write. |
| ├─├─ readerWorkerNum | integer | The number of concurrent readers for Kafka. This parameter is supported only when the source is an Oracle database and incremental data is obtained from Kafka. |
| ├─├─ writerWorkerNum | integer | The number of concurrent writers for incremental logs. |
| ├─ throttleRps | integer | The maximum allowed transfer rate in records per second during incremental synchronization. Set this parameter to null to indicate no restriction on RPS. |
| ├─ throttleIOPS | integer | The maximum allowed transfer traffic in bytes per second during incremental synchronization. Set this parameter to null to indicate no restriction on IOPS. |
| ├─ supportDDLTypes | array | The supported DDL operations.
|
| reverseIncrTransferConfig | object | The reverse incremental synchronization configuration. |
| ├─ startTimestamp | string | The starting timestamp of reverse incremental synchronization. This parameter is supported only when incremental synchronization is enabled but full migration is not enabled. |
| ├─ recordTypeWhiteList | array | The types of incremental data to be synchronized in the reverse incremental synchronization phase. Valid values: HEARTBEAT, INSERT, UPDATE, DELETE, BEGIN, COMMIT, ROLLBACK, DDL, and ROW. |
| ├─ storeLogKeptHour | integer | The retention period of reverse incremental logs when the incremental log pull component is a store, in hours. The default value is 7, and the maximum value is 365. |
| ├─ enableSequencingWithinTxn | boolean | Specifies whether to enable sequence number arrangement within a transaction when the incremental log pull component is a store. |
| ├─ enableIncrSyncStatistics | boolean | Specifies whether to enable statistics collection for reverse incremental records (DML and DDL operations). |
| ├─ incrOnlineDdlConfig | array | The value of this parameter indicates the name of an online DDL tool. Valid values: GH_OST, PT_OSC, and ALIYUN_DMS, which respectively indicate the gh-ost, pt-osc, and Alibaba Cloud DMS tools.
|
| ├─ incrSyncConcurrency | integer | The number of threads for reverse incremental synchronization. Default value: 64. |
| ├─ incrLogPullSpeedMode | string | The reverse incremental log pull speed. Valid values: STEADY (low), NORMAL (medium), FAST (high), and CUSTOM (custom). |
| ├─ incrLogPullSpeedConfig | string | The reverse incremental log pull resource configuration. This parameter is effective only when incrLogPullSpeedMode is set to CUSTOM. |
| ├─├─ memory | integer | The memory limit for reverse incremental log pull. |
| ├─ incrLogSyncSpeedMode | string | The reverse incremental data write speed. Valid values: STEADY (low), NORMAL (medium), FAST (high), and CUSTOM (custom). |
| ├─ incrLogSyncSpeedConfig | string | The reverse incremental data write resource configuration. This parameter is effective only when incrLogSyncSpeedMode is set to CUSTOM. |
| ├─├─ memory | integer | The memory limit for reverse incremental log write. |
| ├─├─ readerWorkerNum | integer | The number of concurrent readers for Kafka. This parameter is supported only when the destination is an Oracle database and incremental data is obtained from Kafka. |
| ├─├─ writerWorkerNum | integer | The number of concurrent writers for reverse incremental logs. |
| ├─ throttleRps | integer | The maximum allowed transfer rate in records per second during reverse incremental synchronization. Set this parameter to null to indicate no restriction on RPS. |
| ├─ throttleIOPS | integer | The maximum allowed transfer traffic in bytes per second during reverse incremental synchronization. Set this parameter to null to indicate no restriction on IOPS. |
| ├─ supportDDLTypes | array | The supported DDL operations.
|
Parameters in databases/databasesBlack
Parameter Type Description mappedName string The name of the database in the target. name string The name of the database in the source. id string The ID of the database. tables array The configurations of table objects to be migrated in the current database. When modeisWILDCARD, this parameter specifies the configurations for renaming and row filtering of specific table objects.├─ mappedName string The mapped name of the table. ├─ name string The name of the table. ├─ id string The ID of the table. ├─ whereClause string The WHEREclause in the standard SQL statement for configuring row filtering.Notice
When
modeisWILDCARD, this parameter is not supported.├─ filterColumns array The filtered column set. Notice
When
modeisWILDCARD, this parameter is not supported.├─ shardColumns array The sharding column set. Notice
When
modeisWILDCARD, this parameter is not supported.views array The configurations of view objects to be migrated in the current database. ├─ mappedName string The mapped name of the view. ├─ name string The name of the view. ├─ id string The ID of the view. ├─ whereClause string The WHEREclause in the standard SQL statement for configuring row filtering.Notice
When
modeisWILDCARD, this parameter is not supported.├─ filterColumns array The filtered column set. Notice
<When
modeisWILDCARD, this parameter is not supported.├─ shardColumns array The sharding column set. Notice
When
modeisWILDCARD, this parameter is not supported.functions array The configurations of function objects to be migrated in the current database. ├─ name string The name of the function. ├─ mappedName string The mapped name of the function. This parameter cannot be modified. procedures array The configurations of stored procedure objects to be migrated in the current database. ├─ name string The name of the stored procedure. ├─ mappedName string The mapped name of the stored procedure. This parameter cannot be modified. packages array The configurations of package objects to be migrated in the current database. ├─ name string The name of the package. ├─ mappedName string The mapped name of the package. This parameter cannot be modified. synonyms array The configurations of synonym objects to be migrated in the current database. ├─ name string The name of the synonym. ├─ mappedName string The mapped name of the synonym. This parameter cannot be modified. sequences array The configurations of sequence objects to be migrated in the current database. ├─ name string The name of the sequence. ├─ mappedName string The mapped name of the sequence. This parameter cannot be modified. specificTables array The configurations of specific table objects. When modeisWILDCARD, this parameter specifies the renaming and row filtering configurations of specific table objects.├─ mappedName string The mapped name of the table. ├─ name string The name of the table. ├─ id string The ID of the table. ├─ whereClause string The WHEREclause in the standard SQL statement for configuring row filtering.Notice
When
modeisWILDCARD, this parameter is not supported.├─ filterColumns array The filtered column set. Notice
When
modeisWILDCARD, this parameter is not supported.├─ shardColumns array The sharding column set. Notice
When
modeisWILDCARD, this parameter is not supported.specificViews array The configurations of specific view objects. When modeisWILDCARD, this parameter specifies the renaming configurations of specific view objects.├─ mappedName string The mapped name of the view. ├─ name string The name of the view. ├─ id string The ID of the view. ├─ whereClause string The WHEREclause in the standard SQL statement for configuring row filtering.Notice
When
modeisWILDCARD, this parameter is not supported.├─ filterColumns array The filtered column set. Notice
When
modeisWILDCARD, this parameter is not supported.├─ shardColumns array The sharding column set. Notice
When
modeisWILDCARD, this parameter is not supported.specificFunctions array The configurations of specific function objects. When modeisWILDCARD, this parameter specifies the renaming configurations of specific function objects.├─ name string The name of the function. ├─ mappedName string The mapped name of the function. This parameter cannot be modified. specificProcedures array The configurations of specific stored procedure objects. When modeisWILDCARD, this parameter specifies the renaming configurations of specific stored procedure objects.├─ name string The name of the stored procedure. ├─ mappedName string The mapped name of the stored procedure. This parameter cannot be modified. specificPackages array The configurations of specific package objects. When modeisWILDCARD, this parameter specifies the renaming configurations of specific package objects.├─ name string The name of the package. ├─ mappedName string The mapped name of the package. This parameter cannot be modified. specificSynonyms array The configurations of specific synonym objects. When modeisWILDCARD, this parameter specifies the renaming configurations of specific synonym objects.├─ name string The name of the synonym. ├─ mappedName string The mapped name of the synonym. This parameter cannot be modified. specificSequences array The configurations of specific sequence objects. When modeisWILDCARD, this parameter specifies the renaming configurations of specific sequence objects.├─ name string The name of the sequence. ├─ mappedName string The mapped name of the sequence. This parameter cannot be modified.
Response parameters
| Parameter | Type | Description |
|---|---|---|
| success | boolean | Specifies whether the call is successful. |
| errorDetail | object | Details of the error. |
| code | string | The error code. |
| message | string | The description. |
| advice | string | The suggestion. |
| requestId | string | The request ID. |
| pageNumber | integer[int32] | The page number. This parameter is effective only when you query data in pages. |
| pageSize | integer[int32] | The page size. This parameter is effective only when you query data in pages. |
| totalCount | integer[int64] | The total number of records. This parameter is effective only when you query data in pages. |
| cost | string | The time spent in processing the request, in seconds. |
| data | string | The ID of the task. |
Examples
Request example
{
"name":"test",
"type":"MIGRATION",
"labelIds":[
"l_4w****wygjk"
],
"sourceEndpointId":"e_4wg07f****",
"sinkEndpointId":"e_4wfy2****",
"transferMapping":{
"mode":"SPECIFIC",
"databases":[
{
"mappedName":"gez3***",
"name":"ion1****bmysql",
"id":null,
"tables":[
{
"mappedName":"tion12***20obmysql2",
"name":"GRATION129****",
"id":null,
"whereClause":null,
"filterColumns":null,
"shardColumns":null,
}
],
"views":null,
"specificTables":null,
"specificViews":null
}
],
"databasesBlack":null,
"tableAndViewWhiteList":null,
"tableAndViewBlackList":null
},
"commonTransferConfig":{
"tableCategory":null,
"activeActive":false,
"mqSerializerType":null,
"mqPartitionMode":null,
"mqPartition":0,
"datahubTopicType":null,
"rocketMqProducerGroup":null,
"rocketMqMsgTags":null,
"rocketMqEnableMsgTrace":null,
"rocketMqSendMsgTimeout":null,
"dataWorksBusinessName":null
},
"enableStructTransfer":true,
"structTransferConfig":{
"byteCharConvertStrategy":"DO_NOTHING_IF_BYTE_USED",
"autoProcessedToNextStep": true,
"indexMigrationType": "AFTER_FULL_TRANSFER",
"triggerMigrationType":"NOT_MIGRATION"
},
"enableFullTransfer":true,
"enableFullVerify":true,
"fullTransferConfig":{
"nonePkUkTruncateDstTable":null,
"allowDestTableNotEmpty":true,
"fullTransferSpeedMode":"CUSTOM",
"fullTransferSpeedConfig":{
"memory":4,
"readerWorkerNum":1,
"writerWorkerNum":2
},
"fullVerifySpeedMode":"CUSTOM",
"fullVerifySpeedConfig":{
"memory":4,
"readerWorkerNum":1,
"writerWorkerNum":2
},
"throttleRps": 1000,
"throttleIOPS": 1000
},
"enableIncrTransfer":true,
"enableReverseIncrTransfer":true,
"incrTransferConfig":{
"startTimestamp":1689201369,
"storeLogKeptHour":120,
"recordTypeWhiteList":[
"DELETE",
"INSERT",
"UPDATE"
],
"enableSequencingWithinTxn":true,
"incrSyncConcurrency":64,
"incrLogPullSpeedMode":"NORMAL",
"incrLogPullSpeedConfig":{
"memory":4
},
"incrLogSyncSpeedMode":"CUSTOM",
"incrLogSyncSpeedConfig":{
"memory":4,
"readerWorkerNum":1,
"writerWorkerNum":2
},
"throttleRps": 1000,
"throttleIOPS": 1000,
"enableIncrSyncStatistics":true,
"supportDDLTypes":[
"CREATE_TABLE",
"ALTER_TABLE"
],
"incrOnlineDdlConfig":[
"GH_OST",
"PT_OSC",
"ALIYUN_DMS"
]
},
"ReverseIncrTransferConfig":{
"startTimestamp":1689201369,
"recordTypeWhiteList":[
"DELETE",
"INSERT",
"UPDATE"
],
"storeLogKeptHour":120,
"incrSyncConcurrency":64,
"incrLogPullSpeedMode":"NORMAL",
"incrLogPullSpeedConfig":{
"memory":4
},
"incrLogSyncSpeedMode":"CUSTOM",
"incrLogSyncSpeedConfig":{
"memory":4,
"readerWorkerNum":1,
"writerWorkerNum":2
},
"throttleRps": 1000,
"throttleIOPS": 1000,
"enableIncrSyncStatistics":true,
"supportDDLTypes":[
"CREATE_TABLE",
"ALTER_TABLE"
]
}
}
Response example
{
"Success":true,
"ErrorDetail":{
"Code":"CM-RES***11",
"Level":"WARN",
"Message":"A system error occurred.",
"Proposal":"Contact the administrator."
},
"Code":"INNER_ERROR",
"Message":"A system error occurred.",
"Advice":"Contact the administrator.",
"RequestId":"XCVSADG****DSGDS",
"PageNumber":1,
"PageSize":1,
"TotalCount":1,
"Cost":"27 ms",
"Data":"np_****"
}
