Overview
Creates a data migration or data validation project. Corresponds to CommonProjectController.createProject.
API details
Constraints
The caller must have an AccessKey to access the multi-cloud API. For information about how to obtain the AccessKey ID and AccessKey Secret, see Manage AccessKeys.
Request path
POST /api/v2/oms/project
Request parameters
Header
Parameter |
Type |
Required |
Description |
Example value |
|---|---|---|---|---|
| X-Ob-Project-Id | String | Yes | Project ID. | proj-xxxxxxxx |
| X-Ob-Org-Id | String | Yes | Organization ID. | org-xxxxxxxx |
Query
Parameter |
Type |
Required |
Description |
Example value |
|---|---|---|---|---|
| requestId | String | No | Request ID. | d04eabba-acef-4f4b-96f1-c657******** |
Body
Parameter |
Type |
Required |
Description |
Example value |
|---|---|---|---|---|
| name | String | Yes | Project name, 1 to 256 characters long. Spaces are not allowed. | Data migration project |
| type | String | Yes | Project type: MIGRATION (data migration) or VERIFY (data validation). |
MIGRATION |
| sourceDatasourceId | String | Yes | Source data source ID. | resource-123456 |
| sinkDatasourceId | String | Yes | Destination data source ID. | resource-123456 |
| transferMapping | Object | Yes | Transfer object mappings. | |
| commonTransferConfig | Object | No | General transfer settings. | |
| enableStructTransfer | Boolean | No | Whether to enable schema transfer. Default: false. | true |
| structTransferConfig | Object | No | Schema transfer settings. Required when enableStructTransfer=true. | |
| enableFullTransfer | Boolean | No | Whether to enable full data transfer. Default: false. | true |
| fullTransferConfig | Object | No | Full data transfer settings. Required when enableFullTransfer=true. | |
| enableIncrTransfer | Boolean | No | Whether to enable incremental data transfer. Default: false. | true |
| incrTransferConfig | Object | No | Incremental data transfer settings. Required when enableIncrTransfer=true. | |
| enableFullVerify | Boolean | No | Whether to enable full validation. Default: false. For VERIFY tasks, enable either this option or enableCountVerify. | true |
| fullVerifyConfig | Object | No | Full validation settings. Required when enableFullVerify=true. | |
| enableCountVerify | Boolean | No | Whether to enable row count validation. Default: false. For VERIFY tasks, enable either this option or enableFullVerify. | false |
| countVerifyConfig | Object | No | Row count validation settings. Required when enableCountVerify=true. | |
| enableIncrVerify | Boolean | No | Whether to enable incremental validation. Default: false. This creation API does not currently support incremental validation projects. Do not set this field to true; scenario validation will reject the request. | false |
| incrVerifyConfig | Object | No | Incremental validation settings. Omit this field because the creation API does not currently support incremental validation. |
Note
- This API currently supports creating VERIFY projects with either full validation or row count validation (
enableFullVerify=trueorenableCountVerify=true, but not both). - Although
enableIncrVerifyis listed in the parameter table, this scenario is not yet available. SettingenableIncrVerify=truewhen creating a VERIFY project causes scenario validation to reject the request. Validation steps are also not allowed in MIGRATION projects.
Response parameters
Field |
Type |
Description |
Example value |
|---|---|---|---|
| success | Boolean | Whether the request succeeded. | true |
| data | String | ID of the created project. | np_xxx |
| requestId | String | Request ID. | req-xxx-xxx |
| cost | String | Time taken. | 123ms |
Nested object fields
transferMapping (transfer object mappings)
Parameter |
Type |
Required |
Description |
|---|---|---|---|
| mode | String | Yes | Transfer mapping mode. Only COMPLEX is supported. |
| subMode | String | Conditional | Submode. Valid values: WILDCARD and FULLDB. Required when whiteTableRules is nonempty; otherwise, the server raises the validation error "subMode is required when whiteTableRules is configured". |
| schemas | List<TransferSchema> |
Yes | List of schemas (COMPLEX mode). |
| databasesBlack | List<TransferSchema> |
No | Database blocklist. |
| tableAndViewWhiteList | List<String> |
No | Table and view allowlist. |
| tableAndViewBlackList | List<String> |
No | Table and view blocklist. |
| whiteTableRules | List<TransferMatchingRule> |
No | Table allowlist matching rules. |
| whiteViewRules | List<TransferMatchingRule> |
No | View allowlist matching rules. |
| blackTableRules | List<TransferMatchingRule> |
No | Table blocklist matching rules. |
| blackViewRules | List<TransferMatchingRule> |
No | View blocklist matching rules. |
TransferSchema structure
Field |
Type |
Description |
|---|---|---|
| name | String | Original database or schema name. |
| mappedName | String | Mapped database or schema name. |
| tables | List<TransferTable> |
List of table configurations. |
| views | List<TransferView> |
List of view configurations. |
TransferTable structure
Field |
Type |
Description |
|---|---|---|
| name | String | Original table name. |
| mappedName | String | Mapped table name. |
| whereClause | String | Row filter condition (the expression after WHERE). |
| applyFilterSqlToQuerySlice | Boolean | Whether the row filter applies to QuerySlice. Default: false. |
TransferView structure
Field |
Type |
Description |
|---|---|---|
| name | String | Original view name. |
| mappedName | String | Mapped view name. |
TransferMatchingRule structure
Field |
Type |
Description |
|---|---|---|
| schemaMapping | TransferMappingRule | Schema matching and mapping rules. |
| objectMapping | TransferMappingRule | Matching and mapping rules for objects (tables or views) in a schema. |
TransferMappingRule structure
Field |
Type |
Description |
|---|---|---|
| name | String | Source matching pattern. Wildcards such as * and [a-zA-Z]* are supported. |
| mappedName | String | Mapped name at the destination. Optional; if omitted, the source name is used. |
commonTransferConfig (general transfer settings)
Parameter |
Type |
Required |
Description |
|---|---|---|---|
| tableCategory | String | No | Types of tables to transfer: ALL (all tables, default), WITH_UNIQUE_ROW_ID (tables with a unique row identifier), or WITHOUT_UNIQUE_ROW_ID (tables without a unique row identifier). |
| mqSerializerType | String | No | MQ serialization type. Applies when the destination is MQ. Valid values: DEFAULT, CANAL, DATAWORKS_V2, SHAREPLEX, DEFAULT_WITH_SCHEMA, DEBEZIUM, DEBEZIUM_FLATTEN, DEBEZIUM_SMT, and AVRO. |
| mqPartitionMode | String | No | MQ partition routing mode. Valid values: ONE, TABLE, and HASH. |
| mqPartition | Integer | No | Partition position when partitionMode=ONE. Must be ≥0. |
| datahubTopicType | String | No | DataHub topic type. Applies when the destination is DATAHUB. Valid values: TUPLE and BLOB. |
| dataWorksBusinessName | String | No | Business system identifier. Applies when delivering data to MQ. |
| syncSchema | Boolean | No | Whether to synchronize the source schema to the destination. Applies only when the source is PostgreSQL. |
| syncSchemaColumnName | String | No | Name of the field used to synchronize the schema. Applies only when the source is PostgreSQL. |
| customColumns | List<CustomColumn> |
No | Global custom columns. Applies only when the source is PostgreSQL. |
| sinkStoreFormat | String | No | Storage type of destination table objects (OB 4.3+). |
| sourceStoreFormat | String | No | Storage type of source table objects (OB 4.3+). |
| enableHiddenColumnsAndIndex | Boolean | No | Whether to add hidden columns and indexes to tables with a non-null unique key. |
CustomColumn structure
Field |
Type |
Description |
|---|---|---|
| columnName | String | Column name. |
| columnValue | String | Column value. |
| columnType | String | Column type: STRING (string, default) or FUNCTION (function). |
structTransferConfig (schema transfer settings)
Parameter |
Type |
Required |
Description |
|---|---|---|---|
| byteCharConvertStrategy | String | No | Byte/char conversion strategy. Valid values: IGNORE_BYTE_IF_BYTE_USED, FORCE_CHAR_IF_BYTE_USED, EXPAND_LEN_IF_BYTE_USED, and DO_NOTHING_IF_BYTE_USED (default). |
| deferIndexCreation | Boolean | No | Whether to defer index creation. Default: false. |
fullTransferConfig (full data transfer settings)
Parameter |
Type |
Required |
Description |
|---|---|---|---|
| nonePkUkTruncateDstTable | Boolean | No | Whether to truncate the destination table for tables without a primary key or unique index. |
| allowDestTableNotEmpty | Boolean | No | Whether to allow nonempty destination tables. Allowed only when full validation is not selected. |
| writeWorkerNum | Integer | No | Write concurrency, in the range [1, 512]. |
| readWorkerNum | Integer | No | Read concurrency, in the range [1, 512]. |
| indexDDLConcurrencyLimit | Integer | No | Parallelism for a single index DDL statement, in the range [1, 512]. |
| maxConcurrentIndexDDLs | Integer | No | Maximum number of concurrent index DDL statements, in the range [1, 512]. |
| throttleRps | Integer | No | RPS limit. Null means no limit. |
| throttleIOPS | Integer | No | Traffic limit (bytes). Null means no limit. |
| sinkThrottleRps | Integer | No | Destination RPS limit. Null means no limit. |
| sinkThrottleIOPS | Integer | No | Destination traffic limit. Null means no limit. |
incrTransferConfig (incremental data transfer settings)
Parameter |
Type |
Required |
Description |
|---|---|---|---|
| startTimestamp | Long | Conditional | Incremental start position (timestamp in seconds), within the last 30 days. Required when full data transfer is not selected. |
| recordTypeWhiteList | List<String> |
No | Allowlist of incremental synchronization data types: HEARTBEAT, INSERT, UPDATE, DELETE, BEGIN, COMMIT, ROLLBACK, DDL, and ROW. |
| storeLogKeptHour | Integer | No | Store log retention period in hours, in the range [1, 8760]. Default: 24. |
| enableSequencingWithinTxn | Boolean | No | Whether to enable sequence numbering within transactions. Applies only when the source is OceanBase. |
| incrSyncThreadCount | Integer | No | Number of incremental synchronization threads, in the range [1, 512]. Default: 64. Recommended value: four times the number of CPU cores on the machine. |
| enableIncrSyncStatistics | Boolean | No | Whether to enable incremental DML/DDL statistics. Default: true. |
| throttleRps | Integer | No | RPS limit. Null means no limit. |
| throttleIOPS | Integer | No | Traffic limit (bytes). Null means no limit. |
| supportDDLTypes | List<String> |
Conditional | Supported DDL types. Required when DDL synchronization is enabled. Valid values: CREATE_TABLE, DROP_TABLE, TRUNCATE_TABLE, RENAME_TABLE, ALTER_TABLE, CREATE_INDEX, and DROP_INDEX. |
| incrOnlineDdlConfig | List<String> |
No | Online DDL rules. Valid values: GH_OST, PT_OSC, and ALIYUN_DMS. |
| sourceHeartbeatIntervalSecond | Integer | No | MySQL source heartbeat interval in seconds, in the range [0, 60]. Default: 5. Set to 0 to disable heartbeats. |
fullVerifyConfig (full validation settings)
Parameter |
Type |
Required |
Description |
|---|---|---|---|
| mode | String | No | Validation mode: AUTO (adaptive, default) or IN. |
| sourceThreads | Integer | No | Number of source threads, in the range [1, 512]. Default: 4. |
| sourceThrottleBps | Integer | No | Source BPS limit in MiB/s, in the range [1, 1024]. Null means no limit. |
| sinkThreads | Integer | No | Number of destination threads, in the range [1, 512]. Default: 4. |
| sinkThrottleBps | Integer | No | Destination BPS limit in MiB/s, in the range [1, 1024]. Null means no limit. |
countVerifyConfig (row count validation settings)
Parameter |
Type |
Required |
Description |
|---|---|---|---|
| concurrencyTablesNumber | Integer | Yes | Number of tables to validate concurrently, in the range [1, 100]. Required when enableCountVerify=true. The server does not supply a default value; omitting this field causes a parameter error. |
| queryTimeout | Integer | Yes | Query timeout in seconds, in the range [1, 1440]. Required when enableCountVerify=true. The server does not supply a default value; omitting this field causes a parameter error. |
incrVerifyConfig (incremental validation settings)
Parameter |
Type |
Required |
Description |
|---|---|---|---|
| startCheckpoint | String | No | Incremental validation start position (timestamp), in the range [now-7d, now+7d]. |
| endCheckpoint | String | No | Incremental validation end position. If omitted, validation continues indefinitely. |
| initialInspectionCooldownTimeout | Integer | No | Cooldown period for hot data during initial validation, in seconds, in the range [1, 3600]. |
| initialInspectionDelayTimeout | Integer | No | Time to detect inconsistent data during initial validation, in seconds, in the range [1, 86400]. |
| reInspectionThreads | Integer | No | Number of revalidation threads, in the range [1, 512]. |
| reInspectionthrottleBps | Integer | No | Revalidation rate limit in MiB/s, in the range [1, 1024]. |
| reInspectiontolerateTimeout | Integer | No | Maximum revalidation duration in minutes, in the range [1, 720]. |
Supported transfer paths
Source |
Task type |
Supported destinations |
|---|---|---|
| OB_MYSQL | MIGRATION | OB_MYSQL, OB_ORACLE, MYSQL, POSTGRESQL, KAFKA, TIDB |
| OB_ORACLE | MIGRATION | OB_ORACLE, OB_MYSQL, ORACLE, POSTGRESQL, KAFKA |
| MYSQL | MIGRATION | OB_MYSQL |
Examples
Request example
MIGRATION (data migration) example
curl --digest -u '<your AK:SK>' \
--request POST \
--url 'https://api-cloud.oceanbase.com/api/v2/oms/project' \
-H 'Content-Type: application/json' \
-H 'X-Ob-Project-Id: <project ID>' \
-H 'X-Ob-Org-Id: <organization ID>' \
--data @request.json
{
"name": "MySQLToOBMigration",
"type": "MIGRATION",
"sourceDatasourceId": "ds-source-xxx",
"sinkDatasourceId": "ds-sink-xxx",
"transferMapping": {
"mode": "COMPLEX",
"schemas": [
{
"name": "source_db",
"mappedName": "target_db",
"tables": [
{ "name": "table_1", "whereClause": "id > 1", "applyFilterSqlToQuerySlice": true },
{ "name": "table_2", "mappedName": "table_2_dest" }
]
}
]
},
"enableStructTransfer": true,
"enableFullTransfer": true,
"enableIncrTransfer": true,
"structTransferConfig": {
"byteCharConvertStrategy": "DO_NOTHING_IF_BYTE_USED",
"deferIndexCreation": false
},
"fullTransferConfig": {
"writeWorkerNum": 64,
"readWorkerNum": 64
},
"incrTransferConfig": {
"incrSyncThreadCount": 64,
"recordTypeWhiteList": ["INSERT", "UPDATE", "DELETE", "DDL"],
"supportDDLTypes": ["CREATE_TABLE", "DROP_TABLE", "ALTER_TABLE", "CREATE_INDEX", "DROP_INDEX"]
}
}
VERIFY (data validation) example
{
"name": "MySQLToOBVerify",
"type": "VERIFY",
"sourceDatasourceId": "ds-source-xxx",
"sinkDatasourceId": "ds-sink-xxx",
"transferMapping": {
"mode": "COMPLEX",
"schemas": [
{
"name": "source_db",
"mappedName": "target_db",
"tables": [
{ "name": "table_1", "mappedName": "table_1_dest" },
{ "name": "table_2", "mappedName": "table_2_dest" }
]
}
]
},
"enableFullVerify": true,
"fullVerifyConfig": {
"mode": "AUTO",
"sourceThreads": 4,
"sinkThreads": 4
}
}
Response example
{
"success": true,
"data": "np_xxx",
"requestId": "req-xxx-xxx",
"cost": "123ms"
}
