CreateProject

2024-06-26 08:14:11  Updated

This topic describes the CreateProject API provided by OceanBase Migration Service (OMS) Community Edition.

Purpose

You can call this operation to create a data migration or synchronization task.

Call description

Limitations

This operation is supported in OMS Community Edition V4.1.1 and later.

Request path

POST /api/v2?Action=CreateProject

Request parameters

Parameter Type Description
name string The name of the task, which cannot contain spaces.
type string The type of the task. Valid values include MIGRATION and SYNC.
labelIds string The tag IDs. You can specify up to five tag IDs.
sourceEndpointId string The ID of the source data source. You must call the ListDataSource operation to obtain the ID, which is in the e_**** format.
sinkEndpointId string The ID of the destination data source. You must call the ListDataSource operation to obtain the ID, which is in the e_**** format.
transferMapping Object The mapping information about the objects to be transferred.
├─ mode string The mapping mode for the objects to be transferred. Valid values include SPECIFIC and WILDCARD, which indicate specifying specific objects and matching objects by using a rule, respectively.
├─ databases Array The database objects to be transferred. For more information, see Parameters in databases/databasesBlack.
├─ databasesBlack Array The blocklist of database objects used for the WILDCARD mode. For more information, see Parameters in databases/databasesBlack.
├─ tableAndViewWhiteList Array The wildcard-based allowlist of tables and views used for the WILDCARD mode.
├─ tableAndViewBlackList Array The wildcard-based blocklist of tables and views used for the WILDCARD mode.
commonTransferConfig Object The common settings for transfer.
├─ tableCategory string The supported table types. Valid values: ALL (all tables), WITH_UNIQUE_ROW_ID (tables with a unique row ID), and WITHOUT_UNIQUE_ROW_ID (tables without a unique row ID). Default value: ALL.
├─ activeActive boolean Indicates whether active-active disaster recovery is enabled.
├─ mqSerializerType string The JSON data serialization type for delivering data to message queues. Valid values include DEFAULT, CANAL, DATAWORKS_V2, SHAREPLEX, DEFAULT_WITH_SCHEMA, DEBEZIUM, DEBEZIUMFLATTEN, and DEBEZIUMSMT.
├─ mqPartitionMode string The partition routing mode for delivering data to message queues. Valid values include ONE, HASH, and TABLE. ONE specifies to deliver data to a specified partition. HASH specifies to deliver data to HASH partitions based on the primary key column or sharding columns. TABLE specifies to deliver data to HASH partitions based on database and table names.
├─ mqPartition Integer When the value of the partitionMode parameter is ONE, this parameter indicates the location of the partition to which data is delivered in the current topic.
├─ datahubTopicType string The type of the topic for delivering data to DataHub. Valid values include TUPLE and BLOB.
├─ rocketMqProducerGroup string The name of the producer group. This parameter takes effect when the type of destination data source is RocketMQ. The default value is OMS.
├─ rocketMqMsgTags string The tag of the message. This parameter takes effect when the type of destination data source is RocketMQ. For example, when the value is default, the _tag suffix is added to the value by default.
├─ rocketMqEnableMsgTrace boolean Indicates whether message tracing is enabled. This parameter takes effect when the type of destination data source is RocketMQ.
├─ rocketMqSendMsgTimeout Long The message delivery timeout period. This parameter takes effect when the type of destination data source is RocketMQ.
├─ dataWorksBusinessName string The ID of the business system.
enableStructTransfer boolean Indicates whether schema transfer is allowed.
structTransferConfig Object The settings for schema transfer.
├─ byteCharConvertStrategy string The BYTE/CHAR conversion scheme. Valid values include IGNORE_BYTE_IF_BYTE_USED, FORCE_CHAR_IF_BYTE_USED, EXPAND_LEN_IF_BYTE_USED, and DO_NOTHING_IF_BYTE_USED. The default value is DO_NOTHING_IF_BYTE_USED.
├─ deferIndexCreation boolean Indicates whether post-indexing is allowed. The default value is false.
enableFullTransfer boolean Indicates whether full transfer is enabled.
enableFullVerify boolean Indicates whether full verification is enabled.
fullTransferConfig Object The settings for full transfer.
├─ nonePkUkTruncateDstTable boolean Indicates whether the destination table is cleared during full migration from a table without a unique index.
├─ allowDestTableNotEmpty string Specifies the processing strategy when the destination table has records. Valid values: True and False.
  • The value True specifies to ignore existing data in the destination table. If the destination is a MySQL database or OceanBase Database Community Edition, OMS Community Edition overwrites the existing data in the destination table.

    Notice

    If you select Ignore, data is pulled in IN mode for verification. In this case, the scenario where the destination table contains more data than the source table cannot be verified, and the verification efficiency will be decreased.

  • The value False specifies to stop migration. If the destination table has records, an error is returned during full migration. In this case, clear the existing data in the destination table and try again.

    Notice

    After an error is returned, if you click Resume in the dialog box, OMS Community Edition ignores this error and continues to migrate data. Proceed with caution.

├─ fullTransferSpeedMode string The concurrency for full migration. Valid values include STEADY, NORMAL, and FAST.
├─ fullVerifySpeedMode string The concurrency for full verification. Valid values include STEADY, NORMAL, and FAST.
enableIncrTransfer boolean Indicates whether incremental synchronization is enabled.
enableReverseIncrTransfer boolean Indicates whether reverse incremental synchronization is enabled.
incrTransferConfig Object The settings for incremental synchronization.
├─ startTimestamp string The start timestamp for incremental synchronization. This parameter is available only when incremental synchronization is enabled and full transfer is disabled.
├─ recordTypeWhiteList string The data types of incremental data synchronized in incremental synchronization. Supported data types include HEARTBEAT, INSERT, UPDATE, DELETE, BEGIN, COMMIT, ROLLBACK, DDL, and ROW.
├─ storeLogKeptHour Integer The retention period in hours of logs when incremental synchronization is enabled and the incremental log pull component is Store. The default retention period is seven days. The maximum retention period is 365 days.
├─ enableSequencingWithinTxn boolean Indicates whether intra-transaction sequencing is enabled when incremental synchronization is enabled and the incremental log pull component is Store.
├─ incrSyncConcurrency Integer The number of threads for incremental synchronization. The default value is 64.
├─ enableIncrSyncStatistics boolean Indicates whether statistics collection is enabled for incremental DML and DDL operation records.
  • Parameters in databases/databasesBlack

    Parameter Type Description
    mappedName string The name of the destination database.
    name string The name of the source database.
    id string The ID of the database.
    tables Array The settings for the table objects to be transferred in the current database. When the value of the mode parameter is WILDCARD, this parameter indicates the settings for a specific blocklist or allowlist of tables and views.
    ├─ mappedName string The mapped table name.
    ├─ name string The name of the table.
    ├─ id string The ID of the table.
    ├─ whereClause string The WHERE clause in the standard SQL syntax that is used to configure row filtering.
    ├─ filterColumns Array The filter columns.
    ├─ shardColumns Array The sharding columns.
    views Array The settings for the view objects to be transferred in the current database.
    ├─ mappedName string The mapped view name.
    ├─ name string The name of the view.
    ├─ id string The ID of the view.
    ├─ whereClause string The WHERE clause in the standard SQL syntax that is used to configure row filtering.
    ├─ filterColumns Array The filter columns.
    ├─ shardColumns Array The sharding columns.
    specificTables Array The settings for the matched table objects. When the value of the mode parameter is WILDCARD, this parameter indicates the rename and row filtering settings for the matched table objects.
    ├─ mappedName string The mapped table name.
    ├─ name string The name of the table.
    ├─ id string The ID of the table.
    ├─ whereClause string The WHERE clause in the standard SQL syntax that is used to configure row filtering.
    ├─ filterColumns Array The filter columns.
    ├─ shardColumns Array The sharding columns.
    specificViews Array The settings for the matched view objects. When the value of the mode parameter is WILDCARD, this parameter indicates the rename settings for the matched view objects.
    ├─ mappedName string The mapped view name.
    ├─ name string The name of the view.
    ├─ id string The ID of the view.
    ├─ whereClause string The WHERE clause in the standard SQL syntax that is used to configure row filtering.
    ├─ filterColumns Array The filter columns.
    ├─ shardColumns Array The sharding columns.

Response parameters

Parameter Type Description
success boolean Indicates whether the call is successful.
errorDetail Object The error details.
code string The error code.
message string The description.
advice string The suggestions.
requestId string The ID of the request.
pageNumber Integer[int32] The page number, which takes effect in a pagination query.
pageSize Integer[int32] The page size, which takes effect in a pagination query.
totalCount Integer[int64] The total number of pages returned, which takes effect in a pagination query.
cost string The time spent in processing the request, in seconds.
data string The task ID.

Examples

Sample request

{
    "name":"test",
    "type":"MIGRATION",
    "labelIds":[
        "l_4w****wygjk"
    ],
    "sourceEndpointId":"e_4wg07f****",
    "sinkEndpointId":"e_4wfy2****",
    "transferMapping":{
        "mode":"SPECIFIC",
        "databases":[
            {
                "mappedName":"gez3***",
                "name":"ion1****bmysql",
                "id":null,
                "tables":[
                    {
                        "mappedName":"tion12***20obmysql2",
                        "name":"GRATION129****",
                        "id":null,
                        "whereClause":null,
                        "filterColumns":null,
                        "shardColumns":null,
                    }
                ],
                "views":null,
                "specificTables":null,
                "specificViews":null
            }
        ],
        "databasesBlack":null,
        "tableAndViewWhiteList":null,
        "tableAndViewBlackList":null
    },
    "commonTransferConfig":{
        "tableCategory":null,
        "activeActive":false,
        "mqSerializerType":null,
        "mqPartitionMode":null,
        "mqPartition":0,
        "datahubTopicType":null,
        "rocketMqProducerGroup":null,
        "rocketMqMsgTags":null,
        "rocketMqEnableMsgTrace":null,
        "rocketMqSendMsgTimeout":null,
        "dataWorksBusinessName":null
    },
    "enableStructTransfer":true,
    "structTransferConfig":{
        "byteCharConvertStrategy":"DO_NOTHING_IF_BYTE_USED",
        "deferIndexCreation":true
    },
    "enableFullTransfer":true,
    "enableFullVerify":true,
    "fullTransferConfig":{
        "nonePkUkTruncateDstTable":null,
        "allowDestTableNotEmpty":true,
        "fullTransferSpeedMode":"NORMAL",
        "fullVerifySpeedMode":"NORMAL"
    },
    "enableIncrTransfer":true,
    "enableReverseIncrTransfer":true,
    "incrTransferConfig":{
        "startTimestamp":1689201369,
        "recordTypeWhiteList":[
            "DELETE",
            "INSERT",
            "UPDATE"
        ],
        "storeLogKeptHour":120,
        "enableSequencingWithinTxn":true,
        "incrSyncConcurrency":64,
        "enableIncrSyncStatistics":true
    }
}

Sample response

{
    "Success":true,
    "ErrorDetail":{
        "Code":"CM-RES***11",
        "Level":"WARN",
        "Message":"A system error occurred.",
        "Proposal":"Contact the administrator."
    },
    "Code":"INNER_ERROR",
    "Message":"A system error occurred.",
    "Advice":"Contact the administrator.",
    "RequestId":"XCVSADG****DSGDS",
    "PageNumber":1,
    "PageSize":1,
    "TotalCount":1,
    "Cost":"27 ms",
    "Data":"np_****"
}

Contact Us