ListProjects|V4.2.11|OceanBase Migration Service| docs|Distributed Database

This topic describes the ListProjects operation provided by OceanBase Migration Service (OMS) Community Edition.

Description

You can call this operation to query a list of data migration or synchronization tasks in pagination mode.

Call description

Limitations

This operation is supported in OMS Community Edition V4.1.1 and later.

Request path

POST /api/v2?Action=ListProjects

Request parameters

Parameter	Type	Description
searchKey	string	The name or ID of the data migration or synchronization task.
type	string	The type of the task. Valid values: `MIGRATION` and `SYNC`.
status	string	The status of the task.
labelIds	array	The tag IDs. You can specify a maximum of five tag IDs.
sourceEndpointTypes	array	The types of sources.
sinkEndpointTypes	array	The types of targets.
visibleSubProject	boolean	Specifies whether to show subtasks. Default value: `false`.
sortField	string	The sorting field. Valid values: `gmtCreate` and `gmtModified`. Default value: `gmtCreate`.
order	string	The sorting order. Valid values: `asc` and `descend`. Default value: `descend`.
pageNumber	integer	The current page number. Default value: `1`.
pageSize	integer	The number of records to return per page. Default value: `10`. Maximum value: `150`.

Response parameters

Parameter	Type	Description
success	boolean	Indicates whether the call is successful.
errorDetail	object	The error details.
code	string	The error code.
message	string	The error description.
advice	string	The suggestions.
requestId	string	The ID of the request.
pageNumber	integer	The number of the page returned, which takes effect in a pagination query.
pageSize	integer	The number of records returned on each page, which takes effect in a pagination query.
totalCount	integer	The total number of pages returned, which takes effect in a pagination query.
cost	string	The time spent in processing the request, in seconds.
data	object []	The information about the tasks. For more information, see the Parameters in data section.

Parameters in data

Parameter	Type	Description
id	string	The ID of the task.
type	string	The type of the task. Valid values: `MIGRATION` and `SYNC`.
name	string	The name of the task, which cannot contain spaces.
├─ id	string	The ID of the tag.
├─ name	string	The name of the tag.
owner	string	The owner of the task.
importance	string	The alert level of the task. Valid values: `HIGH`, `MEDIUM`, `LOW`, and `IGNORE`.
status	string	The status of the task.
gmtCreate	string	The time when the task was created, in the UTC+0 time zone.
gmtModified	string	The time when the task was updated, in the UTC+0 time zone.
gmtStart	string	The start time of the task.
gmtFinish	string	The end time of the task.
destConnId	string	The connection ID of the task.
isMerging	boolean	Indicates whether the task is in the progress of a major compaction.
isModifying	boolean	Indicates whether the task is being modified.
isSubProject	boolean	Indicates whether the task is a subtask.
sourceEndpointType	string	The type of the source. Valid values: `MYSQL`, `OB_ORACLE`, `OB_MYSQL`, `ORACLE`, `TIDB`, `DB2_LUW`, `POSTGRESQL`, `IDB`, and `DBP_OP_ROUTE`.
sinkEndpointType	string	The type of the target. Valid values: `MYSQL`, `OB_ORACLE`, `OB_MYSQL`, `ORACLE`, `DB2_LUW`, `KAFKA`, `DATAHUB`, and `ROCKETMQ`.
transferMapping	Object	The mapping information about the objects to be transferred.
├─ mode	string	The mapping mode for the objects to be transferred. Valid values: `SPECIFIC` and `WILDCARD`, which indicate specifying specific objects and matching objects by using a rule, respectively.
├─ databases	Array	The list of database objects. For more information, see the Parameters in databases/databasesBlack section.
├─ databasesBlack	Array	The blocklist of database objects used for the `WILDCARD` mode. For more information, see the Parameters in databases/databasesBlack section.
├─ tableAndViewWhiteList	array	The wildcard-based allowlist of tables and views used for the `WILDCARD` mode.
├─ tableAndViewBlackList	array	The wildcard-based blocklist of tables and views used for the `WILDCARD` mode.
commonTransferConfig	Object	The common settings for transfer.
├─ tableCategory	string	The type of tables to be transferred. Valid values: `ALL` (all tables), `WITH_UNIQUE_ROW_ID` (tables with a unique row ID), and `WITHOUT_UNIQUE_ROW_ID` (tables without a unique row ID). Default value: `ALL`.
├─ activeActive	boolean	Indicates whether active-active disaster recovery is enabled.
├─ mqSerializerType	string	The JSON data serialization type for delivering data to message queues. Valid values: `DEFAULT`, `CANAL`, `DATAWORKS_V2`, `SHAREPLEX`, `DEFAULT_WITH_SCHEMA`, `DEBEZIUM`, `DEBEZIUMFLATTEN`, and `DEBEZIUMSMT`.
├─ mqPartitionMode	string	The partition routing mode for delivering data to message queues. Valid values: `ONE`, `HASH`, and `TABLE`. `ONE` specifies to deliver data to a specified partition. `HASH` specifies to deliver data to HASH partitions based on the primary key column or sharding columns. `TABLE` specifies to deliver data to HASH partitions based on database and table names.
├─ mqPartition	integer	When the value of the `partitionMode` parameter is `ONE`, this parameter indicates the location of the partition to which data is delivered in the current topic.
├─ datahubTopicType	string	The type of the topic for delivering data to DataHub. Valid values: `TUPLE` and `BLOB`.
├─ rocketMqProducerGroup	string	The name of the producer group. This parameter takes effect when the type of the target is RocketMQ. The default value is `OMS`.
├─ rocketMqMsgTags	string	The tag of the message. This parameter takes effect when the type of the target is RocketMQ. For example, when the value is `default`, the `_tag` suffix is added to the value by default.
├─ rocketMqEnableMsgTrace	boolean	Indicates whether message tracing is enabled. This parameter takes effect when the type of the target is RocketMQ.
├─ rocketMqSendMsgTimeout	long	The message delivery timeout period. This parameter takes effect when the type of the target is RocketMQ.
├─ dataWorksBusinessName	string	The ID of the business system.
enableStructTransfer	boolean	Indicates whether schema transfer is allowed.
structTransferConfig	object	The settings for schema transfer.
├─ byteCharConvertStrategy	string	The BYTE/CHAR conversion scheme. Valid values: `IGNORE_BYTE_IF_BYTE_USED`, `FORCE_CHAR_IF_BYTE_USED`, `EXPAND_LEN_IF_BYTE_USED`, and `DO_NOTHING_IF_BYTE_USED`. Default value: `DO_NOTHING_IF_BYTE_USED`.
├─ deferIndexCreation	boolean	Indicates whether post-indexing is allowed. The default value is `false`.
enableFullTransfer	boolean	Indicates whether full transfer is enabled.
enableFullVerify	boolean	Indicates whether full verification is enabled.
fullTransferConfig	object	The settings for full transfer.
├─ nonePkUkTruncateDstTable	boolean	Indicates whether the target table is cleared during full migration from a table without a unique index.
├─ allowDestTableNotEmpty	string	The processing strategy to apply when the target table has records, including Ignore and Stop Migration. The value `true` indicates that Ignore is selected. If the target is a MySQL database or OceanBase Database Community Edition, OMS Community Edition overwrites the existing data in the target table. Notice If you set the value to true, data is pulled in IN mode for full verification. In this case, the scenario where the target table contains more data than the source table cannot be verified, and the verification efficiency will be decreased. The value `false` indicates that Stop Migration is selected. If the target table has records, an error is returned during full migration. In this case, clear the existing data in the target table and try again. Notice After an error is returned, if you click Resume in the dialog box, OMS Community Edition ignores this error and continues to migrate data. Proceed with caution.
├─ fullTransferSpeedMode	string	The concurrency for full migration. Valid values: `STEADY`, `NORMAL`, and `FAST`.
├─ fullVerifySpeedMode	string	The concurrency for full verification. Valid values: `STEADY`, `NORMAL`, and `FAST`.
enableIncrTransfer	boolean	Indicates whether incremental synchronization is enabled.
enableIncrVerify	boolean	Indicates whether incremental verification is enabled.
enableReverseIncrTransfer	boolean	Indicates whether reverse increment is enabled.
incrTransferConfig	object	The settings for incremental synchronization.
├─ startTimestamp	long	The start timestamp for incremental synchronization. This parameter is available only when incremental synchronization is enabled and full transfer is disabled.
├─ recordTypeWhiteList	array	The data types of incremental data synchronized in incremental synchronization. Supported data types are HEARTBEAT, INSERT, UPDATE, DELETE, BEGIN, COMMIT, ROLLBACK, DDL, and ROW.
├─ storeLogKeptHour	integer	The retention period of logs when incremental synchronization is enabled and the Store component is used to pull incremental logs. Unit: hours. The default retention period is seven days. The maximum retention period is 365 days.
├─ enableSequencingWithinTxn	boolean	Indicates whether intra-transaction sequencing is enabled when incremental synchronization is enabled and the Store component is used to pull incremental logs.
├─ incrSyncConcurrency	integer	The number of threads for incremental synchronization. Default value: `64`.
├─ enableIncrSyncStatistics	boolean	Indicates whether statistics collection is enabled for incremental DML and DDL operation records.
sourceConnectInfo	object	The connection information of the source.
├─ id	string	The ID of the connection.
├─ endpointName	string	The name of the endpoint.
├─ endpointId	string	The ID of the endpoint.
├─ endpointSide	string	The type of the endpoint. Valid values: `source` and `dest`.
├─ dbEngine	string	The database engine of the endpoint.
├─ connectionInfo	string	The connection information of the endpoint.
├─ username	string	The username used to connect to the database.
├─ version	string	The version of the database.
├─ timezone	string	The time zone of the database.
├─ charset	string	The character set.
├─ nlsLengthSemantics	string	This attribute is available only when the data source is an Oracle database or an Oracle-compatible tenant of OceanBase Database.
├─ operatingSystem	string	The operating system. This parameter is unavailable for OceanBase Database.
├─ region	string	The region.
├─ ocpName	string	The name of the OceanBase Cloud Platform (OCP) instance.
├─ connExtraAttributes	object	The additional attributes of the connection. For more information about the nested response parameters, see Appendix.
├─ owner	string	The owner.
├─ host	string	The IP address.
├─ port	integer	The port number.
sinkConnectInfo	object	The connection information of the target.
├─ id	string	The ID of the connection.
├─ endpointName	string	The name of the endpoint.
├─ endpointId	string	The ID of the endpoint.
├─ endpointSide	string	The type of the endpoint. Valid values: `source` and `dest`.
├─ dbEngine	string	The database engine of the endpoint.
├─ connectionInfo	string	The connection information of the endpoint.
├─ username	string	The username used to connect to the database.
├─ version	string	The version of the database.
├─ timezone	string	The time zone of the database.
├─ charset	string	The character set.
├─ nlsLengthSemantics	string	This attribute is available only when the data source is an Oracle database or an Oracle-compatible tenant of OceanBase Database.
├─ operatingSystem	string	The operating system. This parameter is unavailable for OceanBase Database.
├─ region	string	The region.
├─ ocpName	string	The name of the OCP instance.
├─ connExtraAttributes	object	The additional attributes of the connection. For more information about the nested response parameters, see Appendix.
├─ owner	string	The owner.
├─ host	string	The IP address.
├─ port	integer	The port number.
steps	Array	The steps.
├─ order	integer	The sequence number of the step.
├─ name	string	The name of the step.
├─ description	string	The description of the step.
├─ status	string	The status of the step.
├─ extraInfo	object	The additional information. For more information, see the Parameters in extraInfo section.
├─ startTime	string	The start time of the step, in the UTC+0 time zone.
├─ finishTime	string	The end time of the step, in the UTC+0 time zone.
├─ progress	integer	The progress of the step.
├─ stepInfo	object	The execution details of each step. For more information, see DescribeProjectSteps.
extraInfo	Object	The error messages and other additional information.
├─ runningStep	string	The step in the Running state.
├─ runningProgress	integer	The progress of the step in the Running state.
├─ monitoringIncr	boolean	Indicates whether incremental data monitoring is started.
├─ subtopics	string	The subtopics of the store associated with the task.
├─ reverseSubtopics	string	The subtopics of the reverse store associated with the task.
├─ logServiceStartCheckpoint	integer	The start timestamp.
├─ sourceStoreKeptHour	integer	The retention period of logs.
├─ syncDelay	integer	The delay of incremental synchronization.
├─ syncDelaySampleTimestamp	integer	The timestamp when the synchronization delay is collected, in seconds.
├─ accessObSource	boolean	Indicates whether to allow direct connection to the source OceanBase database.
├─ overwriteConfig	boolean	Indicates whether the overwriting of all new configurations is allowed.
├─ ignoreUnsupportDdl	boolean	Indicates whether unsupported DDL statements are ignored.
├─ subDbs	map	The table configurations of different downstream consumers. This parameter is used for combining configurations when the data synchronization task is reused.
├─ subConds	map	The configuration objects.
├─ subIds	map	The IDs of the configuration objects.
├─ maxConnectorCount	integer	The maximum number of connectors. The value `-1` indicates that the number of connectors is not limited.
├─ storeIncr	boolean	Indicates whether incremental data is saved.
├─ incrSyncTimestamp	integer	The current timestamp for incremental synchronization, in seconds.
alarmStats	object	The alert statistics.
├─ target	string	The object for which the alert is generated.
├─ alarming	boolean	Indicates whether alerts exist in the system.
├─ recentlyTriggerCount	integer	The total number of recent alert messages.
├─ ruleToRecentlyTriggerCount	map	The number of recent alert messages by rule.
├─ alarmContent	string	The content of recent alert messages.

Parameters in databases/databasesBlack

Parameter	Type	Description
mappedName	string	The name of the target database.
name	string	The name of the source database.
id	string	The ID of the database.
tables	array	The settings for the table objects to be transferred in the current database. When the value of the `mode` parameter is `WILDCARD`, this parameter indicates the settings for a specific blocklist or allowlist of tables and views.
├─ mappedName	string	The mapped-to table name.
├─ name	string	The name of the table.
├─ id	string	The ID of the table.
├─ whereClause	string	The `WHERE` clause in the standard SQL syntax that is used to configure row filtering.
├─ filterColumns	array	The filter columns.
├─ shardColumns	array	The sharding columns.
├─ columns		For more information, see the Parameters in columns section.
views	array	The settings for the view objects to be transferred in the current database.
├─ mappedName	string	The mapped-to view name.
├─ name	string	The name of the view.
├─ id	string	The ID of the view.
├─ whereClause	string	The `WHERE` clause in the standard SQL syntax that is used to configure row filtering.
├─ filterColumns	array	The filter columns.
├─ shardColumns	array	The sharding columns.
├─ columns	Array	The columns to be transferred. For more information, see the Parameters in columns section.
specificTables	array	The settings for the matched table objects. When the value of the `mode` parameter is `WILDCARD`, this parameter indicates the renaming and row filtering settings for the matched table objects.
├─ mappedName	string	The mapped-to table name.
├─ name	string	The name of the table.
├─ id	string	The ID of the table.
├─ whereClause	string	The `WHERE` clause in the standard SQL syntax that is used to configure row filtering.
├─ filterColumns	array	The filter columns.
├─ shardColumns	array	The sharding columns.
├─ columns	Array	The columns to be transferred. For more information, see the Parameters in columns section.
specificViews	array	The settings for the matched view objects. When the value of the `mode` parameter is `WILDCARD`, this parameter indicates the renaming settings for the matched view objects.
├─ mappedName	string	The mapped-to view name.
├─ name	string	The name of the view.
├─ id	string	The ID of the view.
├─ whereClause	string	The `WHERE` clause in the standard SQL syntax that is used to configure row filtering.
├─ filterColumns	array	The filter columns.
├─ shardColumns	array	The sharding columns.
├─ columns	Array	The columns to be transferred. For more information, see the Parameters in columns section.

Parameters in columns

Parameter	Type	Description
columnName	string	The name of the column.
mappedName	string	The mapped-to column name.
position	integer	The position of the column.
columnType	string	The type of the column.
recordFieldType	string	The extension of the column type.
rawColumnType	string	The original string of the column type recorded in the database.
columnKey	string	The type of the index.
nullable	boolean	Indicates whether null values are supported.
defaultValue	string	The default value.
dataLength	integer	The length of the column.
dataPrecision	integer	The precision of the column.
dataScale	integer	The number of decimal places.
encoding	string	The character encoding format for the column.
columnComment	string	The comment on the column.
isGenerateField	boolean	Indicates whether the column is a hidden column.

Parameters in extraInfo

Parameter	Type	Description
errorDetails	array	The error details.
errorCode	string	The error code.
errorMsg	string	The error message.
errorParam	Map	The error-related parameters.
failedTime	string	The time when the error occurred.

Examples

Sample request

{
    "searchKey":"j_oms41_ob",
    "type":"MIGRATION",
    "status":[
        "RUNNING"
    ],
    "labelIds":[
        "l_4w****wygjk"
    ],
    "sourceEndpointTypes":[
        "OB_MYSQL"
    ],
    "sinkEndpointTypes":[

    ],
    "visibleSubProject":false,
    "sortField":"gmtCreate",
    "order":"asc",
    "pageNumber":1,
    "pageSize":5
}

Sample response

{
    "success":"true",
    "errorDetail":null,
    "code":null,
    "message":null,
    "advice":null,
    "requestId":"e8dec2fe-40****",
    "pageNumber":"1",
    "pageSize":"5",
    "totalCount":"151",
    "cost":"54 ms",
    "data":[
        {
            "id":"np_4w****",
            "type":"MIGRATION",
            "name":"j_oms4****9_56_obo",
            "labels":[
                {
                    "id":"l_***",
                    "name":"boq***"
                }
            ],
            "owner":"oms_api_te***",
            "importance":"LOW",
            "status":"RUNNING",
            "gmtCreate":"2023-07-13T06:26:15",
            "gmtModified":"2023-07-13T07:01:27",
            "gmtStart":"2023-07-13T07:01:27",
            "gmtFinish":null,
            "destConnId":null,
            "isMerging":false,
            "isModifying":false,
            "isSubProject":false,
            "sourceEndpointType":"OB_MYSQL",
            "sinkEndpointType":"MYSQL",
            "transferMapping":{
                "mode":"SPECIFIC",
                "databases":[
                    {
                        "mappedName":"n12956****mysql",
                        "name":"striG***UTF115ng",
                        "id":null,
                        "tables":[
                            {
                                "mappedName":"CHAR****001_",
                                "name":"CHARA****PK_001",
                                "id":null,
                                "whereClause":null,
                                "filterColumns":null,
                                "shardColumns":null,
                                "columns":null
                            }
                        ],
                        "views":[
                            {
                                "mappedName":"HIGH_SAL****EASE_",
                                "name":"HIGH_SA****LEASE",
                                "id":null,
                                "whereClause":null,
                                "filterColumns":null,
                                "shardColumns":null,
                                "columns":null
                            }
                        ],
                        "specificTables":null,
                        "specificViews":null
                    }
                ]
            },
            "commonTransferConfig":null,
            "enableStructTransfer":true,
            "structTransferConfig":null,
            "enableFullTransfer":true,
            "enableFullVerify":true,
            "fullTransferConfig":null,
            "enableIncrTransfer":true,
            "enableIncrVerify":false,
            "enableReverseIncrTransfer":true,
            "incrTransferConfig":null,
            "sourceConnectInfo":{
                "id":"c_4w3aq****",
                "endpointName":"ration12****0obmysql4116892***",
                "endpointId":"e_4w3al***",
                "endpointSide":null,
                "dbEngine":"OB_MYSQL_PUBLIC",
                "connectionInfo":null,
                "username":"oms931mys**",
                "version":"5.7.27-log",
                "timezone":"UTC",
                "charset":"utf8mb4",
                "nlsLengthSemantics":null,
                "operatingSystem":"Linux",
                "region":"cn-beijing",
                "ocpName":"",
                "connExtraAttributes":null,
                "owner":"oms_api_te***",
                "host":"xxx.xxx.xxx.1",
                "port":"2883"
            },
            "sinkConnectInfo":{
                "id":"c_4w3akx***",
                "endpointName":"ation129***9apimysql5616****",
                "endpointId":"e_4w3aic***",
                "endpointSide":null,
                "dbEngine":"MYSQL_PUBLIC",
                "connectionInfo":null,
                "username":"oms**obmys**",
                "version":"2.2.77",
                "timezone":"+08:00",
                "charset":"utf8mb4",
                "nlsLengthSemantics":null,
                "operatingSystem":null,
                "region":"cn-anhui",
                "ocpName":"107",
                "connExtraAttributes":null,
                "owner":"oms_api_te***",
                "host":"xxx.xxx.xxx.2",
                "port":"2883"
            },
            "steps":[
                {
                    "order":"1",
                    "name":"TRANSFER_PRECHECK",
                    "description":"Precheck",
                    "status":"FINISHED",
                    "extraInfo":{
                        "errorDetails":null,
                        "errorCode":null,
                        "errorMsg":null,
                        "errorParam":null,
                        "failedTime":null
                    },
                    "startTime":null,
                    "finishTime":null,
                    "progress":100,
                    "stepInfo":{

                    }
                }
            ],
            "extraInfo":{
                "errorDetails":null,
                "errorCode":null,
                "errorMsg":null,
                "errorParam":null,
                "failedTime":null
            },
            "alarmStats":{
                "target":null,
                "alarming":false,
                "recentlyTriggerCount":null,
                "ruleToRecentlyTriggerCount":null,
                "alarmContent":null
            }
        }
    ]
}

Appendix

Kafka

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "username":null,
        "enableSSL":null,
        "enableSASL":null,
        "saslMechanism":null,
        "disableIdentificationAlgorithm":false
    }
}

Parameter	Type	Description
endpoint	string	The service endpoint.
username	string	The username. This parameter is required when the authentication method is `PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`.
enableSSL	boolean	Indicates whether SSL authentication is enabled. If you want to enable SSL authentication, you must upload an SSL certificate.
enableSASL	boolean	Indicates whether SASL authentication is enabled.
saslMechanism	KafkaSaslMechanisnEnum	The encryption method. Valid values: `PLAIN`, `SCRAM-SHA-256`, `SCRAM-SHA-512`, and `KERBEROS`.
disableIdentificationAlgorithm	boolean	Indicates whether hostname verification is enabled during SSL encryption.

DataHub

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "project":"detx***"
    }
}

Parameter	Type	Description
endpoint	string	The service endpoint.
project	string	The name of the task.

RocketMQ

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "endpointType":null,
        "version":null
    }
}

Parameter	Type	Description
endpoint	string	The service endpoint.
endpointType	CloudRocketMQEndpoint.Type	The type of the service endpoint.
version	CloudRocketMQInst.Version	The version of RocketMQ.

OceanBase Database

{
    "connExtraAttributes":{
        "cluster":"xgez***",
        "tenant":"gexr**",
        "isLogicSource":false,
        "useLogProxy":true,
        "drcUser":"tes***",
        "configUrl":"http://xxx.xxx.xxx.1:2883/service****",
        "logProxyIp":"xxx.xxx.xxx.2",
        "logProxyPort":2883,
        "noUserAuth":false
    }
}

Parameter	Type	Description
cluster	string	The name of the cluster.
tenant	string	The name of the tenant.
isLogicSource	boolean	Indicates whether the data source is a logical data source.
useLogProxy	boolean	Indicates whether the oblogproxy service is used.
drcUser	string	The configuration of drc_user.
configUrl	string	Indicates whether the oblogproxy service is used.
logProxyIp	string	The IP address of the oblogproxy service.
logProxyPort	integer	The port number of the oblogproxy service.
noUserAuth	boolean	When you synchronize data to DataWorks, OMS Community Edition automatically generates a data source. The username and password are required for connecting to OceanBase Database.

TiDB

{
    "connExtraAttributes":{
        "kafkaId":"e_edz****",
        "kafkaTopic":"test**",
        "kafkaName":"denz***"
    }
}

Parameter	Type	Description
kafkaId	string	The ID of the bound Kafka data source.
kafkaTopic	string	The name of the bound Kafka topic.
kafkaName	string	The name of the bound Kafka data source.

Enterprise Edition

Community Edition

ListProjects