ListProjects|V4.2.3|OceanBase Migration Service|OMS docs|Distributed Database

This topic describes the ListProjects API provided by OceanBase Migration Service (OMS) Community Edition.

Purpose

You can call this operation to query a list of data migration or synchronization projects.

Call description

Limitations

This operation is applicable to OMS Community Edition V4.1.1 and later versions.

Request path

POST /api/v2?Action=ListProjects

Request parameters

Parameter	Type	Description
searchKey	string	The name or ID of the data migration or synchronization project.
type	string	The project type. Valid values: `MIGRATION` and `SYNC`.
status	string	The status of the project.
labelIds	string	A collection of at most five label IDs.
sourceEndpointTypes	string	The type of the source data source. Valid values: MYSQL, OB_ORACLE, OB_MYSQL, ORACLE, TIDB, DB2_LUW, POSTGRESQL, IDB, and DBP_OP_ROUTE.
sinkEndpointTypes	string	The type of the destination data source. Valid values: MYSQL, OB_ORACLE, OB_MYSQL, ORACLE, DB2_LUW, KAFKA, DATAHUB, and ROCKETMQ.
visibleSubProject	boolean	Specifies whether to display sub-projects. Default value: `false`.
sortField	string	The sorting field. Valid values: `gmtCreate` and `gmtModified`. Default value: `gmtCreate`.
order	string	The sorting order. Valid values: `asc` and `descend`. Default value: `descend`.
pageNumber	integer	The number of the page to return. Default value: `1`.
pageSize	integer	The number of entries to return on each page. Default value: `10`. Maximum value: `150`.

Return result

Parameter	Type	Description
success	boolean	Indicates whether the call is successful.
errorDetail	object	The error details.
code	string	The error code.
message	string	The error description.
advice	string	The suggestions about how to resolve the error.
requestId	string	The request ID.
pageNumber	integer	The number of the page to return.
pageSize	integer	The number of entries to return on each page.
totalCount	integer	The total number of entries returned.
cost	string	The time spent in processing the request, in seconds.
data	string	The information about projects. For more information, see the "Parameters in data" section of this topic.

Parameters in data

Parameter	Type	Description
id	string	The project ID.
type	string	The project type. Valid values: `MIGRATION` and `SYNC`.
name	string	The project name, which must not contain spaces.
labels	string	A collection of at most five labels.
├─ id	string	The label ID.
├─ name	string	The name of the label.
owner	string	The owner of the project.
importance	string	The alert level of the project. Valid values: `HIGH`, `MEDIUM`, `LOW`, and `IGNORE`.
status	string	The status of the project.
gmtCreate	string	The time when the project was created, which is presented in the UTC+0 time zone.
gmtModified	string	The time when the project was modified, which is presented in the UTC+0 time zone.
gmtStart	string	The time when the project was started.
gmtFinish	string	The time when the project was finished.
destConnId	string	The connection ID of the project.
isMerging	boolean	Indicates whether the project is being merged.
isModifying	boolean	Indicates whether the project is being modified.
isSubProject	boolean	Indicates whether the project is a sub-project.
sourceEndpointType	string	The type of the source data source. Valid values: MYSQL, OB_ORACLE, OB_MYSQL, ORACLE, TIDB, DB2_LUW, POSTGRESQL, IDB, and DBP_OP_ROUTE.
sinkEndpointType	string	The type of the destination data source. Valid values: MYSQL, OB_ORACLE, OB_MYSQL, ORACLE, DB2_LUW, KAFKA, DATAHUB, and ROCKETMQ.
transferMapping	object	The migration objects and their mapping relationships.
├─ mode	string	The mapping mode of migration or synchronization objects. Valid values: `SPECIFIC` (Specify Objects mode) `WILDCARD` (Match Rules mode).
├─ databases	array	A list of database objects. For more information, see the "Parameters in databases/databasesBlack" section of this topic.
├─ databasesBlack	array	The database object blocklist in Match Rules mode. For more information, see the "Parameters in databases/databasesBlack" section of this topic.
├─ tableAndViewWhiteList	array	The table/view allowlist in Match Rules mode.
├─ tableAndViewBlackList	array	The table/view blocklist in Match Rules mode.
commonTransferConfig	object	The general migration or synchronization configurations.
├─ tableCategory	string	The type of tables to be migrated. Valid values: ALL (all tables), WITHUNIQUEROWID (tables with unique ROWIDs), and WITHOUTUNIQUEROWID (tables without unique ROWIDs). Default value: ALL.
├─ activeActive	boolean	Indicates whether the scenario is an active-active disaster recovery scenario.
├─ mqSerializerType	string	The JSON serialization type when data is delivered to MQ. Valid values: DEFAULT, CANAL, DATAWORKS_V2, SHAREPLEX, DEFAULT_WITH_SCHEMA, DEBEZIUM, DEBEZIUMFLATTEN, and DEBEZIUMSMT.
├─ mqPartitionMode	string	The partition routing mode when data is delivered to MQ. Valid values: ONE, HASH, and TABLE. ONE indicates that data is delivered to specified partitions. HASH indicates that data is delivered to HASH partitions based on values of the primary key or sharding columns. TABLE indicates that data is delivered to HASH partitions based on database and table names.
├─ mqPartition	integer	The position of the destination partition in the current topic when the value of `partitionMode` is set to `ONE`.
├─ datahubTopicType	string	The type of the topic when data is delivered to a DataHub instance. Valid values: `TUPLE` and `BLOB`.
├─ rocketMqProducerGroup	string	The name of the producer group. This parameter is valid only when the destination data source is RocketMQ. Default value: OMS.
├─ rocketMqMsgTags	string	The message tag. This parameter is valid only when the destination data source is RocketMQ. The default value is default->default_tag, which indicates that when the value is `default`, a `_tag` suffix is automatically added to the value.
├─ rocketMqEnableMsgTrace	boolean	Indicates whether message tracing is enabled. This parameter is valid only when the destination data source is RocketMQ.
├─ rocketMqSendMsgTimeout	long	The message delivery timeout period. This parameter is valid only when the destination data source is RocketMQ.
├─ dataWorksBusinessName	string	The name of the business system.
enableStructTransfer	boolean	Indicates whether schema migration is allowed.
structTransferConfig	object	The schema migration configurations.
├─ byteCharConvertStrategy	string	The conversion strategy between bytes and characters. Valid values: `IGNORE_BYTE_IF_BYTE_USED`, `FORCE_CHAR_IF_BYTE_USED`, `EXPAND_LEN_IF_BYTE_USED`, and `DO_NOTHING_IF_BYTE_USED`. Default value: `DO_NOTHING_IF_BYTE_USED`.
├─ deferIndexCreation	boolean	Indicates whether post-indexing is allowed. Default value: `false`.
enableFullTransfer	boolean	Indicates whether full migration is enabled.
enableFullVerify	boolean	Indicates whether full verification is enabled.
fullTransferConfig	object	The full migration configurations.
├─ nonePkUkTruncateDstTable	boolean	Specifies whether to clear destination tables during the full migration of tables without unique indexes.
├─ allowDestTableNotEmpty	string	Indicates whether a nonempty destination table is allowed. This parameter is required when you migrate data from multiple source tables to a single destination table.
├─ fullTransferSpeedMode	string	The concurrency for full migration. Valid values: STEADY, NORMAL, and FAST.
├─ fullVerifySpeedMode	string	The concurrency for full verification. Valid values: STEADY, NORMAL, and FAST.
enableIncrTransfer	boolean	Indicates whether incremental synchronization is enabled.
enableIncrVerify	boolean	Indicates whether incremental verification is enabled.
enableReverseIncrTransfer	boolean	Indicates whether reverse incremental synchronization is enabled.
incrTransferConfig	object	The incremental synchronization configurations.
├─ startTimestamp	string	The start timestamp for incremental synchronization. This parameter is supported only when incremental synchronization is enabled but full migration is disabled.
├─ recordTypeWhiteList	string	The type of incremental data synchronized during incremental synchronization. Valid values: HEARTBEAT, INSERT, UPDATE, DELETE, BEGIN, COMMIT, ROLLBACK, DDL, and ROW.
├─ storeLogKeptHour	integer	The retention period of logs when incremental synchronization is enabled and the incremental log pull component is Store. Unit: hours. Default value: 7 days. The maximum retention period is 365 days.
├─ enableSequencingWithinTxn	boolean	Indicates whether intra-transaction sequencing is enabled when incremental synchronization is enabled and the incremental log pull component is Store.
├─ incrSyncConcurrency	integer	The number of threads for incremental synchronization. Default value: `64`.
├─ enableIncrSyncStatistics	boolean	Indicates whether statistics on incremental records (DML and DDL operations) are collected.
sourceConnectInfo	object	The connection information of the source node.
├─ id	string	The connection ID.
├─ endpointName	string	The name of the endpoint.
├─ endpointId	string	The endpoint ID.
├─ endpointSide	string	The type of the endpoint. Valid values: `source` and `dest`.
├─ dbEngine	string	The database engine of the endpoint.
├─ connectionInfo	string	The connection information of the endpoint.
├─ username	string	The username used to connect to the database.
├─ version	string	The version of the database.
├─ timezone	string	The time zone of the database.
├─ charset	string	The character encoding format.
├─ nlsLengthSemantics	string	The nlsLengthSemantics attribute. This parameter is valid only when the data source is an Oracle database or an Oracle tenant of OceanBase Database.
├─ operatingSystem	string	The operating system. This parameter is invalid in OceanBase Database.
├─ region	string	The region.
├─ ocpName	string	The name of the associated OceanBase Cloud Platform (OCP) instance.
├─ connExtraAttributes	string	Extra connection attributes. For more information about the fields returned, see the Appendix section.
├─ owner	string	The owner.
├─ host	string	The IP address.
├─ port	integer	The port.
sinkConnectInfo	object	The connection information of the destination node.
├─ id	string	The connection ID.
├─ endpointName	string	The name of the endpoint.
├─ endpointId	string	The endpoint ID.
├─ endpointSide	string	The type of the endpoint. Valid values: `source` and `dest`.
├─ dbEngine	string	The database engine of the endpoint.
├─ connectionInfo	string	The connection information of the endpoint.
├─ username	string	The username used to connect to the database.
├─ version	string	The version of the database.
├─ timezone	string	The time zone of the database.
├─ charset	string	The character encoding format.
├─ nlsLengthSemantics	string	The nlsLengthSemantics attribute. This parameter is valid only when the data source is an Oracle database or an Oracle tenant of OceanBase Database.
├─ operatingSystem	string	The operating system. This parameter is invalid in OceanBase Database.
├─ region	string	The region.
├─ ocpName	string	The name of the associated OCP instance.
├─ connExtraAttributes	string	Extra connection attributes. For more information about the fields returned, see the Appendix section.
├─ owner	string	The owner.
├─ host	string	The IP address.
├─ port	integer	The port.
steps	array	The step list.
├─ order	integer	The execution sequence of the step.
├─ name	string	The name of the step.
├─ description	string	The description of the step.
├─ status	string	The execution status of the step.
├─ extraInfo	object	The additional information. For more information, see the "Parameters in extraInfo" section of this topic.
├─ startTime	string	The start time presented in the UTC+0 time zone.
├─ finishTime	string	The end time presented in the UTC+0 time zone.
├─ progress	integer	The execution progress of the step.
├─ stepInfo	object	The execution details of each step.
extraInfo	object	The extra information, such as error information.
├─ runningStep	string	The running step.
├─ runningProgress	integer	The progress of the running step.
├─ monitoringIncr	boolean	Indicates whether incremental synchronization monitoring is started.
├─ subtopics	string	The subtopics of the Store associated with the project.
├─ reverseSubtopics	string	The subtopics of the reverse Store associated with the project.
├─ logServiceStartCheckpoint	integer	The start checkpoint.
├─ sourceStoreKeptHour	integer	The time when the log was recorded.
├─ syncDelay	integer	The incremental synchronization latency.
├─ syncDelaySampleTimestamp	integer	The timestamp when the synchronization latency is collected, which is accurate to seconds.
├─ accessObSource	boolean	Indicates whether a direct connection to OceanBase Database is supported.
├─ overwriteConfig	boolean	Indicates whether all new configurations are overwritten.
├─ ignoreUnsupportDdl	boolean	Indicates whether unsupported DDL operations are ignored.
├─ subDbs	map	The table configurations for different downstream consumers, which will be merged with those of a reused data synchronization project.
├─ subConds	map	The list of configuration objects.
├─ subIds	map	The list of configuration object IDs.
├─ maxConnectorCount	integer	The maximum number of connectors. The value -1 indicates the number of connectors is not limited.
├─ storeIncr	boolean	Indicates whether incremental data is saved.
├─ incrSyncTimestamp	integer	The current timestamp of incremental synchronization. The value is a second-level timestamp.
alarmStats	object	The alert statistics.
├─ target	string	The target of the alert.
├─ alarming	boolean	Indicates whether alerts exist in the system.
├─ recentlyTriggerCount	integer	The total number of recent alert messages.
├─ ruleToRecentlyTriggerCount	map	The number of recent alert messages triggered by rules.
├─ alarmContent	string	The content of recent alert messages.

Parameters in databases/databasesBlack

Parameter	Type	Description
mappedName	string	The name of the destination database.
name	string	The name of the source database.
id	string	The database ID.
tables	array	The list of tables in the current database for migration. When the value of `mode` is set to `WILDCARD`, the value of this parameter is a blocklist or allowlist of tables and views.
├─ mappedName	string	The name of the destination table.
├─ name	string	The name of the source table.
├─ id	string	The table ID.
├─ whereClause	string	A standard SQL `WHERE` clause used to filter data by row.
├─ filterColumns	array	A collection of filter columns.
├─ shardColumns	array	A collection of sharding columns.
├─ columns		For more information, see the "Parameters in columns" section of this topic.
views	array	The list of views in the current database for migration.
├─ mappedName	string	The name of the destination view.
├─ name	string	The name of the source view.
├─ id	string	The view ID.
├─ whereClause	string	A standard SQL `WHERE` clause used to filter data by row.
├─ filterColumns	array	A collection of filter columns.
├─ shardColumns	array	A collection of sharding columns.
├─ columns	array	The list of columns for migration. For more information, see the "Parameters in columns" section of this topic.
specificTables	array	The list of table objects. When the value of `mode` is set to `WILDCARD`, the value of this parameter is the renaming or row-based filtering configurations of specific table objects.
├─ mappedName	string	The name of the destination table.
├─ name	string	The name of the source table.
├─ id	string	The table ID.
├─ whereClause	string	A standard SQL `WHERE` clause used to filter data by row.
├─ filterColumns	array	A collection of filter columns.
├─ shardColumns	array	A collection of sharding columns.
├─ columns	array	The list of columns for migration. For more information, see the "Parameters in columns" section of this topic.
specificViews	array	The list of specific view objects. When the value of `mode` is set to `WILDCARD`, the value of this parameter is the renaming configurations of specific view objects.
├─ mappedName	string	The name of the destination view.
├─ name	string	The name of the source view.
├─ id	string	The view ID.
├─ whereClause	string	A standard SQL `WHERE` clause used to filter data by row.
├─ filterColumns	array	A collection of filter columns.
├─ shardColumns	array	A collection of sharding columns.
├─ columns	array	The list of columns for migration. For more information, see the "Parameters in columns" section of this topic.

Parameters in columns

Parameter	Type	Description
type	string	The type. Valid values: DATABASE and TABLE.
columnName	string	The column name.
mappedName	string	The name of the destination column.
position	integer	The position of the parameter.
columnType	string	The type of the column.
recordFieldType	string	The derivation of the column type.
rawColumnType	string	The original string of the column type recorded in the database.
columnKey	string	The type of the index.
nullable	boolean	Indicates whether null values are supported.
defaultValue	string	The default value of the column.
dataLength	integer	The length of the column.
dataPrecision	integer	The precision of the column.
dataScale	integer	The number of decimal places.
encoding	string	The character encoding format of the column.
columnComment	string	The comment of the column.
isGenerateField	boolean	Specifies whether the column is a hidden column.

Parameters in extraInfo

Parameter	Type	Description
errorDetails	array	The error details.
errorCode	string	The error code.
errorMsg	string	The error message.
errorParam	map	The error-related parameter.
failedTime	string	The time when the error occurred.

Examples

Sample request

{
    "searchKey":"j_oms41_ob",
    "type":"MIGRATION",
    "status":[
        "RUNNING"
    ],
    "labelIds":[
        "l_4w****wygjk"
    ],
    "sourceEndpointTypes":"OB_MYSQL",
    "sinkEndpointTypes":"OB_MYSQL",
    "visibleSubProject":false,
    "sortField":"gmtCreate",
    "order":1,
    "pageNumber":1,
    "pageSize":5
}

Sample response

{
    "success":"true",
    "errorDetail":null,
    "code":null,
    "message":null,
    "advice":null,
    "requestId":"e8dec2fe-40****",
    "pageNumber":"1",
    "pageSize":"5",
    "totalCount":"151",
    "cost":"54 ms",
    "data":[
        {
            "id":"np_4w****",
            "type":"MIGRATION",
            "name":"j_oms4****9_56_obo",
            "labels":[
                {
                    "id":"l_***",
                    "name":"boq***"
                }
            ],
            "owner":"oms_api_te***",
            "importance":"LOW",
            "status":"RUNNING",
            "gmtCreate":"2023-07-13T06:26:15",
            "gmtModified":"2023-07-13T07:01:27",
            "gmtStart":"2023-07-13T07:01:27",
            "gmtFinish":null,
            "destConnId":null,
            "isMerging":false,
            "isModifying":false,
            "isSubProject":false,
            "sourceEndpointType":"OB_MYSQL",
            "sinkEndpointType":"MYSQL",
            "transferMapping":{
                "mode":"SPECIFIC",
                "databases":[
                    {
                        "clusterName":null,
                        "tenantName":null,
                        "mappedName":"n12956****mysql",
                        "name":"striG***UTF115ng",
                        "id":"3aq****kw",
                        "tables":[
                            {
                                "mappedName":"CHAR****001_",
                                "name":"CHARA****PK_001",
                                "id":null,
                                "whereClause":null,
                                "filterColumns":null,
                                "shardColumns":null,
                                "columns":null
                            }
                        ],
                        "views":[
                            {
                                "mappedName":"HIGH_SAL****EASE_",
                                "name":"HIGH_SA****LEASE",
                                "id":null,
                                "whereClause":null,
                                "filterColumns":null,
                                "shardColumns":null,
                                "columns":null
                            }
                        ],
                        "specificTables":null,
                        "specificViews":null
                    }
                ]
            },
            "commonTransferConfig":null,
            "enableStructTransfer":true,
            "structTransferConfig":null,
            "enableFullTransfer":true,
            "enableFullVerify":true,
            "fullTransferConfig":null,
            "enableIncrTransfer":true,
            "enableIncrVerify":false,
            "enableReverseIncrTransfer":true,
            "incrTransferConfig":null,
            "sourceConnectInfo":{
                "id":"c_4w3aq****",
                "endpointName":"ration12****0obmysql4116892***",
                "endpointId":"e_4w3al***",
                "endpointSide":null,
                "dbEngine":"OB_MYSQL_PUBLIC",
                "connectionInfo":null,
                "username":"oms931mys**",
                "version":"5.7.27-log",
                "timezone":"UTC",
                "charset":"utf8mb4",
                "nlsLengthSemantics":null,
                "operatingSystem":"Linux",
                "region":"cn-beijing",
                "ocpName":"",
                "connExtraAttributes":null,
                "owner":"oms_api_te***",
                "host":"xxx.xxx.xxx.1",
                "port":"2883"
            },
            "sinkConnectInfo":{
                "id":"c_4w3akx***",
                "endpointName":"ation129***9apimysql5616****",
                "endpointId":"e_4w3aic***",
                "endpointSide":null,
                "dbEngine":"MYSQL_PUBLIC",
                "connectionInfo":null,
                "username":"oms**obmys**",
                "version":"2.2.77",
                "timezone":"+08:00",
                "charset":"utf8mb4",
                "nlsLengthSemantics":null,
                "operatingSystem":null,
                "region":"cn-anhui",
                "ocpName":"107",
                "connExtraAttributes":null,
                "owner":"oms_api_te***",
                "host":"xxx.xxx.xxx.2",
                "port":"2883"
            },
            "steps":[
                {
                    "order":"1",
                    "name":"TRANSFER_PRECHECK",
                    "description":"Precheck",
                    "status":"FINISHED",
                    "extraInfo":{
                        "errorDetails":null,
                        "errorCode":null,
                        "errorMsg":null,
                        "errorParam":null,
                        "failedTime":null
                    },
                    "startTime":null,
                    "finishTime":null,
                    "progress":100,
                    "stepInfo":{

                    }
                }
            ],
            "extraInfo":{
                "errorDetails":null,
                "errorCode":null,
                "errorMsg":null,
                "errorParam":null,
                "failedTime":null
            },
            "alarmStats":{
                "target":null,
                "alarming":false,
                "recentlyTriggerCount":null,
                "ruleToRecentlyTriggerCount":null,
                "alarmContent":null
            }
        }
    ]
}

Appendix

Kafka

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "username":null,
        "enableSSL":null,
        "enableSASL":null,
        "saslMechanism":null,
        "disableIdentificationAlgorithm":false
    }
}

Parameter	Type	Description
endpoint	string	The service endpoint.
username	string	The username. This parameter is required when you select the PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512 authentication method.
enableSSL	boolean	Indicates whether Secure Sockets Layer (SSL) authentication is enabled. If SSL authentication is enabled, you must upload an SSL certificate.
enableSASL	boolean	Indicates whether Simple Authentication and Security Layer (SASL) authentication is enabled.
saslMechanism	KafkaSaslMechanisnEnum	The encryption method. Valid values: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, and KERBEROS.
disableIdentificationAlgorithm	boolean	Indicates whether hostname verification is enabled during SSL encryption.

DataHub

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "project":"detx***"
    }
}

Parameter	Type	Description
endpoint	string	The service endpoint.
project	string	The name of the project.

RocketMQ

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "endpointType":null,
        "version":null
    }
}

Parameter	Type	Description
endpoint	string	The service endpoint.
endpointType	CloudRocketMQEndpoint.Type	The type of the endpoint.
version	CloudRocketMQInst.Version	The RocketMQ version.

OceanBase

{
    "connExtraAttributes":{
        "cluster":"xgez***",
        "tenant":"gexr**",
        "isLogicSource":false,
        "useLogProxy":true,
        "drcUser":"tes***",
        "configUrl":"http://xxx.xxx.xxx.1:2883/service****",
        "logProxyIp":"xxx.xxx.xxx.2",
        "logProxyPort":2883,
        "noUserAuth":false
    }
}

Parameter	Type	Description
cluster	string	The name of the cluster.
tenant	string	The name of the tenant.
isLogicSource	boolean	Indicates whether the data source is a logical data source.
useLogProxy	boolean	Indicates whether LogProxy is used.
drcUser	string	The drc_user user.
configUrl	string	The actual URL of the server that runs OceanBase Database.
logProxyIp	string	The IP address of the oblogproxy service.
logProxyPort	integer	The port of the oblogproxy service.
noUserAuth	boolean	For an API for synchronizing data from DataWorks, the data source is automatically generated by OMS. If you do not have a user account and password, you cannot connect to OceanBase Database.

TiDB

{
    "connExtraAttributes":{
        "kafkaId":"e_edz****",
        "kafkaTopic":"test**",
        "kafkaName":"denz***"
    }
}

Parameter	Type	Description
kafkaId	string	The ID of the bound Kafka data source.
kafkaTopic	string	The name of the bound Kafka topic.
kafkaName	string	The name of the bound Kafka data source.

Customer Stories

Documentation

Enterprise Edition

Community Edition

ListProjects