ListProjects

This topic describes the ListProjects operation provided by OceanBase Migration Service (OMS).

Purpose

You can call this operation to query data migration or synchronization projects by page.

Call description

Limitations

This operation is supported in OMS V4.1.0 and later.

Request path

POST /api/v2?Action=ListProjects

Request parameters

Response parameters

• Parameters in data

  Parameter	Type	Description
  id	string	The project ID.
  type	string	The type of the project. Valid values: MIGRATION and SYNC.
  name	string	The name of the project, which cannot contain spaces.
  labels	object []	The tags.
  ├─ id	string	The ID of the tag.
  ├─ name	string	The name of the tag.
  owner	string	The owner of the project.
  importance	string	The alert level of the project. Valid values: HIGH, MEDIUM, LOW, and IGNORE.
  status	string	The status of the project.
  gmtCreate	string	The time when the project was created, in the UTC+0 time zone.
  gmtModified	string	The time when the project was updated, in the UTC+0 time zone.
  gmtStart	string	The start time of the project.
  gmtFinish	string	The end time of the project.
  destConnId	string	The connection ID of the project.
  isMerging	boolean	Indicates whether the project is in the progress of a major compaction.
  isModifying	boolean	Indicates whether the project is being modified.
  isSubProject	boolean	Indicates whether the project is a subproject.
  sourceEndpointType	string	The type of the source data source. Valid values: MYSQL, OB_ORACLE, OB_MYSQL, ORACLE, TIDB, DB2_LUW, POSTGRESQL, IDB, and DBP_OP_ROUTE.
  sinkEndpointType	string	The type of the destination data source. Valid values: MYSQL, OB_ORACLE, OB_MYSQL, ORACLE, DB2_LUW, KAFKA, DATAHUB, and ROCKETMQ.
  transferMapping	object	This parameter is deprecated. We recommend that you call the DescribeProject operation.
  ├─ mode	string	The mapping mode for the objects to be transferred. Valid values are SPECIFIC and WILDCARD, which indicate specifying specific objects and matching objects by using a rule, respectively.
  ├─ databases	array	The list of database objects. For more information, see Parameters in databases/databasesBlack.
  ├─ databasesBlack	array	The blocklist of database objects used for the WILDCARD mode. For more information, see Parameters in databases/databasesBlack.
  ├─ tableAndViewWhiteList	array	The wildcard-based allowlist of tables and views used for the WILDCARD mode.
  ├─ tableAndViewBlackList	array	The wildcard-based blocklist of tables and views used for the WILDCARD mode.
  commonTransferConfig	object	This parameter is deprecated. We recommend that you call the DescribeProject operation.
  ├─ tableCategory	string	The type of tables to be transferred. Valid values: ALL (all tables), WITH_UNIQUE_ROW_ID (tables with a unique row ID), and WITHOUT_UNIQUE_ROW_ID (tables without a unique row ID). Default value: ALL.
  ├─ activeActive	boolean	Indicates whether active-active disaster recovery is enabled.
  ├─ mqSerializerType	string	The JSON data serialization type for delivering data to message queues. Valid values: DEFAULT, CANAL, DATAWORKS_V2, SHAREPLEX, DEFAULT_WITH_SCHEMA, DEBEZIUM, DEBEZIUMFLATTEN, and DEBEZIUMSMT.
  ├─ mqPartitionMode	string	The partition routing mode for delivering data to message queues. Valid values: ONE, HASH, and TABLE. ONE specifies to deliver data to a specified partition. HASH specifies to deliver data to HASH partitions based on the primary key column or sharding columns. TABLE specifies to deliver data to HASH partitions based on database and table names.
  ├─ mqPartition	integer	When the value of the partitionMode parameter is ONE, this parameter indicates the location of the partition to which data is delivered in the current topic.
  ├─ datahubTopicType	string	The type of the topic for delivering data to DataHub. Valid values: TUPLE and BLOB.
  ├─ rocketMqProducerGroup	string	The name of the producer group. This parameter takes effect when the type of destination data source is RocketMQ. The default value is OMS.
  ├─ rocketMqMsgTags	string	The tag of the message. This parameter takes effect when the type of destination data source is RocketMQ. For example, when the value is default, the _tag suffix is added to the value by default.
  ├─ rocketMqEnableMsgTrace	boolean	Indicates whether message tracing is enabled. This parameter takes effect when the type of destination data source is RocketMQ.
  ├─ rocketMqSendMsgTimeout	long	The message delivery timeout period. This parameter takes effect when the type of destination data source is RocketMQ.
  ├─ dataWorksBusinessName	string	The ID of the business system.
  enableStructTransfer	boolean	Indicates whether schema transfer is allowed.
  structTransferConfig	object	This parameter is deprecated. We recommend that you call the DescribeProject operation.
  ├─ byteCharConvertStrategy	string	The BYTE/CHAR conversion scheme. Valid values: IGNORE_BYTE_IF_BYTE_USED, FORCE_CHAR_IF_BYTE_USED, EXPAND_LEN_IF_BYTE_USED, and DO_NOTHING_IF_BYTE_USED. The default value is DO_NOTHING_IF_BYTE_USED.
  ├─ deferIndexCreation	boolean	Indicates whether post-indexing is allowed. The default value is false.
  enableFullTransfer	boolean	Indicates whether full transfer is enabled.
  enableFullVerify	boolean	Indicates whether full verification is enabled.
  fullTransferConfig	object	This parameter is deprecated. We recommend that you call the DescribeProject operation.
  ├─ nonePkUkTruncateDstTable	boolean	Indicates whether the destination table is cleared during full migration from a table without a unique index.
  ├─ allowDestTableNotEmpty	boolean	The processing strategy to apply when the destination table has records. Valid values: True and False. The value True specifies to ignore the existing data in the destination table. If the destination is a MySQL database or a MySQL tenant of OceanBase Database, OMS overwrites the existing data in the destination table. Notice If you select Ignore, data is pulled in IN mode for full verification. In this case, the scenario where the destination contains more data than the source cannot be verified, and the verification efficiency will be decreased. The value False specifies to stop migration. If the destination table has records, an error is returned during full migration. In this case, clear the existing data in the destination table and try again. Notice After an error is returned, if you click Resume in the dialog box, OMS ignores this error and continues to migrate data. Proceed with caution.
  ├─ fullTransferSpeedMode	string	The concurrency for full migration. Valid values: STEADY, NORMAL, and FAST.
  ├─ fullVerifySpeedMode	string	The concurrency for full verification. Valid values: STEADY, NORMAL, and FAST.
  enableIncrTransfer	boolean	Indicates whether incremental synchronization is enabled.
  enableIncrVerify	boolean	Indicates whether incremental verification is enabled.
  enableReverseIncrTransfer	boolean	Indicates whether reverse incremental synchronization is enabled.
  incrTransferConfig	object	This parameter is deprecated. We recommend that you call the DescribeProject operation.
  ├─ startTimestamp	long	The start timestamp for incremental synchronization. This parameter is available only when incremental synchronization is enabled and full transfer is disabled.
  ├─ recordTypeWhiteList	array	The data types of incremental data synchronized in incremental synchronization. Supported data types include HEARTBEAT, INSERT, UPDATE, DELETE, BEGIN, COMMIT, ROLLBACK, DDL, and ROW.
  ├─ storeLogKeptHour	integer	The retention period in hours of logs when incremental synchronization is enabled and the incremental log pull component is Store. The default retention period is seven days. The maximum retention period is 365 days.
  ├─ enableSequencingWithinTxn	boolean	Indicates whether intra-transaction sequencing is enabled when incremental synchronization is enabled and the incremental log pull component is Store.
  ├─ incrSyncConcurrency	integer	The number of threads for incremental synchronization. The default value is 64.
  ├─ enableIncrSyncStatistics	boolean	Indicates whether statistics collection is enabled for incremental DML and DDL operation records.
  sourceConnectInfo	object	The connection information of the source data source.
  ├─ id	string	The ID of the connection.
  ├─ endpointName	string	The name of the endpoint.
  ├─ endpointId	string	The ID of the endpoint.
  ├─ endpointSide	string	The type of the endpoint. Valid values: source and dest.
  ├─ dbEngine	string	The database engine of the endpoint.
  ├─ connectionInfo	string	The connection information of the endpoint.
  ├─ username	string	The username used to connect to the database.
  ├─ version	string	The version of the database.
  ├─ timezone	string	The time zone of the database.
  ├─ charset	string	The character encoding format.
  ├─ nlsLengthSemantics	string	This attribute is available only when the data source is an Oracle database or an Oracle tenant of OceanBase Database.
  ├─ operatingSystem	string	The operating system. This parameter is unavailable for OceanBase Database.
  ├─ region	string	The region.
  ├─ ocpName	string	The name of the OceanBase Cloud Platform (OCP) instance.
  ├─ connExtraAttributes	Object	The additional attributes of the connection. For more information about the nested response parameters, see Appendix.
  ├─ owner	string	The owner.
  ├─ host	string	The IP address.
  ├─ port	integer	The port number.
  sinkConnectInfo	object	The connection information of the destination data source.
  ├─ id	string	The ID of the connection.
  ├─ endpointName	string	The name of the endpoint.
  ├─ endpointId	string	The ID of the endpoint.
  ├─ endpointSide	string	The type of the endpoint. Valid values: source and dest.
  ├─ dbEngine	string	The database engine of the endpoint.
  ├─ connectionInfo	string	The connection information of the endpoint.
  ├─ username	string	The username used to connect to the database.
  ├─ version	string	The version of the database.
  ├─ timezone	string	The time zone of the database.
  ├─ charset	string	The character encoding format.
  ├─ nlsLengthSemantics	string	This attribute is available only when the data source is an Oracle database or an Oracle tenant of OceanBase Database.
  ├─ operatingSystem	string	The operating system. This parameter is unavailable for OceanBase Database.
  ├─ region	string	The region.
  ├─ ocpName	string	The name of the OCP instance.
  ├─ connExtraAttributes	Object	The additional attributes of the connection. For more information about the nested response parameters, see Appendix.
  ├─ owner	string	The owner.
  ├─ host	string	The IP address.
  ├─ port	integer	The port number.
  steps	array	The steps.
  ├─ order	integer	The sequence number of the step.
  ├─ name	string	The name of the step.
  ├─ description	string	The description of the step.
  ├─ status	string	The status of the step.
  ├─ extraInfo	object	The additional information. For more information, see Parameters in extraInfo.
  ├─ startTime	string	The start time of the step, in the UTC+0 time zone.
  ├─ finishTime	string	The end time of the step, in the UTC+0 time zone.
  ├─ progress	integer	The progress of the step.
  ├─ stepInfo	object	For more information, see DescribeProjectSteps.
  extraInfo	object	The error messages and other additional information.
  ├─ runningStep	string	The step in the Running state.
  ├─ runningProgress	integer	The progress of the step in the Running state.
  ├─ monitoringIncr	boolean	Indicates whether incremental data monitoring is started.
  ├─ subtopics	string	The subtopics of the store associated with the project.
  ├─ reverseSubtopics	string	The subtopics of the reverse store associated with the project.
  ├─ logServiceStartCheckpoint	integer	The start timestamp.
  ├─ sourceStoreKeptHour	integer	The retention period of logs.
  ├─ syncDelay	integer	The delay of incremental synchronization.
  ├─ syncDelaySampleTimestamp	integer	The timestamp when the synchronization delay is collected, in seconds.
  ├─ accessObSource	boolean	Indicates whether to allow direct connection to the source OceanBase database.
  ├─ overwriteConfig	boolean	Indicates whether the overwriting of all new configurations is allowed.
  ├─ ignoreUnsupportDdl	boolean	Specifies whether to ignore unsupported DDL statements.
  ├─ subDbs	map	The table configurations of different downstream consumers. This parameter combines configurations when a synchronization project is reused.
  ├─ subConds	map	The configuration objects.
  ├─ subIds	map	The IDs of the configuration objects.
  ├─ maxConnectorCount	integer	The maximum number of connectors. The value -1 indicates that the number of connectors is not limited.
  ├─ storeIncr	boolean	Indicates whether incremental data is saved.
  ├─ incrSyncTimestamp	integer	The current timestamp for incremental synchronization, in seconds.
  alarmStats	object	The alert statistics.
  ├─ target	string	The target of the alert.
  ├─ alarming	boolean	Indicates whether alerts exist in the system.
  ├─ recentlyTriggerCount	integer	The total number of recent alert messages.
  ├─ ruleToRecentlyTriggerCount	map	The number of recent alert messages by rule.
  ├─ alarmContent	string	The content of recent alert messages.

• Parameters in databases/databasesBlack

  Parameter	Type	Description
  mappedName	string	The name of the destination database.
  name	string	The name of the source database.
  id	string	The ID of the database.
  tables	array	The settings for the table objects to be transferred in the current database. When the value of the mode parameter is WILDCARD, this parameter indicates the settings for a specific blocklist or allowlist of tables and views.
  ├─ mappedName	string	The mapped table name.
  ├─ name	string	The name of the table.
  ├─ id	string	The ID of the table.
  ├─ whereClause	string	The WHERE clause in the standard SQL syntax that is used to configure row filtering.
  ├─ filterColumns	array	The filter columns.
  ├─ shardColumns	array	The sharding columns.
  ├─ columns		For more information, see Parameters in columns.
  views	array	The settings for the view objects to be transferred in the current database.
  ├─ mappedName	string	The mapped view name.
  ├─ name	string	The name of the view.
  ├─ id	string	The ID of the view.
  ├─ whereClause	string	The WHERE clause in the standard SQL syntax that is used to configure row filtering.
  ├─ filterColumns	array	The filter columns.
  ├─ shardColumns	array	The sharding columns.
  ├─ columns	array	The columns to be transferred. For more information, see Parameters in columns.
  specificTables	array	The settings for the matched table objects. When the value of the mode parameter is WILDCARD, this parameter indicates the renaming and row filtering settings for the matched table objects.
  ├─ mappedName	string	The mapped table name.
  ├─ name	string	The name of the table.
  ├─ id	string	The ID of the table.
  ├─ whereClause	string	The WHERE clause in the standard SQL syntax that is used to configure row filtering.
  ├─ filterColumns	array	The filter columns.
  ├─ shardColumns	array	The sharding columns.
  ├─ columns	array	The columns to be transferred. For more information, see Parameters in columns.
  specificViews	array	The settings for the matched view objects. When the value of the mode parameter is WILDCARD, this parameter indicates the renaming settings for the matched view objects.
  ├─ mappedName	string	The mapped view name.
  ├─ name	string	The name of the view.
  ├─ id	string	The ID of the view.
  ├─ whereClause	string	The WHERE clause in the standard SQL syntax that is used to configure row filtering.
  ├─ filterColumns	array	The filter columns.
  ├─ shardColumns	array	The sharding columns.
  ├─ columns	array	The columns to be transferred. For more information, see Parameters in columns.

• Parameters in columns

  Parameter	Type	Description
  columnName	string	The name of the column.
  mappedName	string	The mapped column name.
  position	integer	The position of the parameter.
  columnType	string	The type of the column.
  recordFieldType	string	The extension of the column type.
  rawColumnType	string	The original string of the column type recorded in the database.
  columnKey	string	The type of the index.
  nullable	boolean	Indicates whether null values are supported.
  defaultValue	string	The default value.
  dataLength	integer	The length of the column.
  dataPrecision	integer	The precision of the column.
  dataScale	integer	The number of decimal places.
  encoding	string	The character encoding format for the column.
  columnComment	string	The comment on the column.
  isGenerateField	boolean	Indicates whether the column is a hidden column.

• Parameters in extraInfo

  Parameter	Type	Description
  errorDetails	array	The error details.
  errorCode	string	The error code.
  errorMsg	string	The error message.
  errorParam	map	The error-related parameters.
  failedTime	string	The time when the error occurred.

Examples

Sample request

{
    "searchKey":"j_oms41_ob",
    "type":"MIGRATION",
    "status":[
        "RUNNING"
    ],
    "labelIds":[
        "l_4w****wygjk"
    ],
    "sourceEndpointTypes":[
        "OB_MYSQL"
    ],
    "sinkEndpointTypes":[

    ],
    "visibleSubProject":false,
    "sortField":"gmtCreate",
    "order":"asc",
    "pageNumber":1,
    "pageSize":5
}

Sample response

{
    "success":"true",
    "errorDetail":null,
    "code":null,
    "message":null,
    "advice":null,
    "requestId":"e8dec2fe-40****",
    "pageNumber":"1",
    "pageSize":"5",
    "totalCount":"151",
    "cost":"54 ms",
    "data":[
        {
            "id":"np_4w****",
            "type":"MIGRATION",
            "name":"j_oms4****9_56_obo",
            "labels":[
                {
                    "id":"l_***",
                    "name":"boq***"
                }
            ],
            "owner":"oms_api_te***",
            "importance":"LOW",
            "status":"RUNNING",
            "gmtCreate":"2023-07-13T06:26:15",
            "gmtModified":"2023-07-13T07:01:27",
            "gmtStart":"2023-07-13T07:01:27",
            "gmtFinish":null,
            "destConnId":null,
            "isMerging":false,
            "isModifying":false,
            "isSubProject":false,
            "sourceEndpointType":"OB_MYSQL",
            "sinkEndpointType":"MYSQL",
            "transferMapping":{
                "mode":"SPECIFIC",
                "databases":[
                    {
                        "mappedName":"n12956****mysql",
                        "name":"striG***UTF115ng",
                        "id":null,
                        "tables":[
                            {
                                "mappedName":"CHAR****001_",
                                "name":"CHARA****PK_001",
                                "id":null,
                                "whereClause":null,
                                "filterColumns":null,
                                "shardColumns":null,
                                "columns":null
                            }
                        ],
                        "views":[
                            {
                                "mappedName":"HIGH_SAL****EASE_",
                                "name":"HIGH_SA****LEASE",
                                "id":null,
                                "whereClause":null,
                                "filterColumns":null,
                                "shardColumns":null,
                                "columns":null
                            }
                        ],
                        "specificTables":null,
                        "specificViews":null
                    }
                ]
            },
            "commonTransferConfig":null,
            "enableStructTransfer":true,
            "structTransferConfig":null,
            "enableFullTransfer":true,
            "enableFullVerify":true,
            "fullTransferConfig":null,
            "enableIncrTransfer":true,
            "enableIncrVerify":false,
            "enableReverseIncrTransfer":true,
            "incrTransferConfig":null,
            "sourceConnectInfo":{
                "id":"c_4w3aq****",
                "endpointName":"ration12****0obmysql4116892***",
                "endpointId":"e_4w3al***",
                "endpointSide":null,
                "dbEngine":"OB_MYSQL_PUBLIC",
                "connectionInfo":null,
                "username":"oms931mys**",
                "version":"5.7.27-log",
                "timezone":"UTC",
                "charset":"utf8mb4",
                "nlsLengthSemantics":null,
                "operatingSystem":"Linux",
                "region":"cn-beijing",
                "ocpName":"",
                "connExtraAttributes":null,
                "owner":"oms_api_te***",
                "host":"xxx.xxx.xxx.1",
                "port":"2883"
            },
            "sinkConnectInfo":{
                "id":"c_4w3akx***",
                "endpointName":"ation129***9apimysql5616****",
                "endpointId":"e_4w3aic***",
                "endpointSide":null,
                "dbEngine":"MYSQL_PUBLIC",
                "connectionInfo":null,
                "username":"oms**obmys**",
                "version":"2.2.77",
                "timezone":"+08:00",
                "charset":"utf8mb4",
                "nlsLengthSemantics":null,
                "operatingSystem":null,
                "region":"cn-anhui",
                "ocpName":"107",
                "connExtraAttributes":null,
                "owner":"oms_api_te***",
                "host":"xxx.xxx.xxx.2",
                "port":"2883"
            },
            "steps":[
                {
                    "order":"1",
                    "name":"TRANSFER_PRECHECK",
                    "description":"Precheck",
                    "status":"FINISHED",
                    "extraInfo":{
                        "errorDetails":null,
                        "errorCode":null,
                        "errorMsg":null,
                        "errorParam":null,
                        "failedTime":null
                    },
                    "startTime":null,
                    "finishTime":null,
                    "progress":100,
                    "stepInfo":{

                    }
                }
            ],
            "extraInfo":{
                "errorDetails":null,
                "errorCode":null,
                "errorMsg":null,
                "errorParam":null,
                "failedTime":null
            },
            "alarmStats":{
                "target":null,
                "alarming":false,
                "recentlyTriggerCount":null,
                "ruleToRecentlyTriggerCount":null,
                "alarmContent":null
            }
        }
    ]
}

Appendix

Kafka data source

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "username":null,
        "enableSSL":null,
        "enableSASL":null,
        "saslMechanism":null,
        "disableIdentificationAlgorithm":false
    }
}

DataHub data source

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "project":"detx***"
    }
}

RocketMQ data source

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "endpointType":null,
        "version":null
    }
}

OceanBase data source

{
    "connExtraAttributes":{
        "cluster":"xgez***",
        "tenant":"gexr**",
        "isLogicSource":false,
        "useLogProxy":true,
        "drcUser":"tes***",
        "configUrl":"http://xxx.xxx.xxx.1:2883/service****",
        "logProxyIp":"xxx.xxx.xxx.2",
        "logProxyPort":2883,
        "noUserAuth":false
    }
}

TiDB data source

{
    "connExtraAttributes":{
        "kafkaId":"e_edz****",
        "kafkaTopic":"test**",
        "kafkaName":"denz***"
    }
}