DescribeProject

This topic describes the DescribeProject operation provided by OceanBase Migration Service (OMS) Community Edition.

Description

You can call this operation to query details of a data migration or synchronization task.

Call description

Limitations

This operation is supported in OMS Community Edition V4.1.1 and later.

Request path

POST /api/v2?Action=DescribeProject

Request parameters

Response parameters

• Parameters in data

  Parameter	Type	Description
  id	string	The ID of the task.
  type	string	The type of the task. Valid values: MIGRATION and SYNC.
  name	string	The name of the task, which cannot contain spaces.
  labels	object []	The tags.
  ├─ id	string	The ID of the tag.
  ├─ name	string	The name of the tag.
  owner	string	The owner of the task.
  importance	string	The alert level of the task. Valid values: HIGH, MEDIUM, LOW, and IGNORE.
  status	string	The status of the task.
  gmtCreate	string	The time when the task was created, in the UTC+0 time zone.
  gmtModified	string	The time when the task was updated, in the UTC+0 time zone.
  gmtStart	string	The start time of the task.
  gmtFinish	string	The end time of the task.
  destConnId	string	The connection ID of the task.
  isMerging	boolean	Indicates whether the task is in the progress of a major compaction.
  isModifying	boolean	Indicates whether the task is being modified.
  isSubProject	boolean	Indicates whether the task is a subtask.
  sourceEndpointType	String	The type of the source data source.
  sinkEndpointType	String	The type of the target data source.
  transferMapping	Object	The mapping information about the objects to be transferred.
  ├─ mode	string	The mapping mode for the objects to be transferred. Valid values: SPECIFIC and WILDCARD, which indicate specifying specific objects and matching objects by using a rule, respectively.
  ├─ databases	object []	The list of database objects. For more information, see the Parameters in databases/databasesBlack section.
  ├─ databasesBlack	Object []	The blocklist of database objects used for the WILDCARD mode. For more information, see the Parameters in databases/databasesBlack section.
  ├─ tableAndViewWhiteList	array	The wildcard-based allowlist of tables and views used for the WILDCARD mode.
  ├─ tableAndViewBlackList	array	The wildcard-based blocklist of tables and views used for the WILDCARD mode.
  commonTransferConfig	Object	The common settings for transfer.
  ├─ tableCategory	string	The type of tables to be transferred. Valid values: ALL (all tables), WITH_UNIQUE_ROW_ID (tables with a unique row ID), and WITHOUT_UNIQUE_ROW_ID (tables without a unique row ID). Default value: ALL.
  ├─ activeActive	boolean	Indicates whether active-active disaster recovery is enabled.
  ├─ mqSerializerType	string	The JSON data serialization type for delivering data to message queues. Valid values: DEFAULT, CANAL, DATAWORKS_V2, SHAREPLEX, DEFAULT_WITH_SCHEMA, DEBEZIUM, DEBEZIUMFLATTEN, and DEBEZIUMSMT.
  ├─ mqPartitionMode	string	The partition routing mode for delivering data to message queues. Valid values: ONE, HASH, and TABLE. ONE specifies to deliver data to a specified partition. HASH specifies to deliver data to HASH partitions based on the primary key column or sharding columns. TABLE specifies to deliver data to HASH partitions based on database and table names.
  ├─ mqPartition	integer	When the value of the partitionMode parameter is ONE, this parameter indicates the location of the partition to which data is delivered in the current topic.
  ├─ datahubTopicType	string	The type of the topic for delivering data to DataHub. Valid values: TUPLE and BLOB.
  ├─ rocketMqProducerGroup	String	The name of the producer group. This parameter takes effect only when the type of the target data source is RocketMQ. Default value: OMS.
  ├─ rocketMqMsgTags	string	The tag of the message. This parameter takes effect when the type of the target data source is RocketMQ. For example, when the value is default, the _tag suffix is added to the value by default.
  ├─ rocketMqEnableMsgTrace	boolean	Indicates whether message tracing is enabled. This parameter takes effect when the type of the target data source is RocketMQ.
  ├─ rocketMqSendMsgTimeout	long	The message delivery timeout period. This parameter takes effect when the type of the target data source is RocketMQ.
  ├─ dataWorksBusinessName	string	The ID of the business system.
  enableStructTransfer	boolean	Indicates whether schema transfer is allowed.
  structTransferConfig	object	The settings for schema transfer.
  ├─ byteCharConvertStrategy	string	The BYTE/CHAR conversion scheme. Valid values: IGNORE_BYTE_IF_BYTE_USED, FORCE_CHAR_IF_BYTE_USED, EXPAND_LEN_IF_BYTE_USED, and DO_NOTHING_IF_BYTE_USED. Default value: DO_NOTHING_IF_BYTE_USED.
  ├─ deferIndexCreation	boolean	Indicates whether post-indexing is allowed. The default value is false.
  enableFullTransfer	boolean	Indicates whether full transfer is enabled.
  enableFullVerify	boolean	Indicates whether full verification is enabled.
  fullTransferConfig	object	The settings for full transfer.
  ├─ nonePkUkTruncateDstTable	boolean	Indicates whether the target table is cleared during full migration from a table without a unique index.
  ├─ allowDestTableNotEmpty	string	The processing strategy to apply when the target table has records. Valid values: true and false . The value true specifies to ignore existing data in the target table. If the target is a MySQL database or OceanBase Database Community Edition, OMS Community Edition overwrites the existing data in the target table. Notice If you set the value to ture, data is pulled in IN mode for full verification. In this case, the scenario where the target table contains more data than the source table cannot be verified, and the verification efficiency will be decreased. The value false specifies to stop migration. If the target table has records, an error is returned during full migration. In this case, clear the existing data in the target table and try again. Notice After an error is returned, if you click Resume in the dialog box, OMS Community Edition ignores this error and continues to migrate data. Proceed with caution.
  ├─ fullTransferSpeedMode	string	The concurrency for full migration. Valid values: STEADY, NORMAL, and FAST.
  ├─ fullVerifySpeedMode	string	The concurrency for full verification. Valid values: STEADY, NORMAL, and FAST.
  enableIncrTransfer	boolean	Indicates whether incremental synchronization is enabled.
  enableIncrVerify	boolean	Indicates whether incremental verification is enabled.
  enableReverseIncrTransfer	boolean	Indicates whether reverse increment is enabled.
  incrTransferConfig	object	The settings for incremental synchronization.
  ├─ startTimestamp	string	The start timestamp for incremental synchronization. This parameter is available only when incremental synchronization is enabled and full transfer is disabled.
  ├─ recordTypeWhiteList	Array	The data types of incremental data synchronized in incremental synchronization.
  ├─ storeLogKeptHour	integer	The retention period of logs when incremental synchronization is enabled and the Store component is used to pull incremental logs. Unit: hours. The default retention period is seven days. The maximum retention period is 365 days.
  ├─ enableSequencingWithinTxn	boolean	Indicates whether intra-transaction sequencing is enabled when incremental synchronization is enabled and the Store component is used to pull incremental logs.
  ├─ incrSyncConcurrency	integer	The number of threads for incremental synchronization. Default value: 64.
  ├─ enableIncrSyncStatistics	boolean	Indicates whether statistics collection is enabled for incremental DML and DDL operation records.
  sourceConnectInfo	object	The connection information of the source data source.
  ├─ id	string	The ID of the connection.
  ├─ endpointName	string	The name of the endpoint.
  ├─ endpointId	string	The ID of the endpoint.
  ├─ endpointSide	string	The type of the endpoint. Valid values: source and dest.
  ├─ dbEngine	string	The database engine of the endpoint.
  ├─ connectionInfo	string	The connection information of the endpoint.
  ├─ username	string	The username used to connect to the database.
  ├─ version	string	The version of the database.
  ├─ timezone	string	The time zone of the database.
  ├─ charset	string	The character set.
  ├─ nlsLengthSemantics	string	This attribute is available only when the data source is an Oracle database or an Oracle-compatible tenant of OceanBase Database.
  ├─ operatingSystem	string	The operating system. This parameter is unavailable for OceanBase Database.
  ├─ region	string	The region.
  ├─ ocpName	string	The name of the OceanBase Cloud Platform (OCP) instance.
  ├─ connExtraAttributes	object	The additional attributes of the connection. For more information about the nested response parameters, see Appendix.
  ├─ owner	string	The owner.
  ├─ host	string	The IP address.
  ├─ port	integer	The port number.
  sinkConnectInfo	object	The connection information of the target data source.
  ├─ id	string	The ID of the connection.
  ├─ endpointName	string	The name of the endpoint.
  ├─ endpointId	string	The ID of the endpoint.
  ├─ endpointSide	string	The type of the endpoint. Valid values: source and dest.
  ├─ dbEngine	string	The database engine of the endpoint.
  ├─ connectionInfo	string	The connection information of the endpoint.
  ├─ username	string	The username used to connect to the database.
  ├─ version	string	The version of the database.
  ├─ timezone	string	The time zone of the database.
  ├─ charset	string	The character set.
  ├─ nlsLengthSemantics	string	This attribute is available only when the data source is an Oracle database or an Oracle-compatible tenant of OceanBase Database.
  ├─ operatingSystem	string	The operating system. This parameter is unavailable for OceanBase Database.
  ├─ region	string	The region.
  ├─ ocpName	string	The name of the OCP instance.
  ├─ connExtraAttributes	object	The additional attributes of the connection. For more information about the nested response parameters, see Appendix.
  ├─ owner	string	The owner.
  ├─ host	string	The IP address.
  ├─ port	integer	The port number.
  steps	object []	The steps.
  ├─ order	integer	The sequence number of the step.
  ├─ name	string	The name of the step.
  ├─ description	string	The description of the step.
  ├─ status	string	The status of the step.
  ├─ extraInfo	object	The additional information. For more information, see the Parameters in extraInfo section.
  ├─ startTime	string	The start time of the step, in the UTC+0 time zone.
  ├─ finishTime	string	The end time of the step, in the UTC+0 time zone.
  ├─ progress	integer	The progress of the step.
  ├─ stepInfo	Object	The details of the step. For more information, see the Parameters in stepInfo section.
  extraInfo	Object	The error messages and other additional information.
  ├─ runningStep	string	The step in the Running state.
  ├─ runningProgress	integer	The progress of the step in the Running state.
  ├─ monitoringIncr	boolean	Indicates whether incremental data monitoring is started.
  ├─ subtopics	string	The subtopics of the store associated with the task.
  ├─ reverseSubtopics	string	The subtopics of the reverse store associated with the task.
  ├─ logServiceStartCheckpoint	integer	The start timestamp.
  ├─ sourceStoreKeptHour	integer	The retention period of logs.
  ├─ syncDelay	integer	The delay of incremental synchronization.
  ├─ syncDelaySampleTimestamp	integer	The timestamp when the synchronization delay is collected, in seconds.
  ├─ accessObSource	boolean	Indicates whether to allow direct connection to the source OceanBase database.
  ├─ overwriteConfig	boolean	Indicates whether the overwriting of all new configurations is allowed.
  ├─ ignoreUnsupportDdl	boolean	Indicates whether unsupported DDL statements are ignored.
  ├─ subDbs	map	The table configurations of different downstream consumers. This parameter is used for combining configurations when the data synchronization task is reused.
  ├─ subConds	map	The configuration objects.
  ├─ subIds	map	The IDs of the configuration objects.
  ├─ maxConnectorCount	integer	The maximum number of connectors. The value -1 indicates that the number of connectors is not limited.
  ├─ storeIncr	boolean	Indicates whether incremental data is saved.
  ├─ incrSyncTimestamp	integer	The current timestamp for incremental synchronization, in seconds.
  alarmStats	object	The alert statistics.
  ├─ target	string	The object for which the alert is generated.
  ├─ alarming	boolean	Indicates whether alerts exist in the system.
  ├─ recentlyTriggerCount	integer	The total number of recent alert messages.
  ├─ ruleToRecentlyTriggerCount	map	The number of recent alert messages by rule.
  ├─ alarmContent	string	The content of recent alert messages.

• Parameters in databases/databasesBlack

  Parameter	Type	Description
  mappedName	string	The name of the target database.
  name	string	The name of the source database.
  id	string	The ID of the database.
  tables	array	The settings for the table objects to be transferred in the current database. When the value of the mode parameter is WILDCARD, this parameter indicates the settings for a specific blocklist or allowlist of tables and views.
  ├─ mappedName	string	The mapped-to table name.
  ├─ name	string	The name of the table.
  ├─ id	string	The ID of the table.
  ├─ whereClause	string	The WHERE clause in the standard SQL syntax that is used to configure row filtering.
  ├─ filterColumns	Array	The filter columns.
  ├─ shardColumns	array	The sharding columns.
  ├─ columns		For more information, see the Parameters in columns section.
  views	Array	The settings for the view objects to be transferred in the current database.
  ├─ mappedName	string	The mapped-to view name.
  ├─ name	string	The name of the view.
  ├─ id	string	The ID of the view.
  ├─ whereClause	string	The WHERE clause in the standard SQL syntax that is used to configure row filtering.
  ├─ filterColumns	array	The filter columns.
  ├─ shardColumns	array	The sharding columns.
  ├─ columns	object []	The columns to be transferred. For more information, see the Parameters in columns section.
  specificTables	array	The settings for the matched table objects. When the value of the mode parameter is WILDCARD, this parameter indicates the renaming and row filtering settings for the matched table objects.
  ├─ mappedName	string	The mapped-to table name.
  ├─ name	string	The name of the table.
  ├─ id	string	The ID of the table.
  ├─ whereClause	string	The WHERE clause in the standard SQL syntax that is used to configure row filtering.
  ├─ filterColumns	array	The filter columns.
  ├─ shardColumns	array	The sharding columns.
  ├─ columns	object []	The columns to be transferred. For more information, see the Parameters in columns section.
  specificViews	array	The settings for the matched view objects. When the value of the mode parameter is WILDCARD, this parameter indicates the renaming settings for the matched view objects.
  ├─ mappedName	string	The mapped-to view name.
  ├─ name	string	The name of the view.
  ├─ id	string	The ID of the view.
  ├─ whereClause	string	The WHERE clause in the standard SQL syntax that is used to configure row filtering.
  ├─ filterColumns	array	The filter columns.
  ├─ shardColumns	array	The sharding columns.
  ├─ columns	object []	The columns to be transferred. For more information, see the Parameters in columns section.

• Parameters in columns

  Parameter	Type	Description
  columnName	string	The name of the column.
  mappedName	string	The mapped-to column name.
  position	Integer	The position of the column.
  columnType	string	The type of the column.
  recordFieldType	string	The extension of the column type.
  rawColumnType	string	The original string of the column type recorded in the database.
  columnKey	string	The type of the index.
  nullable	boolean	Indicates whether null values are supported.
  defaultValue	string	The default value.
  dataLength	integer	The length of the column.
  dataPrecision	integer	The precision of the column.
  dataScale	integer	The number of decimal places.
  encoding	string	The character encoding format for the column.
  columnComment	string	The comment on the column.
  isGenerateField	boolean	Indicates whether the column is a hidden column.

• Parameters in extraInfo

  Parameter	Type	Description
  errorDetails	array	The error details.
  errorCode	string	The error code.
  errorMsg	string	The error message.
  errorParam	Object	The error-related parameters.
  failedTime	string	The time when the error occurred.

• Parameters in stepInfo

  Different parameters are nested under stepInfo for different steps.

  • TRANSFER_APP_SWITCH (forward switchover)

    Parameter	Type	Description
    checkpointSampleTimestamp	Long	The checkpoint collection timestamp. The value is a Unix timestamp in seconds.

  • INCR_TRANSFER (incremental synchronization)

    Parameter	Type	Description
    incrTimestampCheckpoint	Long	The checkpoint timestamp. The value is a Unix timestamp in seconds.
    checkpointSampleTimestamp	Long	The checkpoint collection timestamp. The value is a Unix timestamp in seconds.
    enableIncrStatistics	Boolean	Indicates whether DDL/DML statistics collection is enabled.

  • FULL_VERIFIER (full verification) and FULL_TRANSFER (full migration)

    Parameter	Type	Description
    processedRecords	long	The ratio of migrated rows to total rows.
    capacity	Long	The estimated total number of rows.
    srcRps	Long	The read requests per second (RPS) of the source data source.
    srcRt	String	The read response time (RT) per record of the source data source, in milliseconds.
    srcIops	long	The read throughput of the source data source, in bytes per second.
    dstRps	Long	The read/write RPS of the target data source.
    dstRt	Long	The read/write RT per record of the target data source, in milliseconds.
    dstIops	long	The read/write throughput of the target data source, in bytes per second.
    srcRpsRef	Long	The read RPS baseline of the source data source.
    dstRpsRef	Long	The read/write RPS baseline of the target data source.
    dstRtRef	Long	The read/write RT baseline of the target data source.

Examples

Sample request

{
    "id":"np_4wh****"
}

Sample response

{
    "success":true,
    "errorDetail":null,
    "code":null,
    "message":null,
    "advice":null,
    "requestId":"a7b57****",
    "pageNumber":null,
    "pageSize":null,
    "totalCount":null,
    "cost":"27 ms",
    "data":[
        {
            "id":"np_4w****",
            "type":"MIGRATION",
            "name":"oms41_mig***",
            "labels":null,
            "owner":"oms_api_t***",
            "importance":"MEDIUM",
            "status":"RUNNING",
            "gmtCreate":"2023-07-13T03:59:15",
            "gmtModified":"2023-07-13T10:12:44",
            "gmtStart":"2023-07-13T06:47:35",
            "gmtFinish":null,
            "destConnId":null,
            "isMerging":false,
            "isModifying":false,
            "isSubProject":false,
            "sourceEndpointType":"DB2_LUW",
            "sinkEndpointType":"OB_MYSQL",
            "transferMapping":{
                "mode":"SPECIFIC",
                "databases":[
                    {
                        "mappedName":"tion12***20obmysql2",
                        "name":"GRATION129****",
                        "id":null,
                        "tables":[
                            {
                                "mappedName":"TEST_MA***",
                                "name":"TEST_M***",
                                "id":null,
                                "whereClause":null,
                                "filterColumns":null,
                                "shardColumns":null,
                                "columns":null
                            }
                        ],
                        "views":null,
                        "specificTables":null,
                        "specificViews":null
                    }
                ],
                "databasesBlack":null,
                "tableAndViewWhiteList":null,
                "tableAndViewBlackList":null
            },
            "commonTransferConfig":{
                "tableCategory":null,
                "activeActive":false,
                "mqSerializerType":null,
                "mqPartitionMode":null,
                "mqPartition":0,
                "datahubTopicType":null,
                "rocketMqProducerGroup":null,
                "rocketMqMsgTags":null,
                "rocketMqEnableMsgTrace":null,
                "rocketMqSendMsgTimeout":null,
                "dataWorksBusinessName":null
            },
            "enableStructTransfer":true,
            "structTransferConfig":{
                "byteCharConvertStrategy":"DO_NOTHING_IF_BYTE_USED",
                "deferIndexCreation":true
            },
            "enableFullTransfer":true,
            "enableFullVerify":true,
            "fullTransferConfig":{
                "nonePkUkTruncateDstTable":null,
                "allowDestTableNotEmpty":true,
                "fullTransferSpeedMode":"NORMAL",
                "fullVerifySpeedMode":"NORMAL"
            },
            "enableIncrTransfer":true,
            "enableIncrVerify":false,
            "enableReverseIncrTransfer":true,
            "incrTransferConfig":{
                "startTimestamp":1689201369,
                "recordTypeWhiteList":[
                    "INSERT",
                    "DELETE",
                    "UPDATE"
                ],
                "storeLogKeptHour":120,
                "enableSequencingWithinTxn":null,
                "incrSyncConcurrency":null,
                "enableIncrSyncStatistics":null
            },
            "sourceConnectInfo":{
                "id":"c_4w1qs****",
                "endpointName":"GRATION12948*****",
                "endpointId":"e_4w1q****",
                "endpointSide":null,
                "dbEngine":"DB2_LUW_PUBLIC",
                "connectionInfo":null,
                "username":"db2i***",
                "version":"11.5.0.0",
                "timezone":"+08:00",
                "charset":"UTF8",
                "nlsLengthSemantics":null,
                "operatingSystem":"Linux",
                "region":"cn-anhui",
                "ocpName":"",
                "connExtraAttributes":null,
                "owner":"oms_api_t***",
                "host":"xxx.xxx.xxx.1",
                "port":"2883"
            },
            "sinkConnectInfo":{
                "id":"c_4w1qs***",
                "endpointName":"tion1294820obmysq****",
                "endpointId":"e_4w1qr2***",
                "endpointSide":null,
                "dbEngine":"OB_MYSQL_PUBLIC",
                "connectionInfo":null,
                "username":"oms2027obmysql@oms_***",
                "version":"2.2.77",
                "timezone":"+08:00",
                "charset":"utf8mb4",
                "nlsLengthSemantics":null,
                "operatingSystem":null,
                "region":"cn-anhui",
                "ocpName":"107",
                "connExtraAttributes":null,
                "owner":"oms_api_t***",
                "host":"xxx.xxx.xxx.4",
                "port":"2883"
            },
            "steps":[
                {
                    "order":1,
                    "name":"TRANSFER_PRECHECK",
                    "description":"Precheck",
                    "status":"FINISHED",
                    "extraInfo":{
                        "errorDetails":null,
                        "errorCode":null,
                        "errorMsg":null,
                        "errorParam":null,
                        "failedTime":null
                    },
                    "startTime":"2023-07-12T22:33:42",
                    "finishTime":"2023-07-12T22:33:48",
                    "progress":100,
                    "stepInfo":null
                },
                {
                    "order":2,
                    "name":"STRUCT_TRANSFER",
                    "description":"Schema migration",
                    "status":"FINISHED",
                    "extraInfo":{
                        "errorDetails":null,
                        "errorCode":null,
                        "errorMsg":null,
                        "errorParam":null,
                        "failedTime":"2023-07-12T22:35:48.881"
                    },
                    "startTime":"2023-07-12T22:35:24",
                    "finishTime":"2023-07-12T22:36:04",
                    "progress":100,
                    "stepInfo":null
                },
                {
                    "order":6,
                    "name":"INCR_TRANSFER",
                    "description":"Incremental synchronization",
                    "status":"MONITORING",
                    "extraInfo":{
                        "errorDetails":null,
                        "errorCode":null,
                        "errorMsg":null,
                        "errorParam":null,
                        "failedTime":null
                    },
                    "startTime":"2023-07-12T22:41:20",
                    "finishTime":"2023-07-12T22:46:57",
                    "progress":99,
                    "stepInfo":{
                        "incrTimestampCheckpoint":1689213946,
                        "checkpointSampleTimestamp":1689214358,
                        "enableIncrStatistics":true
                    }
                }
            ],
            "extraInfo":null,
            "alarmStats":{
                "target":"np_4w1q****",
                "alarming":false,
                "recentlyTriggerCount":null,
                "ruleToRecentlyTriggerCount":null,
                "alarmContent":null
            }
        }
    ]
}

Appendix

Kafka

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "username":null,
        "enableSSL":null,
        "enableSASL":null,
        "saslMechanism":null,
        "disableIdentificationAlgorithm":false
    }
}

DataHub

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "project":"detx***"
    }
}

RocketMQ

{
    "connExtraAttributes":{
        "endpoint":"xxx.xxx.xxx.1:2883",
        "endpointType":null,
        "version":null
    }
}

OceanBase Database

{
    "connExtraAttributes":{
        "cluster":"xgez***",
        "tenant":"gexr**",
        "isLogicSource":false,
        "useLogProxy":true,
        "drcUser":"tes***",
        "configUrl":"http://xxx.xxx.xxx.1:2883/service****",
        "logProxyIp":"xxx.xxx.xxx.2",
        "logProxyPort":2883,
        "noUserAuth":false
    }
}

TiDB

{
    "connExtraAttributes":{
        "kafkaId":"e_edz****",
        "kafkaTopic":"test**",
        "kafkaName":"denz***"
    }
}