This topic describes the DescribeProject operation provided by OceanBase Migration Service (OMS) Community Edition.
Description
You can call this operation to query details of a data migration or synchronization task.
Call description
Limitations
This operation is supported in OMS Community Edition V4.1.1 and later.
Request path
POST /api/v2?Action=DescribeProject
Request parameters
| Parameter | Type | Description |
|---|---|---|
| id | string | The ID of the data migration or synchronization task. |
Response parameters
| Parameter | Type | Description |
|---|---|---|
| success | boolean | Indicates whether the call is successful. |
| errorDetail | object | The error details. |
| code | string | The error code. |
| message | string | The error description. |
| advice | string | The suggestions. |
| requestId | string | The ID of the request. |
| pageNumber | integer | The number of the page returned, which takes effect in a pagination query. |
| pageSize | integer | The number of records returned on each page, which takes effect in a pagination query. |
| totalCount | integer | The total number of pages returned, which takes effect in a pagination query. |
| cost | string | The time spent in processing the request, in seconds. |
| data | String | The information about the task. For more information, see the Parameters in data section. |
Parameters in data
Parameter Type Description id string The ID of the task. type string The type of the task. Valid values: MIGRATIONandSYNC.name string The name of the task, which cannot contain spaces. labels object [] The tags. ├─ id string The ID of the tag. ├─ name string The name of the tag. owner string The owner of the task. importance string The alert level of the task. Valid values: HIGH,MEDIUM,LOW, andIGNORE.status string The status of the task. gmtCreate string The time when the task was created, in the UTC+0 time zone. gmtModified string The time when the task was updated, in the UTC+0 time zone. gmtStart string The start time of the task. gmtFinish string The end time of the task. destConnId string The connection ID of the task. isMerging boolean Indicates whether the task is in the progress of a major compaction. isModifying boolean Indicates whether the task is being modified. isSubProject boolean Indicates whether the task is a subtask. sourceEndpointType String The type of the source. sinkEndpointType String The type of the target. transferMapping Object The mapping information about the objects to be transferred. ├─ mode string The mapping mode for the objects to be transferred. Valid values: SPECIFICandWILDCARD, which indicate specifying specific objects and matching objects by using a rule, respectively.├─ databases object [] The list of database objects. For more information, see the Parameters in databases/databasesBlack section. ├─ databasesBlack Object [] The blocklist of database objects used for the WILDCARDmode. For more information, see the Parameters in databases/databasesBlack section.├─ tableAndViewWhiteList array The wildcard-based allowlist of tables and views used for the WILDCARDmode.├─ tableAndViewBlackList array The wildcard-based blocklist of tables and views used for the WILDCARDmode.commonTransferConfig Object The common settings for transfer. ├─ tableCategory string The type of tables to be transferred. Valid values: ALL(all tables),WITH_UNIQUE_ROW_ID(tables with a unique row ID), andWITHOUT_UNIQUE_ROW_ID(tables without a unique row ID). Default value:ALL.├─ activeActive boolean Indicates whether active-active disaster recovery is enabled. ├─ mqSerializerType string The JSON data serialization type for delivering data to message queues. Valid values: DEFAULT,CANAL,DATAWORKS_V2,SHAREPLEX,DEFAULT_WITH_SCHEMA,DEBEZIUM,DEBEZIUMFLATTEN, andDEBEZIUMSMT.├─ mqPartitionMode string The partition routing mode for delivering data to message queues. Valid values: ONE,HASH, andTABLE.ONEspecifies to deliver data to a specified partition.HASHspecifies to deliver data to HASH partitions based on the primary key column or sharding columns.TABLEspecifies to deliver data to HASH partitions based on database and table names.├─ mqPartition integer When the value of the partitionModeparameter isONE, this parameter indicates the location of the partition to which data is delivered in the current topic.├─ datahubTopicType string The type of the topic for delivering data to DataHub. Valid values: TUPLEandBLOB.├─ rocketMqProducerGroup String The name of the producer group. This parameter takes effect only when the type of the target is RocketMQ. Default value: OMS.├─ rocketMqMsgTags string The tag of the message. This parameter takes effect when the type of the target is RocketMQ. For example, when the value is default, the_tagsuffix is added to the value by default.├─ rocketMqEnableMsgTrace boolean Indicates whether message tracing is enabled. This parameter takes effect when the type of the target is RocketMQ. ├─ rocketMqSendMsgTimeout long The message delivery timeout period. This parameter takes effect when the type of the target is RocketMQ. ├─ dataWorksBusinessName string The ID of the business system. enableStructTransfer boolean Indicates whether schema transfer is allowed. structTransferConfig object The settings for schema transfer. ├─ byteCharConvertStrategy string The BYTE/CHAR conversion scheme. Valid values: IGNORE_BYTE_IF_BYTE_USED,FORCE_CHAR_IF_BYTE_USED,EXPAND_LEN_IF_BYTE_USED, andDO_NOTHING_IF_BYTE_USED. Default value:DO_NOTHING_IF_BYTE_USED.├─ deferIndexCreation boolean Indicates whether post-indexing is allowed. The default value is false.enableFullTransfer boolean Indicates whether full transfer is enabled. enableFullVerify boolean Indicates whether full verification is enabled. fullTransferConfig object The settings for full transfer. ├─ nonePkUkTruncateDstTable boolean Indicates whether the target table is cleared during full migration from a table without a unique index. ├─ allowDestTableNotEmpty string The processing strategy to apply when the target table has records, including Ignore and Stop Migration. - The value
trueindicates that Ignore is selected. If the target is a MySQL database or OceanBase Database Community Edition, OMS Community Edition overwrites the existing data in the target table.Notice
If you set the value to true, data is pulled in IN mode for full verification. In this case, the scenario where the target table contains more data than the source table cannot be verified, and the verification efficiency will be decreased.
- The value
falseindicates that Stop Migration is selected. If the target table has records, an error is returned during full migration. In this case, clear the existing data in the target table and try again.Notice
After an error is returned, if you click Resume in the dialog box, OMS Community Edition ignores this error and continues to migrate data. Proceed with caution.
├─ fullTransferSpeedMode string The concurrency for full migration. Valid values: STEADY,NORMAL, andFAST.├─ fullVerifySpeedMode string The concurrency for full verification. Valid values: STEADY,NORMAL, andFAST.enableIncrTransfer boolean Indicates whether incremental synchronization is enabled. enableIncrVerify boolean Indicates whether incremental verification is enabled. enableReverseIncrTransfer boolean Indicates whether reverse increment is enabled. incrTransferConfig object The settings for incremental synchronization. ├─ startTimestamp string The start timestamp for incremental synchronization. This parameter is available only when incremental synchronization is enabled and full transfer is disabled. ├─ recordTypeWhiteList Array The data types of incremental data synchronized in incremental synchronization. ├─ storeLogKeptHour integer The retention period of logs when incremental synchronization is enabled and the Store component is used to pull incremental logs. Unit: hours. The default retention period is seven days. The maximum retention period is 365 days. ├─ enableSequencingWithinTxn boolean Indicates whether intra-transaction sequencing is enabled when incremental synchronization is enabled and the Store component is used to pull incremental logs. ├─ incrSyncConcurrency integer The number of threads for incremental synchronization. Default value: 64.├─ enableIncrSyncStatistics boolean Indicates whether statistics collection is enabled for incremental DML and DDL operation records. sourceConnectInfo object The connection information of the source. ├─ id string The ID of the connection. ├─ endpointName string The name of the endpoint. ├─ endpointId string The ID of the endpoint. ├─ endpointSide string The type of the endpoint. Valid values: sourceanddest.├─ dbEngine string The database engine of the endpoint. ├─ connectionInfo string The connection information of the endpoint. ├─ username string The username used to connect to the database. ├─ version string The version of the database. ├─ timezone string The time zone of the database. ├─ charset string The character set. ├─ nlsLengthSemantics string This attribute is available only when the data source is an Oracle database or an Oracle-compatible tenant of OceanBase Database. ├─ operatingSystem string The operating system. This parameter is unavailable for OceanBase Database. ├─ region string The region. ├─ ocpName string The name of the OceanBase Cloud Platform (OCP) instance. ├─ connExtraAttributes object The additional attributes of the connection. For more information about the nested response parameters, see Appendix. ├─ owner string The owner. ├─ host string The IP address. ├─ port integer The port number. sinkConnectInfo object The connection information of the target. ├─ id string The ID of the connection. ├─ endpointName string The name of the endpoint. ├─ endpointId string The ID of the endpoint. ├─ endpointSide string The type of the endpoint. Valid values: sourceanddest.├─ dbEngine string The database engine of the endpoint. ├─ connectionInfo string The connection information of the endpoint. ├─ username string The username used to connect to the database. ├─ version string The version of the database. ├─ timezone string The time zone of the database. ├─ charset string The character set. ├─ nlsLengthSemantics string This attribute is available only when the data source is an Oracle database or an Oracle-compatible tenant of OceanBase Database. ├─ operatingSystem string The operating system. This parameter is unavailable for OceanBase Database. ├─ region string The region. ├─ ocpName string The name of the OCP instance. ├─ connExtraAttributes object The additional attributes of the connection. For more information about the nested response parameters, see Appendix. ├─ owner string The owner. ├─ host string The IP address. ├─ port integer The port number. steps object [] The steps. ├─ order integer The sequence number of the step. ├─ name string The name of the step. ├─ description string The description of the step. ├─ status string The status of the step. ├─ extraInfo object The additional information. For more information, see the Parameters in extraInfo section. ├─ startTime string The start time of the step, in the UTC+0 time zone. ├─ finishTime string The end time of the step, in the UTC+0 time zone. ├─ progress integer The progress of the step. ├─ stepInfo Object The details of the step. For more information, see the Parameters in stepInfo section. extraInfo Object The error messages and other additional information. ├─ runningStep string The step in the Running state. ├─ runningProgress integer The progress of the step in the Running state. ├─ monitoringIncr boolean Indicates whether incremental data monitoring is started. ├─ subtopics string The subtopics of the store associated with the task. ├─ reverseSubtopics string The subtopics of the reverse store associated with the task. ├─ logServiceStartCheckpoint integer The start timestamp. ├─ sourceStoreKeptHour integer The retention period of logs. ├─ syncDelay integer The delay of incremental synchronization. ├─ syncDelaySampleTimestamp integer The timestamp when the synchronization delay is collected, in seconds. ├─ accessObSource boolean Indicates whether to allow direct connection to the source OceanBase database. ├─ overwriteConfig boolean Indicates whether the overwriting of all new configurations is allowed. ├─ ignoreUnsupportDdl boolean Indicates whether unsupported DDL statements are ignored. ├─ subDbs map The table configurations of different downstream consumers. This parameter is used for combining configurations when the data synchronization task is reused. ├─ subConds map The configuration objects. ├─ subIds map The IDs of the configuration objects. ├─ maxConnectorCount integer The maximum number of connectors. The value -1indicates that the number of connectors is not limited.├─ storeIncr boolean Indicates whether incremental data is saved. ├─ incrSyncTimestamp integer The current timestamp for incremental synchronization, in seconds. alarmStats object The alert statistics. ├─ target string The object for which the alert is generated. ├─ alarming boolean Indicates whether alerts exist in the system. ├─ recentlyTriggerCount integer The total number of recent alert messages. ├─ ruleToRecentlyTriggerCount map The number of recent alert messages by rule. ├─ alarmContent string The content of recent alert messages. - The value
Parameters in databases/databasesBlack
Parameter Type Description mappedName string The name of the target database. name string The name of the source database. id string The ID of the database. tables array The settings for the table objects to be transferred in the current database. When the value of the modeparameter isWILDCARD, this parameter indicates the settings for a specific blocklist or allowlist of tables and views.├─ mappedName string The mapped-to table name. ├─ name string The name of the table. ├─ id string The ID of the table. ├─ whereClause string The WHEREclause in the standard SQL syntax that is used to configure row filtering.├─ filterColumns Array The filter columns. ├─ shardColumns array The sharding columns. ├─ columns For more information, see the Parameters in columns section. views Array The settings for the view objects to be transferred in the current database. ├─ mappedName string The mapped-to view name. ├─ name string The name of the view. ├─ id string The ID of the view. ├─ whereClause string The WHEREclause in the standard SQL syntax that is used to configure row filtering.├─ filterColumns array The filter columns. ├─ shardColumns array The sharding columns. ├─ columns object [] The columns to be transferred. For more information, see the Parameters in columns section. specificTables array The settings for the matched table objects. When the value of the modeparameter isWILDCARD, this parameter indicates the renaming and row filtering settings for the matched table objects.├─ mappedName string The mapped-to table name. ├─ name string The name of the table. ├─ id string The ID of the table. ├─ whereClause string The WHEREclause in the standard SQL syntax that is used to configure row filtering.├─ filterColumns array The filter columns. ├─ shardColumns array The sharding columns. ├─ columns object [] The columns to be transferred. For more information, see the Parameters in columns section. specificViews array The settings for the matched view objects. When the value of the modeparameter isWILDCARD, this parameter indicates the renaming settings for the matched view objects.├─ mappedName string The mapped-to view name. ├─ name string The name of the view. ├─ id string The ID of the view. ├─ whereClause string The WHEREclause in the standard SQL syntax that is used to configure row filtering.├─ filterColumns array The filter columns. ├─ shardColumns array The sharding columns. ├─ columns object [] The columns to be transferred. For more information, see the Parameters in columns section. Parameters in columns
Parameter Type Description columnName string The name of the column. mappedName string The mapped-to column name. position Integer The position of the column. columnType string The type of the column. recordFieldType string The extension of the column type. rawColumnType string The original string of the column type recorded in the database. columnKey string The type of the index. nullable boolean Indicates whether null values are supported. defaultValue string The default value. dataLength integer The length of the column. dataPrecision integer The precision of the column. dataScale integer The number of decimal places. encoding string The character encoding format for the column. columnComment string The comment on the column. isGenerateField boolean Indicates whether the column is a hidden column. Parameters in extraInfo
Parameter Type Description errorDetails array The error details. errorCode string The error code. errorMsg string The error message. errorParam Object The error-related parameters. failedTime string The time when the error occurred. Parameters in stepInfo
Different parameters are nested under
stepInfofor different steps.TRANSFER_APP_SWITCH (forward switchover)
Parameter Type Description checkpointSampleTimestamp Long The checkpoint collection timestamp. The value is a Unix timestamp in seconds. INCR_TRANSFER (incremental synchronization)
Parameter Type Description incrTimestampCheckpoint Long The checkpoint timestamp. The value is a Unix timestamp in seconds. checkpointSampleTimestamp Long The checkpoint collection timestamp. The value is a Unix timestamp in seconds. enableIncrStatistics Boolean Indicates whether DDL/DML statistics collection is enabled. FULL_VERIFIER (full verification) and FULL_TRANSFER (full migration)
Parameter Type Description processedRecords long The ratio of migrated rows to total rows. capacity Long The estimated total number of rows. srcRps Long The read requests per second (RPS) of the source. srcRt String The read response time (RT) per record of the source, in milliseconds. srcIops long The read throughput of the source, in bytes per second. dstRps Long The read/write RPS of the target. dstRt Long The read/write RT per record of the target, in milliseconds. dstIops long The read/write throughput of the target, in bytes per second. srcRpsRef Long The read RPS baseline of the source. dstRpsRef Long The read/write RPS baseline of the target. dstRtRef Long The read/write RT baseline of the target.
Examples
Sample request
{
"id":"np_4wh****"
}
Sample response
{
"success":true,
"errorDetail":null,
"code":null,
"message":null,
"advice":null,
"requestId":"a7b57****",
"pageNumber":null,
"pageSize":null,
"totalCount":null,
"cost":"27 ms",
"data":[
{
"id":"np_4w****",
"type":"MIGRATION",
"name":"oms41_mig***",
"labels":null,
"owner":"oms_api_t***",
"importance":"MEDIUM",
"status":"RUNNING",
"gmtCreate":"2023-07-13T03:59:15",
"gmtModified":"2023-07-13T10:12:44",
"gmtStart":"2023-07-13T06:47:35",
"gmtFinish":null,
"destConnId":null,
"isMerging":false,
"isModifying":false,
"isSubProject":false,
"sourceEndpointType":"DB2_LUW",
"sinkEndpointType":"OB_MYSQL",
"transferMapping":{
"mode":"SPECIFIC",
"databases":[
{
"mappedName":"tion12***20obmysql2",
"name":"GRATION129****",
"id":null,
"tables":[
{
"mappedName":"TEST_MA***",
"name":"TEST_M***",
"id":null,
"whereClause":null,
"filterColumns":null,
"shardColumns":null,
"columns":null
}
],
"views":null,
"specificTables":null,
"specificViews":null
}
],
"databasesBlack":null,
"tableAndViewWhiteList":null,
"tableAndViewBlackList":null
},
"commonTransferConfig":{
"tableCategory":null,
"activeActive":false,
"mqSerializerType":null,
"mqPartitionMode":null,
"mqPartition":0,
"datahubTopicType":null,
"rocketMqProducerGroup":null,
"rocketMqMsgTags":null,
"rocketMqEnableMsgTrace":null,
"rocketMqSendMsgTimeout":null,
"dataWorksBusinessName":null
},
"enableStructTransfer":true,
"structTransferConfig":{
"byteCharConvertStrategy":"DO_NOTHING_IF_BYTE_USED",
"deferIndexCreation":true
},
"enableFullTransfer":true,
"enableFullVerify":true,
"fullTransferConfig":{
"nonePkUkTruncateDstTable":null,
"allowDestTableNotEmpty":true,
"fullTransferSpeedMode":"NORMAL",
"fullVerifySpeedMode":"NORMAL"
},
"enableIncrTransfer":true,
"enableIncrVerify":false,
"enableReverseIncrTransfer":true,
"incrTransferConfig":{
"startTimestamp":1689201369,
"recordTypeWhiteList":[
"INSERT",
"DELETE",
"UPDATE"
],
"storeLogKeptHour":120,
"enableSequencingWithinTxn":null,
"incrSyncConcurrency":null,
"enableIncrSyncStatistics":null
},
"sourceConnectInfo":{
"id":"c_4w1qs****",
"endpointName":"GRATION12948*****",
"endpointId":"e_4w1q****",
"endpointSide":null,
"dbEngine":"DB2_LUW_PUBLIC",
"connectionInfo":null,
"username":"db2i***",
"version":"11.5.0.0",
"timezone":"+08:00",
"charset":"UTF8",
"nlsLengthSemantics":null,
"operatingSystem":"Linux",
"region":"cn-anhui",
"ocpName":"",
"connExtraAttributes":null,
"owner":"oms_api_t***",
"host":"xxx.xxx.xxx.1",
"port":"2883"
},
"sinkConnectInfo":{
"id":"c_4w1qs***",
"endpointName":"tion1294820obmysq****",
"endpointId":"e_4w1qr2***",
"endpointSide":null,
"dbEngine":"OB_MYSQL_PUBLIC",
"connectionInfo":null,
"username":"oms2027obmysql@oms_***",
"version":"2.2.77",
"timezone":"+08:00",
"charset":"utf8mb4",
"nlsLengthSemantics":null,
"operatingSystem":null,
"region":"cn-anhui",
"ocpName":"107",
"connExtraAttributes":null,
"owner":"oms_api_t***",
"host":"xxx.xxx.xxx.4",
"port":"2883"
},
"steps":[
{
"order":1,
"name":"TRANSFER_PRECHECK",
"description":"Precheck",
"status":"FINISHED",
"extraInfo":{
"errorDetails":null,
"errorCode":null,
"errorMsg":null,
"errorParam":null,
"failedTime":null
},
"startTime":"2023-07-12T22:33:42",
"finishTime":"2023-07-12T22:33:48",
"progress":100,
"stepInfo":null
},
{
"order":2,
"name":"STRUCT_TRANSFER",
"description":"Schema migration",
"status":"FINISHED",
"extraInfo":{
"errorDetails":null,
"errorCode":null,
"errorMsg":null,
"errorParam":null,
"failedTime":"2023-07-12T22:35:48.881"
},
"startTime":"2023-07-12T22:35:24",
"finishTime":"2023-07-12T22:36:04",
"progress":100,
"stepInfo":null
},
{
"order":6,
"name":"INCR_TRANSFER",
"description":"Incremental synchronization",
"status":"MONITORING",
"extraInfo":{
"errorDetails":null,
"errorCode":null,
"errorMsg":null,
"errorParam":null,
"failedTime":null
},
"startTime":"2023-07-12T22:41:20",
"finishTime":"2023-07-12T22:46:57",
"progress":99,
"stepInfo":{
"incrTimestampCheckpoint":1689213946,
"checkpointSampleTimestamp":1689214358,
"enableIncrStatistics":true
}
}
],
"extraInfo":null,
"alarmStats":{
"target":"np_4w1q****",
"alarming":false,
"recentlyTriggerCount":null,
"ruleToRecentlyTriggerCount":null,
"alarmContent":null
}
}
]
}
Appendix
Kafka
{
"connExtraAttributes":{
"endpoint":"xxx.xxx.xxx.1:2883",
"username":null,
"enableSSL":null,
"enableSASL":null,
"saslMechanism":null,
"disableIdentificationAlgorithm":false
}
}
| Parameter | Type | Description |
|---|---|---|
| endpoint | string | The service endpoint. |
| username | string | The username. This parameter is required when the authentication method is PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512. |
| enableSSL | boolean | Indicates whether SSL authentication is enabled. If you want to enable SSL authentication, you must upload an SSL certificate. |
| enableSASL | boolean | Indicates whether SASL authentication is enabled. |
| saslMechanism | KafkaSaslMechanisnEnum | The encryption method. Valid values: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, and KERBEROS. |
| disableIdentificationAlgorithm | boolean | Indicates whether hostname verification is enabled during SSL encryption. |
DataHub
{
"connExtraAttributes":{
"endpoint":"xxx.xxx.xxx.1:2883",
"project":"detx***"
}
}
| Parameter | Type | Description |
|---|---|---|
| endpoint | string | The service endpoint. |
| project | string | The name of the task. |
RocketMQ
{
"connExtraAttributes":{
"endpoint":"xxx.xxx.xxx.1:2883",
"endpointType":null,
"version":null
}
}
| Parameter | Type | Description |
|---|---|---|
| endpoint | string | The service endpoint. |
| endpointType | CloudRocketMQEndpoint.Type | The type of the service endpoint. |
| version | CloudRocketMQInst.Version | The version of RocketMQ. |
OceanBase Database
{
"connExtraAttributes":{
"cluster":"xgez***",
"tenant":"gexr**",
"isLogicSource":false,
"useLogProxy":true,
"drcUser":"tes***",
"configUrl":"http://xxx.xxx.xxx.1:2883/service****",
"logProxyIp":"xxx.xxx.xxx.2",
"logProxyPort":2883,
"noUserAuth":false
}
}
| Parameter | Type | Description |
|---|---|---|
| cluster | string | The name of the cluster. |
| tenant | string | The name of the tenant. |
| isLogicSource | boolean | Indicates whether the data source is a logical data source. |
| useLogProxy | boolean | Indicates whether the oblogproxy service is used. |
| drcUser | string | The configuration of drc_user. |
| configUrl | string | Indicates whether the oblogproxy service is used. |
| logProxyIp | string | The IP address of the oblogproxy service. |
| logProxyPort | integer | The port number of the oblogproxy service. |
| noUserAuth | boolean | When you synchronize data to DataWorks, OMS Community Edition automatically generates a data source. The username and password are required for connecting to OceanBase Database. |
TiDB
{
"connExtraAttributes":{
"kafkaId":"e_edz****",
"kafkaTopic":"test**",
"kafkaName":"denz***"
}
}
| Parameter | Type | Description |
|---|---|---|
| kafkaId | string | The ID of the bound Kafka data source. |
| kafkaTopic | string | The name of the bound Kafka topic. |
| kafkaName | string | The name of the bound Kafka data source. |