Linkis is an open-source distributed computing middleware that offers unified management of computing resources, task scheduling, and execution services. Linkis creates an intermediary layer that allows different data processing applications (such as Apache Spark, Hadoop, and TensorFlow) to interact with data storage and computing resources.
Linkis V3.1.2 supports the OceanBase database. This article introduces how to create OceanBase database tasks in Linkis. OceanBase database is compatible with most of the features and syntax of MySQL 5.7. Therefore, you can use OceanBase Database as if it were MySQL.
Prerequisites
Before integrating OceanBase Database, make sure that you have installed and deployed Linkis. For more information, see Linkis Deployment Guide.
Submit tasks using shell
sh ./bin/linkis-cli -engineType jdbc-4 -codeType jdbc -code "show tables" -submitUser hadoop -proxyUser hadoop -runtimeMap wds.linkis.jdbc.connect.url=jdbc:mysql://${ip}:${port}/${db_name} -runtimeMap wds.linkis.jdbc.driver=com.mysql.jdbc.Driver -runtimeMap wds.linkis.jdbc.username=${username} -runtimeMap wds.linkis.jdbc.password=${password}
Submit tasks using Linkis SDK
Linkis provides SDKs for Java and Scala to submit tasks to the Linkis server. For more information, see Java SDK Guide. For OceanBase tasks, you only need to modify the EngineConnType and CodeType parameters in the Demo :
Map<String, Object> labels = new HashMap<String, Object>();
labels.put(LabelKeyConstant.ENGINE_TYPE_KEY, "jdbc-4"); // required engineType Label
labels.put(LabelKeyConstant.USER_CREATOR_TYPE_KEY, "hadoop-IDE");// required execute user and creator
labels.put(LabelKeyConstant.CODE_TYPE_KEY, "jdbc"); // required codeType
Submit OceanBase tasks through the Linkis console
Add OceanBase Data Source
Log in to the Linkis console.
In the left navigation bar, click on Data Source.
In Data Source page, click New in the upper right corner.
In New Data Source form, select MySQL from Type.
Configure parameters.
Parameter Description name Data source identifier Host IP address of the OceanBase tenant Port Port of the OceanBase tenant Username Username of the OceanBase tenant Password Password of the OceanBase tenant Click Test Connection, Verify the network connection between Linkis and the database, as well as the validity of the username and password.
After the connection test is passed, click Finish.
Publish Data Source.
In Data Source Page, click OceanBase version, in the edit version pop-up window, click Publish.
Submit OceanBase Tasks
Submit an OceanBase task by specifying the data source name.
Request URL: http://${gateway_url}:${port}/api/rest_j/v1/entrance/submit
Method: POST
Parameter:
{
"executionContent": {
"code": "show databases",
"runType": "jdbc"
},
"params": {
"variable": {},
"configuration": {
"startup": {},
"runtime": {
"wds.linkis.engine.runtime.datasource": "ob-test"
}
}
},
"labels": {
"engineType": "jdbc-4"
}
}
Response:
{
"method": "/api/entrance/submit",
"status": 0,
"message": "OK",
"data": {
"taskID": 93,
"execID": "exec_id018017linkis-cg-entrance000830fb1364:9104IDE_hadoop_jdbc_0"
}
}
Reference
For more information, see following documents:
