Integrate OceanBase Database with Linkis

Linkis is an open-source distributed computing middleware that provides unified computing resource management, task scheduling, and execution services. It creates an intermediary layer that allows different data processing applications, such as Apache Spark, Hadoop, and TensorFlow, to interact with data storage and computing resources.

Linkis V3.1.2 supports OceanBase Database. This topic introduces how to create OceanBase tasks in Linkis. OceanBase Database is compatible with most features and syntax of MySQL 5.7. Therefore, you can use OceanBase Database just like MySQL.

Prerequisites

Before integrating OceanBase Database with Links, make sure that:

• You have installed and deployed OceanBase Database. For more information, see Get started with OceanBase Database.
• You have installed and deployed Linkis. For more information, see Linkis deployment guide.

Submit tasks by using shell

 sh ./bin/linkis-cli -engineType jdbc-4 -codeType jdbc -code "show tables" -submitUser hadoop -proxyUser hadoop -runtimeMap wds.linkis.jdbc.connect.url=jdbc:mysql://${ip}:${port}/${db_name} -runtimeMap wds.linkis.jdbc.driver=com.mysql.jdbc.Driver -runtimeMap wds.linkis.jdbc.username=${username} -runtimeMap wds.linkis.jdbc.password=${password}

Submit tasks by using Linkis SDK

Linkis provides SDKs for Java and Scala to submit tasks to the Linkis server. For more information, see Linkis SDK Manual. For OceanBase tasks, you only need to modify the EngineConnType and CodeType parameters in the Demo file.

Map<String, Object> labels = new HashMap<String, Object>();
labels.put(LabelKeyConstant.ENGINE_TYPE_KEY, "jdbc-4"); // required engineType Label
labels.put(LabelKeyConstant.USER_CREATOR_TYPE_KEY, "hadoop-IDE");// required execute user and creator
labels.put(LabelKeyConstant.CODE_TYPE_KEY, "jdbc"); // required codeType

Submit OceanBase tasks by using the Linkis console

Add an OceanBase data source

1. Log on to the Linkis console.

2. In the left navigation bar, click on Data Source.

3. On the Data Source page, click New in the upper right corner.

4. In the New Data Source section, select MySQL from Type.

5. Configure the parameters.

   Parameter	Description
   Name	The data source identifier.
   Host	The IP address of the OceanBase tenant.
   Port	The port of the OceanBase tenant.
   Username	The username of the OceanBase tenant.
   Password	The password of the OceanBase tenant.

6. Click Test Connection to check the network connection between Linkis and the database, as well as the validity of the username and password.

7. After the connection test is passed, click Finish.

8. Publish the data source.

   On the Data Source page, click the OceanBase version. In the edit version pop-up window, click Publish.

Submit OceanBase tasks

You can submit an OceanBase task by specifying the data source name.

Request URL: http://${gateway_url}:${port}/api/rest_j/v1/entrance/submit

Method: POST

Parameters:

{
    "executionContent": {
        "code": "show databases",
        "runType": "jdbc"
    },
    "params": {
        "variable": {},
        "configuration": {
            "startup": {},
            "runtime": {
                "wds.linkis.engine.runtime.datasource": "ob-test"
            }
        }
    },
    "labels": {
        "engineType": "jdbc-4"
    }
}

Response:

{
    "method": "/api/entrance/submit",
    "status": 0,
    "message": "OK",
    "data": {
        "taskID": 93,
        "execID": "exec_id018017linkis-cg-entrance000830fb1364:9104IDE_hadoop_jdbc_0"
    }
}

References

For more information, see the following documents:

• Linkis 1.3.2 Integration with OceanBase
• Linkis Data Source Manual