Integrate Linkis with OceanBase Database

Linkis is an open-source distributed computing middleware service that allows you to manage computing resources and schedule and execute tasks in a unified manner. Linkis provides an intermediate layer for data processing applications such as Apache Spark, Hadoop, and TensorFlow to interact with data storage and computing resources.

Linkis V3.1.2 supports OceanBase Database. This topic describes how to submit a task to OceanBase Database from Linkis to implement the integration between Linkis and OceanBase Database. OceanBase Database is compatible with most features and statements of MySQL 5.7. You can use an OceanBase database as a MySQL database.

Prerequisites

Before you integrate Linkis with OceanBase Database, make sure that the following conditions are met:

• You have deployed OceanBase Database. For more information, see Quick experience.
• You have deployed Linkis. For more information, see Quick Deployment.

Submit tasks by using shell

 sh ./bin/linkis-cli -engineType jdbc-4 -codeType jdbc -code "show tables" -submitUser hadoop -proxyUser hadoop -runtimeMap wds.linkis.jdbc.connect.url=jdbc:mysql://${ip}:${port}/${db_name} -runtimeMap wds.linkis.jdbc.driver=com.mysql.jdbc.Driver -runtimeMap wds.linkis.jdbc.username=${username} -runtimeMap wds.linkis.jdbc.password=${password}

Submit tasks by using Linkis SDKs

Linkis provides SDKs for Java and Scala to submit tasks to the Linkis server. For more information, see Linkis SDK Manual. For OceanBase tasks, you need to modify only the EngineConnType and CodeType parameters in Demo:

Map<String, Object> labels = new HashMap<String, Object>();
labels.put(LabelKeyConstant.ENGINE_TYPE_KEY, "jdbc-4"); // required engineType Label
labels.put(LabelKeyConstant.USER_CREATOR_TYPE_KEY, "hadoop-IDE");// required execute user and creator
labels.put(LabelKeyConstant.CODE_TYPE_KEY, "jdbc"); // required codeType

Submit OceanBase tasks through the Linkis console

Add an OceanBase data source

1. Log in to the Linkis console.

2. In the left-side navigation pane, click DataSource Management.

3. On the DataSource Management page, click Add Source in the upper-right corner.

4. In the New Data Source dialog box, select MySQL for Data Source Type.

5. Configure the following parameters.

   Parameter	Description
   DataSource Name	The identifier of the data source.
   Host	The IP address of the host where OceanBase Database is deployed.
   Port	The port number of the host where OceanBase Database is deployed.
   Username	The username of the account for connecting to OceanBase Database.
   Password	The password of the account.

6. Click Test Connection to check whether Linkis can connect to OceanBase Database and whether the specified username and password are valid.

7. After the test is passed, click Complete.

8. Publish the data source.

   On the DataSource Management page, find the created OceanBase data source and click the version number in the Version column. In the EditVersion list dialog box, click Publish.

Submit an OceanBase task

Submit an OceanBase task by specifying the data source name.

Request URL: http://${gateway_url}:${port}/api/rest_j/v1/entrance/submit

Method: POST

Request parameters:

{
    "executionContent": {
        "code": "show databases",
        "runType": "jdbc"
    },
    "params": {
        "variable": {},
        "configuration": {
            "startup": {},
            "runtime": {
                "wds.linkis.engine.runtime.datasource": "ob-test"
            }
        }
    },
    "labels": {
        "engineType": "jdbc-4"
    }
}

Response:

{
    "method": "/api/entrance/submit",
    "status": 0,
    "message": "OK",
    "data": {
        "taskID": 93,
        "execID": "exec_id018017linkis-cg-entrance000830fb1364:9104IDE_hadoop_jdbc_0"
    }
}

References

For more information, see the following topics in the Linkis documentation:

• Linkis 1.3.2 Integration with OceanBase
• Data Source Manual