You must configure data sources before you create a data migration task. To migrate data to a Kafka instance, you must create the Kafka instance as the target. This topic describes how to create a Kafka data source in OceanBase Migration Service (OMS) Community Edition.
Limitations
OMS Community Edition allows you to create only a Kafka instance as the target for data synchronization.
Prerequisites
You have obtained the endpoint of the self-managed Kafka instance.
Background
Data security is a major concern throughout service connectivity verification, link creation, and data transmission. Based on the security system provided by the Kafka service, OMS Community Edition can meet most security requirements in data encryption and user authentication.
Kafka authentication methods supported by OMS Community Edition:
GSSAPI
GSSAPI is short for Generic Security Services Application Program Interface. GSSAPI is a framework that provides general security services to callers. It supports the Kerberos protocol.
PLAIN
PLAIN authentication is simple and does not support dynamic change of users. In this authentication mode, usernames and passwords are configured in plaintext, resulting in low security.
SCRAM-SHA-256
Salted Challenge Response Authentication Mechanism (SCRAM) authenticates users by username and password. SCRAM-SHA-256 can be used together with Transport Layer Security (TLS) for security authentication.
In this authentication method, users can be dynamically changed, and the user data is stored in Zookeeper. Before a broker is started, a user for communication between the broker and Zookeeper must be created. However, usernames and passwords are configured in plaintext in this authentication method.
SCRAM-SHA-512
SCRAM-SHA-512 can be used together with TLS for security authentication.
Procedure
Log in to the OMS Community Edition console.
In the left-side navigation pane, click Data Source Management .
On the Data Sources page, click New Data Source in the upper-right corner.
In the New Data Source dialog box, select Kafka for Data Source Type.
Specify the following parameters.
| Parameter | Description |
|---|---|
| Data Source Identifier | We recommend that you use Chinese, English, and numbers to name the data source. It must not contain any spaces and cannot exceed 32 characters in length.
NoticeThe data source name is globally unique in OMS Community Edition. |
| Region | Select the region where the data source resides from the drop-down list. The region is the value that you set for the cm_region parameter when you deployed OMS Community Edition.
Notice
|
| Kafka Endpoint | The endpoint of the self-managed Kafka cluster, which is in the IP address:port number format. The Kafka data source supports configuring information for multiple nodes, with Broker addresses separated by commas (,), for example, xxx.xxx.xxx.1:2883,xxx.xxx.xxx.2:2883. |
| Enable SSL | Choose whether to enable SSL based on the business needs. To enable SSL, click Upload File and upload an SSL certificate suffixed with .jks. |
| Enable Authentication | Choose whether to enable authentication based on business needs. Kafka provides data encryption and multiple identity authentication mechanisms to ensure the security of user data and services. |
| Authentication Method | If authentication is enabled, you must specify the authentication method. OMS Community Edition supports the following authentication methods: GSSAPI, PLAIN, SCRAM-SHA-256, and SCRAM-SHA-512. |
| KDC Server Address | The IP address or domain name of the Kerberos Key Distribution Center (KDC) server. Note: This parameter is displayed only when you set Authentication Method to GSSAPI. |
| User Entity | Enter the username. Note: This parameter is displayed only when you set Authentication Method to GSSAPI. |
| Service Name | The value of sasl.kerberos.service.name on the Kafka server, which is the name of the Kafka service principal configured in Kerberos. |
| keytab file | Click Upload File and upload a key file suffixed with .keytab. Note: This parameter is displayed only when you set Authentication Method to GSSAPI. |
| Username | The username of the account used for data migration or synchronization. Note: This parameter is not displayed when you set Authentication Method to GSSAPI. |
| Password | The password of the account used for data migration or synchronization. Note: This parameter is not displayed when you set Authentication Method to GSSAPI. |
| Remarks (optional) | Additional information about the data source. |
Click Test Connection to test the network connection between OMS Community Edition and the self-managed Kafka instance.
After the connection test succeeds, click OK.