Create a Kafka data source

You must configure data sources before you create a data migration project. To migrate data to a Kafka instance, you must create the Kafka instance as the destination data source. This topic describes how to create a Kafka data source in OceanBase Migration Service (OMS).

Limits

OMS allows you to create only a Kafka instance as the destination for data synchronization.

Prerequisites

You have obtained the endpoint of the self-managed Kafka instance.

Background

Data security is a major concern throughout service connectivity verification, link creation, and data transmission. Based on the security system provided by the Kafka service, OMS can meet most security requirements in data encryption and user authentication.

Kafka authentication methods supported by OMS:

• GSS-API

  GSS-API is short for Generic Security Services Application Program Interface. GSS-API is a framework that provides general security services to callers. It supports the Kerberos protocol.

• PLAIN

  PLAIN authentication is simple and does not support dynamic change of users. In this authentication mode, usernames and passwords are configured in plaintext, resulting in low security.

• SCRAM-SHA-256

  Salted Challenge Response Authentication Mechanism (SCRAM) authenticates users by username and password. SCRAM-SHA-256 can be used together with Transport Layer Security (TLS) for security authentication.

  In this authentication method, users can be dynamically changed, and the user data is stored in Zookeeper. Before a broker is started, a user for communication between the broker and Zookeeper must be created. However, usernames and passwords are configured in plaintext in this authentication method.

• SCRAM-SHA-512

  SCRAM-SHA-512 can be used together with TLS for security authentication.

Procedure

1. Log on to the OMS console.

2. In the left-side navigation pane, click Data Source Management .

3. On the Data Sources page, click New Data Source in the upper-right corner.

4. In the New Data Source dialog box, select Kafka for Data Source Type.

5. Specify the following parameters.

   Parameter	Description
   Data Source Identifier	The name of the data source. The name must be 1 to 32 characters in length and can contain Chinese characters, digits, and case-sensitive letters.
   Region	Select the region where the data source resides from the drop-down list. The region is the value you set for the cm_region parameter when you deploy OMS. Notice: This parameter is displayed only when multiple regions are available. Make sure that the regions specified for the data source and configured in OMS are the same. Otherwise, the migration and synchronization performance may be poor.
   Kafka Endpoint	The endpoint of the self-managed Kafka cluster, which is in the IP address:port number format.
   Enable SSL	Choose whether to enable SSL based on the business needs. To enable SSL, click Upload File and upload an SSL certificate suffixed with .jks.
   Enable Authentication	Choose whether to enable authentication based on business needs. Kafka provides data encryption and multiple identity authentication mechanisms to ensure the security of user data and services.
   Authentication Method	If authentication is enabled, you must specify the authentication method. OMS supports the following authentication methods: GSS-API, PLAIN, SCRAM-SHA-256, and SCRAM-SHA-512.
   KDC Server Address	The IP address or domain name of the Kerberos Key Distribution Center (KDC) server. This parameter is displayed only when you set Authentication Method to GSS-API.
   User Entity	Enter the username. This parameter is displayed only when you set Authentication Method to GSS-API.
   SSL Certificate	Click Upload File and upload a key file suffixed with .keytab. This parameter is displayed only when you set Authentication Method to GSS-API.
   Username	The username of the account used for data migration or synchronization. This parameter is not displayed when you set Authentication Method to GSS-API.
   Password	The password of the account used for data migration or synchronization. This parameter is not displayed when you set Authentication Method to GSS-API.
   Remarks (optional)	Additional information about the data source.

6. Click Test Connection to test the network connection between OMS and the self-managed Kafka instance.

7. After the connection test succeeds, click OK.