Read data from OceanBase Database by using Maxwell

Maxwell reads and converts MySQL binlogs into the JSON format. It is used to capture all data change events in a database. This topic describes how to use Maxwell to export data of a MySQL tenant of OceanBase Database in the JSON format.

Prerequisites

Before you use Maxwell, make sure that the following conditions are met:

• You have installed oblogproxy.
• You have installed Java 1.8 or later.
• You have downloaded Maxwell. For more information, see the description of Maxwell at GitHub.

Procedure

1. Configure Maxwell. Edit the config.properties file in the working directory of Maxwell. If this file does not exist, copy config.properties.example, rename it to config.properties, and edit the file.

   user=maxwell
   password=yourpassword
   host=IP address of the server of the source MySQL tenant of OceanBase Database
   producer=stdout
   

   stdout indicates the standard output mode. With this configuration, Maxwell directly exports data to the console. To enable Maxwell to export data to other targets such as Kafka, change the value of producer and adjust related configurations.

2. Start Maxwell.

   Run the following command to start Maxwell:

   bin/maxwell --config config.properties
   

3. View data changes. Change data in OceanBase Database, for example, insert, update, or delete data. Then, observe the JSON data exported by Maxwell. The output mode of Maxwell is set to stdout in the configuration file in Step 1. Therefore, a sample return result is as follows:

    {
    "database": "mydb",
    "table": "mytable",
    "type": "insert",
    "ts": 1589372847,
    "xid": 12345678,
    "commit": true,
    "data": {
        "id": 1,
        "column1": "value1",
        "column2": "value2"
    }
    }
   

   The return result shows that data was inserted into the mydb.mytable table.

In the following example, the latin1 character set is deleted from the CREATE TABLE statement of Maxwell.

CREATE TABLE IF NOT EXISTS `databases` (
  id        bigint auto_increment NOT NULL primary key,
  schema_id bigint,
  name      varchar(255) charset 'utf8',
  charset   varchar(255),
  index (schema_id)
);

This example describes how to operate a single metadata table. You need to perform the same operations on other metadata tables.