Read OceanBase data by using Maxwell

Maxwell reads and converts MySQL binlogs into the JavaScript Object Notation (JSON) format. It is used to capture all data change events in a database. This topic describes how to use Maxwell to export data of a MySQL tenant of OceanBase Database in the JSON format.

Prerequisites

Before you use Maxwell, make sure that:

• You have installed OBLogProxy.
• You have installed Java 1.8 or later.
• You have downloaded Maxwell. For more information, see the description of Maxwell at GitHub.

Procedure

1. Configure Maxwell.

   Edit the config.properties file in the working directory of Maxwell. If this file does not exist, copy config.properties.example, rename it into config.properties, and edit the file.

   user=maxwell
   password=yourpassword
   host=Server IP address of the source MySQL tenant in OceanBase Database
   producer=stdout
   

   stdout indicates the standard output mode. With this configuration, Maxwell directly exports data to the console. To enable Maxwell to export data to other targets such as Kafka, change the value of producer and related configurations.

2. Start Maxwell.

   Run the following command to start Maxwell:

   bin/maxwell --config config.properties
   

3. View data changes.

   1. Perform data changes operations, for example, insert, update, or delete data in OceanBase Database.
   2. Observe the JSON data exported by Maxwell.
   3. Set the output mode of Maxwell to stdout.

   The possible return results are as follows:

    {
    "database": "mydb",
    "table": "mytable",
    "type": "insert",
    "ts": 1589372847,
    "xid": 12345678,
    "commit": true,
    "data": {
        "id": 1,
        "column1": "value1",
        "column2": "value2"
    }
    }
   

   The return results show that data was inserted into the mydb.mytable table.

In the following example, the latin1 character set is deleted from the CREATE TABLE statement of Maxwell.

CREATE TABLE IF NOT EXISTS `databases` (
  id        bigint auto_increment NOT NULL primary key,
  schema_id bigint,
  name      varchar(255) charset 'utf8',
  charset   varchar(255),
  index (schema_id)
);

This example only describes how to operate a single metadata table. You need to perform the same operations on other metadata tables.