Add a Store component

OceanBase Migration Service (OMS) allows you to add a Store component on the Store page of the OPS & Monitoring module.

Background information

The incremental fetch component (Store) reads raw data from the source instance, parses, filters, and formats the data, and then persists the data locally. If load balancing or other scenarios require a Store component to perform a pullback, we recommend that you add a new Store component. For example:

• Data in the database logs still exists, and data from several days ago needs to be pulled back.

  In this case, the current Store components for this data timestamp have expired. If you restart the current Store components, all downstream processes will be delayed. Therefore, you need to create a Store component that starts from the pullback timestamp.

• Load balancing needs to be performed for a subtopic, and the subtopic can be a data source at the instance, tenant, or database level.

  The purpose is to prevent excessive downstream consumption of Store component resources, which could affect stability and lead to large-scale delays.

Procedure

1. Log in to the OMS console.

2. In the left-side navigation pane, choose OPS & Monitoring > Component. The Store page automatically appears.

3. On the Store page, click Add in the upper-right corner.

4. In the Add dialog box, configure the parameters.

   Parameter	Required	Description
   Subtopic	Required	The instance- or tenant-level data source. Select a Subtopic as needed.
   Node	Optional	The server on which the Store component is to be created. OMS obtains all servers in the resource group of the creator of the selected Subtopic. The drop-down list displays the servers in the obtained server list. In single-node deployment, only one server is available, and the Store component is created on this server by default. If no server is specified when the high availability feature is enabled, the system automatically schedules the Store component to an available server that has not reached the performance bottleneck.
   Timestamp	Required	The timestamp from which database logs are to be pulled upon Store component startup. Make sure that database logs exist from the specified timestamp. Otherwise, the Store component startup will fail.

   You can check whether database logs exist as follows:

   • For MySQL databases, you can use the mysqlbinlog tool to view the binlog file at the source.

   • For Oracle databases, the log retention period depends on the archive retention period at the source. You can check whether an archive log exists within a specific date range.

     To check whether logs existed before a specific date, run the following command:

     SELECT * FROM V$ARCHIVED_LOG WHERE NAME IS NOT NULL AND NEXT_TIME < to_timestamp('Date','yyyy-MM-dd hh24:mi:ss') AND STATUS = 'A' AND DELETED = 'NO' AND ARCHIVED = 'YES' AND STANDBY_DEST = 'NO'
     

     To check whether logs existed after a specific date, run the following command:

     SELECT * FROM V$ARCHIVED_LOG WHERE NAME IS NOT NULL AND FIRST_TIME >= to_timestamp('Date','yyyy-MM-dd hh24:mi:ss') AND STATUS = 'A' AND DELETED = 'NO' AND ARCHIVED = 'YES' AND STANDBY_DEST = 'NO'
     

   • For PostgreSQL databases, OMS by default saves logs of the current time within 48 hours after the first successful Store component startup. If the first Store component startup fails, you cannot specify a timestamp earlier than the current time to pull logs.

     Note

     • If the first Store component startup was within 48 hours ago, you can specify a timestamp after the startup time to pull logs.

     • If the first Store component startup was more than 48 hours ago, you can specify a timestamp within the next 48 hours to pull logs.

5. Click Add.