Store performance diagnostics

The Store component of OceanBase Migration Service (OceanBase Migration Service,OMS) provides the performance diagnostics feature. You can use the performance diagnostic tool to tune the component parameters and improve the troubleshooting efficiency. This topic describes how to use the performance diagnostic tool for performance tuning.

Limitations

• The Store performance diagnostic tool supports oboms-xlog-store V5.5.0 and later.

• To use the performance diagnostic tool, the OMS container where the Store component runs must contain the store-command.jar and store_utils.sh files.

• At present, the performance diagnostic tool supports diagnostics of only Oracle Store.

Procedure

1. Log on to the OMS server where the Store component runs.

2. Run the following command to enter the Docker container.

   docker exec -it ${CONTAINER_NAME} bash
   

3. Verify that the container contains the store-command.jar and store_utils.sh files.

   • Go to the /home/ds/store/store{port}/kafka/libs/ directory and verify that the store-command.jar file exists.

   • Go to the /home/ds/store/store{port}/kafka/bin/ directory and verify that the store_utils.sh file exists.

   If the store-command.jar or store_utils.sh file does not exist, update the Store component. For more information, see How do I upgrade Store?.

4. Run the following command to perform performance diagnostics:

   ./home/ds/store/store{port}/kafka/bin/store_utils.sh perf_diagnose -t 'oracle' -p '<port>' -s 'YYYY-MM-DDTHH:mm:ss' -e 'YYYY-MM-DDTHH:mm:ss' -f '<json>' -c '<config_address>'
   

   Option	Required?	Description
   -t	Yes	The type of the database. Only Oracle is supported.
   -p	No	The port of the Store instance. This parameter is required in an online environment.
   -s	No	The start time of log analysis. The default value is 10 minutes before the current time. Specify the timestamp in the 'YYYY-MM-DDTHH:mm:ss' format, such as '2023-06-01T12:00:00'.
   -e	No	The end time of log analysis. The default value is the current time. Specify the timestamp in the 'YYYY-MM-DDTHH:mm:ss' format, such as '2023-06-01T13:00:00'.
   -f	No	The output format of the return result. Default value: JSON. NORMAL specifies to return the result by line. JSON specifies to return the result in JSON format.
   -c	No	The path of the configuration file.

5. Modify the parameters based on the return result.

   Note

   • In the return result, you can view the prompt about the impact of parameter modifications. We recommend that you evaluate the impact before you modify a parameter.

   • The performance bottlenecks vary over time. Therefore, parameter modification based on the suggestions may not solve all issues. You can collect performance metrics from time to time and use the performance diagnostic tool to perform diagnostics.

   1. Log on to the OMS console.

   2. In the left-side navigation pane, choose O&M and Monitoring > Components. The Store page automatically appears.

   3. On the Store page, click Update in the Actions column of the target Store task.

   4. In the Update Configurations dialog box, add a parameter or update the configurations based on the suggestions in the diagnostic results.

   5. Click OK.

      After the update is completed, the Store task automatically restarts and runs with the updated parameters.

Sample return result

{
  "metricResult": {
    "fetcher stage": {
      "Queue size": "20000.0",
      "RPS": "87721.0",
      "Redo size avg": "1113.0",
      "Redo size max": "2214.0"
    },
    "analyse stage": {
      "Queue size": "20000.0",
      "RPS": "38534.0",
      "Entry RPS": "39008.0"
    },
    "aggregator stage": {
      "Queue size": "20000.0",
      "Local store size": "0",
      "Disk transaction num": "0"
    },
    "converter stage": {
      "Queue size": "20000.0",
      "RPS": "29093.0",
      "TPS": "9861.0"
    },
    "after select stage": {
      "Queue size": "3.0",
      "RPS": "9373.0"
    },
    "JVM": {
      "TOO_OFTEN_FULL_GC": "false",
      "TOO_OFTEN_YOUNG_GC": "false",
      "MAX_HEAP_MEMORY": "10881"
    }
  },
  "suggestions": [{
    "key": "selector_thread_num",
    "value": "2",
    "comment": "how much threads to back select log",
    "changeValue": "3",
    "affect": "use more thread to back select log"
  }],
  "modifiedTimestamp": 1678346615422
}

The preceding sample return result provides information in the metricResult and suggestions sections.

• The metricResult section displays information such as the performance metrics of each stage to help with manual troubleshooting. In most cases, you can skip this section.

• The suggestions section provides modification suggestions based on the diagnostic results.

  Parameter	Description
  key	The name of a parameter that can be modified. In general, the parameter name is prefixed with deliver2store.logminer..
  value	The current value of the parameter.
  comment	The description of the parameter.
  changeValue	The recommended value of the parameter.
  affect	The possible impact after the parameter value is modified. We recommend that you carefully evaluate the impact before you change the parameter value.

The preceding sample return result indicates that the performance of the log flashback query is low. Therefore, the performance diagnostic tool recommends that you increase the number of threads for the log flashback query to process more tasks.