You can tune the performance of OBLOADER from three aspects: command line options, virtual machine memory, and database kernel.
Command line options-based performance tuning
Decrease the value of the --batch option when you import wide tables or tables with columns that contain long values. Default value of the --batch option: 200.
Increase the value of the --batch option when you import narrow tables or tables with columns that contain short values. Default value of --batch: 200.
You can adjust the value of the --thread option when the server load and network workload are relatively low.
Note
During performance tuning, take into account the resource utilization on the server where OBLOADER run and on the nodes in the OBServer cluster, and the network condition on the node where OBProxy runs.
VM memory-based performance tuning
Modify the VM memory parameter in the script. We recommend that you set the value of this parameter to 60% the size of the available physical memory when you import a large amount of data.
The following table describes the VM memory parameter:
| Parameter | Default value |
|---|---|
| -Xms4G -Xmx4G | 4G |
Database kernel-based performance tuning
You can tune the performance of OBLOADER by adjusting the following database kernel parameters:
| Option | Default value | Description |
|---|---|---|
| set global ob_sql_work_area_percentage=30; | 5 | The memory utilization during the execution of SQL statements. |
| alter system set freeze_trigger_percentage=30; | 70 | major_freeze_trigger_percent = major_freeze trigger threshold/Memstore capacity. The memstore capacity is calculated based on the value of memstore_lmt_percent by using the following formula: memstore_lmt_percent=memstore_limit/min_memory. Value range: (0, 100). |
| alter system set enable_syslog_recycle='True'; | False | Specifies whether to recycle old system logs. This parameter takes effect when max_syslog_file_count is specified. |
| alter system set max_syslog_file_count=100; | 0 | Specifies the maximum number of system log files that can exist at the same time. Each system log file is 256 MB in size. If you set this parameter to 0, system log files will not be deleted. Value range: [0, + ∞). |
| alter system set minor_freeze_times=5;alter system set minor_freeze_times=500; (when the tenant has sufficient memory) | 3 | If the memory utilization reaches a predefined threshold, a minor freeze or a major freeze will be triggered. This parameter specifies the number of minor freezes triggered between two consecutive major freezes. If you set this parameter to 0, automatic triggering of minor freezes is disabled. Value range:[0, 65536). |
| alter system set minor_compact_trigger=5; | Specifies the number of mini SSTables in a layer that will trigger the compaction of all the mini SSTables in the current layer to new SSTables in the next layer. | |
| alter system set merge_thread_count=45; | 0 | Specifies the number of worker threads for daily major compactions. When the default value is used, the number of threads for major compaction is MIN[ No. of logical CPUs * 30%, 10]. This parameter takes effect immediately after modification and does not require a restart. Value range: [0, 64]. |
| alter system set minor_merge_concurrency=20; | 0 | The number of concurrent threads for a minor compaction. |
| alter system set writting_throttling_trigger_percentage=85; | 10 | Specifies the threshold of server memory usage that will trigger write throttling. OBServer supports this parameter in V2.2.30 and later versions. Therefore, the tool also needs to have the memory explosion prevention capability. |