Memory leak diagnostics

This topic describes how to use the memory leak diagnostics feature.

Background information

OceanBase Database Proxy (ODP) of earlier versions records limited memory information. Once a memory leak occurs, it will be difficult to locate the leak location. Therefore, the memory leak diagnostics feature is introduced since ODP V4.2.3 to help solve memory leak issues.

Prerequisites

Before using the memory leak diagnostics feature, make sure that:

• You have deployed ODP V4.2.3 or later.

• The binary file obproxy in the ODP installation directory is of the debug edition. You can contact OceanBase Technical Support to obtain this file.

Procedure

1. Log on to OceanBase Database in proxy mode through ODP as the root@sys user or log on to ODP as the root@proxysys user.

   For more information, see View parameters in View and modify parameters.

2. Run the following commands to configure related parameters for the feature.

   Configure the mem_leak_check_mod_name parameter. We recommend that you set the parameter to ALL to locate memory leaks. For more information about the parameter, see mem_leak_check_mod_name.

   obclient> alter proxyconfig set mem_leak_check_mod_name='ALL';
   

   Configure the mem_leak_check_class_name parameter. We recommend that you leave the parameter empty to locate memory leaks. For more information about the parameter, see mem_leak_check_class_name.

   obclient> alter proxyconfig set mem_leak_check_class_name='';
   

3. Reproduce scenarios with high memory usage.

4. View the memory usage and memory allocation stack.

   obclient> show proxymemory 3\G
   

   If there is a memory allocation stack, the output is as follows:

   *************************** 10. row ***************************
                                                             mod_name: OB_CONCURRENCY_OBJ_POOL
                                                             mod_type: user
                                                                 hold:  34,552,064
                                                                 used:  33,573,536
                                                                count:  173
                                                             avg_used:  194,066
   hold bytes, alloc times and backtraces(addr2line -Cfe obproxy 0xXXXX):
   17097232 33
   0xf13db9 0xf8994e 0xf85e1b 0xf0973f 0xf25332 0xf26867 0xef75c2 0x10aa1ba 0x10b509a 0x1e71111 0x20ff1a2 0x1e5529d 0x140a655 0xec6ae0 0xec6157 0xec6976 0xec1fa7 0xeaccdf 0xea6201 0xea5ee1 0xea36fc 0xea196e 0xea0dfe 0xe9dbc9 0x20972b1 0x207c559 0x2086a4c 0xf077be 0x7fd3d417c555 0xef6fd5
   7176416 14
   0xf13db9 0xf8994e 0xf85e1b 0xf0973f 0xf25a75 0xf265f8 0x2281b1e 0x2283de6 0x1dfefee 0x7fd3d55b88e1 0x1e0aed0 0x1e0b73d 0x209e3fa 0x207c546 0x2086a4c 0xf077be 0x7fd3d417c555 0xef6fd5
   1319264 22
   0xf13db9 0xf8994e 0xf85e1b 0xf0973f 0xf25332 0xf26867 0x10b55fa 0x1e71111 0x20ff1a2 0x1e5529d 0x140a655 0xec6ae0 0xec6157 0xec6976 0xec1fa7 0xeaccdf 0xea6201 0xea5ee1 0xea36fc 0xea196e 0xea0dfe 0xe9dbc9 0x20972b1 0x207c559 0x2086a4c 0xf077be 0x7fd3d417c555 0xef6fd5
   

5. In the ODP installation directory, run the addr2line command to check the memory allocation location, and then determine the cause of the memory leak.

   [admin@obtest1 bin]$ addr2line -Cfe obproxy 0xf13db9 0xf8994e 0xf85e1b 0xf0973f 0xf25332 0xf26867 0xef75c2 0x10aa1ba 0x10b509a 0x1e71111 0x20ff1a2 0x1e5529d 0x140a655 0xec6ae0 0xec6157 0xec6976 0xec1fa7 0xeaccdf 0xea6201 0xea5ee1 0xea36fc 0xea196e 0xea0dfe 0xe9dbc9 0x20972b1 0x207c559 0x2086a4c 0xf077be 0x7fd3d417c555 0xef6fd5
   

   Note

   • When the memory usage reaches the warning threshold (proxy_mem_limited × 80%), ODP will print top 10 memory consumers to obproxy.log. You can query them by using the MEMORY LEAK CHECK (START) keyword.

   • When the memory usage reaches 90% of the upper limit (proxy_mem_limited × 90%), ODP will disable the memory leak diagnostics feature and clean up the memory allocation information stored internally to save memory.

References

For more information about the proxy_mem_limited parameter, see proxy_mem_limited.