FAQ

Question 1: What do I do if the memory usage of a tenant exceeds the upper limit?

The error message is as follows:

ERROR 4030 (HY000): OB-4030:Over tenant memory limits

If you receive the preceding error message, check whether the MemStore usage exceeds the upper limit. If yes, check whether an excessive amount of data was written to the MemStore and whether throttling is disabled. This error message is returned if a large amount of data is written into the MemStore and the minor compaction speed does not keep up with the data write speed. Execute the following statement in the sys tenant to check the memory usage:

obclient> SELECT /*+ READ_CONSISTENCY(WEAK),query_timeout(100000000) */ TENANT_ID,IP,
           round(ACTIVE/1024/1024/1024,2)ACTIVE_GB,
           round(TOTAL/1024/1024/1024,2) TOTAL_GB,
           round(FREEZE_TRIGGER/1024/1024/1024,2) FREEZE_TRIGGER_GB,
           round(TOTAL/FREEZE_TRIGGER*100,2) percent_trigger,
           round(MEM_LIMIT/1024/1024/1024,2) MEM_LIMIT_GB
FROM oceanbase.gv$memstore
WHERE tenant_id >1000 OR TENANT_ID=1
ORDER BY tenant_id,TOTAL_GB DESC;

To resolve this issue at the earliest opportunity, increase the tenant memory. For more information, contact OceanBase Technical Support. After the issue is resolved, analyze the root cause. If the issue is caused because throttling is not implemented, implement throttling and roll back the operation that increases the tenant memory. If the issue is caused because the tenant memory fails to support business growth, set a proper memory size for the tenant based on the minor compaction frequency. If the MemStore memory does not exceed the upper limit, execute the following statement to check which memory module exceeds the upper limit:

obclient> SELECT tenant_id,svr_ip,sum(hold) module_sum
FROM oceanbase.__all_virtual_memory_info
WHERE tenant_id>1000 AND hold<>0 AND
             mod_name NOT IN ( 'OB_KVSTORE_CACHE','OB_MEMSTORE')
GROUP BY tenant_id,svr_ip;

If the value of module_sum of a memory module is greater than the difference between the value of min_memory of the tenant and the size of the MemStore of the tenant, it is considered an overrun has occurred on the memory module. If an overrun has occurred on a memory module, the memory for the individual memory modules needs to be adjusted, such as modifying the ob_sql_work_area_percentage parameter. If the memory size of the tenant is too small, increase the memory size.

Question 2: What do I do if the hit rate of the plan cache is lower than 90%?

The hit rate of the plan cache must reach at least 90% in an online transaction processing (OLTP) system. You can execute the following statements to view the hit rate of the plan cache:

obclient> SELECT hit_count,executions,(hit_count/executions) as hit_ratio
FROM oceanbase.v$plan_cache_plan_stat
WHERE (hit_count/executions) < 0.9;

obclient> SELECT hit_count,executions,(hit_count/executions) AS hit_ratio
FROM oceanbase.v$plan_cache_plan_stat
WHERE (hit_count/executions) < 0.9 AND executions > 1000;

Check for similar statements, such as statements with a random number of parameters following in or not in, that consume a lot of memory unnecessarily. If such statements do not exist, the issue may occur when a sudden increase in traffic or sessions results in insufficient memory. In this case, you need to increase the memory of the tenant.

Question 3: What do I do if logs contain "fail to allocate memory" or "allocate memory fail"?

The error message contains the tenant_id (tenant ID) and mod_id (memory module ID) information, with which you can obtain the information about the specific memory module by executing the following statement:

obclient> SELECT * FROM oceanbase.__all_virtual_memory_info WHERE mod_id=xxx AND tenant_id = xxx

If a memory overrun occurs in the memory module, you must adjust the memory size for the module. If the memory of the tenant is insufficient, increase the memory of the tenant. For more information, contact OceanBase Technical Support.

Question 4: What do I do if the memory of the sys500 tenant exceeds the upper limit?

The tenant whose ID is 500 is an internal tenant of OceanBase Database, also known as the sys500 tenant. In normal cases, this error may be reported multiple times. The most possible reason is that the server has used up its system memory or you cannot increase the memory of the sys500 tenant because the remaining memory of the server is already pre-allocated.

To resolve this issue, you must check the memory usage of the sys500 tenant and the server. The memory usage of the sys500 tenant is not recorded by the v$memory and gv$memory fields. Therefore, you must query the __all_virtual_memory_info table. If the server has used up its system memory, check whether this is an occasional situation. If yes and it has no impact on your business, you can wait for the system to release memory. If your business is affected, or the normal activities of system processes lead to memory occupation or pre-allocation, you can increase the memory. For more information, contact OceanBase Technical Support.

SELECT * FROM oceanbase.__all_virtual_memory_info WHERE tenant_id=500;
// Check the system memory usage.