Memory-related errors

2024-03-05 01:54:26  Updated

The following table describes two memory-related errors.

Internal error code Error message Description
-4013 No memory or reach tenant memory limit Insufficient workspace memory
-4030 Over tenant memory limits Insufficient tenant memory

Insufficient tenant memory

When the MemTable memory usage of a tenant reaches the specified threshold, no more memory can be allocated for applications to write or replay SQL statements, which results in the error of insufficient tenant memory.

You can view the MemTable memory usage of a tenant by querying the GV$OB_MEMSTORE view.

obclient> select * from GV$OB_MEMSTORE limit 1\G
*************************** 1. row ***************************
        SVR_IP: xx.xx.xx.xx
      SVR_PORT: 2882
     TENANT_ID: 1
   ACTIVE_SPAN: 0
FREEZE_TRIGGER: 4294967250
    FREEZE_CNT: 0
 MEMSTORE_USED: 0
MEMSTORE_LIMIT: 8589934550
1 row in set (0.01 sec)

The key fields are described as follows:

  • active_span: the range of memory used by active MEMTables, which is used to estimate the memory usage of active MEMTables.

  • freeze_trigger: the memory usage threshold at which a freeze is triggered.

  • memstore_used: the total amount of memory used by frozen MemTables and active MemTables.

  • memstore_limit: the maximum memory of the tenant available for MemTables.

The following situations can arise based on the preceding fields:

  • If the value of the memstore_used field approaches or exceeds that of the memstore_limit field, the memory of the tenant for MemTables is fully occupied.

  • If the value of the active_span field is greater than that of the freeze_trigger field, a freeze is triggered.

The memory for MemTables can be fully occupied in the following two cases:

  • Minor compactions of MemTables are slower than data writes. In this case, the memory for MemTables is automatically released a while after the client reports an error. To resolve this issue, you must first increase the memory size of the tenant. Then, you can find and solve the bottleneck that causes slow minor compactions to increase the minor compaction speed, or slow down data writing.

  • The memory for MemTables cannot be released after minor compactions. In this case, contact OceanBase Technical Support for diagnostics.

If the error 4030 is reported by applications even when the memory for MemTables is not fully occupied, it indicates that other modules have occupied parts of the memory. This issue is likely caused by memory leak or high memory usage by modules other than MemTables. To resolve this issue, you need to isolate the node for troubleshooting and then restart the OBServer node.

Insufficient workspace memory

The workspace memory is used by blocking operators such as SQL Sort, and is specified by the system variable ob_sql_work_area_percentage, whose default value is 5%. The maximum memory for a workspace is calculated by using the following formula: Tenant memory × ob_sql_work_area_percentage.

The error of insufficient workspace memory is often reported when a great number of concurrent requests are involved and each request occupies a large amount of workspace memory, such as when the Union, Sort, and Group By operators are used. To resolve this issue, you can increase the value of ob_sql_work_area_percentagevariable. For example:

obclient> set global ob_sql_work_area_percentage = 10;

Contact Us