INDEX_VECTOR_MEMORY_ESTIMATE

The INDEX_VECTOR_MEMORY_ESTIMATE function is used to analyze the memory usage of vector indexes in the table.

Syntax

FUNCTION INDEX_VECTOR_MEMORY_ESTIMATE (
    IN     TABLE_NAME        VARCHAR(65535),
    IN     COLUMN_NAME       VARCHAR(65535),
    IN     IDX_TYPE          VARCHAR(65535),
    IN     IDX_PARAMETERS    LONGTEXT DEFAULT NULL
);

The function does the following:

• The INDEX_VECTOR_MEMORY_ESTIMATE function reads the dimension and data type information of the vector column based on table_name and column_name. It then calculates the total number of vectors and the maximum number of vectors in each partition based on the statistics. The memory calculation process is consistent with that of the INDEX_VECTOR_MEMORY_ADVISOR function.
• For sparse vectors, this function randomly samples 100 vectors from the table and calculates the average number of non-zero dimensions among these 100 vectors as the average length of sparse vectors. It then estimates the memory usage based on this average length.

Parameters

The returned value is a string that describes the estimated memory usage.

Examples

1. Create a table named t1 and insert sample data.

   CREATE TABLE t1 (id int auto_increment, c1 vector(3));
   INSERT INTO t1(c1) SELECT array(1/UNIFORM(1,1000,RANDOM(1)), 1/UNIFORM(1,1000,RANDOM(2)) , 1/UNIFORM(1,1000,RANDOM(3))) FROM table(generator(10000));
   

2. Update the statistics information of t1.

   ANALYZE TABLE t1;
   

3. Query the memory usage estimation result of the vector index of t1.

   -- Query the memory usage estimation result of the HNSW index.
   SELECT dbms_vector.index_vector_memory_estimate('t1','c1','HNSW','M=10,TYPE=HNSW');
   +-----------------------------------------------------------------------------+
   | dbms_vector.index_vector_memory_estimate('t1','c1','HNSW','M=10,TYPE=HNSW') |
   +-----------------------------------------------------------------------------+
   | Suggested minimum vector memory is 11.5 MB                                  |
   +-----------------------------------------------------------------------------+
   1 row in set
   
   -- Query the memory usage estimation result of the IVF index.
   SELECT dbms_vector.index_vector_memory_estimate('t1','c1','IVF_FLAT','SAMPLE_PER_NLIST=100');
   +---------------------------------------------------------------------------------------------------------+
   | dbms_vector.index_vector_memory_estimate('t1','c1','IVF_FLAT','SAMPLE_PER_NLIST=100')                   |
   +---------------------------------------------------------------------------------------------------------+
   | Suggested minimum vector memory is 151.5 KB, memory consumption when providing search service is 1.5 KB |
   +---------------------------------------------------------------------------------------------------------+
   1 row in set
   
   -- Query the memory usage estimation result of the SINDI index.
   SELECT dbms_vector.index_vector_memory_estimate('t1','c1','SINDI','TYPE=SINDI,LIB=VSAG,DISTANCE=INNER_PRODUCT');
   +----------------------------------------------------------------------------------------------------------+
   | dbms_vector.index_vector_memory_estimate('t1','c1','SINDI','TYPE=SINDI,LIB=VSAG,DISTANCE=INNER_PRODUCT') |
   +----------------------------------------------------------------------------------------------------------+
   | Suggested minimum vector memory is 73.6 MB                                                               |
   +----------------------------------------------------------------------------------------------------------+
   1 row in set