This topic describes the concepts and types of dense vector indexes in OceanBase Database. Below is the term "dense index".
A dense index is an efficient index type provided by OceanBase Database for dense vector data (vectors with most elements being non-zero). Dense vector indexes are mainly used to accelerate vector searches and support nearest neighbor search through distance functions (L2, inner product, cosine distance).
Index types
OceanBase Database provides the HNSW series and IVF series of dense indexes.
HNSW series
The HNSW series is a graph-based index that requires complete loading into memory. It supports DML operations and real-time search, and offers high search performance and recall rate.
| Index type | Description |
|---|---|
| HNSW | The maximum dimension of the indexed column is 4096. HNSW indexes are memory-based and require complete loading into memory. |
| HNSW_SQ | HNSW_SQ indexes provide similar construction speed, search performance, and recall rate as HNSW indexes, but reduce the total memory usage to 1/2 to 1/3 of the original. |
| HNSW_BQ | HNSW_BQ indexes have slightly lower recall rates than HNSW indexes but significantly reduce memory usage. The BQ quantization compression algorithm (Rabitq) can compress vectors to 1/32 of their original size. As the vector dimension increases, the memory optimization effect of HNSW_BQ indexes becomes more pronounced. |
For more information, see the relevant documents at the end of this topic.
IVF series
The IVF series is a disk-based index that does not require resident memory. It is suitable for scenarios with large data volumes and cost sensitivity.
| Index type | Description |
|---|---|
| IVF | The IVF index is implemented based on database tables and does not require resident memory. |
| IVF_PQ | The IVF_PQ index is implemented based on database tables and does not require resident memory. It applies the PQ quantization technique to the IVF index, resulting in slightly lower recall rates but higher performance than the IVF index. The PQ quantization compression algorithm can compress vectors to 1/16 to 1/32 of their original size in most scenarios. |
For more information, see the relevant documents at the end of this topic.
Syntax
Dense indexes support two creation methods: creating during table creation and creating after table creation. The search and deletion syntax are consistent with those of regular vector indexes. For specific syntax, parameters, search syntax, and examples for different index types, see:
- HNSW series indexes, which includes the creation, search, and deletion syntax and examples for HNSW, HNSW_SQ, and HNSW_BQ indexes.
- IVF series indexes, which includes the creation, search, and deletion syntax and examples for IVF and IVF_PQ indexes.
References
- For index selection recommendations, see Index type selection.
- Vector index memory management
- Monitor and maintain vector indexes