This topic introduces the concept and types of dense vector indexes in OceanBase Database. In the following sections, dense vector indexes are referred to as dense indexes.
A dense index is an efficient index type for dense vector data—vectors in which most elements are non-zero. Dense indexes are primarily used to accelerate vector searches and support nearest neighbor search using distance functions (L2, inner product, and cosine distance).
Index types
OceanBase Database provides two series of dense indexes: the HNSW series and the IVF series.
HNSW series
The HNSW series uses a graph-based index that resides in memory. It supports DML operations and real-time search, delivering high search performance and recall.
Notice
The HNSW_BQ index is supported starting from OceanBase Database V4.3.5 BP2.
| Index type | Description |
|---|---|
| HNSW | Supports up to 4096 dimensions for the index column. The HNSW index is memory-based and must be fully loaded into memory. |
| HNSW_SQ | Offers similar construction speed, search performance, and recall as the HNSW index, but reduces total memory usage to one-half or one-third of HNSW. |
| HNSW_BQ | Has slightly lower recall than HNSW, but significantly reduces memory usage. The BQ quantization compression algorithm (Rabitq) can compress vectors to 1/32 of their original size. As vector dimension increases, the memory optimization effect of HNSW_BQ becomes more pronounced. |
For more details, see the related topics at the end of this document.
IVF series
The IVF series is a disk-based index that does not require permanent memory residency. It is suitable for scenarios with large data volumes and cost sensitivity.
Notice
The IVF/IVF_PQ index is supported starting from OceanBase Database V4.3.5 BP1.
| Index type | Description |
|---|---|
| IVF | Implemented as a database table, the IVF index does not require permanent memory residency. |
| IVF_PQ | Also implemented as a database table and does not require permanent memory. Based on IVF, it applies PQ quantization technology. IVF_PQ has slightly lower recall than IVF, but offers higher performance. In general scenarios, the PQ quantization compression algorithm can compress vectors to 1/16 to 1/32 of their original size. |
For more details, see the related topics at the end of this document.
Syntax
Dense indexes can be created during table creation or after the table is created. The search and deletion syntax is the same as for regular vector indexes. For more information about creation syntax, parameter descriptions, search syntax, and examples for different index types, see:
- HNSW series indexes, which covers creation, search, and deletion syntax and provides examples for HNSW, HNSW_SQ, and HNSW_BQ.
- IVF series indexes, which covers creation, search, and deletion syntax and provides examples for IVF and IVF_PQ.
References
- For information about how to choose an index type, see Choose an index type. This topic applies only to OceanBase Database V4.3.5 BP3 and later.
- Manage memory for vector indexes
- Monitor and maintain vector indexes