Integrate OceanBase vector search with Ollama

OceanBase OceanBase offers features such as vector storage, vector indexing, and embedding-based vector search. You can store vectorized data in OceanBase Database, making it available for fast and efficient search.

Ollama is an open-source tool that allows you to easily run and manage large language models (LLMs) such as GPT-oss, Gemma 3, DeepSeek-R1, and Qwen3 locally.

Prerequisites

• You have deployed OceanBase Database V4.4.0 or later, and created a MySQL-compatible tenant. After creating the tenant, continue with the steps below.

• Your environment includes an active MySQL-compatible tenant, a MySQL database, and a user account with read and write privileges.

• Python 3.11 or above is installed.

• Required dependencies are installed:

  python3 -m pip install pyobvector requests sqlalchemy ollama tqdm
  

• Make sure you have set the ob_vector_memory_limit_percentage parameter in your tenant to enable vector search. A recommended value is 30. For details on configuring this parameter, refer to ob_vector_memory_limit_percentage.

Step 1: Get your database connection information

Reach out to your OceanBase administrator or deployment team to obtain the database connection string, for example:

obclient -h$host -P$port -u$user_name -p$password -D$database_name

Parameters:

• $host: The IP address for connecting to OceanBase Database. If you are using OceanBase Database Proxy (ODP), use the ODP address. For direct connections, use the OBServer node IP.

• $port: The port number for connecting to OceanBase Database. The default for ODP is 2883 (can be customized during ODP deployment). For direct connections, the default is 2881 (customizable during OceanBase deployment).

• $database_name: The name of the database you want to access.

  Notice

  The user connecting to the tenant must have CREATE, INSERT, DROP, and SELECT privileges on the database. For more details on user privileges, see privilege types in MySQL-compatible mode.

• $user_name: The user account for connecting to the tenant. For ODP, common formats are username@tenant_name#cluster_name or cluster_name:tenant_name:username; for direct connections, use username@tenant_name.

• $password: The password for the account.

For more details about connection strings, see Connect to an OceanBase tenant using OBClient.

Step 2: Build your AI assistant

Set environment variables

Obtain the OceanBase Database connection information and configure it to the environment variables.

export OCEANBASE_DATABASE_URL=YOUR_OCEANBASE_DATABASE_URL
export OCEANBASE_DATABASE_USER=YOUR_OCEANBASE_DATABASE_USER
export OCEANBASE_DATABASE_DB_NAME=YOUR_OCEANBASE_DATABASE_DB_NAME
export OCEANBASE_DATABASE_PASSWORD=YOUR_OCEANBASE_DATABASE_PASSWORD

Install Ollama and pull the qwen3-embedding model

curl -fsSL https://ollama.ai/install.sh | sh
export PATH="/usr/local/bin:$PATH"
ollama --version
ollama pull qwen3-embedding:latest

Sample code snippets

Load data

In this example, the qwen3-embedding embedding model is used to load the vector search FAQs as private knowledge, and the content in the file is separated by "#".

import requests,os,json,ollama
from tqdm import tqdm

from sqlalchemy import Column, Integer, String
from pyobvector import ObVecClient, VECTOR, IndexParam, l2_distance

# text URL
url = "https://raw.githubusercontent.com/oceanbase/oceanbase-doc/refs/heads/V4.3.5/en-US/640.ob-vector-search/800.ob-vector-search-faq.md"
response = requests.get(url)
text_lines = []
file_text = response.text
text_lines += file_text.split("# ")

def emb_text(text):
    response = ollama.embeddings(model="qwen3-embedding", prompt=text)
    return response["embedding"]


data = []
for i, line in enumerate(tqdm(text_lines, desc="Creating embeddings")):
    data.append({"text": line, "embedding": emb_text(line)})

Define the vector table structure and store the vectors in OceanBase

Create a table named ollama_oceanbase_demo_documents that contains a text column for storing text, an embedding column for storing embedding vectors, and a vector index. Use the qwen3-embedding model to generate embedding vectors for each text segment and store them in OceanBase:

OCEANBASE_DATABASE_URL = os.getenv('OCEANBASE_DATABASE_URL')
OCEANBASE_DATABASE_USER = os.getenv('OCEANBASE_DATABASE_USER')
OCEANBASE_DATABASE_DB_NAME = os.getenv('OCEANBASE_DATABASE_DB_NAME')
OCEANBASE_DATABASE_PASSWORD = os.getenv('OCEANBASE_DATABASE_PASSWORD')

client = ObVecClient(uri=OCEANBASE_DATABASE_URL, user=OCEANBASE_DATABASE_USER,password=OCEANBASE_DATABASE_PASSWORD,db_name=OCEANBASE_DATABASE_DB_NAME)
table_name = "ollama_oceanbase_demo_documents"
client.drop_table_if_exist(table_name)

cols = [
    Column("id", Integer, primary_key=True, autoincrement=True),
    Column("text", String(255), nullable=False),
    Column("embedding", VECTOR(4096))
]

# Create vector index
vector_index_params = IndexParam(
    index_name="idx_question_embedding",
    field_name="embedding",  
    index_type="HNSW",
    distance_metric="l2"
)

client.create_table_with_index_params(
    table_name=table_name,
    columns=cols, 
    vidxs=[vector_index_params]
)

print('- Inserting Data to OceanBase...')
client.insert(table_name, data=data)

Perform semantic search

Use the qwen3-embedding model to generate an embedding vector for the query text. Then, search for the most relevant documents based on the l2 distance between the query text's embedding vector and each embedding vector in the vector table:

question = "Which mode supports vector search?"
search_res = client.ann_search(
    table_name,
    vec_data=emb_text(question),
    vec_column_name="embedding",
    distance_func=l2_distance,
    with_dist=True,
    topk=1, 
    output_column_names=["id","text"], 
)

print('- The Most Relevant Document and Its Distance to the Query:')
for row in search_res.fetchall():
    print(f'  - ID: {row[0]}\n'
          f'    content: {row[1]}\n'
          f'    distance: {row[2]}')

Expected result

- ID: 3
  content: Which mode supports vector search?

  Vector search is supported only in the MySQL-compatible mode of OceanBase Database.

  distance: 0.509979758468682