OceanBase AI Database: Built on a Lakebase Architecture for the AI Era

Over the past few decades, enterprise data systems evolved along several major lines: OLTP databases for online transactions, data warehouses for analytics, and big data systems for large-scale processing. Yet most of them were built around the same assumption: applications, designed around human-defined workflows, would perform deterministic transactions and structured analytics.

That assumption is changing. AI agents are no longer limited to reading data and answering questions. They call tools, generate code, execute tasks, modify state, and participate in business processes. The users of databases are expanding from human-driven applications to large numbers of autonomous agents. This raises a fundamental question: when agents need to read, write, search, experiment, roll back, and continuously build context, should the database remain what it was?

I believe the answer is no. AI changes three things at once. First, who uses the database: from applications alone to applications and agents. Second, what the database manages: from mostly structured data to structured, semi-structured, unstructured, and multimodal data. Third, what workloads the database must support: from transactions and analytics to search, context engineering, AI computation, and agent experimentation.

An AI database is not a traditional database with a few AI functions bolted on. Nor is it a vector database with SQL capabilities retrofitted. It must solve the data infrastructure problem that emerges when AI enters production systems. multimodal data must be managed on a unified foundation. Online serving and offline compute must converge. Agents need real-time, trustworthy, continuous context. And all of this—reads, writes, experimentation, rollback, governance—must maintain database-grade consistency and reliability.

This is not a feature enhancement. It is a redesign of database architecture for the AI era.

Moving Toward Unified Data Platforms

Across the industry, different vendors are approaching this problem from different starting points. Databricks and Snowflake started from the lakehouse side and are steadily adding OLTP transaction capabilities. OceanBase and Oracle started from transactional databases and are expanding into OLAP and big data territory. MongoDB, Milvus, and Elasticsearch started as specialized systems and are becoming more general-purpose. Regardless of starting point, all these paths are converging toward a unified data platform capable of handling transactions, analytics, search, vectors, and AI compute simultaneously.

OceanBase has pursued a unified architecture from the beginning. We started with distributed OLTP, solving scalability and reliability for online transactions. We then added real-time OLAP support on top of OLTP, eliminating the data movement pipeline between transactional and analytical workloads. Last year, we shipped multimodal integration—bringing vector, full-text and JSON capabilities into the same database engine. Today, we are releasing lakebase integration: bringing the real-time transaction capabilities of a database and the open storage and open compute capabilities of a data lake into a single unified platform.

What a Unified Lakebase Architecture Means

A real Lakebase architecture cannot stop at bolting a data lake onto the side of a database. Nor can it merely add a few online query interfaces to a lakehouse. To be production-ready, it must merge three boundaries.

First, data boundaries must be unified. Structured data, semi-structured data, unstructured data, vectors, graphs, and full-text indexes cannot live in separate systems with separate management. They should be governed under a single table semantic.

Second, compute boundaries must be unified. SQL queries, real-time analytics, hybrid search, Spark ETL, and Ray-based AI compute should all operate on the same data—without constant export, transformation, and intermediate materialization.

Third, governance boundaries must be unified. Metadata, permissions, row-level controls, auditing, versioning, and lifecycle policies must apply consistently across all data types. If structured columns have access controls but vector retrieval bypasses them, that system cannot enter enterprise production.

This is the design philosophy behind OceanBase Lakebase:

At the bottom layer, Lakebase uses a disaggregated storage-compute architecture. Data resides on object storage; the compute layer runs independently. AI agent workloads are inherently bursty—any agent might see traffic spikes on any given day. Every day is potentially a mini-Double Eleven Day for some agent. Disaggregated storage-compute allows the compute layer to scale independently: instant scale-out when load surges, scale-to-zero when idle.

At the middle layer, Lakebase introduces multimodal tables to unify structured, semi-structured, unstructured, and multimodal data in a single table abstraction.

At the compute layer, Lakebase supports open compute. Beyond the native SQL engine for transactions, analytics, and search, it can work with engines such as Spark for ETL and Ray-based frameworks for AI processing. The goal is not to force every workload into one compute engine, but to let different engines operate on a shared data foundation.

The value of a lake lies in openness, elasticity, and cost. The value of a database lies in transactions, consistency, low latency, and governance. The AI era demands both sets of capabilities in one system.

But there's a key value that's easy to overlook: real-time availability. In the traditional approach, data processing is offline. Processed results must be moved back to online systems before they can serve applications—introducing T+1 or longer delays. A unified lakebase architecture unifies offline processing and online serving on the same data: the output of a Spark ETL job is immediately queryable by the SQL engine; vectors generated by model inference are immediately available for hybrid search. Real-time data access isn't achieved by accelerating data movement. It's achieved by eliminating data movement.

Multimodal Tables: The Core Data Structure of an AI Database

In traditional relational databases, the fundamental structure is a relational table—containing Int, Float, Varchar columns, all structured data.

In an AI database, the fundamental structure should be a multimodal table.

A multimodal table contains both traditional relational columns for structured data and multimodal columns plus AI columns for unstructured data. Unstructured data can be written into the multimodal table as vectors or text after external embedding or tagging, or written directly as LOBs (large objects). OceanBase supports highly flexible LOB storage: if the LOB is small, it's stored inline within the row to minimize I/O; if the LOB is large, it's chunked and stored in object storage with only chunk location metadata kept inline; if the LOB is very large, it supports referencing existing files in external object storage while the database stores only metadata. To the application layer, it still looks like a single table.

Why not use other open source lake format directly? Iceberg format can not support multi modal data, Lance format is not good for structured data and both of them need lots of additional work to achieve realtime and consistency.

On top of multimodal tables, we designed AI columns. Think of them as real-time computed columns on a table: when data is written, embedding, tagging, or other model computations are automatically triggered, and results are written back into the table. The critical property here is transactional consistency semantics. When a batch of audio files is written, either all of them complete embedding and tagging, or none of them do. Partial success is not acceptable.

Hybrid Search: Search as a First-Class Workload in AI Database

With multimodal tables in place, the next step is executing AI workloads on them. In an AI database, the fundamental query pattern evolves from relational lookup to hybrid search—completing relational filtering, full-text search, vector search, graph traversal, and AI computation within a single table.

Why isn't vector search alone sufficient? Vector search will certainly be the most common computation pattern in AI databases. But in real-world scenarios, we typically first apply relational filters to narrow the global dataset to a smaller candidate set (e.g., "only orders from the last 30 days"), then perform vector, full-text, and graph hybrid search on that candidate set. The database narrows the scope first; the model processes only high-value candidates—lower inference cost, more accurate results, more controllable pipelines.

In the AI era, search will return to the database as a native workload, alongside OLTP and OLAP.

On performance, we've conducted systematic benchmarks. Using the HNSW algorithm in 768-dimension and 1536-dimension test scenarios, OceanBase's vector search performance significantly outperforms Milvus, Elasticsearch, and pgvector at equivalent recall rates. On hybrid search, using the MS MARCO dataset, OceanBase's hybrid search performance exceeds Elasticsearch by over 30%. These aren't theoretical projections—they're reproducible results on standard datasets.

Open Compute and Unified Catalog

An AI data pipeline goes beyond SQL queries—it includes ETL processing, AI inference, and multimodal understanding. The traditional approach uses many separate systems: Kafka for ingestion, Flink for stream processing, Spark for batch processing, HDFS for persistence, ClickHouse for analytics, HBase for wide tables, Elasticsearch for search, and Presto for federated queries.

OceanBase Lakebase takes a different approach. The bottom layer uses multimodal tables on object storage to enable data sharing across compute engines—one copy of data simultaneously serves all compute engines. OceanBase's SQL engine handles online queries and transactions. Spark handles PB-scale batch ETL. Daft on Ray handles AI inference. This eliminates the data consistency gaps and compute latency that plague multi-system architectures.

Different compute engines do what they are good at, while sharing the same governed data foundation. This requires a unified, open Catalog to manage data. Tables, views, schemas, lineage, row-level permissions, and column-level authorization are all managed in one place. Every operation entering the OceanBase AI database passes through a unified metadata and access control plane—preventing unauthorized data access. We also support the finest-grained row-level security (RLS) controls.
In production AI systems, governance is not optional. It is part of the architecture.

Designed for Agents: Branching, Isolation and Elastic Scale

For agents to enter production systems, the database must provide more than read access. Agents need a place to experiment, write intermediate state, compare results, and roll back safely.

This is where Fork Database becomes important. Fork Database is designed to create lightweight branches of a database using copy-on-write, like pulling a branch in git. Even a PB-scale database can be forked in under a second, consuming only incremental space (Copy-on-Write: unmodified data blocks point to the same storage on the lake, no physical copies). After branching, agents can perform all kinds of AI development, testing, and experimentation on the branch. If the experiment succeeds, commit. If it fails, roll back—essentially zero cost.

Combined with DIFF and MERGE, agents gain complete data version control capabilities—Fork to create branches, DIFF to inspect changes (down to row and value level), MERGE to integrate back according to strategy. This isn't an analogy to Git. It's a native SQL-level implementation.

The other dimension is scale. The volume of agent data will be enormous—potentially billions or even trillions of agents running in parallel in the future, each with its own schema and tables. This creates a schema explosion problem. Traditional databases optimize for "few databases + massive data"—one cluster hosting dozens of databases, each with billions of rows. The agent scenario is the inverse: millions of agents, each with only hundreds of rows, but an astronomical number of instances.

OceanBase's logical table design gives each agent the appearance of independent logical tables, while the underlying storage is a single physical table. The logical abstraction layer solves schema explosion. Fork Database provides isolated environments; logical tables handle instance scale—together, they enable individual agents to safely experiment while massive numbers of agents run in parallel at low cost.

The Context Layer: Helping AI Understand Enterprise and Users

An AI database needs more than just an engine. Between the engine and applications, there's a missing layer—context.

The context layer has two components. Data context addresses data semantics and data governance, helping AI understand the enterprise. Application context addresses Memory and RAG, helping AI understand users.

On the memory dimension, an agent's memory cannot simply be a growing pile of context. It must be an evolvable, structured asset. For this we developed PowerMem, and its cloud-based product seekdb M0. PowerMem is built on top of the AI database—memory retrieval itself is a hybrid query combining structured filtering with semantic similarity. More critically, it supports memory self-evolution—including experience evolution and skill evolution.

We validated this using the AppWorld fair distillation experiment—same trajectories, same model, with the only variable being the distillation and retrieval approach. Results: seekdb M0 achieved a 39% pass rate versus Hermes at 22%; to complete the same tasks, M0 required 6.2 steps versus Hermes at 10.4; overall token consumption dropped by 32%.

On the semantic dimension, high-quality data semantics enable AI applications to truly understand the enterprise. OceanBase OSI doesn't aim to build yet another BI semantic layer—it unifies metrics, definitions, raw data, context graphs, and ontology layers. The bottom layer is the semantic layer—metrics, calculation definitions, and raw data, designed on the Ant-OSI semantic layer standard, compatible with the open OSI standard, and validated extensively within Ant Group. The middle layer is the context graph—large models reason against the context graph to improve accuracy. The top layer is the ontology layer—unifying business semantics with underlying database semantics, balancing global and local semantics. The core philosophy is "semantics as code"—data semantics defined once, consumed everywhere: BI tools render reports, agents generate SQL, governance tools perform lineage analysis—all reading the same definition.

We also developed the OceanBase DataPilot product based on OceanBase OSI. In POC tests across different industry customers, feedback consistently shows accuracy far exceeding other products in the market. The difference isn't in the model—it's in the quality of semantic context. When AI has access to precise business metric definitions, the accuracy of natural language to SQL translation improves fundamentally.

One Stack to Reduce Engineering Complexity

The most direct value of this architecture is reducing system complexity.

In the traditional approach, enterprises often need many systems to process multimodal data and support AI workloads: databases, data lakes, warehouses, search engines, vector databases, ETL systems, governance tools, and serving layers. Stitching them together creates a cascade of problems: CDC latency, ETL retries, inconsistent permissions, duplicated metadata, multiple monitoring systems, and multiple operational teams.

A unified Lakebase architecture does not mean every component disappears. But it does mean the core data foundation becomes simpler. One copy of data can support multiple workloads. Governance can be defined once and applied consistently. Online serving and offline processing can operate closer to the same data. Hybrid search can run with fresher and more trusted context.

For AI applications, this simplicity matters. The gap between a demo and a production system is often not the model. It is the data infrastructure around the model.

Conclusion: The AI Database for the Agent Era

When you put all these capabilities together, the OceanBase Lakebase architecture can be summarized in three layers. The bottom layer is the lakebase engine—multimodal tables running on object storage, supporting open compute: OceanBase's SQL compute (OLTP, OLAP, search) plus Spark ETL and Daft on Ray AI compute. The middle layer is the context layer—data context helps AI understand the enterprise, application context helps AI understand users. The top layer consists of the application agents we've developed: a data development agent for data engineers, and a data analysis agent for business analysts.gent for data engineers, and a data analysis agent for business analysts.

For enterprises to truly put AI to work, the most important and fundamental step is to manage their own data through a unified AI database. Only after managing their data well can enterprise AI become more accurate, more efficient, faster, and more secure.

OceanBase Lakebase is our answer to this challenge: a unified Lakebase architecture designed for the agent era.