In OceanBase Database versions earlier than V4.3.5 BP2, accessing external data sources required manually creating mapping tables with CREATE EXTERNAL TABLE. When an external data source contained many tables, each one had to be mapped individually, which was inflexible.
To simplify querying data stored in various external data sources, OceanBase Database introduces the Catalog feature. It adds a new data hierarchy level for unified management of external data sources.
Catalog types
Catalogs are divided into two types:
Internal Catalog
Manages data stored within OceanBase Database. Each tenant has exactly one internal catalog named
internal. Databases and objects (such as tables) created before V4.3.5 BP2 are considered to be stored ininternal. The internal catalog cannot be created, dropped, or modified.External Catalog
Connects to external data sources and retrieves their metadata. You can query external data directly through an external catalog without importing or migrating the data.
Advantages
Multi-level data management architecture
A Catalog level is added on top of the existing Database-Table hierarchy, forming a Catalog-Database-Table three-level structure that supports direct access to the directory tree of external data sources.
Automatic mapping and on-demand querying
By declaring a connection to an external data source with
CREATE EXTERNAL CATALOG, you can query external tables transparently without manually creating mapping tables.Unified query experience
The syntax for querying external data sources is the same as for local tables, and cross-source join analysis is supported.
Features
Multi-source support
Supports external data sources such as ODPS (MaxCompute) and HMS (Hive, Iceberg).
Permission isolation
Controls access permissions to external data sources at the Catalog level to ensure data security.
Performance optimization
Optimizes query plans based on Catalog metadata to reduce intermediate data transfer and improve cross-source query efficiency.
Use cases
Federated analysis
Scenarios that require joint analysis of OceanBase Database data and external data sources.
Data lake/warehouse integration
Build a unified data lake architecture and directly query massive data stored in external systems such as ODPS and Hive.
ETL pipeline optimization
Reduce data migration steps by accessing source system data directly through Catalog, lowering ETL development complexity.
Limitations
In the current version, data in a Catalog is read-only and cannot be modified. All statements that modify objects under a Catalog are blocked, such as DROP TABLE and INSERT INTO.