FAQs

FAQs about the architecture and features of the product

Do I need one or multiple instances?

If multiple subsystems are deployed and the subsystems do not interact at the database level, we recommend that each subsystem use a different instance.

How do users use OceanBase Database?

OceanBase Database provides an SQL interface so that users can access and operate the database through SQL statements.

Does OceanBase Database support Java Persistence API (JPA)?

Java Persistence API (JPA) is a set of object-relational mapping (ORM) specifications in Java standards. With JPA, you can describe the mappings between objects and relational tables through annotations or XML files, and persist entity objects to a database (i.e., map the object model to the data model). OceanBase Database is a native distributed relational database independently developed by Alibaba and Ant Group. It does not build on any open-source product. Therefore, there is no conflict between OceanBase Database and JPA.

What level of database management do data files belong to?

At present, OceanBase Database has two types of data files, both of which belong to the cluster level:

• Data files: store data of all partitions, including the checkpoint (CP) data of all partitions.
• clog-related files: contain clogs (also known as redo logs or write-ahead logs, or WAL logs) and their index files.

How does OceanBase Database support HTAP?

OceanBase Database has a distributed computing engine that allows multiple computing nodes to run OLTP applications and complex OLAP applications simultaneously. This enables the engine to support mixed workloads with one system and address 80% of users' business needs. In this way, users can fully utilize their computing resources and reduce the costs of purchasing additional hardware resources and software licenses.

What are instances and tenants, and what is the relationship between them?

OceanBase Database is a multi-tenant system. An instance is a tenant in OceanBase Database. Data cannot be accessed between tenants.

What is the relationship between the number of servers and the performance of OceanBase Database?

The TPC-C report of OceanBase Database shows that the system performance generally scales linearly with the increase in the number of servers.

What should I pay attention to when I use OceanBase Database for development?

Here are some points to consider during development:

• Pay special attention to memory usage when importing a large amount of data.
• Since the index becomes effective after the database is rebuilt, it is recommended to include the index statement in the table creation statement.
• We recommend that you use mysql-connector-java 5.1.30 or later.
• When you modify the column type, keep the column length the same or increase it. Decreasing the column length is not supported.
• If a connection is idle for more than 15 minutes, the server will proactively disconnect the connection. When using a connection pool, set the maximum idle time of a connection. For example, set the minEvictableIdleTimeMillis parameter of the Druid connection pool to less than 15 minutes.

How does OceanBase Database achieve higher compression ratio compared with traditional databases?

OceanBase Database uses data encoding, a compression technique that is tailored for relational database data. Data encoding generates a series of encoding schemes based on the value ranges and types of different fields in the relational tables of the database. Compared with general compression algorithms, data encoding understands the data better and can achieve a higher compression ratio.

How much data is required for AP capabilities of OceanBase Database?

The amount of data is not limited. For AP, OceanBase Database can adapt to business needs ranging from hundreds of GBs to several PBs. You can set a smaller degree of parallelism for a smaller amount of data and a larger degree of parallelism for a larger amount of data. This way, OceanBase Database can make full use of computing resources and generate efficient parallel execution plans regardless of the actual data volumes.

What level of standard SQL support does OceanBase Database have in the latest version?

Most business applications in MySQL mode can be smoothly migrated with no modifications or only slight modifications. Oracle mode of OceanBase Database supports most basic Oracle features. Applications can be smoothly migrated from Oracle databases to OceanBase Database with slight modifications.

What is the cost of migrating a business application from a MySQL database to OceanBase Database?

OceanBase Database is compatible with general MySQL features and MySQL frontend and backend protocols. Applications can be migrated from MySQL databases to OceanBase Database with zero or slight modifications.

How does OceanBase Database, which is a TP-dominant database, perform in AP computing?

AP in OceanBase Database leverages hybrid row-column storage, compilation-based execution, vectorized processing, cost-based query rewriting and optimization, and other cutting-edge technologies to deliver top-notch real-time analytical capabilities. For offline big data processing, Spark and other big data solutions are a better choice.

When you use OceanBase Database as a distributed database, is there any difference from connecting to a traditional database?

As a distributed database, OceanBase Database can deploy multiple replicas across servers. To minimize cross-server data access, OceanBase Database provides obproxy. As a dedicated reverse proxy server for OceanBase Database, obproxy offers high-performance and highly accurate routing and forwarding of frontend user requests, and provides high availability and scalability of disaster recovery for backend servers. Compared with proxy servers in other databases, obproxy, which is designed based on the asynchronous framework and stream-based forwarding, and adopts memory solutions based on FastParse and LockFree, can process millions of QPS with limited resources and offer efficient O&M support for a large number of deployed proxy servers.

What are the technical features of OceanBase Database's architecture?

As a native distributed database, OceanBase Database has the following technical features:

• Elastic scaling

  OceanBase Database supports online elastic scaling. When the storage capacity or processing capability of a cluster is insufficient, you can add new OBServer nodes to the cluster. The system automatically migrates data and distributes appropriate partitions to the new nodes based on the processing capability of each node. Similarly, when the system capacity and processing capability are abundant, you can remove nodes to reduce costs. For example, during the Double 11 Grand Sale, the database can provide excellent elastic scaling capabilities.

• Load balancing

  OceanBase Database manages multiple OBServer nodes in an OBServer cluster to provide data services to multiple tenants. All OBServer nodes managed by the OceanBase cluster can be regarded as a huge "resource cake". When allocating resources, the system allocates resources based on the resource requests of tenants. The load balancing feature of OceanBase Database ensures that resources are occupied relatively evenly among multiple tenants in the OBServer cluster. Even in dynamic scenarios such as adding or removing OBServer nodes, adding or removing tenants, or data skew during partition reassignment, the load balancing algorithm can still balance resources on existing nodes.

  OceanBase Database uses Root Service to manage the load balancing among nodes. Different types of replicas have different resource requirements. Therefore, when Root Service performs partition management operations, it needs to consider the CPU usage, disk usage, memory usage, and IOPS of each OBServer node. Root Service needs to avoid having all partitions of a table fall on a few OBServer nodes. It also needs to place memory-consuming replicas and memory-efficient replicas on the same OBServer node and disk-consuming replicas and disk-efficient replicas on the same OBServer node. After load balancing, the various types of resources on all nodes are relatively balanced, so as to make full use of all resources on each node.

• Distributed transaction ACIDity

  The architecture of OceanBase Database ensures the ACIDity of transactions in the following ways:

  • Atomicity: The two-phase commit protocol is used to ensure the atomicity of snapshot transactions.
  • Consistency: The consistency of the transaction data is ensured.
  • Isolation: The multiversion mechanism is used for concurrency control.
  • Durability: The Paxos protocol is used for multi-replica synchronization of transaction logs.

• High availability

  Each partition of OceanBase Database maintains multiple replicas. Log synchronization between replicas of the same partition is performed using the Paxos protocol. Each partition and its replicas form an independent Paxos group, with one of the replicas serving as the leader and the others as followers. Some partitions of an OBServer node serve as leaders, and the remaining partitions serve as followers. If an OBServer node fails, the follower partitions on the node are not affected. The write service of the leader partitions is interrupted for a short time. Then, a follower is selected as the new leader through the Paxos protocol. This process takes no more than 30s. The Paxos protocol ensures high availability and performance in a strong-consistency scenario.

  In addition, OceanBase Database supports primary/standby database architectures. The multi-replica mechanism of an OceanBase cluster provides rich disaster recovery capabilities. It can automatically switch to a standby IDC or region in the case of failures at the IDC or region level, without data loss (RPO = 0). If the OceanBase cluster becomes unavailable due to planned or unplanned outages (the failure of the majority of replicas), the standby cluster can take over the services. It offers lossless switchover (RPO = 0) and lossy switchover (RPO > 0) disaster recovery capabilities to minimize service interruption.

  You can create, manage, maintain, and monitor one or more standby clusters in OceanBase Database. A standby cluster serves as a hot backup for a production database. You can allocate resource-intensive reporting operations to a standby cluster to improve system performance and resource efficiency.

• Efficient storage engine

  The storage engine of OceanBase Database is built on the LSM-tree architecture. Data in the engine is stored in MemTables (also known as MemStores) and SSTables. MemTables support read and write operations, while SSTables support only read operations. Your data is first written to a MemTable. Then, the redo logs ensure transactionality. The redo logs are synchronized among three replicas using the Paxos protocol. If a single server crashes, the Paxos protocol ensures data integrity and high availability of data with a short recovery time.

  When the size of a MemTable exceeds the specified threshold, you need to flush the data in the MemTable to a Mini SSTable to release the memory space. This process is called a mini compaction. As you write user data, the number of Mini SSTables increases. When the number of Mini SSTables exceeds the specified threshold, the system automatically triggers a minor compaction in the background. A mini compaction generates a new Mini SSTable. A major SSTable is the baseline SSTable in a minor compaction. Minor SSTables and Mini SSTables generated after the minor compaction are incremental SSTables. The system merges the baseline SSTable and incremental SSTables into a new major SSTable during a major compaction. OceanBase Database optimizes the data storage space through mini and major compactions, ensuring efficient read and write services and transactionality and data integrity.

• Multi-tenant

  OceanBase Database is a distributed database that supports the multi-tenant feature. One cluster serves multiple business systems, namely, tenants.

  The multi-tenant architecture makes it possible to make full use of system resources. The same resources can serve more business lines. You can deploy different business systems in one cluster to make full use of system resources. The multi-tenant architecture also ensures the isolation of tenants in applications and protects tenant data to prevent data breaches. In addition, it ensures the stability of performance for frontend applications using allocated resources, regardless of the load fluctuations of other tenants.

• Oracle and MySQL compatibility

  OceanBase Database supports Oracle-compatible mode and MySQL-compatible mode. You can choose a different mode based on your business needs.

Memory FAQs

What memory areas are included in OceanBase Database?

OceanBase Database includes the following memory areas:

• kv cache: the cache for SSTables and database tables in the LSM-tree.
• memory store: the memory of MemStores in the LSM-tree.
• sql work area: the memory used by operator work areas during SQL execution. When the memory is insufficient, the data is written to disk.
• system memory: memory reserved for network I/O, disk I/O, elections, load balancing, and other purposes.

Which resources in a tenant are shared and which are not?

In a tenant, the sql work area, memory store, and kv cache are dedicated resources, whereas system memory is shared among multiple tenants. sql worker resources are isolated between tenants, but net I/O, disk I/O, and clog writer resources are shared among tenants.

What are the memory usage characteristics of OceanBase Database?

When OceanBase Database starts, it needs to load about several GB of memory. During operation, it will gradually apply for memory as needed until the memory_limit is reached. After an OBServer node applies for memory from the OS, the memory is typically not released back to the OS but maintained in the usage list and Free List for memory management. This is the memory management mechanism designed in OceanBase Database.

If OceanBase Database is running and its memory usage is close to memory_limit, is this behavior normal?

If the memory consumption of an OceanBase cluster approaches the memory_limit and remains high after running for a period of time, this behavior is as expected.

However, if the memory_chunk_cache_size parameter is set, the OBServer node will attempt to return memory blocks exceeding memory_chunk_cache_size in the Free List to the OS to increase the reuse rate of the Free List in OceanBase Database and reduce the risk of timeout in memory operations that trigger RPCs. This can help reduce the likelihood of memory-related RPC timeouts. Typically, the memory_chunk_cache_size parameter does not need to be configured. In specific scenarios, you can perform a scenario analysis with OceanBase Database to determine whether to dynamically adjust the memory_chunk_cache_size parameter.

Can the memory upper limit of an OBServer node be dynamically adjusted?

Yes. You can dynamically adjust the memory_limit or memory_limit_percentage to change the memory upper limit of an OBServer node. However, before you adjust the parameters, make sure that the available memory resources are sufficient and the target memory upper limit is not lower than the total memory initially allocated to all tenants and 500 tenants. The unit of memory_limit is MB. For example, if you want to set the memory upper limit of an OBServer node to 64 GB, you can specify the memory_limit parameter as 64G or 65536.

On the other hand, you can use the memory_limit parameter to set the maximum effective memory of an OBServer node. If you set the memory_limit to 0, you can use the memory_limit_percentage parameter to flexibly constrain the memory usage of the OBServer node in percentage.

Can memory be oversold when you define resource units and resource pools in OceanBase Database?

In OceanBase Database, the Root Service is responsible for allocating resources (units). When the Root Service allocates a unit, it determines whether memory can be oversold based on the value of resource_hard_limit (resource_hard_limit indicates the percentage of memory that can be oversold. A value greater than 100 indicates that memory can be oversold). The Root Service then allocates resources to the unit based on the defined resource units.

However, if the resources are relatively tight, the system allows the interleaved operation of different tenant loads. When you configure a tenant, CPU resources can be oversold. If CPU overselling takes effect, when the OceanBase cluster is under heavy load, the system may experience overloaded conditions. In such cases, threads from different tenants compete for CPU resources. The result is that the business scenarios of tenants slow down. If memory is oversold, the total memory specifications of tenants can exceed memory_limit when they are created. However, the memory usage of OceanBase Database is still constrained by memory_limit. For example, if the total memory usage of running tenants exceeds memory_limit, the tenants will experience memory overrun or even encounter the OOM error.

What checks does OceanBase Database perform during a memory allocation?

The OceanBase Database kernel limits the size of a single memory allocation to 4 GB, which is an internal restriction to avoid unreasonable memory allocation. In addition, each time the kernel allocates memory, it performs the following checks. When an error occurs, check the corresponding error message for analysis:

What are the KV caches in OceanBase Database, and what are their purposes?

You can query the GV$OB_KVCACHE view for the KV caches in OceanBase Database. Generally, there are the following types:

• BloomFilter Cache: The Bloom filter cache in OceanBase Database is built on macroblocks. It is automatically built on a macroblock when the number of missed queries exceeds the specified threshold. The Bloom filter is then stored in the cache.
• Row Cache: The row cache stores specific data rows. During a GET or MULTI_GET query, the database often stores the found data rows in the Row Cache to significantly improve the performance of hot row queries.
• Block Index Cache: Caches the index of microblocks, similar to the intermediate layer of a Btree. Since the intermediate layer is usually not large, the hit rate of the Block Index Cache is typically quite high.
• Block Cache: The block cache is the same as the buffer cache in OceanBase Database. It stores specific data blocks. In fact, the data blocks in the Block Cache are decompressed.
• Partition Location Cache: The partition location cache stores the location information of partitions to help route a query.
• Schema Cache: The schema cache stores the metadata of data tables for execution plan generation and subsequent queries.
• clog Cache: The clog cache stores clog data to accelerate the pull of Paxos logs in some scenarios.

How does KV Cache achieve dynamic scaling, and what are the eviction rules?

The main part of dynamically scalable memory is the KV cache. OceanBase Database manages most caches in the KV format centrally in the KV cache.

Generally, you do not need to configure the KV cache. In special scenarios, you can set the priorities of different types of KV. KVs with higher priority are more likely to be retained in the cache. If you want to change the default priorities, contact OceanBase Database technical support engineers.

What are some common memory issues in OceanBase Database, and what are the possible causes?

Some common memory issues in OceanBase Database are described as follows:

• Out of memory in the work area

  The memory space for the work area limit = tenant memory * ob_sql_work_area_percentage (default value: 5%). If a large number of concurrent requests each occupy a significant amount of memory in the work area, an out-of-memory error may occur for the work area. Union, sort, and group by operations are scenarios where this error frequently occurs.

  You can increase the work area system variable, for example, set global ob_sql_work_area_percentage = 10, to avoid this error.

• Insufficient tenant memory

  An error "Over tenant memory limits" (error code 4013) is returned to the client. This indicates that the tenant memory is insufficient. Generally, you need to analyze the current memory consumption first. You can check the GV$OB_MEMORY view or the logs in the observer.log file to see whether a specific module consumes a significantly large proportion of the memory.

• MemStore memory exhausted

  If the write speed exceeds the dump speed, the MemStore is exhausted. For example, in high-concurrency data import, the MemStore writes data too fast. If the system cannot dump the MemStore in time, the MemStore will be exhausted and error 4030 will be returned to the user.

  You can identify the bottlenecks in the dump speed, increase the dump speed, or slow down the write speed to address or resolve this issue.

• Overall memory usage exceeds the limit on the OBServer node

  When the OBServer node process starts, it calculates the maximum physical memory that the process can use based on the configuration parameters. If the memory_limit parameter is set, the memory used by the OBServer node process is directly set to the value of the memory_limit parameter. If the memory_limit parameter is set to 0, the memory used by the OBServer node process is calculated as physical memory * memory_limit_percentage. If you find that the memory usage on an OBServer node exceeds the limit, you can query the _all_virtual_server_stat view for the actual memory limit on the OBServer node, and then check whether the memory_limit and memory_limit_percentage parameters are correctly configured. If the memory usage continues to exceed the limit in an upward trend, it may indicate a memory leak. In this case, contact OceanBase Database technical support engineers for troubleshooting.

Frequently asked questions about multi-tenant threads

How does OceanBase Database manage threads?

In OceanBase Database, the OBServer node avoids dynamic thread creation. Generally, after a thread is created, its number will not change unless the relevant parameters are adjusted. The number of sql worker and transaction worker threads for a specific tenant on an OBServer node is determined by the unit specifications, whereas the other threads are specified by relevant parameters.

What is the principle behind resource allocation for threads in OceanBase Database?

The resource allocation principle for threads in OceanBase Database is as follows:

• Network and I/O resources are not controlled.
• Memory: Memory is managed separately for different tenants. The memory allocator of the current tenant is used to allocate memory for the executed SQL query.
• CPU: The number of threads that can be executed concurrently for a tenant is limited. For example, if a tenant has 16 CPU cores and the cpu_quota_concurrency parameter is set to 2, the tenant can have a maximum of 16 × 2 = 32 active SQL threads. The actual number of concurrent threads is controlled by the user-space thread scheduler of the OBServer node.

Is OceanBase Database running in a single process or multiple processes?

OceanBase Database runs in a single process, and the main threads are as follows:

• election worker: the election thread.
• net io: the network I/O thread.
• disk io: the disk I/O thread.
• clog writer: the clog writing thread.
• misc timer: the background timer thread, which is responsible for resource cleanup.
• compaction worker: the minor merge and major merge thread.

What are the background threads in an OBServer node, and what are they used for?

Generally, you do not need to manually manage threads. As the OBServer node version evolves, the threads may also be updated. The following table describes some common threads in an OBServer node.

How does an OBServer node ensure effective CPU resource isolation in a multi-tenant architecture?

In an earlier version of the OBServer node, it controlled the number of active threads (which are the threads actually consuming CPU resources) to control CPU usage. When creating a tenant, you can specify the tenant resource pool. The max_cpu parameter in the resource pool definition specifies the maximum number of active threads allowed for the tenant, achieving CPU usage isolation among tenants. In the latest version of the OBServer node, the kernel implements cgroup isolation to effectively control and limit CPU, memory, and resource usage. This feature requires operating system-level configuration. OceanBase Database is integrating this capability into the tool platform, so that cgroup isolation can be directly used in the future for resource control and limitation.

How can you read the number of worker threads of an OBServer node? Does the OBServer node dynamically start new threads to handle loads when it is under high load?

In the log fragment with dump tenant info as the keyword in observer.log, the current and maximum values of worker threads are described. Specifically:

tocken_count = allocated_cpu_count (>min_cpu&&<max_cpu)  _cpu_quota_concurrency.

For example, on a server running OceanBase Database, Tenant T1 is configured with unit_min_cpu = 2, unit_max_cpu=8. However, the CPU resources on the server are oversold. Actually, 5 CPU cores are allocated to T1. In this case, token_count = 5_ cpu_quota_concurrency.

How does an OBServer node control the number of active threads and the total number of threads of a tenant?

An OBServer node calculates the maximum number of active threads for a tenant based on the formula cpu_count * cpu_quota_concurrency, and creates the threads upon tenant creation. For example, if thread A is an active thread and is suspended while executing a large query, the number of active threads decreases by one. To make up for the lost thread, the tenant creates an additional active thread. However, the total number of threads that a tenant can create is limited and specified by the parameter cpu_count * workers_per_cpu_quota. If a new thread cannot be created when the tenant reaches the thread creation upper limit, an attempt to create a new thread fails, and thread A is not suspended. This ensures that the number of active threads remains unchanged.

For example, suppose a tenant is configured with 16 CPUs and the cluster's cpu_quota_concurrency is set to 2, then this tenant will have a maximum of 16 * 2 = 32 active SQL threads. The control of the number of active threads is implemented by the user-state thread scheduler of the OBServer node.

What is the principle for allocating CPU resources to large queries? Can CPU resources be preempted when both large queries and OLTP workloads are running?

In OceanBase Database, you can configure the parameter large_query_threshold to define a query operation whose execution time exceeds the threshold as a large query. If large queries and regular queries are running at the same time, the database allocates some CPU resources to the large queries and limits the maximum number of active threads that can be used for executing large queries based on the parameter large_query_worker_percentage (default value: 30%).

OceanBase Database constrains the CPU resources used by large queries by limiting the number of active worker threads available to tenant large queries, ensuring the system retains sufficient CPU resources to handle OLTP (e.g., transactional small workloads). This approach guarantees that latency-sensitive OLTP workloads receive adequate CPU resources for timely execution. Additionally, it is important to note that OceanBase Database can allocate resources between large queries and OLTP workloads. The large_query_threshold parameter should be set within a reasonable range and not to an excessively high value. Otherwise, large queries may easily consume excessive CPU resources, potentially causing slow OLTP responses or even request backlogs.