ODP routing

ODP allows you to access different servers of different tenants in different clusters through cluster routing, tenant routing, and intra-tenant routing. This topic describes the routing features of ODP in terms of cluster routing and tenant routing. For more information about intra-tenant routing, see Intra-tenant routing.

Cluster routing

Cluster routing allows you to access different clusters. The key to cluster routing is to obtain the mapping between a cluster name and an RS list.

• In the startup method in which the RS list is specified by using a startup parameter, the mapping between a cluster name and the RS list is also specified by using the startup parameter.

• In the startup method in which the obproxy_config_server_url parameter is specified in the startup command, the mapping between a cluster name and an RS list is obtained from the access URL.

As can be seen from the preceding figure, OceanBase Cloud Platform (OCP) is a key module in cluster routing. Common issues are as follows if the OCP module is abnormal:

• The service is unavailable: ODP fails to obtain the mapping between the cluster name and the RS list from OCP. As a result, logon fails.

• The returned result is incorrect: If ODP accesses OCP over HTTP to obtain a result, the obtained result must be in the JSON format. If the format is not JSON, the result cannot be parsed correctly.

ODP obtains the RS list when you log on to access a cluster for the first time. ODP then saves the RS list to the memory. In subsequent accesses to the cluster, ODP directly obtains the RS list from the memory.

Tenant routing

In OceanBase Database, a cluster has multiple tenants. The tenant routing feature of ODP allows you to access different tenants. Different from other tenants, the sys tenant is similar to an administrator tenant, and is related to cluster management. This section separately describes routing for the sys tenant and routing for a user tenant.

Routing for the sys tenant

After completing cluster routing, ODP obtains the RS list of the cluster. In this case, ODP logs on to a server in the RS list with the proxyro@sys account and obtains all the servers in the cluster from the DBA_OB_SERVERS view. In the implementation of OceanBase Database, the sys tenant is distributed on each node. Therefore, the result retrieved from the DBA_OB_SERVERS view is the routing information of the sys tenant.

ODP accesses the DBA_OB_SERVERS view every 15 seconds to maintain the latest routing information and to perceive node changes in the cluster.

In addition to the cluster server list, ODP also obtains information such as the partition distribution information, zone information, and tenant information from the sys tenant. Therefore, the sys tenant is crucial to ODP.

Routing for a user tenant

Unlike the routing information of the sys tenant, the routing information of a user tenant is about the servers where the tenant resources are located.

ODP stores the obtained tenant information in the local memory and updates the cache information according to certain strategies. For the sys tenant, ODP initiates a pull task every 15 seconds to obtain the latest information. For user tenants, ODP refreshes the routing cache in accordance with the following strategies:

• Creation: When you access a tenant for the first time, ODP accesses the __all_virtual_proxy_schema table to obtain the routing information of a user tenant and creates the routing cache.

• Eviction: ODP disables the cache when the OBServer node returns the OB_TENANT_NOT_IN_SERVER error code.

• Update: ODP accesses the __all_virtual_proxy_schema table again to obtain the routing information of the user tenant when the cache is disabled.

In summary, in the multi-tenant architecture, ODP obtains metadata information from the sys tenant. The routing information of the sys tenant is a list of servers in the cluster. Then, ODP obtains the routing information of tenants from the metadata information. With the tenant routing feature, ODP supports the multi-tenant architecture of OceanBase Database.

Based on the routing feature, a query can be accurately routed to the OBServer node where the desired data is located. In addition, a read request without high consistency requirements can be routed to an OBServer node where a replica is located. This maximizes the usage of server resources. During the routing process, SQL statements, configuration rules, and OBServer node status are used as the input, and an available OBServer node IP address is generated as the output. The following figure shows the data routing process carried out by ODP for a user tenant.

The custom parser module of ODP is used to parse SQL statements. ODP needs only to parse the database names, table names, and hints in DML statements. No complex expression analysis is required.

In the routing rule determination module, ODP needs to determine the optimal routing rule based on the actual situation. For example, DML requests for strong consistency reads are routed to the OBServer node where the leader of the partition log stream is located, whereas DML requests for weak consistency reads and other requests are evenly distributed to the OBServer nodes where the leader and followers are located. If an OceanBase cluster is deployed across regions, ODP provides logical data center (LDC)-based routing to preferentially route requests to an OBServer node in the same IDC, then an OBServer node in the same region, and finally an OBServer node in a different region. If the OceanBase cluster adopts the read-write splitting mode, ODP provides rules such as RZ-first, RZ-only, and non-major-compaction-first. You can set these rules based on your business needs. Here, RZ refers to read-only zone. In practice, the previous rules can be combined to generate a specific routing rule.

In the routing table obtaining module, ODP obtains the location of the replica involved in the SQL request from a user. ODP attempts to obtain the routing table first from the local cache, then from the global cache, and finally from the OBServer node by initiating an asynchronous query task. ODP uses the trigger mechanism to update the routing table. ODP forwards requests to OBServer nodes based on the routing table. If a request cannot be locally processed by an OBServer node, the OBServer node places the feedback in the response packet and sends the packet to ODP. ODP determines whether to forcibly update the routing table in the local cache based on the feedback. Generally, the routing table changes only when an OBServer node undergoes a major compaction or when leader switchover is caused by load balancing.

ODP selects an OBServer node from the routing table obtained in the previous step based on the determined routing rule and forwards requests to the OBServer node after this OBServer node passes the blocklist and graylist checks.