For the physical connections of a client, OceanBase Database Proxy (ODP) maintains its connections to multiple OBServer nodes at the backend. It also adopts the incremental synchronization solution based on version numbers to maintain the session status of connections to each OBServer node. This ensures efficient client access to each OBServer node. ODP also helps keep connections alive. When an OBServer node is down, being upgraded, or being restarted, the connection between the relevant client and ODP is not terminated. In addition, ODP rapidly switches to a normal OBServer node. This process is transparent to applications.
Create a connection
ODP sessions are classified into client sessions and server sessions. A client session is a connection established between a client and ODP. A server session is a connection established between ODP and an OBServer node. When ODP forwards a client request to an OBServer node, if no connection is established between ODP and the OBServer node, a session instance needs to be initialized.
Authentication is required when a session is being established. ODP cannot decide which OBServer node this session will access. Therefore, ODP selects an arbitrary OBServer node for authentication. ODP forwards the authentication packet sent from the client to the selected OBServer node and forwards the result returned by the OBServer node to the client. ODP also caches the authentication packet of the client within the client session. This way, when ODP establishes a server session that is associated with this client session to another OBServer node, ODP can forward the authentication packet to the target OBServer node for authentication.
Store a connection
When a client sends a request to ODP, ODP queries a client session based on the client connection information. In non-connection pool mode, a server session cannot independently exist without being associated with a client session. You can query the server sessions that are associated with a client session or check whether a server session is associated with a client session.
When ODP receives an SQL request from a client, ODP queries the partition table cache for the IP address of the OBServer node, and then checks whether a server session saved in the client session is associated with this OBServer node. If yes, the server session is used. If not, ODP establishes a connection to this OBServer node and adds the server session to the server session storage structure of the client session.
Maintain the transaction status
ODP binds client sessions to server sessions based on transactions. When the first statement of a transaction arrives at ODP, ODP selects a server session and associates it with a client session. Then data is forwarded to an OBServer node by using this server session during the entire transaction processing process. The server session cannot be changed during the process, and therefore the transaction status needs to be recorded to the client session.
When the transaction starts, the initial transaction status is recorded in the client session. When the transaction ends, the transaction status is reset. You can use the following method to determine the end of the transaction: If the autocommit mode is enabled to determine the transaction status, the return packet of the OBServer node is parsed for each request. Once data forwarding is complete, the transaction status of the current request is reset. For a long transaction, the return packet of the OBServer node is parsed after the request is committed or rolled back to determine whether the transaction ends.
Transaction status maintenance aims to ensure that the server session is not changed during the transaction processing process. In other words, if the OBServer node is down during the process, ODP needs to check the status of the OBServer node and ends the transaction. The MySQL protocol has no timeout mechanism. Therefore, ODP must proactively notify the client that the transaction is ended. Transactions are not synchronized among OBServer nodes and cannot be migrated. In other words, a transaction must be processed only by the OBServer node that accepts the request and cannot be processed by other OBServer nodes. Before primary/secondary ODP switchover is performed, the ongoing transaction must be ended, or a special error must be returned so that ODP will notify the client that the transaction is ended.
Transaction status maintenance serves another purpose during an ODP upgrade. After ODP runs for a period of time, fewer active client sessions exist. ODP processes need to be killed to stop services. Before services are stopped, transactions must be ended to minimize the impact on users.
Manage connection variables
A client session records all variables that are set by the client in this session. Each time a variable is modified, the modification time is recorded in the client session. When ODP is selecting a server session to forward requests, ODP first checks whether the modification time of variables in this server session is later than that recorded in the client session. If not, this server session does not use the latest session variable settings. In this case, ODP first resets all session variables in the server session. Then, ODP applies the session variable settings recorded in the current client session to the server session and forwards requests by using the server session. If the modification time of variables in the server session is later than that recorded in the client session, ODP directly forwards requests by using the server sessions.
The client may frequently set commonly used session variables, such as autocommit. Therefore, ODP cannot decide whether to reset multiple session variables at a time only based on the modification time of the variables. Instead, each server session stores the values of commonly used session variables. ODP checks whether the variable values in the client session and those in the server session are consistent for each request, and resets the variable values if they are inconsistent.
Avoid connection interruption
Connection interruption avoidance ensures that server session exceptions between ODP and OBServer nodes are imperceptible to clients, client sessions are normal, and clients can properly read and write data. The following part describes the scenarios where server session exceptions must be handled:
ODP does not obtain the new leader during the follower-to-leader switchover. Before the follower-to-leader switchover is performed, make sure that the ongoing transaction is ended. ODP only forwards a request to the OBServer node where the data is located. After the leader OBServer node is switched, the leader OBServer node needs to process the request and returns the result to ODP even if the data is not on the leader OBServer node.
The OBServer node is down. The connection between ODP and the OBServer node is closed. If the current server session is processing a transaction, ODP needs to send an error response to the client. If the current server session is idle, ODP needs only to disassociate the server session from the associated client session. Then, new requests are no longer forwarded by using this server session.
The communication between ODP and the OBServer node times out. The MySQL protocol does not have a timeout mechanism, but OBServer nodes do. Therefore, the OBServer node first notifies ODP of the timeout error, and then ODP notifies the client of the error. If the OBServer node notices that the server session has been idle for a long period of time, the OBServer node kills this server session. In this case, the handling strategy for the second scenario applies as well.
The network connection between ODP and the OBServer node is closed, or the network is partitioned. In this case, the handling strategy for the second scenario applies as well. If the server session is processing a transaction when network partitioning occurs, the OBServer node terminates this server session after a period of time. Then, ODP reports an error to the client and ends the uncompleted transaction. This avoids longtime occupancy of the session.
After an ODP upgrade is performed, the new ODP is responsible for new sessions initiated by the clients, and the number of sessions on the original ODP is decreased. If the number of sessions on the original ODP is smaller than the specified threshold, the sessions will be terminated, and the original ODP will be stopped. The sessions will be terminated after all ongoing transactions are completed. Long-running transactions may consume a long period of time. Transactions that time out must be forcibly killed. During this process, client connection interruption is unavoidable.
The current ODP is down. In this case, all connections to the current ODP are interrupted. The current ODP may be restarted within a short period of time, or the connections will be taken over by another ODP. Connection interruption is also unavoidable during the process.
If you use ODP, you can avoid connection interruption in most cases and minimize the impact on clients. Normal client connections can be kept during backend maintenance operations, such as the follower-to-leader switchover of OBServer nodes, primary/secondary cluster switchover, and ODP upgrade.
Reuse sessions
In most cases, when a client session is closed, all associated server sessions will be closed as well. If the same client attempts to reconnect to the relevant ODP, a server session needs to be established again. Session establishment is time-consuming. If you use client code that involves applications, you must first establish a client session and then close the session after data is obtained by sending an SQL request. Repeated session establishment and closing operations consume a lot of resources on ODP. To resolve this problem, you can reuse sessions.
To implement session reuse, after a client session is closed, do not close the associated server session but add the client session to the free list. If the client attempts to establish a client session again, ODP first checks whether this client has an idle client session. If yes, the client session is reused. If a server session associated with this client session needs to be used to forward client requests, the session variables recorded in this server session are reset to new values.
As an optimization method, session reuse reduces the frequency of session establishment between ODP and OBServer nodes. On ODP, client sessions and server sessions are not strongly associated but constitute a public session pool. A client session can obtain an available server session from this session pool and release the server session if it is no longer needed. This way, the released server session can be used by other requests.
Kill sessions
MySQL has no timeout mechanism, but it kills a session that has been idle for a long period of time. If a client does not receive a response from the OBServer, the client keeps waiting. In this case, a DBA will intervene in the process and call the kill session command of MySQL to close the session.
OBServer nodes use a timeout mechanism. Generally, ODP can notify the client session of a transaction processing failure in the event of server downtime, network partitioning, and ODP disconnection from the OBServer node. If a client session is suspended because a long timeout period is specified or ODP misses specific objects to be processed, the client will send a request to kill the session. To kill a session, ODP needs to process the recorded client session status and notify the OBServer node to close the server session.