V4.3.2 BP1
Version information
Release date: December 08, 2024
Version: V4.3.2 BP1
RPM version: obproxy-4.3.2.0-752024120610
Overview
The persistence capabilities of OBKV-Redis are supported.
Common Redis commands and cross-partition Redis data processing are supported.
OBKV-Redis and OBKV can share a remote procedure call (RPC) port (port
2885by default).
Bug fixes
Fixed the issue where the error code
4007is returned for an ObTableLSOpRequest request if it is the first request initiated after reconnection.Fixed the issue where an information parsing exception occurs if RANGE partitions are obtained by calling the
getPartitionAPI operation.Fixed the issue where a scheduling exception occurs if default partitioning information is used when you call the
getPartitionAPI operation to obtain partition information for the first time.Fixed the issue where an exception occurs if a cross-partition request is repeatedly scheduled.
Fixed the issue where a core dump is generated if the return result set of OceanBase Database does not contain a response (RESP) flag.
V4.3.2
Version information
Release date: October 08, 2024
Version: V4.3.2
RPM version: obproxy-4.3.2.0-432024100811
Overview
OceanBase Database Proxy (ODP) V4.3.2 is the latest version of the ODP V4.3.x series and is recommended for on-cloud and on-premises business in a proof of concept (POC) test or production environment. The SQL proxy provides the weight-based load-balanced routing and replica type-based routing features for weak-consistency read requests. The SQL firewall and VIP-based connection strings on Google Cloud Platform (GCP) are supported to improve the competitiveness in clouds. Columnstore replica-based routing is supported for online analytical processing (OLAP) requests in OceanBase Database V4.3.3. Routing of the DOUBLE type and invalid date types is supported in MySQL mode, further improving the routing feature. ODP performance is further optimized through a series of means such as disabling distributed transaction routing in standalone mode, optimizing log printing, and optimizing CPU consumption. The RPC proxy adapts to the getPartition API operation to allow the client to obtain and cache all partition routing information of the specified table. Threads for processing cross-partition subrequests are isolated from regular worker threads to reduce the impact of cross-partition requests on the response time of regular single-partition requests, thereby optimizing the performance.
Key features
SQL proxy
Weight-based load-balanced routing
In earlier versions, ODP supports two routing strategies for weak-consistency read requests: logical data center (LDC)-based routing and random routing based on the routing strategy specified by proxy_route_policy, which can meet the routing requirements for weak-consistency read requests in most scenarios. After cloud databases support replica types such as read-only replicas and read-only columnstore replicas, more flexible load balancing requirements are raised for weak-consistency read requests. ODP V4.3.2 supports the weight-based load-balanced routing strategy to meet the new requirements and improve the competitiveness. The WEAKREAD_WEIGHT_LOAD_BALANCE value is added to the proxy_route_policy parameter. You can configure a weight ranging from 0 to 100 for a zone in a cluster. ODP forwards weak-consistency read requests based on the replica weights to achieve load balancing.
Replica type-based routing for weak-consistency read requests
OceanBase Database V4.3.3 supports a new replica type, namely, columnstore replica, to improve the query performance of OLAP requests. Columnstore replicas strongly rely on the routing capabilities of ODP. In terms of deployment architecture, an independent ODP is deployed for columnstore replicas, and the proxy_primary_zone_name parameter is provided to specify to route all requests destined for this ODP to specified columnstore replicas, isolating OLAP requests from online transaction processing (OLTP) requests in the architecture. In terms of routing strategies, the TARGET_REPLICA_TYPE_WITH_LEADER, TARGET_REPLICA_TYPE_FOLLOWER_FIRST, and TARGET_REPLICA_TYPE_FOLLOWER_ONLY values, which are all replica type-based routing strategies, are added to the proxy_route_policy parameter for weak-consistency read requests destined for columnstore replicas. On top of that, the VIP-level parameter route_target_replica_type is provided to specify replicas for routing.
The replica type-based routing strategy applies not only to columnstore replicas but also to full-featured replicas and read-only replicas. This strategy is designed to be applicable to all replicas. You can configure routing strategies for different replica types to support more business scenarios such as read/write splitting, disaster recovery, and double leader addresses.
SQL firewall
ODP already supports SQL throttling and termination features in V1.9.x. However, these features are unsuitable for massive popularization due to the complex usage process. ODP V4.3.2 modifies and improves the SQL firewall to improve the ease of use and stability. You can configure, view, and delete parameter settings at multiple levels. The following types of SQL statements can be blocked:
Certain types of SQL statements, such as
SELECT,INSERT, andDELETESQL statements without a
WHEREconditionSQL statements with a
LIKEclauseCustom SQL statements
Note that after the SQL firewall is enabled, the performance may deteriorate by less than 10%. Proceed with caution in actual business scenarios.
Routing improvement
To further improve the compatibility with MySQL, OceanBase Database supports a column of the DOUBLE type as the partitioning key and the invalid date types of MySQL. Accordingly, ODP V4.3.2 supports accurate routing for tables that use a column of the DOUBLE type as the partitioning key, and for invalid date types of MySQL. To meet the requirements of tools in the ecosystem, ODP V4.3.2 supports the static variable port and can forward SHOW MASTER STATUS statements with comments to the binlog service. Furthermore, ODP V4.3.2 implements special routing for the select last_trace_id() statement. Specifically, ODP routes this statement to the last successfully connected OBServer node to obtain the accurate trace ID of the previous SQL statement, so as to locate SQL logs based on the trace ID, thereby improving the troubleshooting efficiency.
RPC proxy
Expose the table partitioning API operation
OBKV provides a new API operation named getPartition. With this API operation, you can obtain related information about certain or all partitions of the specified table to make good use of the atomicity of TableAPI of OBKV or improve the data synchronization performance of HBase. ODP V4.3.2 adapts to this API operation. If you connect to your OceanBase cluster by using ODP, the client needs to obtain the table entry information of the specified table through multiple processes. ODP encapsulates all the obtained table entry information as an RPC packet and returns the packet to the client, which then caches the packet. ODP can provide routing information without affecting the performance.
Isolate cross-partition subrequests
In earlier ODP versions, if a large number of cross-partition requests are initiated, the response time of other regular single-partition requests increases, which is unacceptable to users. ODP V4.3.2 isolates the threads for processing cross-partition subrequests from worker threads for processing single-partition requests, to reduce the impact on the response time of single-partition requests.
Parameter changes
| Parameter | Change type | Description |
|---|---|---|
| proxy_route_policy | Modified | The WEAKREAD_WEIGHT_LOAD_BALANCE value indicating weight-based load-balanced routing, as well as TARGET_REPLICA_TYPE_WITH_LEADER, TARGET_REPLICA_TYPE_FOLLOWER_FIRST, and TARGET_REPLICA_TYPE_FOLLOWER_ONLY indicating replica type-based routing, have been added to the value range of this parameter since V4.3.2. |
| route_target_replica_type | New | The replica type for routing weak-consistency read requests. This parameter takes effect only when proxy_route_policy is set to a value starting with TARGET_REPLICA_TYPE_. |
| weakread_weight_zone | New | The weights of zones. ODP routes weak-consistency read requests based on the specified weights to achieve load balancing. This parameter takes effect only when proxy_route_policy is set to WEAKREAD_WEIGHT_LOAD_BALANCE. |
| sql_firewall_config | New | The blocking rules of the firewall. |
| rpc_sub_request_isolation_mode | New | The processing mode of RPC subrequests. You can set an appropriate processing mode to control the number of threads used for processing subrequests and reduce the impact on the response time of general requests. |
| rpc_async_task_thread_num | New | The number of asynchronous threads for OBKV. You can set an appropriate number of asynchronous threads to reduce the impact of subrequest forwarding and large request processing on general requests. |
| rpc_sub_request_weight | New | The weight of RPC subrequests. The higher the weight, the more the general threads occupied by subrequests. |
| rpc_request_throttle_waiting_time | New | The time to wait before reading RPC requests after throttling is triggered. |
| rpc_throttle_limit_qps_qa | New | The QPS for RPC requests after throttling is triggered. |
| rpc_throttle_trigger_percentage | New | The memory usage threshold that triggers throttling. When the memory usage of RPC requests reaches the specified threshold, RPC requests are throttled. |
| enable_rpc_throttle | New | Specifies whether to perform throttling based on the memory usage of RPC requests. |
Supported OceanBase Database versions
ODP supports OceanBase Database V4.x and OceanBase Database V3.x and earlier. The recommended ODP version varies based on the OceanBase Database version.
For OceanBase Database V3.x and earlier, ODP V3.2.11 is recommended.
For OceanBase Database V4.x, ODP V4.3.1, which is a Long-Term Support (LTS) version, is recommended.
ODP V4.3.2 provides multiple new features. We recommend that you use ODP V4.3.2 if your environment relies on the new features, or the LTS version V4.3.1 if otherwise.
Bug fixes
Fixed the issue where inaccurate routing occurs because ODP fails to obtain location information after case sensitivity is enabled in the tenant.
Fixed the issue where, if
lower_case_table_names = 0is configured in a tenant, a disconnection occurs due to aSYNC_DATABASE_COMMON_ERRORerror when the database name contains both uppercase and lowercase letters.Fixed the issue where an exception is thrown due to a memory allocation failure when memory overrun occurs.
Fixed the issue where, if a tenant is dropped and created again, ODP keeps trying to connect to the tenant in HBase mode after the client passes in the new tenant ID.