You can deploy oblogproxy by downloading the installation package or using the source code. This topic describes how to deploy oblogproxy by downloading the installation package.
Background
oblogproxy is a proxy service for managing incremental logs of OceanBase Database. It is a part of OceanBase Migration Service (OMS). Designed based on obcdc, oblogproxy provides links for applications to access and manage real-time incremental logs of OceanBase Database.
oblogproxy allows you to subscribe to incremental logs in isolated networks and supports multiple access methods. You can perform the following steps to configure and use oblogproxy.
Install oblogproxy
Notice
- You do not need to specify information about OceanBase clusters when you configure oblogproxy. Technically, oblogproxy can subscribe to multiple OceanBase clusters when it is connected to all OBServer nodes by using the specified account and password of the sys tenant of each cluster. For security, the account and password are not exposed to oblogclient users.
- oblogproxy is stateless. It obtains information about databases and tables for data subscription from oblogclient. The information such as the timestamps of incremental logs is also stored by oblogclient. When oblogproxy resumes the connection to an OBServer node after disconnection, it is equivalent to creating a new connection.
- oblogproxy consumes a large amount of memory. We recommend that you deploy oblogproxy and the OBServer nodes on different servers to avoid affecting the database performance.
Configure the YUM repository of OceanBase Database
yum install -y yum-utils
yum-config-manager --add-repo https://mirrors.aliyun.com/oceanbase/OceanBase.repo
Download and install the precompiled package
Go to the download page of open-source images and download the precompiled package named "oblogproxy-xxxx.OS version.x86_64.rpm". You must select the package based on the version of your operating system and run the following command to install the package:
yum install -y oblogproxy-xxxx.OS version.x86_64.rpm
Note
By default, oblogproxy will be installed in the /usr/local/oblogproxy directory.
Configure the sys tenant.
For security, you must specify the username and password of a user for oblogproxy. Only a user in the sys tenant of OceanBase Database can be connected.
Notice
The username of this user cannot contain the cluster name or tenant name.
After you obtain the account and password of the sys tenant of your OBServer node, you must encrypt them and configure them for oblogproxy. Run the following commands to obtain the encrypted strings of the account and password:
# Assume that the account and password are user and pswd.
./logproxy -x user
# After you run the command, you will get the following string: 4B9C75F64934174F4E77EE0E9A58****
./logproxy -x pswd
# After you run the command, you will get the following string: DCE2AF09D006D6A440816880B938****
Copy the encrypted account and password strings to the ob_sys_username and ob_sys_password fields of the conf.json file in the /usr/local/oblogproxy/conf/ directory. For example:
{
"ob_sys_username": "4B9C75F64934174F4E77EE0E9A58****",
"ob_sys_password": "DCE2AF09D006D6A440816880B938****"
}
Run oblogproxy
You can run the following commands to start the service.
cd /usr/local/oblogproxy
bash ./run.sh start
By default, the 2983 port is listened to. You can set the service_port field in the conf.json file to change the port. The service logs of oblogproxy are stored in the logs/ directory. The service logs of the LogReader thread are stored in the run/{client-id}/logs/ directory.
Note
You can then use oblogclient for OceanBase data subscription. For more information, see Documentation.
Use oblogclient for data subscription
You can use oblogclient to subscribe to data from OceanBase Database. Make sure that you have installed Maven dependencies before use.
<dependency>
<groupId>com.oceanbase.logclient</groupId>
<artifactId>logproxy-client</artifactId>
<version>1.0.7</version>
</dependency>
After that, you can configure the oblogclient by referencing the following sample code:
ObReaderConfig config = new ObReaderConfig();
// Specify the IP addresses of OceanBase root servers and separate multiple IP addresses with semicolons (;) in the following format: ip1:rpc_port1:sql_port1;ip2:rpc_port2:sql_port2
config.setRsList("xxx.xxx.xxx.1:2882:2881;xxx.xxx.xxx.2:2882:2881");
// Specify the username and password (user tenant).
config.setUsername("root");
config.setPassword("root");
// Specify the UNIX start timestamp in seconds. A value of 0 specifies to start the client from the current time.
config.setStartTimestamp(0L);
// Specify the subscription allowlist in the tenant.db.table format. Use asterisks (*) as wildcards.
config.setTableWhiteList("sys.*.*");
// Specify the IP address of the server where oblogproxy is deployed to create the client instance.
LogProxyClient client = new LogProxyClient("xxx.xxx.xxx.1", 2983, config);
// Add the RecordListener
client.addListener(new RecordListener() {
@Override
public void notify(LogMessage message){
// Process the message.
}
@Override
public void onException(LogProxyClientException e) {
// Handle the error.
if (e.needStop()) {
// Stop the client when an unrecoverable exception occurs.
client.stop();
}
}
});
// Start the client
client.start();
client.join();
More information
For more information about oblogclient, see oblogclient.