This topic describes how to deploy oblogproxy by using the CLI in full deployment mode or by adding oblogproxy. In full deployment mode, oblogproxy is deployed with OceanBase Database.
Note
This topic assumes that the server mentioned in this topic can connect to the Internet or the required software programs such as OceanBase Database, OceanBase Database Proxy (ODP), oblogproxy, and obconfigserver are installed on the server.If obd is installed by using the OceanBase All in One package, the local image repository contains all installation packages required for deployment by default.
Full deployment: If no OceanBase Database component is deployed in the current environment, you can deploy OceanBase Database Community Edition, ODP Community Edition, obconfigserver, and oblogproxy at a time by using OceanBase Deployer (obd). This mode is supported by obd V2.5.0 and later.
Add oblogproxy: If an OceanBase cluster has been deployed in the current environment, you can add oblogproxy to the cluster by using obd. This mode is supported by obd V2.9.0 and later.
Components
ODP
ODP is a proxy server for OceanBase Database. For more information, see ODP documentation.
obconfigserver
obconfigserver allows you to register, store, and query metadata of OceanBase Database. For more information, see ob-configserver.
oblogproxy
oblogproxy is a proxy service for managing incremental logs of OceanBase Database. It connects to OceanBase Database and reads incremental logs. It provides the change data capture (CDC) capability for downstream services.
Configuration files
obd provides sample configuration files required for deploying oblogproxy. You can modify and use them based on your server resources.
If you have installed obd by downloading the RPM package for obd, you can view the sample configuration files for oblogproxy in the
/usr/obd/example/oblogproxydirectory.If you have installed obd by using the all-in-one installation package, you can view the sample configuration files for oblogproxy in the
~/.oceanbase-all-in-one/obd/usr/obd/example/oblogproxydirectory.You can also view the sample configuration files for oblogproxy on GitHub.
obd provides the following sample configuration files for deploying oblogproxy:
Sample configuration file for full deployment:
distributed-with-obproxy-and-oblogproxy-example.yamlSample configuration file for deploying oblogproxy only:
oblogproxy-only-example.yamlSample configuration file for adding oblogproxy:
oblogproxy-component-add.yaml
Procedure
Scenarios
If no OceanBase Database component is deployed in the current environment, you can deploy OceanBase Database Community Edition, ODP Community Edition, obconfigserver, and oblogproxy in a cluster by using obd.
Note
If you want to use the binlog service of oblogproxy, you must deploy ODP and obconfigserver.
Prerequisites
You have installed obd V2.5.0 or later (latest version recommended). For more information, see Install obd.
Procedure
This section describes how to deploy oblogproxy by using the distributed-with-obproxy-and-oblogproxy-example.yaml configuration file.
Configure user information.
The following sample code shows user information parameters in the configuration file:
## Only need to configure when remote login is required user: username: admin # password: your password if need key_file: /home/admin/.ssh/id_rsa # port: your ssh port, default 22 # timeout: ssh connection timeout (second), default 30usernamespecifies the username of the account used to log on to the target server. Make sure that this account has the write privilege onhome_path.passwordandkey_fileare used for user verification. Generally, you need to specify only one of them.Notice
After you specify the path of the key, comment out or delete the
passwordparameter if your key does not require a password. Otherwise, the value of thepasswordparameter will be taken as the password of the key and used for logon, leading to a logon verification failure.Configure OceanBase Database Community Edition.
Create a user named
cdcro@sysand grant the read-only privilege on oblogproxy to the user. Thecdcro@sysuser belongs to OceanBase Database. oblogproxy only uses this user.The following sample code shows parameters of OceanBase Database Community Edition in the configuration file:
oceanbase-ce: servers: - name: server1 # Please don't use hostname, only IP can be supported ip: 10.10.10.1 - name: server2 ip: 10.10.10.2 - name: server3 ip: 10.10.10.3 depends: - ob-configserver global: memory_limit: 64G # The maximum running memory for an observer # The reserved system memory. system_memory is reserved for general tenants. The default value is 30G. system_memory: 30G datafile_size: 192G # Size of the data file. log_disk_size: 192G # The size of disk space used by the clog files. enable_syslog_wf: false # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true. enable_syslog_recycle: true # Enable auto system log recycling or not. The default value is false. max_syslog_file_count: 4 # The maximum number of reserved log files before enabling auto recycling. The default value is 0. # observer cluster name, consistent with obproxy's cluster_name appname: obcluster scenario: htap root_password: ******** # root user password, can be empty proxyro_password: ******** # proxyro user password, consistent with obproxy's observer_sys_password, can be empty cdcro_password: ******** # cdcro user password, consistent with oblogproxy's observer_sys_password, can be empty server1: mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started. rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started. # The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field. home_path: /home/admin/observer # The directory for data storage. The default value is $home_path/store. # data_dir: /data # The directory for clog, ilog, and slog. The default value is the same as the data_dir value. # redo_dir: /redo zone: zone1 server2: mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started. rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started. # The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field. home_path: /home/admin/observer # The directory for data storage. The default value is $home_path/store. # data_dir: /data # The directory for clog, ilog, and slog. The default value is the same as the data_dir value. # redo_dir: /redo zone: zone2 server3: mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started. rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started. # The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field. home_path: /home/admin/observer # The directory for data storage. The default value is $home_path/store. # data_dir: /data # The directory for clog, ilog, and slog. The default value is the same as the data_dir value. # redo_dir: /redo zone: zone3For more information about the parameters, see the OceanBase Database part of the Parameters section in Configuration file. Compared with the configuration file for deploying only an OceanBase cluster, this configuration file provides the
cdcro_passwordparameter. You can use this parameter to specify the password of thecdcro@sysuser. If you do not specify the user password, obd automatically generates a random password string.Configure ODP Community Edition and obconfigserver.
If you do not use the binlog service of oblogproxy, you do not need to deploy ODP Community Edition or obconfigserver.
The following sample code shows parameters of ODP Community Edition and obconfigserver in the configuration file:
obproxy-ce: depends: - oceanbase-ce servers: - 10.10.10.1 global: listen_port: 2883 # External port. The default value is 2883. prometheus_listen_port: 2884 # The Prometheus port. The default value is 2884. rpc_listen_port: 2885 home_path: /home/admin/obproxy enable_cluster_checkout: false # observer cluster name, consistent with oceanbase-ce's appname. When a depends exists, obd gets this value from the oceanbase-ce of the depends. # cluster_name: obcluster skip_proxy_sys_private_check: true enable_strict_kernel_release: false obproxy_sys_password: ******** # obproxy sys user password, can be empty. When a depends exists, obd gets this value from the oceanbase-ce of the depends. # observer_sys_password: # proxyro user password, consistent with oceanbase-ce's proxyro_password, can be empty. When a depends exists, obd gets this value from the oceanbase-ce of the depends. ob-configserver: servers: - 10.10.10.1 global: listen_port: 8080 # The port of ob-configserver web # server_ip: 0.0.0.0 # Listen to the ob-configserver server IP. When you want to listen to the specified IP address,use it. home_path: /home/admin/ob-configserver # The working directory for prometheus. ob-configserver is started under this directory. This is a required field. ## log config # log_level: info # Log printing level of ob-configserver. The default value is `info` # log_maxsize: 30 # The total size of manager ob-configserver.Log size is measured in Megabytes.The default value is 30 # log_maxage: 7 # The days of manager expired ob-configserver.Log retention days. The default value is 7 # log_maxbackups: 10 #The number of manager expired ob-configserver.Log. The default value is 10 # log_localtime: true # Switch of ob-configserver.Log naming with localtime. The default value is true # log_compress: true # Compress ob-configserver.Log switch. The default value is true ## vip config, configserver will generate url with vip_address and port and return it to the client ## do not use some random value that can't be connected # vip_address: "10.10.10.1" # vip_port: 8080 ## storage config # storage: # database_type: sqlite3 # sqlite3 or mysql. Default sqlite3 # connection_url: "" # When database_type is set to sqlite3, the connection_url parameter can be left empty. If it is empty, the default value $home_path/.data.db?cache=shared&_fk=1 will be used. When database_type is set to mysql, the connection_url parameter must be configured, with a sample value of user:password@tcp(10.10.10.1:2883)/test?parseTime=true.For more information about the parameters, see the ODP and obconfigserver parts of the Parameters section in Configuration file.
Configure oblogproxy.
The following sample code shows parameters of oblogproxy in the configuration file:
oblogproxy: depends: - oceanbase-ce - obproxy-ce servers: - 10.10.10.1 global: home_path: /home/admin/oblogproxy service_port: 2983 # binlog_dir: /home/admin/oblogproxy/run # The directory for binlog file. The default value is $home_path/run. # binlog_mode: true # enable binlog mode, default trueoblogproxy depends on OceanBase Database Community Edition and ODP Community Edition. The following table describes parameters in the configuration file:
Parameter Required? Default value Description version No Latest version The version of the component to deploy. Generally, you do not need to specify this parameter. The latest version is deployed by default. servers Yes N/A The servers. Specify each server in the format of - name: server identifier(line break)ip: server IP address. You can also specify a server in the- <ip>format. In this case,- <ip>is equivalent to- name: server IP address (line break) ip: server IP address.home_path Yes N/A The installation path of oblogproxy. service_port Yes 2983 The listening port of oblogproxy. binlog_dir No $home_path/run The absolute path of the binlog service. binlog_mode No true Specifies whether to enable the binlog service. Take note of the following considerations:
When you deploy oblogproxy by using obd, the·
binlog_dirparameter in the configuration file of obd is equivalent to thebinlog_log_bin_basenameparameter in the configuration file of oblogproxy.When you deploy oblogproxy by using obd, the default value of the
binlog_modeparameter istrue.
Deploy a cluster.
[admin@test001 ~]$ obd cluster deploy obtest -c distributed-with-obproxy-and-oblogproxy-example.yamlAfter you run the
obd cluster deploycommand, if your server is connected to the Internet, obd checks whether the desired installation package exists in the local image repository. If no, obd automatically obtains the installation package from the YUM repository.Start the cluster.
[admin@test001 ~]$ obd cluster start obtestQuery the cluster status.
[admin@test001 ~]$ obd cluster display obtest
Scenarios
If an OceanBase cluster has been deployed in the current environment, you can add oblogproxy to the cluster by using obd.
Prerequisites
You have installed obd V2.9.0 or later. For information about how to upgrade obd, see the How do I upgrade obd? section in FAQ.
You have deployed ODP and obconfigserver. ODP and obconfigserver are not required if you do not want to use the binlog service of oblogproxy.
Procedure
This section describes how to deploy oblogproxy by using the oblogproxy-component-add.yaml configuration file.
Configure oblogproxy.
In this example, the configuration file is named
oblogproxy-component-add.yaml. You can change the file name. The file content is as follows:oblogproxy: depends: - oceanbase-ce servers: - 10.10.10.1 version: 2.0.0 global: home_path: /root/oblogproxy service_port: 2983 # External port. The default value is 2983.The following table describes parameters in the configuration file:
Parameter Required? Default value Description version No Latest version The version of the component to deploy. Generally, you do not need to specify this parameter. The latest version is deployed by default. servers Yes N/A The servers. Specify each server in the format of - name: server identifier(line break)ip: server IP address. You can also specify a server in the- <ip>format. In this case,- <ip>is equivalent to- name: server IP address (line break) ip: server IP address.home_path Yes N/A The installation path of oblogproxy. service_port Yes 2983 The listening port of oblogproxy. Add oblogproxy.
[admin@test001 ~]$ obd cluster component add obtest -c oblogproxy-component-add.yamlIn this example, the target cluster is named
obtest. You must replace the cluster name with the actual one. For more information about theobd cluster component addcommand, see Cluster commands.The output is as follows:
Check before start oblogproxy ok start oblogproxy ok +--------------------------------+ | oblogproxy | +---------------+-------+--------+ | ip | port | status | +---------------+-------+--------+ | 10.10.10.1 | 2983 | active | +---------------+-------+--------+ obclient -h10.10.10.1 -P2983 test running Trace ID: b082e760-2889-11ef-9e33-00163e0808cc If you want to view detailed obd logs, please run: obd display-trace b082e760-2889-11ef-9e33-00163e0808ccCheck whether oblogproxy is successfully added.
[admin@test001 ~]$ obd cluster display obtestIf the output contains oblogproxy information, oblogproxy is successfully deployed and added to the cluster.
Verify the deployment
Depending on whether the binlog service is enabled, you can use one of the following two methods to verify the deployment:
Method 1: If you set the
binlog_modeparameter totrue, you can copy and execute the connection string exported by obd after oblogproxy is started to verify whether obd can connect to oblogproxy.obclient -h10.10.10.1 -P2983Method 2: If you set the
binlog_modeparameter tofalse, you can log on to the server where oblogproxy is located and run the following command to check whether the oblogproxy process exists:ps axu | grep logproxyThe output is as follows:
admin 10493 1.1 0.0 634812 2940 ? Sl 16:21 0:07 /home/admin/oblogproxy/bin/logproxy -f /home/admin/oblogproxy/conf/conf.json admin 18071 0.0 0.0 112812 980 pts/1 S+ 16:32 0:00 grep --color=auto logproxy
References
- You can use obd to manage oblogproxy. For example, you can start, stop, restart, and destroy oblogproxy. For more information, see Cluster commands. Note that if you stop oblogproxy by running the
obd cluster stopcommand, obd stops only the oblogproxy process. The binlog converter process of the binlog service is not stopped.
Note
When you deploy oblogproxy in full deployment mode, obd automatically creates the
cdcrouser in OceanBase Database and specifies the user information in the configuration file of oblogproxy. You do not need to specify the user information when you enable the binlog service.When you deploy oblogproxy by adding oblogproxy to your cluster, if you have specified a valid username and password, obd automatically specifies them in the configuration file of oblogproxy. You do not need to specify the username or password when you enable the binlog service.