Deploy ODC in high-availability mode

Load and run an ODC image

After you initialize the MetaDB of OceanBase Developer Center (ODC), you need to obtain and run the ODC image on each node to install ODC.

Load the image

1. To obtain the ODC image on the host, select the required image from the Maintenance Tools of the OceanBase Download Center.

2. After you obtain the image, run the following command in the command-line tool to load it:

   gunzip -c obodc-{$version}.tar.gz | docker load
   

Run the image

Run a command in the command-line tool on the host to run the obtained ODC image. The sample command is as follows:

#!/usr/bin/env bash
docker run -v /var/log/odc:/opt/odc/log -v /var/data/odc:/opt/odc/data \
-d -i --net host --cpu-period 100000 --cpu-quota 400000 --memory 8G --name "obodc" \
-e "DATABASE_HOST=xxx.xx.xx.xx" \
-e "DATABASE_PORT=60805" \
-e "DATABASE_USERNAME=[username]@[tenant name]#[cluster name]" \
-e "DATABASE_PASSWORD=******" \
-e "DATABASE_NAME=odc_metadb" \
-e "ODC_PROFILE_MODE=alipay" \
-e "ODC_ADMIN_INITIAL_PASSWORD=******" \
oceanbase/odc:4.2.2

The following table describes the parameters.

In addition to the preceding parameters, you can also use the following parameters when running the image.

Deploy an SSL certificate

When you log on to Web ODC in a browser over HTTP for the first time, you will receive a message indicating that the connection is not secure. To ensure access security, you can deploy an SSL certificate to access ODC over HTTPS. You can apply for a CA certificate or configure an HTTPS self-signed certificate. If you do not want to access ODC over HTTPS, you can skip this step and directly deploy Nginx.

Apply for a CA certificate

To apply for a CA certificate, we recommend that you purchase an SSL certificate and an SSL certificate service on Alibaba Cloud. For more information, see SSL certificate.

In the intranet environment, you can consult your IT administrator for the procedure of applying for a CA certificate.

Configure an SSL self-signed certificate

The ODC image package contains an executable file that can generate a certificate and private keys: generate-odc-ssl-certificate.sh. You can run this file on the host to generate a certificate. A self-signed certificate can implement encrypted communication. However, the operating system does not trust a self-signed certificate by default. You can add the certificate to Trusted Root Certificate Authorities in the browser to configure certificate trust.

The following code shows how to run the executable file. The value for domain must be consistent with the domain name used by your ODC. Otherwise, the browser considers that the certificate does not match the site.

$./generate-odc-ssl-certificate.sh
Enter your domain [www.example.com]:
Enter your province [Zhejiang]:
Enter your city [Hangzhou]:
Enter your company name [OceanBase]:
Enter your company unit name [DBA]:
generate certificate...
Generating a 2048 bit RSA private key
.......................................+++
.......................
......+++
writing new private key to '/etc/pki/nginx/odcserver.key'
-----
end of string encountered while processing type of subject name element #5
problems making Certificate Request
check generated files:
-rw-r--r-- 1 root root 1424 Jun  3 15:18 server.crt
-rw-r--r-- 1 root root 1708 Jun  3 15:18 server.key

The following code shows the content of the generate-odc-ssl-certificate.sh file. You can directly run the following command to generate a self-signed certificate.

#!/usr/bin/env bash
# for generate self certificated ssl certificate files

echo try create directory '/etc/pki/nginx' if not exists...
sudo mkdir -p /etc/pki/nginx
cd /etc/pki/nginx || exit

#read configurations
read -rp "Enter your domain [www.example.com]:
read -rp "Enter your province [Zhejiang]:
read -rp "Enter your city [Hangzhou]:
read -rp "Enter your company name [OceanBase]:
read -rp "Enter your company unit name [DBA]:

echo generate certificate...
SUBJ="/C=CN/ST=${PROVINCE}/L=${CITY}/O=${COMPANY}/OU=${UNIT}/CN=${DOMAIN}"
sudo openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
    -keyout /etc/pki/nginx/odcserver.key \
    -out /etc/pki/nginx/odcserver.crt \
    -subj "${SUBJ}"

echo "check generated files (use ls -l odcserver.
ls -l odcserver.

Deploy the Nginx proxy

After you deploy the ODC image on all nodes, you need to deploy the Nginx proxy on one of the ODC nodes to route your requests to applications on different nodes. To deploy the Nginx proxy, you need to obtain and run the Nginx image on one node, copy the configuration file locally, and then modify the file. After the modification, stop running the Nginx image and then restart it to apply the modified configuration file.

Load the image

Select a node and load and run the Nginx image. You can use the following methods to load the Nginx image:

• Pull the image from the Docker warehouse on the host.

• Download the image package to a local device, copy the package to the host, and decompress the package.

Pull the image from the Docker warehouse

Ensure that Docker is installed on the host and the host has access to the Internet. Then, run the following command in the command-line tool to pull the Nginx image from the Docker warehouse:

docker pull nginx

Copy the image package to the host and decompress the package

1. Ensure that Docker is installed on the host and the host has access to the Internet. Then, run the following command on the host to download the image package to a local device:

   docker save nginx:latest | gzip - >nginx.tar.gz
   

2. Copy the image package to the host and run the following command to load the image:

   gunzip -c nginx.tar.gz | docker load
   

Configure the configuration file

Before you start running the Nginx image, you must modify settings of the configuration file. The ODC image package contains templates of the configuration file. You can copy a template to a local device and edit the configuration where necessary. The templates of the configuration file vary, depending on whether you have deployed a self-signed certificate. We recommend that you copy a template based on the deployment and modify the configuration where necessary.

Configure the configuration file that does not contain the self-signed certificate

Run the following command to copy the configuration file from the image package to a custom path on the host for editing. The tilde (~) indicates the local path where the configuration file is located.

docker cp -a <odc_image_name>:/opt/odc/conf/nginx.conf.template  ~/<conf_local_path>

The following code shows a template of the configuration file. We recommend that you modify the settings based on the comments.

# Nginx conf template for http deployment
# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    #set 0 to disable request body size check, for support large size file upload
    client_max_body_size 0;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # for websocket configuration
    map $http_upgrade $connection_upgrade {
        default upgrade;
        '' close;
    }

    # load balancing configuration
    # notice under_score character are not allowed for upsteram name, 400 Bad Request happens if used
    # please use ip_hash strategy
    # one server line for each odc-server node
    upstream odcbackends {
      ip_hash;
      # Change it to the actual ODC server address. This operation is optional.
      server xxx.x.x.x:8989;
      # add more servers here
    }

    #https server, proxy to odc-server 8989 port
    server {
        listen 80;
        # If IPv6 is enabled, you must uncomment the following content:
        #listen [::]:80;

        # Change it to the actual domain name of the site. This operation is optional.
        server_name  odc.oceanbase.com;

        location / {
            proxy_pass http://odcbackends;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-For  $proxy_add_x_forwarded_for;
            proxy_read_timeout 1800;
            proxy_send_timeout 1800;
            proxy_connect_timeout 75;
            proxy_next_upstream off;
        }
    }

}

Configure the configuration file that contains a self-signed certificate

Run the following command to copy the configuration file from the image package to a custom path on the host for editing. The tilde (~) indicates the local path where the configuration file is located.

docker cp -a <odc_image_name>:/opt/odc/conf/nginx.conf.https.template  ~/<conf_local_path>

The following code shows a template of the configuration file. We recommend that you modify the settings based on the comments.

# Nginx conf template for https deployment
# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    #set 0 to disable request body size check, for support large size file upload
    client_max_body_size 0;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # for websocket configuration
    map $http_upgrade $connection_upgrade {
        default upgrade;
        '' close;
    }

    # load balancing configuration
    # notice under_score character are not allowed for upsteram name, 400 Bad Request happens if used
    # please use ip_hash strategy
    # one server line for each odc-server node
    upstream odcbackends {
      ip_hash;
      # Change it to the actual ODC server address. This operation is optional.
      server xxx.x.x.x:8989;
      #add more servers here
    }

    # redirect http to https
    server {
        listen 80 default_server;

        # If IPv6 is enabled, you must uncomment the following content:
        #listen [::]:80 default_server;

        location / {
            return 301 https://$host$request_uri;
        }
    }

    # https server, proxy to odc-server 8989 port
    server {
        listen 443 ssl http2;
        # If IPv6 is enabled, you must uncomment the following content:
        #listen [::]:443 ssl http2;

        # Change it to the actual domain name of the site. This operation is optional.
        server_name  odc.oceanbase.com;

        # You can use /opt/odc/bin/generate-odc-ssl-certificate.sh to generate a self-signed certificate.

        # Change the path of the certificate file in the case of a mismatch.
        ssl_certificate /etc/pki/nginx/odcserver.crt;
        ssl_certificate_key /etc/pki/nginx/odcserver.key;
        ssl_session_timeout 1d;
        ssl_session_cache shared:MozSSL:10m;  # about 40000 sessions
        ssl_session_tickets off;

        # intermediate configuration
        ssl_protocols TLSv1.2 TLSv1.3;
        ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
        ssl_prefer_server_ciphers off;

        # HSTS (ngx_http_headers_module is required) (63072000 seconds)
        add_header Strict-Transport-Security "max-age=63072000" always;

        location / {
            proxy_pass http://odcbackends;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-For  $proxy_add_x_forwarded_for;
            proxy_read_timeout 1800;
            proxy_send_timeout 1800;
            proxy_connect_timeout 75;
            proxy_next_upstream off;
        }
    }

}

Start the image

After the configuration file is modified locally, run the following command in the command-line tool to restart the Nginx image. The tilde (~) indicates the local path where the configuration file is located.

docker run --network host --name nginx -v ～/<conf_local_path>:/etc/nginx/nginx.conf -v /etc/pki/nginx/:/etc/pki/nginx/ -d nginx

The following table describes the parameters in the command.

Configure certificate trust

A browser does not trust a self-signed certificate. You must manually configure certificate trust in the browser.

Configure certificate trust in Windows

Google Chrome is used as an example to describe the certificate trust configuration procedure.

1. Log on to ODC in Chrome. When the system prompts that the connection is not secure, click Not Secure on the left side of the address bar and click Certificate in the list that appears to view the certificate details.

   

2. In the Certificate window, click the Details tab and click Copy to File in the lower-right corner of the tab.

   

3. In the Certificate Export Wizard step, click Next.

4. In the Export File Format step, select DER encoded binary X.509 (CER) and click Next.

5. In the file path selection step, click Browse to open the file explorer, select an export path, enter a file name in the File Name field, and click Save. Then, click Next.

6. In the Completing the Certificate Export Wizard step, click Finish in the lower-right corner to export the file to a local device.

7. Find the certificate file in the path specified in Step 5, right-click the certificate file, and then select Install Certificate from the context menu that appears.

8. In the Certificate Import Wizard window, click Next.

9. In the Certificate Store step, select Place all certificates in the following store and click Browse. In the Select Certificate Store dialog box, select Trusted Root Certification Authorities and click OK. In the Certificate Store step, click Next.

10. In the Completing the Certificate Import Wizard step, click Finish in the lower-right corner.

11. After the certificate trust configuration is completed, log on to ODC again in the browser.

Configure certificate trust in macOS

Google Chrome is used as an example to describe the certificate trust configuration procedure.

1. Log on to ODC in Chrome. When the system prompts that the connection is not secure, click Not Secure on the left side of the address bar and click Certificate in the list that appears to view the certificate details.

   

2. Details of the certificate are displayed.

   

3. Create a file with the extension of .cer, copy the certificate details to this file, and save the file.

   

4. In macOS, choose Applications > Utilities > Keychain Access. The Keychain Access window appears. In the left-side navigation pane of the window, click System and then click the plus sign (+) in the navigation bar on top of the page. In the window that appears, select the created certificate file to add the certificate to the system.

   

5. Double-click the odc.oceanbase.com,odc.oceanbse.net certificate. In the certificate details window that appears, expand the Trust section. In the configuration list that appears, set When using this certificate to Always Trust.

   

6. Close the certificate details window. When you try to close the window, the system reminds you to enter the password to make the configuration take effect. Enter the password to finish the configuration.

7. After the certificate trust configuration is completed, log on to ODC again in the browser.