Common issues and solutions

Background information

After the JNI framework is introduced as the bridge for connecting to external data lakes, various environment initialization issues may occur.

Issue 1: An error is returned when you create an HDFS or ODPS JNI external table.

Error 1: ERROR 11056: JNI env could not be found

Cause

The JNI environment initialization fails for the following reasons:

1. The ob_java_home parameter is incorrectly set, which prevents the system from loading the libjvm.so dynamic library. This dynamic library is essential for initializing the environment and starting the JVM.

2. The ob_java_opts parameter is incorrectly set. Due to the limitations in the integration between the CPP framework and the JNI framework, it is necessary to forcibly set the relevant parameters for starting the JVM process. For more information, see ob_java_opts.

3. The path where the libhdfs.so library file (the communication bridge between OceanBase Database and the JVM) is stored does not match the _ob_additional_lib_path parameter. Ensure that the path is set accurately.

Solution

1. Manually verify whether the java home setting is correct and check whether the current user has access permissions to the specified path. It is possible that the java home exists, but the OBServer startup user does not have access permissions to the path.

2. Check whether the ob_java_opts parameter is correctly set by referring to ob_java_opts.

3. Check whether the path specified by the _ob_additional_lib_path parameter exists and whether the OBServer startup user has access permissions to the path.

Error 2: ERROR 11032: HDFS: path not found

Cause

The HDFS: path not found error typically occurs in the following scenarios:

1. In a HDFS environment with Kerberos authentication enabled, if the user does not have sufficient permissions to access a specific HDFS external table, the -11032 error will be returned.

2. If the Kerberos authentication configuration is correct, but the HDFS: path not found error still occurs, it is likely that the specified external table path in the CREATE EXTERNAL TABLE statement is incorrect, pointing to a non-existent path.

3. If the external table access is initially normal but the 11032 error occurs after a period of query execution, it may be due to a bad block in the corresponding external table path on HDFS. This means that the HDFS DataNode reports that the data block is faulty and inaccessible, triggering the error.

Solution

1. Confirm that the Kerberos authentication keytab and principal are valid and that the corresponding user has access permissions to the HDFS cluster. Reconfigure the external table permissions and set up Kerberos authentication accordingly.

2. Verify that the specified HDFS path is correct and exists. Re-establish the HDFS path mapping for the external table.

3. Contact the customer's operations and maintenance team to confirm the data block status. You can then choose to adjust the bad block or replace the environment for further verification.

Error 3: ERROR 4016: Internal Error

Cause

The Internal Error may occur for the following reasons:

1. During an external table query, an Internal Error may occur if the actual data types of the external table do not match the expected types. For example, if a String type column is mapped to an Int type, some field data conversion may fail.
2. Other unexpected execution paths in the program may trigger an Internal Error.

Solution

1. Verify that the metadata type mapping in the CREATE EXTERNAL TABLE statement and the corresponding external table data files is correct.

2. Check the relevant logs, capture the error log stack trace, and identify the specific details of the error.