OBLOADER & OBDUMPER V4.2.6

OBLOADER & OBDUMPER V4.2.6 released in October 2023 enhances the capability to parse Parquet files, improves the performance on servers with low specifications, supports the configuration of the JDBC high-availability mode, and fixes some known issues.

Version information

• Current version: V4.2.6

• Previous version: V4.2.5

• Release date: October 16, 2023

Supported OceanBase Database versions

The following table lists the OceanBase Database versions supported by OBLOADER & OBDUMPER V4.2.6.

Feature updates

• OBLOADER:

  • The DECIMAL, DATE, TIME, and TIMESTAMP types are supported when you import Parquet files.

  • You can convert column separators in a CSV file to hexadecimal characters before you import the file, so that OBLOADER takes the column separators as special (invisible) characters.

  • The --direct, --rpc-port, and --parallel options are provided so that you can connect to an OBServer node to import data in bypass mode. For more information, see Command-line options of OBLOADER.

• When you use OBLOADER to import data in Oracle mode of OceanBase Database, you can specify the --replace-data option to process files that contain duplicate data.

• OBDUMPER:

  • The --enable-hidden-pk option is provided so that tables without a primary key can use a hidden primary key to increase the export speed. For more information, see Command-line options of OBDUMPER.

• OBLOADER and OBDUMPER:

  • The connection configuration file session.properties is replaced with session.config.json so that you can manage session parameters more flexibly. For more information, see Connection settings.

  • You can configure the JDBC high-availability mode in the following ways:

    • Method 1: Specify the jdbc_hamode parameter in the session.config.json file. For more information, see Failover and load balancing modes.

    • Method 2: Specify the -h or --host command-line option. You can enter one or more IP addresses or domain names separated with commas (,).

    • Method 3: Specify the -P or --port command-line option. You can enter one or more ports separated with commas (,). If you enter only one port, all hosts share the same port.

Bug fixes

• Fixed the issue where DDL files imported by OBLOADER contain Chinese garbled characters.

• Fixed the issue where a parsing failure occurs during the import if the default or comment column in a CSV file imported by OBLOADER contains comment characters and the --empty-string option is specified in the command line.

• Fixed the issue where the table is locked and data import is affected when OBLOADER becomes abnormal or forcibly exits due to an exception during bypass import.

• Fixed the issue where an error indicating primary key conflicts is returned when the --replace-object option is specified for OBLOADER.

• Fixed the issue where a PreparedStatement object is garbage collected before being closed when OBLOADER uses soft references for caches.

• Fixed the issue where partition calculation fails when the --public-cloud command-line option is specified for OBLOADER to import data to ApsaraDB for OceanBase.

• Fixed the issue where an export failure occurs when the --query-sql command-line option is specified for OBDUMPER and no alias is specified for empty fields.

• Fixed the issue where the export fails if the name of a sequence is in lowercase when OBDUMPER is used to export data from OceanBase Database in Oracle mode.

• Fixed the issue where garbled characters are displayed in a DDL file when the file is exported by OBDUMPER with the --file-encoding 'GBK' option specified and then imported by OBLOADER with the --file-encoding 'GBK' option specified.

• Fixed the issue where sequences declared with NOCACHE are recorded as sequences declared with CACHE 1 when OBDUMPER is used to export sequences.

• Fixed the security vulnerabilities introduced by HDFS 3.3.4.

Known issues

• The exported database object definitions may contain the name of the database to which the object belongs.

• OBDUMPER cannot export PL object definitions from OceanBase Database of versions earlier than V2.2.30 in MySQL mode.

• If you do not specify the password of the sys tenant on the command line, OBDUMPER cannot export table group definitions from OceanBase Database of versions earlier than V2.2.70.

• If you do not specify the password of the sys tenant on the command line, OBDUMPER cannot export index definitions from OceanBase Database of versions earlier than V2.2.50 in Oracle mode.

• If you do not specify the password of the sys tenant on the command line, OBDUMPER cannot export partition information of unique indexes from OceanBase Database of versions earlier than V2.2.70 in Oracle mode.

• If you do not specify the password of the sys tenant on the command line, OBDUMPER cannot export unique index definitions of partitioned tables from OceanBase Database V2.2.70 and later in Oracle mode.

• When you use OBLOADER to import data, if the specified file format, such as --sql and --csv, does not match the actual file format, an error occurs.

• In MySQL mode of OceanBase Database, when you explicitly specify object names, such as the names of tables and views, in a case-sensitive manner to import or export only objects of the case-sensitive names, objects are imported or exported in a case-insensitive manner.

• The JDBC high-availability mode “sequential” is unavailable for now. You can use the “loadbalance” mode instead.

Considerations

• In a CUT file, each data record is stored in an entire line. When you specify the --cut option on the OBDUMPER command line, if the exported data contains a single-character field separator, OBDUMPER escapes special characters in the data, such as separators, carriage returns, and line breaks. For example, if the data is abc|def and the separator is |, the exported data is abc\|def.

• When you specify the --cut option on the OBLOADER command line, do not use the --trail-delimiter option if no field separator or separator string exists at the end of the data line in the file. Otherwise, a serious error occurs on OBLOADER.

• When you specify the --logical-database option on the command line, the definition of a random physical database shard is exported and the shard cannot be directly imported into the database. You need to manually convert the exported physical shard to a logical one before you import it to the database for business use.

• When you specify the --partition option on the command line to export data in a partition, you need to specify the name of the subpartition for a composite partitioned table. OBDUMPER cannot export data in a partition of a composite partitioned table. If the specified partition name does not exist, OBDUMPER returns an error.

• When you specify the --add-extra-message option on the command line to export table definitions, OBDUMPER exports the name of the table group to which each table belongs. This option depends on privileges of the sys tenant. If OBDUMPER does not have privileges of the sys tenant, do not specify this option.

• Before you use OBDUMPER & OBLOADER on OceanBase Database V3.2.4 or later, set the system parameter open_cursors to a large value. Otherwise, an error may occur during the import or export. After the data is imported or exported, reset the system parameter to the initial value, for example, ALTER SYSTEM SET open_cursors = 65535;.

• If the schema of a table has been changed in OceanBase Database V4.0.0 or later, you cannot use OBDUMPER to export the baseline data, namely the consistent snapshot data, obtained after the last major compaction. You can manually initiate a major compaction and then re-export the most recent baseline data.

• When you use OBDUMPER of a version earlier than V4.2.0 to export data from a MySQL tenant of OceanBase Database, the Java Database Connectivity (JDBC) driver converts the zero date and time values in the database to NULL. If a column has a NOT NULL constraint, an error is reported during the export. OBDUMPER V4.2.0 can export zero date and time values, but cannot identify whether the original data is NULL or zero values during export and forcibly converts the data to zero values by default. In addition, exported zero values of the DATETIME and TIMESTAMP data types may be distorted into non-zero approximate values. If the sql_mode variable in the database is specified with the NO_ZERO_DATE or NO_ZERO_IN_DATE constraint, an error is reported when zero-value data is exported. In a MySQL tenant of OceanBase Database, data types that involve the preceding zero-value issues include DATE, DATETIME, TIME, YEAR, and TIMESTAMP.

• Feature differences in schema export