What's New|V3.2.0|OceanBase Loader and Dumper| docs|Distributed Database

OBLOADER and OBDUMPER V3.2.0 released in August 2022 provide optimized data import performance and improved formats of exported date and time data types.

New features

Tool Description

OBLOADER

Tool	Description
OBLOADER	The ob-partition-calculator component is upgraded to V1.1.9. The performance in processing the imported data is improved by 20%, and the resource requirements for the Java virtual machine (JVM) and the system are reduced by 50%. A confirmation step is added for you to confirm risky operations such as TRUNCATE TABLE during a full database import. The `-Dfile.split` parameter is added. By default, the parameter is set to UNSAFE in the script for coarse-grained splitting of files. You can set it to SAFE for fine-grained splitting of files. A generated column can be imported and used as a partitioning key. A generated column is a virtual column, whose values are calculated based on the expression only when the column is used. Therefore, a generated column does not occupy any storage space.
OBDUMPER	The `--time-value-format` option is added to specify the export format of the TIME data type in MySQL mode. By default, the TIME data type is exported in the HH:mm:ss format. The precision is the same as that in the table definition. The `--datetime-value-format` option is added to specify the export format of the DATETIME data type in MySQL mode. By default, the DATETIME data type is exported in the yyyy-MM-dd HH:mm:ss.SSSSSS format. The precision is the same as that in the table definition. The `--timestamp-value-format` option is added to specify the export format of the TIMESTAMP data type in MySQL or Oracle mode. By default, the TIMESTAMP data type is exported in the yyyy-MM-dd HH:mm:ss format. The precision is the same as that in the table definition. The `--timestamp-tz-value-format` option is added to specify the export format of the TIMESTAMP WITH TIME ZONE data type in Oracle mode. By default, the TIMESTAMP WITH TIME ZONE data type is exported in the yyyy-MM-dd HH:mm:ss.SSSSSS format. The precision is the same as that in the table definition. The `--date-value-format` option is added to specify the export format of the DATE data type. By default, the DATE data type is exported in the yyyy-MM-dd format in MySQL mode and in the yyyy-MM-dd HH:mm:ss format in Oracle mode. The `--timestamp-ltz-value-format` option is added to specify the export format of the TIMESTAMP WITH LOCAL TIME ZONE data type in Oracle mode. By default, the TIMESTAMP WITH LOCAL TIME ZONE data type is exported in the yyyy-MM-dd HH:mm:ss format. The precision is the same as that in the table definition. The `--exclude-virtual-columns` option is added to specify not to export the data of generated columns. By default, the data of generated columns is exported. For more information, see Command-line options of OBDUMPER.

The ob-partition-calculator component is upgraded to V1.1.9.
The performance in processing the imported data is improved by 20%, and the resource requirements for the Java virtual machine (JVM) and the system are reduced by 50%.
A confirmation step is added for you to confirm risky operations such as TRUNCATE TABLE during a full database import.
The -Dfile.split parameter is added. By default, the parameter is set to UNSAFE in the script for coarse-grained splitting of files. You can set it to SAFE for fine-grained splitting of files.
A generated column can be imported and used as a partitioning key. A generated column is a virtual column, whose values are calculated based on the expression only when the column is used. Therefore, a generated column does not occupy any storage space.

OBDUMPER

The --time-value-format option is added to specify the export format of the TIME data type in MySQL mode. By default, the TIME data type is exported in the HH:mm:ss format. The precision is the same as that in the table definition.
The --datetime-value-format option is added to specify the export format of the DATETIME data type in MySQL mode. By default, the DATETIME data type is exported in the yyyy-MM-dd HH:mm:ss.SSSSSS format. The precision is the same as that in the table definition.
The --timestamp-value-format option is added to specify the export format of the TIMESTAMP data type in MySQL or Oracle mode. By default, the TIMESTAMP data type is exported in the yyyy-MM-dd HH:mm:ss format. The precision is the same as that in the table definition.
The --timestamp-tz-value-format option is added to specify the export format of the TIMESTAMP WITH TIME ZONE data type in Oracle mode. By default, the TIMESTAMP WITH TIME ZONE data type is exported in the yyyy-MM-dd HH:mm:ss.SSSSSS format. The precision is the same as that in the table definition.
The --date-value-format option is added to specify the export format of the DATE data type. By default, the DATE data type is exported in the yyyy-MM-dd format in MySQL mode and in the yyyy-MM-dd HH:mm:ss format in Oracle mode.
The --timestamp-ltz-value-format option is added to specify the export format of the TIMESTAMP WITH LOCAL TIME ZONE data type in Oracle mode. By default, the TIMESTAMP WITH LOCAL TIME ZONE data type is exported in the yyyy-MM-dd HH:mm:ss format. The precision is the same as that in the table definition.
The --exclude-virtual-columns option is added to specify not to export the data of generated columns. By default, the data of generated columns is exported. For more information, see Command-line options of OBDUMPER.

Fixed issues

OBLOADER does not support the latest kernel version of OceanBase Database.
OBLOADER terminates when it splits a file.
The monitoring logs are repeatedly printed when you use OBLOADER to import data.
OBLOADER terminates when it imports a table group.
An error occurs when you use OBLOADER to import data to a table definition with a generated column as a partitioning key.
Duplicate data is generated when you use OBLOADER to import data, but no discard file is generated.
The empty values of the TINYTEXT, MEDIUM TEXT, TEXT, and LANGTEXT data types imported by OBLOADER change to NULL.
The --ignore-unhex option becomes invalid when you use OBLOADER to import data in the CUT, SQL, or POS format.
By default, when the --ctrl-path option is not specified, OBLOADER searches and uses control files from the data/ directory.
When OBDUMPER exports snapshot data, the data exported from non-template-based subpartitions is duplicate.
A syntax error occurs when you use OBDUMPER to export the result set of a conditional query in Oracle mode.
OBDUMPER repeatedly exports data and cannot exit when the conditional query contains an OR clause.
OBDUMPER cannot export the FLOAT data type in Oracle mode.
The table name must be specified when you use OBDUMPER to export the result set of a custom query.
OBDUMPER terminates when it exports a table with a primary key but the primary key column is excluded.
When you use OBDUMPER to export the result set of a custom query in the SQL format, the table name is missing.
OBDUMPER cannot export the definitions of generated columns in OceanBase Database V1.4.x in MySQL mode.
The constraint definitions exported by OBDUMPER lack spaces, which can cause syntax parsing errors.
OBDUMPER cannot export table group definitions in OceanBase Database V1.4.x, V2.x, or V3.x in MySQL mode.
The precision of the TIMESTAMP, TIME, DATETIME, TIMESTAMP WITH TIME ZONE, and TIMESTAMP WITH LOCAL TIME ZONE data types exported by OBDUMPER is inconsistent with that in the database object definition.