OBLOADER & OBDUMPER V4.2.8.1

OBLOADER & OBDUMPER V4.2.8.1 released in February 2024 adds direct load parameters in the session.config.json file.

Version information

• Current version: V4.2.8.1

• Previous version: V4.2.8

• Release date: February 21, 2024

Supported OceanBase Database versions

The following table lists the OceanBase Database versions supported by OBLOADER & OBDUMPER V4.2.8.1.

Upgrade considerations

• OBLOADER & OBDUMPER V4.2.8 and later allow you to specify the value of -u/--user in the <user>@<tenant>#<cluster> format. To ensure backward compatibility, you can still use the -t/--tenant and -c/--cluster options.

• In a scenario of importing data to Alibaba Cloud Object Storage Service (OSS) or Hadoop Distributed File System (HDFS), if the --storage-uri option is specified, the -f/--file-path option is no longer a must. OBLOADER & OBDUMPER will automatically create a directory named tmp in its root directory as a temporary directory for storing logs and checkpoint files.

  Notice

  This modification applies only to a debugging environment. In a production environment, we recommend that you use the -f or --log-path option to specify the log storage directory.

• The parsing behavior in the POS format is modified. At present, the parser will parse a whole line. To be specific, in addition to the byte length defined in the control file, the parser will also read a line break in each line. You may need to modify the control file to support this behavior. The command for calculating the number of bytes in the first line of the example.dat file is as follows: expr $(head -n 1 example.dat | wc -c) - 1.

• Performance optimization for schema export in OceanBase Database in Oracle compatible mode applies only to the following scenarios:

  • The --no-sys option is used for limited export in Oracle compatible mode of OceanBase Database of a version earlier than V4.0.0.

  • The --compact-schema option is used for export in Oracle compatible mode of OceanBase Database V4.0.0 and later.

Feature updates

• OBLOADER:

  • Direct load parameters are added in the connection configuration file session.config.json in the {ob-loader-dumper}/conf directory. For more information, see Direct load.

Bug fixes

• Fixed the issue where an alert is repeatedly generated if the .ckpt file is absent during data import by using OBLOADER.

• Fixed the issue where an error indicating unique key conflicts still occurs when the --replace-data option is specified in OBLOADER.

• Fixed the issue where when OBLOADER is used to import data in the POS format, the control file cannot be properly loaded if the --table option contains a wildcard that matches multiple tables, for example, table 'table_name_*'.

• Fixed the issue where the NullPointerException (NPE) error occurs if --table '*' is specified when OBLOADER is used to import data in the POS format.

• Fixed the issue where the --partition option does not take effect if both the --no-sys and --partition options are specified when OBDUMPER is used to export table data from OceanBase Database V3.2.x.

• Fixed the issue where the name of the exported database object file does not have the .sql suffix when OBDUMPER is used to export database object data with the -f option specified.

• Fixed the issue where the pagination of a partitioned table takes a very long time when OBDUMPER is used to export data from OceanBase Database of a version earlier than V4.0.0.

• Fixed the issue where an empty file cannot overwrite a file that contains data when OBDUMPER is used for export with the --partition option specified.

Known issues

• The exported database object definitions may contain the name of the database to which the object belongs.

• OBDUMPER cannot export PL object definitions from OceanBase Database of versions earlier than V2.2.30 in MySQL compatible mode.

• If you do not specify the password of the sys tenant on the command line, OBDUMPER cannot export table group definitions from OceanBase Database of versions earlier than V2.2.70.

• If you do not specify the password of the sys tenant on the command line, OBDUMPER cannot export index definitions from OceanBase Database of versions earlier than V2.2.50 in Oracle compatible mode.

• If you do not specify the password of the sys tenant on the command line, OBDUMPER cannot export partition information of unique indexes from OceanBase Database of versions earlier than V2.2.70 in Oracle compatible mode.

• If you do not specify the password of the sys tenant on the command line, OBDUMPER cannot export unique index definitions of partitioned tables from OceanBase Database V2.2.70 and later versions in Oracle compatible mode.

• When you use OBLOADER to import data, if the specified file format, such as --sql and --csv, does not match the actual file format, an error occurs.

• In MySQL compatible mode of OceanBase Database, when you explicitly specify object names, such as the names of tables and views, in a case-sensitive manner to import or export only objects of the case-sensitive names, objects are imported or exported in a case-insensitive manner.

• The JDBC high-availability mode sequential is unavailable for now. You can use the loadbalance mode instead.

• If you specify the --remove-newline option to remove line breaks, unexpected escape characters are generated. We recommend that you do not use this option.

• If you use direct load in a multi-table restore scenario, the efficiency may be low.

Considerations

• In a CUT file, each data record is stored in an entire line. When you specify the --cut option on the OBDUMPER command line, if the exported data contains a single-character field separator, OBDUMPER escapes special characters in the data, such as separators, carriage returns, and line breaks. For example, if the data is abc|def and the separator is |, the exported data is abc\|def.

• When you specify the --cut option on the OBLOADER command line, do not use the --trail-delimiter option if no field separator or separator string exists at the end of the data line in the file. Otherwise, a serious error occurs on OBLOADER.

• If you specify the --logical-database option on the command line, the definition of a random physical database shard is exported and the shard cannot be directly imported to the database. You need to manually convert the exported physical shard to a logical one before you import it to the database for business use.

• When you specify the --partition option on the command line to export data in a partition, you need to specify the name of the subpartition for a composite partitioned table. OBDUMPER cannot export data in a partition of a composite partitioned table. If the specified partition name does not exist, OBDUMPER returns an error.

• If you specify the --add-extra-message option on the command line to export table definitions, OBDUMPER exports the name of the table group to which each table belongs. This option depends on privileges of the sys tenant. If OBDUMPER does not have privileges of the sys tenant, do not specify this option.

• Before you use OBDUMPER & OBLOADER on OceanBase Database V3.2.4 or later, set the system parameter open_cursors to a large value. Otherwise, an error may occur during the import or export. After the data is imported or exported, reset the system parameter to the initial value, such as ALTER SYSTEM SET open_cursors = 65535;.

• If the schema of a table has been changed in OceanBase Database V4.0.0 or later, you cannot use OBDUMPER to export the baseline data obtained after the last major compaction. The baseline data is also known as the consistent snapshot data. You can manually initiate a major compaction and then export the most recent baseline data.

• When you use OBDUMPER of a version earlier than V4.2.0 to export data from OceanBase Database in MySQL compatible mode, OceanBase Connector/J converts the zero values of date and time columns in the database to NULL. If a column has a NOT NULL constraint, an error is reported during the export. OBDUMPER V4.2.0 can export zero values of date and time columns, but cannot identify whether the original data is NULL or zero values during export and forcibly converts the data to zero values by default. In addition, exported zero values of the DATETIME and TIMESTAMP data types may be distorted into approximate non-zero values. If the sql_mode variable in the database is specified with the NO_ZERO_DATE or NO_ZERO_IN_DATE constraint, an error is reported when zero values are imported.

  Take note of this consideration on zero values when you export data of the DATE, DATETIME, TIME, YEAR, or TIMESTAMP type from OceanBase Database in MySQL compatible mode.

• Feature differences in schema export in the MySQL compatible and Oracle compatible modes:

  Compatibility Mode	Password of the sys tenant provided	Password of the sys tenant not provided
  MySQL compatible	Tables, views, table groups, stored procedures, and functions are supported.	The export behavior is basically the same as that when the password of the sys tenant is provided, except for the following known issues: Table group definitions cannot be exported for OceanBase Database of a version earlier than V2.2.70. Partition information of unique indexes cannot be exported for OceanBase Database of a version earlier than V2.2.70. Index definitions cannot be exported for Oracle-compatible tenants of OceanBase Database V2.2.30 or earlier. Definitions of unique indexes on partitioned tables cannot be exported for Oracle-compatible tenants of OceanBase Database of a version ranging from V2.2.70 (included) to V4.0.0.
  Oracle compatible	Tables, views, triggers, synonyms, sequences, stored procedures, functions, packages, table groups, and types are supported.