Command-line options

OBLOADER allows you to specify the information required for import in command-line options. For more information about the options and their scenarios and examples, see Options and Scenarios and examples.

Options

Connection options

OBLOADER can read data from and write data to an OceanBase database only after connecting to the database. You can connect to an OceanBase database by specifying the following options:

• -h host_name , --host= host_name

  The host IP address for connecting to ODP or a physical OceanBase Database node.
  
  

• -P port_num , --port= port_num

  The host port for connecting to ODP or a physical OceanBase Database node.
  
  

• -c cluster_name , --cluster= cluster_name

  The OceanBase cluster to connect to. If this option is not specified, the database physical node is connected. Relevant options, such as -h and -P, specify the IP address and port of the physical node of the database. When this option is specified, ODP is connected. Relevant options, such as -h and -P, specify the IP address and port of ODP.
  
  

• -t tenant_name , --tenant= tenant_name

  The OceanBase Database tenant to connect to. For more information about tenants, see the official OceanBase documentation.
  
  

• -u user_name , --user= user_name

  The username for connecting to OceanBase Database. If you specify an incorrect username, OBLOADER cannot connect to the database.
  
  

• -p ' password' , --password=' password'

  The user password for connecting to OceanBase Database. If no password is set for the current database account, you do not need to specify this option. To specify this option on the command line, you must enclose the value in single quotation marks (' '). Example: -p'******'.

  Note
  If you are using Windows OS, enclose the value in double quotation marks (" "). This rule also applies to string values of other options.

• -D database_name , --database= database_name

  Specifies to import database object definitions and table data into the specified database.
  
  

• --sys-user sys_username

  The username of a user with required privileges under the sys tenant, for example, root or proxyro. OBLOADER requires a special user under the sys tenant to query the metadata in system tables. Default value: root. You do not need to specify this option for OceanBase Database V4.0.0.0 and later versions.
  
  

• --sys-password ' sys_password'

  The password of a user with required privileges under the sys tenant. This option is used along with --sys-user. By default, the password of the root user under the sys tenant is left empty. When you specify this option on the command line, enclose the value in single quotation marks (' '). Example: --sys-password '******'. You do not need to specify this option for OceanBase Database V4.0.0.0 and later versions.

  Note
  If this option is not specified, OBLOADER cannot query metadata in system tables, and the import features and performance may be greatly affected.

• --public-cloud

  Specifies to import database objects or table data to OceanBase clusters deployed in the public cloud. If you specify this option on the command line, you do not need to specify the -t or -c connection option. OBDUMPER turns on the --no-sys option by default. For more information about the --no-sys option, see the corresponding option description. The use of the --public-cloud or --no-sys option will affect the import features, performance, and stability. OceanBase Database V2.2.30 and later versions support throttling on the server. Therefore, before you use the --public-cloud or --no-sys option, to ensure the stability of data import, you can run the following commands to set throttling thresholds as required on the server:

  alter system set freeze_trigger_percentage=50;
  alter system set minor_merge_concurrence=64;
  alter system set writing_throttling_trigger_percentage=80 tenant='xxx';
  

• --no-sys

  Specifies to import database objects or table data to OceanBase clusters deployed in the private cloud when the user cannot provide the sys tenant password. Unlike the --public-cloud option, when you use the --no-sys option, you need to specify the -t connection option on the command line and also need to add the -c option to connect to the ODP service. In OceanBase Database V4.0.0.0 or earlier, if you do not specify the --public-cloud or --no-sys option, you must specify the --sys-user and --sys-password options for OBLOADER.
  
  

• --logical-database

  Specifies to import data by using ODP (Sharding). When you specify the --logical-database option on the command line, the definition of a random physical database shard is exported and the shard cannot be directly imported to the database. You need to manually convert the exported physical shard to a logical one before you import it to the database for business use.

Feature options

• -f ' file_path ' , --file-path= ' file_path '

  The absolute path on a local disk for storing data files. When you import data files to the OSS bucket, you must specify the -f option to save the generated logs and binary files.
  
  

• --file-suffix ' suffix_name '

  The file name extension of data files to be imported. Generally, the file name extension is correlated with the file format. For example, a CSV file is usually named as xxx.csv. If you do not strictly follow the naming conventions, you may name a CSV file with any extension, such as .txt. In this case, OBLOADER cannot identify the file as a CSV file. This option is optional. A default value is available for each data format. The default file name extension is .csv for a CSV file, .sql for an SQL file, and .dat for a CUT or POS file. When you specify this option on the command line, enclose the value in single quotation marks (''). Example: --file-suffix '.txt'.
  
  

• --file-encoding ' encode_name '

  The file character set used when OBLOADER reads data files, which is not the database character set. When you specify this option on the command line, enclose the value in single quotation marks (' '). Example: --file-encoding 'GBK'. Default value: UTF-8.
  
  

• --ctl-path ' control_path '

  The absolute path on a local disk for storing control files. You can configure built-in preprocessing functions in a control file. Data will be preprocessed by these functions before being imported. For example, the functions can perform case conversion and check the data for empty values. For the use of control files, see Data processing. When you specify this option on the command line, enclose the value in single quotation marks (' '). Example: --ctl-path '/home/controls/'.
  
  

• --log-path ' log_path '

  The output directory for the operational logs of OBLOADER. If this option is not specified, OBLOADER operational logs are stored in the directory specified by the -f option. Redirection is not required for log output, unless otherwise specified.
  
  

• --ddl

  Imports DDL files. A DDL file stores the database object definitions. The file is named in the format of object name-schema.sql. When this option is specified, only database object definitions are imported, and table data is not imported.

  Notice
  Avoid comments or statements to enable/disable a feature in the file. If database objects depend on each other, the import may fail and manual intervention is required.

• --sql

  Imports data files in the SQL format. In an SQL file, data is stored in the format of INSERT statements. The file is named in the format of table name.sql. Each line of table data corresponds to an executable INSERT statement in an SQL file. An SQL file is different from a DDL file in terms of content format. We recommend that you use this option together with the --table option. When it is used together with the --all option, OBLOADER imports only table data but not database object definitions.

  Notice
  The data cannot contain SQL functions, special characters, line breaks, and so on. Otherwise, the file may not be correctly parsed.

• --orc

  Imports data files in the ORC format. An ORC file stores data in the column-oriented format. The file is named in the format of table name.orc. For more information about ORC format definitions, see Apache ORC.
  
  

• --par

  Imports data files in the Parquet format. A Parquet file stores data in the column-oriented format. The file is named in the format of table name.par. For more information about Parquet format definitions, see Apache Parquet.
  
  

• --mix

  Imports MIX files. A MIX file stores a mix of DDL and DML statements, and does not have strict naming conventions.

  Notice
  MIX files do not have a strict format and feature a complex processing process and poor performance. Therefore, we recommend that you do not use MIX files.

• --csv

  Imports data files in the CSV format. In a CSV file, data is stored in the standard CSV format. The file is named in the format of table name.csv. For CSV format specifications, see the definitions in RFC 4180. Delimiter errors are the most common errors that occur in CSV files. Single or double quotation marks are usually used as the delimiter. If data in the file contains the delimiter, you must specify escape characters. Otherwise, OBLOADER fails to parse the data due to its incorrect format. We strongly recommend that you use the CSV format. We recommend that you use this option together with the --table option. When it is used together with the --all option, OBLOADER imports only table data but not database object definitions.
  
  

• --pos

  Imports data files in the POS format. A POS file stores data with a fixed length in bytes. The file is named in the format of table name.dat. Data stored in each column of a POS file occupies a fixed number of bytes. A column value shorter than specified is padded with spaces. A column value longer than specified is truncated, in which case data may be garbled. We recommend that you use this option together with the --table option. When it is used together with the --all option, OBLOADER imports only table data but not database object definitions. This is different from the format that stores data with a fixed length in characters.
  
  

• --cut

  Imports data files in the CUT format. A CUT file uses a character or a character string as the separator string. A CUT file is named in the format of table name.dat. How to distinguish the CUT format from the CSV format? A file in the CSV format uses a single character, which is usually a comma (,), to separate fields. A file in the CUT format usually uses a string, such as |@|, to separate fields. A file in the CSV format uses single quotation marks or double quotation marks as delimiters between fields, while a file in the CUT format does not have delimiters. We recommend that you use this option together with the --table option. When it is used together with the --all option, OBLOADER imports only table data but not database object definitions.

  Notice
  In a CUT file, each data record is stored in an entire line. If a single character is used as the field separator, make sure that the data contains no special characters such as separators, carriage returns, and line breaks. Otherwise, OBLOADER cannot correctly parse the data.
  When you specify the --cut option in the data import command, do not use the --trail-delimiter option if no field separator or separator string exists at the end of the data line in the file. Otherwise, a serious error occurs on OBLOADER.

• --table-group '*table_group_name [,table_group_name...]*'|--table-group '*'

  Imports table group definitions. This option is similar to the --table option, except that this option does not support data import.
  
  

• --all

  Imports all supported database object definitions and table data. When this option is used in combination with --ddl, all database object definition files are imported. When this option is used in combination with --csv, --sql, --cut, or --pos, data in all tables is imported in the specified format. To import all database object definitions and table data, you can specify --all and --ddl along with a data format option.

  Notice
  The --all option is mutually exclusive with any database object options. It cannot be specified together with other database object options. If both the --all option and a database object option are specified, the --all option will be executed first.

• --table ' table_name [,table_name...] ' | --table ' * '

  Imports table definitions or table data. When this option is used in combination with --ddl, only table definitions are imported. When this option is used in combination with any data format option, only table data is imported. To specify multiple tables, separate the table names with commas (,). By default, the names of tables that are imported to Oracle tenants are treated as in uppercase, and the names of tables that are imported to MySQL tenants are treated as in lowercase. For example, in Oracle mode, both --table 'test' and --table 'TEST' indicate the table named TEST. In MySQL mode, both --table 'test' and --table 'TEST' indicate the table named test. If table names are case-sensitive, enclose them in brackets ([ ]). For example, --table '[test]' indicates the table named test, while --table '[TEST]' indicates the table named TEST. If the table name is specified as an asterisk, all table definitions or table data is imported.

  Notice
  When you use a control file during the import, the table name specified in the --table option must be in the same letter case as that in the database. Otherwise, the control file cannot take effect.

• --view ' view_name [, view_name...] ' | --view ' * '

  Imports view definitions. This option is similar to the --table option, except that this option does not support data import.
  
  

• --trigger ' trigger_name [, trigger_name...] ' | --trigger ' * '

  Imports trigger definitions. This option is similar to the --table option, except that this option does not support data import.
  
  

• --sequence ' sequence_name [, sequence_name...] ' | --sequence ' * '

  Imports sequence definitions. This option is similar to the --table option, except that this option does not support data import. This option is supported only for OceanBase Database in Oracle mode.
  
  

• --synonym ' synonym_name [, synonym_name...] ' | --synonym ' * '

  Imports synonym definitions. This option is similar to the --table option, except that this option does not support data import. This option is supported only for OceanBase Database in Oracle mode.
  
  

• --type ' type_name [, type_name...] ' | --type ' * '

  Imports type definitions. This option is similar to the --table option, except that this option does not support data import. This option is supported only for OceanBase Database in Oracle mode.
  
  

• --type-body ' typebody_name [, typebody_name...] ' | --type-body ' * '

  Imports type body definitions. This option is similar to the --table option, except that this option does not support data import. This option is supported only for OceanBase Database in Oracle mode.
  
  

• --package ' package_name [, package_name...] ' | --package ' * '

  Imports package definitions. This option is similar to the --table option, except that this option does not support data import. This option is supported only for OceanBase Database in Oracle mode.
  
  

• --package-body ' packagebody_name [, packagebody_name...] ' | --package-body ' * '

  Imports package body definitions. This option is similar to the --table option, except that this option does not support data import. This option is supported only for OceanBase Database in Oracle mode.
  
  

• --function ' function_name [, function_name...] ' | --function ' * '

  Imports function definitions. This option is similar to the --table option, except that this option does not support data import.
  
  

• --procedure ' procedure_name [, procedure_name...] ' | --procedure ' * '

  Imports stored procedure definitions. This option is similar to the --table option, except that this option does not support data import.
  
  

• --replace-object

  Replaces existing database object definitions during the import. For tables and synonyms, existing definitions are deleted and then new ones are created. For functions and stored procedures, existing definitions are replaced by using the CREATE OR REPLACE statement. This option can be used only in combination with the --ddl or --mix option, and does not take effect for --csv, --sql, and other data format options.

  Notice

  If an object already exists in the destination database, the object definition is forcibly replaced with the object definition stored in the import file.

  If you do not need to replace objects in the destination database, do not use this option. Otherwise, business may be affected.

• --retry

  Resumes the import task from the breakpoint. We recommend that you specify this option to resume an import task that has more than 80% of data imported, so that you do not have to start a new import. Duplicate data errors may occur during the resumed import. If only a small part of the data has been imported, you can clear tables and start a new import, which is more efficient.

  Notice
  The CHECKPOINT.bin file is a savepoint file generated when the tool runs, and is located in the directory specified in the -f option. You cannot use this option if the CHECKPOINT.bin file does not exist.

• --external-data

  Specifies that the data files to be imported are exported by a third-party tool. OBDUMPER generates a MANIFEST.bin file in the directory specified in the -f option during data export. The file stores metadata information. When OBLOADER imports data, it parses the metadata file by default. If the metadata file is missing, or the data is exported by a third-party tool without a metadata file, you can specify this option to skip metadata parsing for the import.
  
  

• --nls-date-format ' date-format-string '

  The datetime format in the connection to OceanBase Database in Oracle mode. Default value: YYYY-MM-DD HH24:MI:SS.
  
  

• --nls-timestamp-format ' timestamp-format-string '

  The timestamp format in the connection to OceanBase Database in Oracle mode. Default value: YYYY-MM-DD HH24:MI:SS:FF9.
  
  

• --nls-timestamp-tz-format ' timestamp-tz-format-string '

  The timestamp format with a time zone in the connection to OceanBase Database in Oracle mode. Default value: YYYY-MM-DD HH24:MI:SS:FF9 TZR.
  
  

• --skip-header

  Skips headers of CSV/CUT files when the files are imported. Headers are data in the first line. This option can be used only in combination with the --cut or --csv option. Only OBLOADER V3.3.0 and later versions support skipping the first line of data in a CUT file.
  
  

• --skip-footer

  Skips footers of CUT files when the files are imported. Footers are data in the last line. This option can be used only in combination with the --cut option.
  
  

• --null-string ' null_replacer_string '

  Replaces the specified character with NULL. This option can be used only in combination with the --cut or --csv option. Default value: \N.
  
  

• --empty-string ' empty_replacer_string '

  Replaces the specified character with an empty string (' '). This option can be used only in combination with the --csv option. Default value: \E.
  
  

• --line-separator ' line_separator_string '

  The line separator. Custom line separators are supported for the import of CUT files. The default value of this option varies with the OS. Valid values: \r, \n, and \r\n.
  
  

• --column-separator ' column_separator_char '

  The column separator. This option can be used only in combination with the --csv option and supports a single character only. Default value: comma (,).
  
  

• --escape-character ' escape_char '

  The escape character. This option can be used only in combination with the --cut or --csv option and supports a single character only. Default value: backslash ().
  
  

• --column-delimiter ' column_delimiter_char '

  The column delimiter. This option can be used only in combination with the --csv option and supports a single character only. Default value: single quotation mark (').
  
  

• --with-trim

  Truncates spaces on the left and right sides of the data. This option can be used only in combination with the --csv option.
  
  

• --trail-delimiter

  Truncates the last column separator in a line. This option can be used only in combination with the --cut or --csv option.
  
  

• --ignore-unhex

  Ignores hexadecimal strings in decoding. This option applies only to binary data types.
  
  

• --exclude-table ' table_name [, table_name...] '

  Excludes the specified tables from the import of table definitions or table data. Fuzzy match on table names is supported. Example: --exclude-table 'test1,test*,*test,te*st' The preceding example specifies to exclude the following tables from the import of table definitions or table data:

  • test1
  • All tables with the table name starting with test
  • All tables with the table name ending with test
  • All tables with the table name starting with te and ending with st
    
    

• --exclude-data-types ' datatype [, datatype...] '

  Excludes the specified data types from the import of table data.
  
  

• --column-splitter ' split_string '

  The column separator. This option can be used only in combination with the --cut option.
  
  

• --endpoint ' oss_endpoint_string '

  The endpoint to access the OSS bucket. This option must be used in combination with --access-key and --secret-key for the access to OSS.

  Note

  In OBLOADER V4.1.0 or earlier, this option is `--oss-point`.

• --access-key ' access_key_string '

  The AccessKey ID used to access the OSS bucket. This option must be used in combination with --endpoint and --secret-key for access to OSS.
  
  

• --secret-key ' secret_key_string '

  The AccessKey secret used to access the OSS bucket. This option must be used in combination with --endpoint and --access-key for access to OSS.
  
  

• --bucket-uri ' bucket_uri_string '

  The URI of the OSS bucket. This option must be used in combination with --endpoint, --access-key, and secret-key for access to OSS.
  
  

• --max-discards int_num

  The maximum amount of duplicate data allowed in a single table to import. If duplicate data in a table exceeds the specified maximum, data import for this table stops. The import failure on the table is logged. Data import for other tables is not affected. Default value: -1, indicating that data import does not stop for duplicate data.

  Note
  This option takes effect only when the table contains a primary key or a unique key and duplicate data exists in the table.

• --max-errors int_num

  The maximum number of errors allowed for an import. If the number of import errors of a table exceeds the specified maximum, data import for this table stops. The import failure on the table is logged. This option can be set to 0, -1, or any positive integer N. If you set this option to -1, the errors are ignored and the import continues. Default value: 1000.
  
  

• --exclude-column-names ' column_name [, column_name...] '

  Excludes the specified columns from the import of table data. Fuzzy match on column names is not supported.

  Notice
  The letter case of the specified column name must be the same as that of the column name in the schema.
  In the imported data file, the excluded columns must have no corresponding data, and the imported columns must be in the same order as the columns in the table.

• --replace-data

  Replaces duplicate data in the table. This option is only applicable to tables that have primary keys or unique keys with the NOT NULL constraint. If a file has a large amount of duplicate data, for example, more than 30% of the total data volume, we recommend that you clear the table and import it again. The performance of data replacement is lower than that of import after table clearing. This option can be used only in combination with the --cut, --csv, and --sql options, and does not take effect for the --ddl option.

  Notice

  When the file and the table has duplicate data, the data in the table is replaced with the data in the file.

  For tables without primary keys or unique keys, this option appends data to the tables.

  If you do not need to replace duplicate data, do not specify this option. Otherwise, business may be affected.

• --truncate-table

  Truncates tables in the destination database. This option can be used only in combination with a data format option. When being used with the --all or --table '*' option, this option specifies to truncate all tables in the destination database. If you want to truncate only some of the tables, you can explicitly specify them in the format of --table 'test1,test2,[....]'. When being used with the --with-data-files option, this option specifies to truncate tables that have the corresponding data files.

  Notice

  If the `--all` or `--table '*'` option is used in combination with the `--truncate-table` option, the tool truncates all tables in the destination database, even if no data files corresponding to the tables exist in the directory specified by the `-f` option.

  Do not use this option to truncate the destination database or destination tables. We recommend that you manually truncate tables to avoid impacts on business.

• --with-data-files

  When being used in combination with the --truncate-table or --delete-from-table option, this option specifies to truncate or clear tables that have corresponding data files before data import. If not used with these options, this option does not take effect.
  
  

• --delete-from-table

  Clears tables in the destination database before the import. This option can be used only in combination with a data format option. When being used with the --all or --table '*' option, this option specifies to clear all tables in the destination database. If you want to clear only some of the tables, you can explicitly specify them in the format of --table 'test1,test2,[....]'. When being used with the --with-data-files option, this option specifies to clear tables that have corresponding data files only.

  Notice
  If the --all or --table '*' option is used in combination with the --delete-from-table option, the tool clears all tables in the destination database, even if no data files corresponding to the tables exist in the directory specified by the -f option. Do not use this option to clear the destination database or destination tables, especially tables with large data volumes. We strongly recommend that you delete table data manually based on your business requirements to avoid impacts on your business.

• --file-regular-expression

  The regular expression that is used to match the file name during an import. This option applies only to single-table import.
  
  

• --ignore-escape

  Ignores escape operations on characters for the import of CUT files. By default, escape operations are not ignored.

Performance options

• --rw float_num

  The proportion of data file parsing threads. Default value: 0.2. You can use this option together with the --thread option to calculate the number of file parsing threads. Formula: Number of file parsing threads = Value of --thread × Value of --rw.
  
  

• --slow float_num

  The threshold for triggering a slow import. When the memory usage of OceanBase Database reaches 75%, OBLOADER slows down to prevent an excessively high memory usage on the database. Default value: 0.75.
  
  

• --pause float_num

  The threshold for triggering an import pause. When the memory usage of OceanBase Database reaches 85%, OBLOADER pauses data import to prevent issues caused by an excessively high memory usage on the database. Default value: 0.85.
  
  

• --batch int_num

  The number of records for a batch of transactions. We recommend that you set this option to a value inversely proportional to the table width. Do not set this option to an excessively high value, which may cause database memory overflow. Default value: 200.
  
  

• --thread int_num

  The number of concurrent export threads allowed. This option corresponds to the number of import threads. You can use this option together with the --rw option to calculate the number of file parsing threads. Formula: Number of file parsing threads = Value of --thread × Value of --rw. Default value: Number of CPU cores x 2. OceanBase Database executes DDL statements in sequence. Therefore, you do not need to specify this option when you import database object definitions.
  
  

• --block-size int_num

  The block size for a file to be imported. When specifying this option, you do not need to explicitly specify the unit. The default unit is MB. By default, OBLOADER automatically splits a large file into multiple logical subfiles (or blocks) sized 64 MB. The logical subfiles do not occupy additional storage space. Default value: 64.
  
  

• --max-tps int_num

  The maximum import speed. You can specify this option to ensure import stability.
  
  

• --max-wait-timeout int_num

  The timeout period of waiting for a database major compaction to complete. When specifying this option, you do not need to explicitly specify the unit. The default unit is hour. When the database is under a major compaction, OBLOADER stops data import and waits up to the specified period. Default value: V3.

Other options

• -h, --help

  Shows the help information about the tool.
  
  

• -V, --version

  Shows the version of the tool.