Description
The statement is used to import data from an external source.
In OceanBase Database, the LOAD DATA statement can load input files from OceanBase Database and Alibaba Cloud OSS, to OBServer nodes.
Note
- The current version of OceanBase Database does not support the use of
LOCAL INFILEto load data files. - Tables with triggers are not supported.
The LOAD DATA statement can currently import text files in the CSV format. The entire import process consists of the following steps:
Parse the file: OceanBase Database reads data from the file with the name specified by the user and parses data in the file, in parallel or in series, based on the specified degree of parallelism.
Distribute data: As OceanBase Database is a distributed database, data in partitions can be distributed across OBServer nodes. The
LOAD DATAstatement calculates and determines to which OBServer node the parsed data should be sent.Insert data: After the data is received by the destination OBServer node, the node executes an
INSERTstatement to insert the data into the corresponding partition.
To import data from external files, you must have the FILE privilege. You can execute the GRANT FILE ON *.* TO $user_name; statement to grant the privilege to the user, where $user_name is the name of the user who needs to execute the LOAD DATA statement.
Syntax
LOAD DATA
[ [/*+ PARALLEL(N) load_batch_size(M) APPEND */]
| [/*+ PARALLEL(N) direct(bool, int) */] ]
[REMOTE_OSS] INFILE 'file_name'
INTO TABLE table_name
[{FIELDS | COLUMNS}
[TERMINATED BY 'string']
[[OPTIONALLY] ENCLOSED BY 'char']
[ESCAPED BY 'char']
]
[LINES
[STARTING BY 'string']
[TERMINATED BY 'string']
]
[IGNORE number {LINES | ROWS}]
[(column_name_var
[, column_name_var] ...)]
Parameter description
Parameter |
Description |
|---|---|
| parallel(N) | The degree of parallelism for loading data. If N is specified, the default value is 4. |
| load_batch_size(M) | The number of rows to be inserted at a time. If M is specified, the default value is 100. We recommend that you set this parameter to a value within the range of [100,1000]. |
| APPEND | Use this hint to enable direct load, which allocates space and writes data directly in the data file. By default, the APPEND hint is equivalent to direct(true, 0). It also enables online collection of optimizer statistics (GATHER_OPTIMIZER_STATISTICS). |
| direct | Use this hint to enable direct load. The bool parameter in the direct(bool, int) specifies whether to sort the given CSV file, where true indicates that sorting is required, and the int parameter specifies the maximum number of allowed error rows. |
| REMOTE_OSS | Specifies whether to read data from the OSS file system.
NoticeIf you specify this parameter, the |
| file_name | The path and name of the input file. The format of the file_name parameter is as follows:
NoteMake sure that the following information is correct before you import a file from OSS:
|
| table_name | The name of the table to which the imported data is loaded. The table can be a partitioned or non-partitioned table. |
| FIELDS | COLUMNS | The formats of fields.
|
| LINES STARTING BY | The delimiter that indicates the start of a line. |
| LINES TERMINATED BY | The delimiter that indicates the end of a line. |
| IGNORE number { LINES | ROWS } | The number of the first few lines or rows to be ignored. LINES indicates the first few lines of the file, and ROWS indicates the first few rows separated by field separators. By default, the server matches each field in the input file with the corresponding column in the table. If the input file does not contain all the columns in the table, the missing columns are filled with default values, which are described in the following example:
|
| column_name_var | The name of the imported column. |
Examples
Example 1: Import data from a file on the server (OBServer node)
Set the global secure path.
Notice
For security reasons, when you set the system variable
secure_file_priv, you can connect to the database only through a local socket to execute the SQL statement that modifies the global variable. For more information, see secure_file_priv.obclient> SET GLOBAL secure_file_priv = "/";Log out.
Note
After you set the
secure_file_privvariable, you must exit and then reconnect to the database because thesecure_file_privvariable is aGLOBALvariable.obclinet> \qThe return result is as follows:
ByeReconnect to the database and use the
LOAD DATAstatement to import data.Use the following syntax to perform regular import:
obclient> LOAD DATA INFILE '/home/admin/test.csv' INTO TABLE t1;Use the
APPENDhint to enable direct load:LOAD DATA /*+ PARALLEL(4) APPEND */ INFILE '/home/admin/test.csv' INTO TABLE t1;
Example 2: Import data from an OSS file
Notice
When you use an object storage path, make sure that the values of all parameters in the object storage path contain only uppercase and lowercase letters, digits, and the \/-_$+= characters and wildcard characters. If you enter characters other than the aforesaid ones, the setting may fail.
Use the direct(bool, int) hint to enable direct load for files that are to be imported from Alibaba Cloud OSS.
LOAD DATA /*+ direct(true,1024) parallel(16) */ REMOTE_OSS INFILE 'oss://antsys-oceanbasebackup/backup_rd/xiaotao.ht/lineitem2.tbl?host=***.oss-cdn.***&access_id=***&access_key=***' INTO TABLE tbl1 FIELDS TERMINATED BY ',';
References
- For more information about how to use the
LOAD DATAstatement, see Import data by using the LOAD DATA statement. - For more information about direct load using the
LOAD DATAstatement, see Import data by using the LOAD DATA statement.
