OceanBase Migration Service (OMS) Community Edition V4.2.8 and later allow you to migrate data from a Hive database to OceanBase. This topic describes the HiveSource parameters.
{
"boosterClass":"com.oceanbase.oms.connector.source.hive.OmsHiveRunner",
"clients":[
{
"clientId":"np_6cm31nfy1d28",
"instance":"jdbc:hive2://sqaoxxxxxxx:10000/default",
"username":"hadoop",
"password": "",
"hiveconf":{
"HADOOP_USER_NAME":"hadoop",
"oms.kerberos.principal":"",
"oms.kerberos.keytab.file":"",
// Only required by Spark
"hive.metastore.uris":"thrift://sqaoxxxxxxx:9083"
},
"fileconf":{
"hive-site.xml":"Base64-encoded XML file content"
}
}
],
// You can customize the SQL statement used to read data from a Hive database. Fields in the SELECT statement here must be in the same order as those in the target table in OceanBase Database Community Edition.
"hivesql":[
{
"table":"schema.table_name",
"sql":"select f1,f2 from schema.table_name where f1='xx'"
}
],
"addjars": [
"/data/1/hadoop/oms-hive-jdbc-handler.jar","/data/1/hive/lib/hive-jdbc-handler-2.3.9.jar","/data/1/hive/lib/commons-dbcp-1.4.jar","/data/1/hive/lib/commons-pool-1.5.4.jar"
],
"jar":"connector-hive.jar:connector-dataflow.jar",
"dbVersion":"1.2.0-cdh5.15.2",
"timezone":"+08:00",
"taskResume":"false",
"type":"HIVE_SOURCE",
"sliceWorkerNum":"4",
"dbType":"HIVE"
}
| Parameter | Description |
|---|---|
| hiveconf | The Hive configurations, which can be the parameters set by using the SET statement in Hive. |
| hivesql | The custom SQL statement to query data from a Hive database. When migrating data from Hive to OceanBase Database Community Edition, OMS Community Edition constructs SQL statements to query data from Hive by using the field names and order as they appear in the target table in OceanBase Database Community Edition. A custom SQL statement is required in either of the following cases. Note that the fields in this SQL statement must be in the same order as those in the target table in OceanBase Database Community Edition.
|
| addjars | Adds JAR packages related to external tables in the HiveServer2 local mode or HTTP mode.
|
| sliceWorkerNum | The number of Hive tables that are concurrently read. Default value: 4. |