使用sqoop从mysql导入数据到Hive报错

第一阶段的到HDFS的阶段没问题,MR作业也能正常完成,但是第二阶段 Loading uploaded data into Hive的过程出错了。
错误的日志如下:

15/10/23 14:34:18 INFO mapreduce.ImportJobBase: Transferred 30 bytes in 31.2422 seconds (0.9602 bytes/sec)
15/10/23 14:34:19 INFO mapreduce.ImportJobBase: Retrieved 5 records.
15/10/23 14:34:19 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `test` AS t LIMIT 1
15/10/23 14:34:19 INFO hive.HiveImport: Loading uploaded data into Hive
15/10/23 14:34:30 INFO hive.HiveImport: 
15/10/23 14:34:30 INFO hive.HiveImport: Logging initialized using configuration in file:/usr/local/apache-hive-1.1.1-bin/conf/hive-log4j.properties
15/10/23 14:34:40 INFO hive.HiveImport: Exception in thread "main" java.lang.AssertionError: Unknown token: [@-1,0:0='TOK_FILEFORMAT_GENERIC',<679>,0:-1]
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:10895)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:10103)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10147)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:192)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:222)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:421)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:307)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1112)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1160)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1039)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:207)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:305)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:403)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:419)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:708)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)
15/10/23 14:34:40 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
15/10/23 14:34:40 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
15/10/23 14:34:40 INFO hive.HiveImport:     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
15/10/23 14:34:40 INFO hive.HiveImport:     at java.lang.reflect.Method.invoke(Method.java:606)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
15/10/23 14:34:40 INFO hive.HiveImport:     at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
15/10/23 14:34:40 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Hive exited with status 1
    at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:389)
    at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:339)
    at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:240)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:514)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
 
 
 
补充一下,我使用的都是Apache版本,sqoop1.4.6 Hadoop2.6.0 hive1.1.1 hbase1.0.2

fish - Hadooper

赞同来自:

这是apache hive1.1.1版本的一个问题:https://issues.apache.org/jira/browse/HIVE-10831 将你机器上的hive拷贝到我的环境中执行也会发生同样的问题。   换用apache hive 1.2.1 或者 CDH版本的就可以正常工作。

编程小梦 - 大数据

赞同来自:

fish - Hadooper

赞同来自:

做的是全表导入操作么?

timchy

赞同来自:

是的,冼老师,这个问题主要出在 Exception in thread "main" java.lang.AssertionError: Unknown token: [@-1,0:0='TOK_FILEFORMAT_GENERIC',<679>,0:-1]  这个百度都找不到

要回复问题请先登录注册