【问题标题】:Having issues when trying to store data into Hive database using HCatStorer()尝试使用 HCatStorer() 将数据存储到 Hive 数据库时出现问题
【发布时间】:2015-07-13 13:57:06
【问题描述】:

我正在使用 PIG 使用 HCatalog 将表数据存储到 hive 数据库中。我能够在 PIG grunt shell 中加载和转储数据,但是当我尝试将加载的数据存储到我创建为 myweather.weathertable1 (myweather-database name ,weathertable1 -table name1) 的 HIVE 表中时,使用以下命令

STORE A INTO 'myweather.weathertable1' USING org.apache.hcatalog.pig.HCatStorer();

我遇到以下错误:

2015-07-11 20:20:09,056 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve org.apache.hcatalog.pig.HCatStorer using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Details at logfile: /X/Y/pig_1436660283762.log

日志文件说:

ERROR 1070: Could not resolve org.apache.hcatalog.pig.HCatStorer using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]

Failed to parse: Pig script failed to parse:
<line 4, column 45> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve org.apache.hcatalog.pig.HCatStorer using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:196)
        at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1660)
        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1633)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:587)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
        at org.apache.pig.Main.run(Main.java:541)
        at org.apache.pig.Main.main(Main.java:156)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

解决上述问题的任何帮助都非常值得赞赏。 提前致谢。

【问题讨论】:

标签: java hadoop hive apache-pig hcatalog


【解决方案1】:

请尝试使用 -useHCatalog 选项启动 pig。它将加载必要的罐子。请看HCat docs

【讨论】:

  • @Fred 感谢您的回复。我在发布此问题之前使用了 pig -useHCatalog。我终于通过用 org.apache.hive.hcatalog.pig.HCatStorer() 替换 org.apache.hcatalog.pig.HCatStorer() 解决了这个问题。遗憾的是,Apache hive 文档尚未更新此错误 (cwiki.apache.org/confluence/display/Hive/…)。
猜你喜欢
  • 2016-02-01
  • 2016-06-20
  • 1970-01-01
  • 2013-02-14
  • 2021-06-30
  • 1970-01-01
  • 2021-02-23
  • 2020-07-08
  • 2021-12-28
相关资源
最近更新 更多