CDH4 中的 Hadoop 和 Hive Home答案

【问题标题】：Hadoop and Hive Homes in CDH4CDH4 中的 Hadoop 和 Hive Home
【发布时间】：2013-05-17 09:30:47
【问题描述】：

我正在尝试在 CDH4 环境中配置 RHive。在 R 中读取包 'RHive' 时，返回以下错误。我猜那是因为住错了家。如果是这样，什么是正确的？或者如果这不是原因，那有什么问题？

任何帮助将不胜感激。

谢谢。

> Sys.setenv(HIVE_HOME="/etc/hive")
> Sys.setenv(HADOOP_HOME="/etc/hadoop")
> library(RHive)
Loading required package: rJava
Loading required package: Rserve
This is RHive 0.0-7. For overview type '?RHive'.
HIVE_HOME=/etc/hive
[1] "there is no slaves file of HADOOP. so you should pass hosts argument when you call rhive.connect()."
Error : .onLoad failed in loadNamespace() for 'RHive', details:
  call: .jnew("org/apache/hadoop/conf/Configuration")
  error: java.lang.ClassNotFoundException
In addition: Warning message:
In file(file, "rt") :
  cannot open file '/etc/hadoop/conf/slaves': No such file or directory
Error: package/namespace load failed for 'RHive'

【问题讨论】：

似乎很容易解释。您缺少指定集群中哪些节点的“从属”文件。

标签： hadoop hive

【解决方案1】：

Had the problems but solved it. Downside is that I have to keep track of a bunch of sym links

After struggling with install RHive_0.0-7.tar.gz on CDH 4.7.x and getting: 
Warning in file(file, "rt") :
cannot open file '/etc/hadoop/conf/slaves': No such file or directory
[1] "there is no slaves file of HADOOP. so you should pass hosts argument when you call rhive.connect()."

In /etc/hadoop/conf
I added a the following sym link ----> ln -s /opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/etc/hadoop/conf.empty/slaves slaves
(why Cloudera CHD 4.7 installs in /opt without creating the proper sym links from /usr/lib is puzzling)

I also defined the followingin /usr/lib64/R/etc/Renviron
## set hive paths
HIVE_HOME='/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hive'
HADOOP_HOME='/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop'
LD_LIBRARY_PATH='/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop'

At a shell prompt I ran R CMD INSTALL RHive_0.0-7.tar.gz 
Installation Happiness!!

++++++
Inside R-Studio (server)

>
> library(RHive)
Loading required package: rJava
Loading required package: Rserve
This is RHive 0.0-7. For overview type ‘?RHive’.
HIVE_HOME=/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hive
call rhive.init() because HIVE_HOME is set.
rhive.init()
>
+++++++

【讨论】：

【解决方案2】：

您应该单独设置 HADOOP_CONF_DIR。尝试导出 $HADOOP_CONF_DIR=/etc/hadoop/conf/conf.pseudo

conf.pseudo 有从属文件。

虽然我很想知道你是否可以让 RHive 与 CDH4 一起工作。

【讨论】：

感谢您的回复！我会试一试并发布结果。
似乎 RHive 确实适用于 CDH4。你可以看看stackoverflow.com/questions/16783549/…。虽然我没有机会再看一遍。如果你确实得到了它，那么请发布你是如何做到的。谢谢。