运行 hive 时出错：org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException hive 错误等等

Question

我的 .bashrc

运行 hive 时出错： org.apache.hadoop.hive.ql.metadata.HiveException： java.lang.RuntimeException 配置单元错误等等

Java

export JAVA_HOME=/usr/lib/jvm/java-8-oracle 

export PATH=$PATH:$JAVA_HOME/bin

Hadoop

export HADOOP_HOME=/usr/local/hadoop 
export HADOOP_MAPRED_HOME=$HADOOP_HOME 
export HADOOP_COMMON_HOME=$HADOOP_HOME 
export HADOOP_HDFS_HOME=$HADOOP_HOME 
export YARN_HOME=$HADOOP_HOME 
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=/usr/local/hadoop/lib/native"
export HADOOP_INSTALL=$HADOOP_HOME
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin

驯兽师

export MAHOUT_HOME=/usr/local/mahout

# HBase
export HBASE_HOME=/usr/local/hbase
export PATH=$PATH:$HBASE_HOME/bin

蜂巢

export HIVE_HOME=/home/yash/hive/apache-hive-2.1.0-bin
export PATH=$PATH:$HIVE_HOME/bin

 My hive-site.xml:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://localhost/metastore?createDatabaseIfNotExist=true</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>root</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>root</value>
</property>
<property>
  <name>datanucleus.autoCreateSchema</name>
  <value>true</value>
</property>
<property>
  <name>datanucleus.fixedDatastore</name>
  <value>true</value>
</property>
<property>
 <name>datanucleus.autoCreateTables</name>
 <value>True</value>
 </property>
<property>
    <name>hive.querylog.location</name>
    <value>/home/yash/hive/apache-hive-2.1.0-bin/iotmp</value>
    <description>Location of Hive run time structured log file</description>
  </property>
  <property>
    <name>hive.exec.local.scratchdir</name>
    <value>/home/yash/hive/apache-hive-2.1.0-bin/iotmp</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/home/yash/hive/apache-hive-2.1.0-bin/iotmp</value>
    <description>Temporary local directory for added resources in the        remote file system.</description>
  </property>
</configuration>
 <i>ERROR<i>

    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/home/yash/hive/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

    Logging initialized using configuration in jar:file:/home/yash/hive/apache-hive-2.1.0-bin/lib/hive-common-2.1.0.jar!/hive-log4j2.properties

异步：真 线程“主”java.lang.RuntimeException 中的异常：org.apache.hadoop.hive.ql.metadata.HiveException： java.lang.RuntimeException：无法实例化 org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient 在 org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578) 在 org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518) 在 org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705) 在 org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641) 在 sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 在 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 在 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 在 java.lang.reflect.Method.invoke(Method.java:498) 在 org.apache.hadoop.util.RunJar.run(RunJar.java:239) 在 org.apache.hadoop.util.RunJar.main(RunJar.java:153) 引起：org.apache.hadoop.hive.ql.metadata.HiveException：java.lang.RuntimeException：无法实例化 org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient 在 org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:226) 在 org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:366) 在 org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:310) 在 org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:290) 在 org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:266) 在 org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:545) ... 9 更多 引起：java.lang.RuntimeException：无法实例化 org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient 在 org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1627) 在 org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:80) 在 org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130) 在 org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101) 在 org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3317) 在 org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3356) 在 org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3336) 在 org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3590) 在 org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:236) 在 org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:221) ... 14 更多 引起：java.lang.reflect.InvocationTargetException 在 sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 在 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 在 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 在 java.lang.reflect.Constructor.newInstance(Constructor.java:423) 在 org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1625) ... 23 更多 原因：java.lang.IllegalArgumentException：无法识别的 Hadoop 主要版本号：3.0.1 在 org.apache.hadoop.hive.shims.ShimLoader.getMajorVersion（ShimLoader.java:169） 在 org.apache.hadoop.hive.shims.ShimLoader.loadShims(ShimLoader.java:136) 在 org.apache.hadoop.hive.shims.ShimLoader.getHadoopShims(ShimLoader.java:95) 在 org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:402) 在 org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:275) 在 org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77) 在 org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137) 在 org.apache.hadoop.hive.metastore.RawStoreProxy.(RawStoreProxy.java:58) 在 org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) 在 org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:581) 在 org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:546) 在 org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:608) 在 org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:398) 在 org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:78) 在 org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) 在 org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6396) 在 org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:236) 在 org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:70) ... 28 更多

Answer 1

您的错误始于Caused by ... Unrecognized Hadoop major version number: 3.0.1...

我认为 Mahout 还不支持 Hadoop 3.x ；您似乎也在使用 Hive 2.x，它也不支持 Hadoop 3

至少使用最新的 Hadoop 2.7 或 2.8