单用户模式是通过网络连接到一个数据库中,是最经常使用到的模式。

Hive单用户模式搭建

使用hive的前提:

(1)启动hadoop集群

(2)启动mysql服务

节点规划:

hadoop01mysql存放元数据

hadoop02搭建hive单用户模式

搭建步骤:

1.上传好tar包,后解压:

tar -zxvf apache-hive-1.2.1-bin.tar.gz

把解压好的文件移动到/opt/software目录

 mv apache-hive-1.2.1-bin hive-1.2.1

 mv hive-1.2.1/ /opt/software/

 Hive单用户模式搭建

 2. 配置环境变量

 Hive单用户模式搭建

记得顺便重启配置文件:

 . /etc/profile

 3. 配置文件hive-site.xml

 cp hive-default.xml.template hive-site.xml

 Hive单用户模式搭建这个命令可以把光标所在行以下除了倒数第一行都删了。

<configuration>  

<property>  

  <name>hive.metastore.warehouse.dir</name>  

  <value>/user/hive_remote/warehouse</value>  

</property>  

 <property>  

  <name>hive.metastore.local</name>  

  <value>true</value>  

</property>  

<property>  

  <name>javax.jdo.option.ConnectionURL</name>  

  <value>jdbc:mysql://hadoop01/hive_remote?createDatabaseIfNotExist=true</value>  

</property>  

<property>  

  <name>javax.jdo.option.ConnectionDriverName</name>  

  <value>com.mysql.jdbc.Driver</value>  

</property>  

<property>  

  <name>javax.jdo.option.ConnectionUserName</name>  

  <value>root</value>  

</property>  

<property>  

  <name>javax.jdo.option.ConnectionPassword</name>  

  <value>123</value>  

</property>  

</configuration>  

 

4. 安装mysql驱动包

 Hive单用户模式搭建

jar包移动到hivejar包目录

mv mysql-connector-java-5.1.32-bin.jar /opt/software/hive-1.2.1/lib/

 5. hive中的jlinejar包拷贝到hadoop

(/opt/software/hadoop-2.6.5/share/hadoop/yarn/lib/)目录的下,并把其中低版本的删掉,不然会报如下错:

 Logging initialized using configuration in

jar:file:/opt/software/hive-1.2.1/lib/hive-common-1.2.1.jar!/hive-log4j.properties

[ERROR] Terminal initialization failed; falling back to unsupported

java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

at jline.TerminalFactory.create(TerminalFactory.java:101)

at jline.TerminalFactory.get(TerminalFactory.java:158)

at jline.console.ConsoleReader.<init>(ConsoleReader.java:229)

at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)

at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)

at org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader(CliDriver.java:787)

at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:721)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

 

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

at jline.console.ConsoleReader.<init>(ConsoleReader.java:230)

at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)

at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)

at org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader(CliDriver.java:787)

at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:721)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

 

[[email protected] lib]# cp /opt/software/hive-1.2.1/lib/jline-2.12.jar ./

Hive单用户模式搭建

 6. 验证是否配置成功:

hive> create table test01(id int,age int);

OK

Time taken: 2.148 seconds

hive> desc test01;

OK

id                   int                                      

age                  int                                      

Time taken: 0.486 seconds, Fetched: 2 row(s)

hive> insert into test01 values(1,23);

Query ID = root_20180429203346_b7a310a3-8ca0-4e2b-af9d-f55486352a98

Total jobs = 3

Launching Job 1 out of 3

Number of reduce tasks is set to 0 since there's no reduce operator

Starting Job = job_1525003393708_0001, Tracking URL = http://hadoop03:8088/proxy/application_1525003393708_0001/

Kill Command = /opt/software/hadoop-2.6.5/bin/hadoop job  -kill job_1525003393708_0001

Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0

2018-04-29 20:34:19,531 Stage-1 map = 0%,  reduce = 0%

2018-04-29 20:35:01,432 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.9 sec

MapReduce Total cumulative CPU time: 1 seconds 900 msec

Ended Job = job_1525003393708_0001

Stage-4 is selected by condition resolver.

Stage-3 is filtered out by condition resolver.

Stage-5 is filtered out by condition resolver.

Moving data to: hdfs://mycluster/user/hive_remote/warehouse/test01/.hive-staging_hive_2018-04-29_20-33-46_610_4511768652560755389-1/-ext-10000

Loading data to table default.test01

Table default.test01 stats: [numFiles=1, numRows=1, totalSize=5, rawDataSize=4]

MapReduce Jobs Launched:

Stage-Stage-1: Map: 1   Cumulative CPU: 1.9 sec   HDFS Read: 3548 HDFS Write: 75 SUCCESS

Total MapReduce CPU Time Spent: 1 seconds 900 msec

OK

Time taken: 78.059 seconds

hive>

 Hive单用户模式搭建

 到hdfs表的存储路径查看:

 Hive单用户模式搭建

使用命令查看这个文件

 Hive单用户模式搭建

 当然也可以登录mysql查看元数据信息

 Hive单用户模式搭建

表名:

 Hive单用户模式搭建

字段:

 Hive单用户模式搭建



相关文章:

  • 2021-06-16
  • 2022-12-23
  • 2022-01-09
  • 2021-12-25
  • 2022-01-25
  • 2022-12-23
  • 2021-11-18
猜你喜欢
  • 2021-10-01
  • 2022-12-23
  • 2022-12-23
  • 2021-10-22
  • 2021-06-02
  • 2021-07-21
相关资源
相似解决方案