假设你想查询某个表的某一列。Hive默认是会启用MapReduce Job来完毕这个任务，例如以下：
01
|
hive>
SELECT id, money FROM m limit 10;
|
02
|
Total
MapReduce jobs = 1
|
03
|
Launching
Job 1 out
of 1
|
04
|
Number
of reduce tasks is set to 0 since
there's no reduce operator
|
05
|
Cannot
run job locally: Input Size (= 235105473)
is larger than
|
06
|
hive.exec.mode.local.auto.inputbytes.max
(= 134217728)
|
07
|
Starting
Job = job_1384246387966_0229, Tracking URL =
|
11
|
Kill
Command = /home/q/hadoop-2.2.0/bin/hadoop
job
|
12
|
-kill
job_1384246387966_0229
|
13
|
hadoop
job information forStage-1:
number of mappers: 1;
|
15
|
2013-11-13 11:35:16,167 Stage-1 map
= 0%,
reduce = 0%
|
16
|
2013-11-13 11:35:21,327 Stage-1 map
= 100%,
reduce = 0%,
|
17
|
Cumulative
CPU 1.26 sec
|
18
|
2013-11-13 11:35:22,377 Stage-1 map
= 100%,
reduce = 0%,
|
19
|
Cumulative
CPU 1.26 sec
|
20
|
MapReduce
Total cumulative CPU time: 1 seconds 260 msec
|
21
|
Ended
Job = job_1384246387966_0229
|
22
|
MapReduce
Jobs Launched:
|
23
|
Job 0:
Map: 1 Cumulative
CPU: 1.26sec
|
24
|
HDFS
Read: 8388865 HDFS
Write: 60 SUCCESS
|
25
|
Total
MapReduce CPU Time Spent: 1 seconds 260 msec
|
37
|
Time
taken: 16.802 seconds,
Fetched: 10 row(s)
|
我们都知道，启用MapReduce Job是会消耗系统开销的。对于这个问题。从Hive0.10.0版本号開始，对于简单的不须要聚合的类似SELECT <col> from <table> LIMIT n语句，不须要起MapReduce job，直接通过Fetch task获取数据，能够通过以下几种方法实现：
方法一：
01
|
hive>
set hive.fetch.task.conversion=more;
|
02
|
hive>
SELECT id, money FROM m limit 10;
|
14
|
Time
taken: 0.138 seconds,
Fetched: 10 row(s)
|
上面 set hive.fetch.task.conversion=more;开启了Fetch任务，所以对于上述简单的列查询不在启用MapReduce job！
方法二：
1
|
bin/hive
--hiveconf hive.fetch.task.conversion=more
|
方法三：
上面的两种方法都能够开启了Fetch任务，可是都是暂时起作用的；假设你想一直启用这个功能。能够在${HIVE_HOME}/conf/hive-site.xml里面增加下面配置：
02
|
<name>hive.fetch.task.conversion</name>
|
05
|
Some
select queries can be converted to single FETCH task
|
06
|
minimizing
latency.Currently the query should be single
|
07
|
sourced
not having any subquery and should not have
|
08
|
any
aggregations or distincts (which incurrs RS),
|
09
|
lateral
views and joins.
|
10
|
1.
minimal : SELECT STAR, FILTER on partition columns, LIMIT only
|
11
|
2.
more : SELECT, FILTER, LIMIT only (+TABLESAMPLE, virtual columns)
|
这样就能够长期启用Fetch任务了，非常不错吧。也赶紧去试试吧！