搭建单机CDH环境，并更新spark环境

搭建单机CDH环境，并更新spark环境

1，安装VMWare Player，http://dlsw.baidu.com/sw-search-sp/soft/90/13927/VMware_player_7.0.0_2305329.1420626349.exe

2，启动BIOS虚拟化，http://www.cnblogs.com/stono/p/8323516.html

3，下载CDH QuickStart版本，https://downloads.cloudera.com/demo_vm/vmware/cloudera-quickstart-vm-5.12.0-0-vmware.zip

4，用vmware player启动CDH，内存8G，CPU4个；root密码cloudera

5，重新安装spark，下载命令 wget http://apache.mirrors.tds.net/spark/spark-2.0.0/spark-2.0.0-bin-hadoop2.7.tgz

下载的时候多下载几次，开始可能出现404问题；

6，下载后配置spark，

tar xzvf spark-2.0.0-bin-hadoop2.7.tgz 
cd spark-2.0.0-bin-hadoop2.7


vi /etc/profile.d/spark2.sh
export SPARK_HOME=/home/cloudera/spark-2.0.0-bin-hadoop2.7
export PATH=$PATH:/home/cloudera/spark-2.0.0-bin-hadoop2.7/bin


cp conf/spark-env.sh.template conf/spark-env.sh
cp conf/spark-defaults.conf.template conf/spark-defaults.conf

vi conf/spark-env.sh
export HADOOP_CONF_DIR=/etc/hadoop/conf
export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera

cp /etc/hive/conf/hive-site.xml conf/

修改conf/log4j.properties中的日志级别为ERROR