Hadoop学习安装spark

1/spark  解压缩

[root@itcast01 local]# tar  -zxvf spark-2.3.1-bin-hadoop2.7.tgz 

2将文件移到/itcast
[root@itcast01 itcast]# mv  spark-2.3.1-bin-hadoop2.7 /itcast
3配置环境变量
[root@itcast01 itcast]# vi /etc/profile
export SPARK_HOME=/itcast/spark-2.3.1-bin-hadoop2.7
生效[root@itcast01 local]# source  /etc/profile

4配置spark spark-env.sh
[root@itcast01 conf]# pwd
/itcast/spark-2.3.1-bin-hadoop2.7/conf

[root@itcast01 conf]# ls
fairscheduler.xml.template  log4j.properties.template 

metrics.properties.template  slaves  spark-defaults.conf.template 

spark-env.sh.template

[root@itcast01 conf]# vi spark-env.sh

export SCALA_HOME=/itcast/scala-2.12.6
export JAVA_HOME=/usr/java/jdk1.8.0_171
export SPARK_MASTER_IP=192.168.117.100
export SPARK_WORKER_MEMORY=512m
export HADOOP_CONF_DIR=/itcast/hadoop-2.7.2/conf

5 vi slaves

已有
localhost

6/启动HADOOP参考https://blog.csdn.net/anaitudou/article/details/80392141

7 启动spark

[root@itcast01 spark-2.3.1-bin-hadoop2.7]# pwd
/itcast/spark-2.3.1-bin-hadoop2.7

[root@itcast01 sbin]# ./start-all.sh

检验

8 开始进入 spark-shell 

[root@itcast01 sbin]#  spark-shell

9在本地物理机浏览器打开spak

http://itcast01:4040/