Flume的下载安装

1、下载flume源文件

       https://archive.apache.org/dist/flume/1.7.0/

2、解压缩

     tar -zxvf apache-flume-1.7.0-bin.tar.gz

3、重命名

      mv apache-flume-1.7.0-bin flume

4、配置环境变量

     cd /etc/profile

     export FLUME_HOME=/usr/local/software/flume

     export PATH=$PATH:$FLUME/bin

     source /etc/profile

5、修改flume-env.sh配置文件

     cd $FLUME_HOME/conf

     cp flume-env.sh.template flume-env.sh

     vi flume-env.sh

6、在flume-env.sh中添加如下内容

     export JAVA_HOME=/usr/local/software/jdk

     export HADOOP_HOME=/usr/local/software/hadoop

7、创建配置文件

     cd $FLUME_HOME/conf

     cp flume-conf.properties.template flume.conf

     vi /etc/flume.conf

8、在flume.conf文件末尾加入以下内容

    

9、启动flume server   

        flume-ng agent --conf ./conf/ -f conf/flume.conf -Dflume.root.logger=DEBUG,console -n agent1

10、开启client

      flume-ng avro-client --conf conf -H localhost -p 41414 -F /etc/passwd -Dflume.root.logger=DEBUG,console

        

11、采集目录文件到hdfs

        配置conf/flume.conf

       

12、启动agent

        flume-ng agent --conf ./conf/ -f ./conf/flume.conf --name
agent1 -Dflume.root.logger=DEBUG,console