【发布时间】：2014-06-25 04:39:51
【问题描述】：
我是Flume 的新手。我在一台服务器上安装了Flume 和Hadoop，并且日志在另一台服务器上可用。
通过Flume，我正在尝试阅读日志。这是我的配置文件。
# Define a memory channel called ch1 on agent1
agent1.channels.ch1.type = memory
# Define an Avro source called avro-source1 on agent1 and tell it
# to bind to 0.0.0.0:41414. Connect it to channel ch1.
agent1.sources.avro-source1.type = syslogtcp
agent1.sources.avro-source1.bind = 10.209.4.224
agent1.sources.avro-source1.port = 5140
# Define a logger sink that simply logs all events it receives
# and connect it to the other end of the same channel.
agent1.sinks.hdfs-sink1.type = hdfs
agent1.sinks.hdfs-sink1.hdfs.path = hdfs://delvmplldsst02:54310/flume/events
agent1.sinks.hdfs-sink1.hdfs.fileType = DataStream
agent1.sinks.hdfs-sink1.hdfs.writeFormat = Text
agent1.sinks.hdfs-sink1.hdfs.batchSize = 20
agent1.sinks.hdfs-sink1.hdfs.rollSize = 0
agent1.sinks.hdfs-sink1.hdfs.rollCount = 0
# Finally, now that we've defined all of our components, tell
# agent1 which ones we want to activate.
agent1.channels = ch1
agent1.sources = avro-source1
agent1.sinks = hdfs-sink1
#chain the different components together
agent1.sinks.hdfs-sink1.channel = ch1
agent1.sources.avro-source1.channels = ch1
我不确定在这种情况下要使用什么确切的源类型。我正在另一台服务器中启动Flume agent，如下所示：
bin/flume-ng agent --conf-file conf/flume.conf -f /var/log/wtmp -Dflume.root.logger=DEBUG,console -n agent1
这是上述命令的日志
14/06/25 00:37:17 INFO node.PollingPropertiesFileConfigurationProvider: Configuration provider starting
14/06/25 00:37:17 INFO node.PollingPropertiesFileConfigurationProvider: Reloading configuration file:conf/flume.conf
14/06/25 00:37:17 INFO conf.FlumeConfiguration: Processing:hdfs-sink1
14/06/25 00:37:17 INFO conf.FlumeConfiguration: Processing:hdfs-sink1
14/06/25 00:37:17 INFO conf.FlumeConfiguration: Processing:hdfs-sink1
14/06/25 00:37:17 INFO conf.FlumeConfiguration: Processing:hdfs-sink1
14/06/25 00:37:17 INFO conf.FlumeConfiguration: Added sinks: hdfs-sink1 Agent: agent1
14/06/25 00:37:17 INFO conf.FlumeConfiguration: Processing:hdfs-sink1
14/06/25 00:37:17 INFO conf.FlumeConfiguration: Processing:hdfs-sink1
14/06/25 00:37:17 INFO conf.FlumeConfiguration: Processing:hdfs-sink1
14/06/25 00:37:17 INFO conf.FlumeConfiguration: Processing:hdfs-sink1
14/06/25 00:37:17 INFO conf.FlumeConfiguration: Post-validation flume configuration contains configuration for agents: [agent1]
14/06/25 00:37:17 INFO node.AbstractConfigurationProvider: Creating channels
14/06/25 00:37:17 INFO channel.DefaultChannelFactory: Creating instance of channel ch1 type memory
14/06/25 00:37:17 INFO node.AbstractConfigurationProvider: Created channel ch1
14/06/25 00:37:17 INFO source.DefaultSourceFactory: Creating instance of source avro-source1, type syslogtcp
14/06/25 00:37:17 INFO sink.DefaultSinkFactory: Creating instance of sink: hdfs-sink1, type: hdfs
14/06/25 00:37:17 INFO hdfs.HDFSEventSink: Hadoop Security enabled: false
14/06/25 00:37:17 INFO node.AbstractConfigurationProvider: Channel ch1 connected to [avro-source1, hdfs-sink1]
14/06/25 00:37:17 INFO node.Application: Starting new configuration:{ sourceRunners:{avro-source1=EventDrivenSourceRunner: { source:org.apache.flume.source.SyslogTcpSource{name:avro-source1,state:IDLE} }} sinkRunners:{hdfs-sink1=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@5954864a counterGroup:{ name:null counters:{} } }} channels:{ch1=org.apache.flume.channel.MemoryChannel{name: ch1}} }
14/06/25 00:37:17 INFO node.Application: Starting Channel ch1
14/06/25 00:37:17 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: CHANNEL, name: ch1: Successfully registered new MBean.
14/06/25 00:37:17 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: ch1 started
14/06/25 00:37:17 INFO node.Application: Starting Sink hdfs-sink1
14/06/25 00:37:17 INFO node.Application: Starting Source avro-source1
14/06/25 00:37:17 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: SINK, name: hdfs-sink1: Successfully registered new MBean.
14/06/25 00:37:17 INFO instrumentation.MonitoredCounterGroup: Component type: SINK, name: hdfs-sink1 started
14/06/25 00:37:17 INFO source.SyslogTcpSource: Syslog TCP Source starting...
这里的进程卡住了，根本没有继续前进。不知道哪里出错了
有人可以帮我解决一下吗
我没有在我有日志文件的服务器上安装水槽。我也要在那里安装水槽吗？？
Flume 版本使用 - 1.5.0 已安装 Hadoop 版本 - 1.0.4
提前致谢
【问题讨论】：