【发布时间】：2014-09-20 08:57:34
【问题描述】：
这是我的 Twitter.conf：
TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks = HDFS
TwitterAgent.sources.Twitter.type = org.apache.flume.source.twitter.TwitterSource
TwitterAgent.sources.Twitter.channels = MemChannel
TwitterAgent.sources.Twitter.consumerKey = <<API key>>
TwitterAgent.sources.Twitter.consumerSecret = <<API secret>>
TwitterAgent.sources.Twitter.accessToken = <<Access token>>
TwitterAgent.sources.Twitter.accessTokenSecret = <<Access token secret>>
TwitterAgent.sources.Twitter.keywords = hadoop, big data, analytics
TwitterAgent.sinks.HDFS.channel = MemChannel
TwitterAgent.sinks.HDFS.type = hdfs
TwitterAgent.sinks.HDFS.hdfs.path = hdfs://Singh:9000/flume/
TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream
TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text
TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000
TwitterAgent.sinks.HDFS.hdfs.rollSize = 0
TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000
TwitterAgent.channels.MemChannel.type = memory
TwitterAgent.channels.MemChannel.capacity = 10000
TwitterAgent.channels.MemChannel.transactionCapacity = 100
这是我的flume.log：
20 Sep 2014 14:22:02,286 INFO [lifecycleSupervisor-1-0] (org.apache.flume.node.PollingPropertiesFileConfigurationProvider.start:61) - Configuration provider starting
20 Sep 2014 14:22:02,297 INFO [conf-file-poller-0] (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:133) - Reloading configuration file:/home/vijay/BigData/flume-1.5.0.1/conf/twitter.conf
20 Sep 2014 14:22:02,307 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:HDFS
20 Sep 2014 14:22:02,308 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:HDFS
20 Sep 2014 14:22:02,308 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:HDFS
20 Sep 2014 14:22:02,309 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:930) - Added sinks: HDFS Agent: TwitterAgent
20 Sep 2014 14:22:02,309 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:HDFS
20 Sep 2014 14:22:02,309 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:HDFS
20 Sep 2014 14:22:02,309 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:HDFS
20 Sep 2014 14:22:02,309 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:HDFS
20 Sep 2014 14:22:02,309 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016) - Processing:HDFS
20 Sep 2014 14:22:02,322 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration.validateConfiguration:140) - Post-validation flume configuration contains configuration for agents: [TwitterAgent]
20 Sep 2014 14:22:02,322 WARN [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.getConfiguration:138) - No configuration found for this host:agent
20 Sep 2014 14:22:02,330 INFO [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:138) - Starting new configuration:{ sourceRunners:{} sinkRunners:{} channels:{} }
你能帮我解决这个问题吗？
【问题讨论】：
-
如何启动flume agent？您的日志中没有错误，那么您能否描述一下您的上传到底是如何不起作用的？
标签： apache hadoop twitter flume