Scrapy Crawled 0 pages (at 0 pages/min)

【问题标题】：Scrapy Crawled 0 pages (at 0 pages/min)Scrapy Crawled 0 pages (at 0 pages/min)
【发布时间】：2015-09-06 14:30:05
【问题描述】：

我的scrapy有问题，它没有返回任何结果，我不知道问题在哪里

我使用 Python 版本：2.7.3（Windows 8.1 64 位）

我的项目是通过这个命令创建的 scrapy startproject craigslist_sample

项目目录：

craigslist_sample/
    scrapy.cfg           

    craigslist_sample/            
        __init__.py

        items.py          

        pipelines.py      

        settings.py       

        spiders/         
            __init__.py
              byub.py

我的蜘蛛文件（byub.py）

    import scrapy

class MySpider(scrapy.Spider):
        name = "craig"
        allowed_domains = ["craigslist.org"]
        start_urls = [
            "http://sfbay.craigslist.org/search/sfc/npo"    ]

        def parse(self, response):
          items = []
          for sel in response.xpath('//p//a[@class="hdrlnk"]'):
               item = CraigslistSampleItem()
               print(  sel.xpath('text()').extract())
               print (sel.xpath('@href').extract())

当我按 F5 查看打印的数据时，我什么都没有显示

我的 init.py 文件在 spiders 文件夹中：

import sys
sys.path.append("../../craigslist_sample/")

我添加了路径 ../../craigslist_sample/ 来调用我的 CraigslistSampleItem 类

我的 items.py 文件

import scrapy


class CraigslistSampleItem(scrapy.Item):
    # define the fields for your item here like:
    title = scrapy.Field()
    link  = scrapy.Field()

我的日志文件：

2015-06-20 22:34:59 [scrapy] INFO: Scrapy 1.0.0 started (bot: craigslist_sample)
2015-06-20 22:34:59 [scrapy] INFO: Optional features available: ssl, http11
2015-06-20 22:34:59 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'craigslist_sample.spiders', 'SPIDER_MODULES': ['craigslist_sample.spiders'], 'LOG_STDOUT': True, 'LOG_FILE': '/tmp/scrapy_output.txt', 'BOT_NAME': 'craigslist_sample'}
2015-06-20 22:35:00 [scrapy] INFO: Enabled extensions: CloseSpider, TelnetConsole, LogStats, CoreStats, SpiderState
2015-06-20 22:35:00 [scrapy] INFO: Enabled downloader middlewares: HttpAuthMiddleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, DefaultHeadersMiddleware, MetaRefreshMiddleware, HttpCompressionMiddleware, RedirectMiddleware, CookiesMiddleware, ChunkedTransferMiddleware, DownloaderStats
2015-06-20 22:35:00 [scrapy] INFO: Enabled spider middlewares: HttpErrorMiddleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddleware
2015-06-20 22:35:00 [scrapy] INFO: Enabled item pipelines: 
2015-06-20 22:35:00 [scrapy] INFO: Spider opened
2015-06-20 22:35:00 [scrapy] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2015-06-20 22:35:00 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023
2015-06-20 22:35:02 [scrapy] DEBUG: Crawled (200) <GET http://sfbay.craigslist.org/search/sfc/npo> (referer: None)
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5083113578.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5083098605.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5083051162.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5083044559.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5083043239.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5083034151.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082961277.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082936118.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082930994.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082908649.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082826886.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082820427.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082808607.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082796023.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082767892.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082699233.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082685178.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082682792.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082674781.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082565558.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082545852.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082466564.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082457151.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082454103.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082452290.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082452087.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082442715.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082368243.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082367400.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082364446.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082206212.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082176091.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5082142295.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081546128.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081544083.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081349969.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081337282.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081329478.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081325271.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081315033.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081284397.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081272495.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081248716.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081242306.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081198308.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081185072.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081182362.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081039111.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081033894.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5081030919.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5080930010.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5080922969.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5080783300.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5080757424.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5080754908.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5080696793.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5080523544.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5080474373.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079764803.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079655298.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079652979.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079651750.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079617063.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079600458.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079484883.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079458099.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079439949.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079434763.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079423265.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079421733.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079345334.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079272799.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079271027.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079130762.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5079058773.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5078791191.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5078784316.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5078657036.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5078096040.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5078022877.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5078018145.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077960434.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077955778.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077927644.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077906229.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077813126.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077799125.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077795848.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077763673.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077582518.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077522272.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077402309.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077397915.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077350438.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5077123591.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5076362090.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5076361296.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5076341213.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5076299050.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: [u'/sfc/npo/5076222757.html']
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [stdout] INFO: []
2015-06-20 22:35:02 [scrapy] INFO: Closing spider (finished)
2015-06-20 22:35:02 [scrapy] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 232,
 'downloader/request_count': 1,
 'downloader/request_method_count/GET': 1,
 'downloader/response_bytes': 15530,
 'downloader/response_count': 1,
 'downloader/response_status_count/200': 1,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2015, 6, 20, 20, 35, 2, 312000),
 'log_count/DEBUG': 2,
 'log_count/INFO': 209,
 'response_received_count': 1,
 'scheduler/dequeued': 1,
 'scheduler/dequeued/memory': 1,
 'scheduler/enqueued': 1,
 'scheduler/enqueued/memory': 1,
 'start_time': datetime.datetime(2015, 6, 20, 20, 35, 0, 968000)}
2015-06-20 22:35:02 [scrapy] INFO: Spider closed (finished)
2015-06-20 23:03:37 [scrapy] INFO: Scrapy 1.0.0 started (bot: craigslist_sample)
2015-06-20 23:03:37 [scrapy] INFO: Optional features available: ssl, http11
2015-06-20 23:03:37 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'craigslist_sample.spiders', 'SPIDER_MODULES': ['craigslist_sample.spiders'], 'LOG_STDOUT': True, 'LOG_FILE': '/tmp/scrapy_output.txt', 'BOT_NAME': 'craigslist_sample'}
2015-06-20 23:03:38 [scrapy] INFO: Enabled extensions: CloseSpider, TelnetConsole, LogStats, CoreStats, SpiderState
2015-06-20 23:03:38 [scrapy] INFO: Enabled downloader middlewares: HttpAuthMiddleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, DefaultHeadersMiddleware, MetaRefreshMiddleware, HttpCompressionMiddleware, RedirectMiddleware, CookiesMiddleware, ChunkedTransferMiddleware, DownloaderStats
2015-06-20 23:03:38 [scrapy] INFO: Enabled spider middlewares: HttpErrorMiddleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddleware
2015-06-20 23:03:38 [scrapy] INFO: Enabled item pipelines: 
2015-06-20 23:03:38 [scrapy] INFO: Spider opened
2015-06-20 23:03:38 [scrapy] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2015-06-20 23:03:38 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023
2015-06-20 23:03:39 [scrapy] DEBUG: Crawled (200) <GET http://sfbay.craigslist.org/search/sfc/npo> (referer: None)
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5083113578.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5083098605.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5083051162.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5083044559.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5083043239.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5083034151.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082961277.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082936118.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082930994.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082908649.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082826886.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082820427.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082808607.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082796023.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082767892.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082699233.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082685178.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082682792.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082674781.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082565558.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082545852.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082466564.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082457151.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082454103.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082452290.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082452087.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082442715.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082368243.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082367400.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082364446.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082206212.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082176091.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5082142295.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081546128.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081544083.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081349969.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081337282.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081329478.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081325271.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081315033.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081284397.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081272495.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081248716.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081242306.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081198308.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081185072.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081182362.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081039111.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081033894.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5081030919.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5080930010.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5080922969.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5080783300.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5080757424.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5080754908.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5080696793.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5080523544.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5080474373.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079764803.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079655298.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079652979.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079651750.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079617063.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079600458.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079484883.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079458099.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079439949.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079434763.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079423265.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079421733.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079345334.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079272799.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079271027.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079130762.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5079058773.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5078791191.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5078784316.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5078657036.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5078096040.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5078022877.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5078018145.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077960434.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077955778.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077927644.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077906229.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077813126.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077799125.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077795848.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077763673.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077582518.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077522272.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077402309.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077397915.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077350438.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5077123591.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5076362090.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5076361296.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5076341213.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5076299050.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: [u'/sfc/npo/5076222757.html']
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [stdout] INFO: []
2015-06-20 23:03:39 [scrapy] INFO: Closing spider (finished)
2015-06-20 23:03:39 [scrapy] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 232,
 'downloader/request_count': 1,
 'downloader/request_method_count/GET': 1,
 'downloader/response_bytes': 15536,
 'downloader/response_count': 1,
 'downloader/response_status_count/200': 1,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2015, 6, 20, 21, 3, 39, 743000),
 'log_count/DEBUG': 2,
 'log_count/INFO': 209,
 'response_received_count': 1,
 'scheduler/dequeued': 1,
 'scheduler/dequeued/memory': 1,
 'scheduler/enqueued': 1,
 'scheduler/enqueued/memory': 1,
 'start_time': datetime.datetime(2015, 6, 20, 21, 3, 38, 303000)}
2015-06-20 23:03:39 [scrapy] INFO: Spider closed (finished)

感谢您的帮助

【问题讨论】：

  • 请粘贴完整的日志输出：您粘贴的行已被截断。尝试输出到日志文件：-l scrapy.log
  • 我已经编辑了我的问题，你能看到我按照你说的那样制作的新日志（完整日志）
  • 请更新上面的蜘蛛代码。该文件中的缩进与此处显示的不一致。
  • 我已经更新了 :)

标签： python scrapy


【解决方案1】：

当您在每个&lt;p&gt; 中搜索时，&lt;a&gt; 标记有 url，但其中没有文本。检查网站的 HTML。所以你得到不正确的&lt;a&gt; 标签。

我正在搜索&lt;a&gt; 类class="hdrlnk" 的标签，它有url 和文本。

for sel in response.xpath('//p//a[@class="hdrlnk"]'):
    print sel.xpath('text()').extract()
    print sel.xpath('@href').extract()

输出：

[u'Resident Services Coordinator']
[u'/sfc/npo/5083113578.html']
[u'Resident Services Coordinator']
[u'/sfc/npo/5083098605.html']
[u'General Manager - 939/951 Eddy']
[u'/sfc/npo/5083051162.html']
[u'General Manager - 430 Turk']
[u'/sfc/npo/5083044559.html']
....

更新

  1. 我创建了与您相同的项目。 (scrapy startproject craigslist_sample)

  2. 我从文件spiders/__init__.py中删除了所有内容

  3. 更新的蜘蛛

byub.py 文件

import scrapy
from craigslist_sample.items import CraigslistSampleItem

class MySpider(scrapy.Spider):
    name = "craig"
    allowed_domains = ["craigslist.org"]
    start_urls = [
        "http://sfbay.craigslist.org/search/sfc/npo"    ]

    def parse(self, response):
        items = []
        for sel in response.xpath('//p//a[@class="hdrlnk"]'):
           item = CraigslistSampleItem()
           item['title'] =  sel.xpath('text()').extract()
           item['link'] = sel.xpath('@href').extract()
           items.append(item)
        return items

日志：

2015-06-20 22:01:09 [scrapy] DEBUG: Scraped from <200 http://sfbay.craigslist.org/search/sfc/npo>
{'link': [u'/sfc/npo/5083113578.html'],
 'title': [u'Resident Services Coordinator']}
2015-06-20 22:01:09 [scrapy] DEBUG: Scraped from <200 http://sfbay.craigslist.org/search/sfc/npo>
{'link': [u'/sfc/npo/5083098605.html'],
 'title': [u'Resident Services Coordinator']}
2015-06-20 22:01:09 [scrapy] DEBUG: Scraped from <200 http://sfbay.craigslist.org/search/sfc/npo>
{'link': [u'/sfc/npo/5083051162.html'],
 'title': [u'General Manager - 939/951 Eddy']}
...

为了运行爬虫，我从 scrapy.cfg 所在的同一文件夹中运行 scrapy crawl --logfile logs craig。

以不同的日志级别运行爬虫scrapy crawl --logfile logs -L DEBUG craig

【讨论】：

  • 我希望你使用item['title'] = 和item['link'] = 而不是print。
  • 是的，我曾经这样使用过：def parse(self, response): items = [] for sel in response.xpath('//p//a[@class="hdrlnk"]' ): item = CraigslistSampleItem() item['title'] = sel.xpath('text()').extract() item['link'] = sel.xpath('@href').extract() 项目。 append(item) 返回项目
  • 我只更改了 xpath 的内容，但问题仍然存在
  • 也许只是尝试添加与我相同的打印行，只是为了检查数据是否存在。
  • 我接受您的回答，因为它可以在您家中使用，我将重新安装到另一个操作系统和另一个版本的 python 谢谢！至少我现在知道问题来自我的 Windows 或我的 python 版本，但我不知道具体在哪里：D
【解决方案2】：

你的蜘蛛 return items 中的最后一行缩进太远，需要：-

for sel in response.xpath('//p'):
    item = CraigslistSampleItem()
    # ...

return items

还可以尝试使用scrapy crawl craig -o results.json 将项目输出到名为results.json 的文件中。

【讨论】：

  • 我已经编辑了我的代码，但总是同样的问题，results.json 文件是空的
猜你喜欢
  • 2021-06-05
  • 2020-07-07
  • 2018-09-09
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2014-11-04
  • 2017-03-19
  • 1970-01-01
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode