使用基于 python scrapy 的爬虫但出现错误

Question

大家好，我用python写了一个爬虫来抓取......

import scrapy

from c2.items import C2Item

try:

    class C2(scrapy.Spider):
            name = 'cn'
            allowed_domains = ['priceraja.com']
            start_urls = ['https://www.priceraja.com']



            def parse_item(self, response):

               Item = []
               Item['url']=response.xpath('//a/@href/text()').extract()
               yield Item

except Exception:
logging.exception("message")

我不断收到 NotImplemented 错误

2017-08-05 01:12:28 [scrapy.core.scraper] ERROR: Spider error processing 
<GET 
https://www.killerfeatures.com> (referer: None)
Traceback (most recent call last):
File "D:\Ana\lib\site-packages\twisted\internet\defer.py", line 653, in _ 
runCallbacks
current.result = callback(current.result, *args, **kw)
File "D:\Ana\lib\site-packages\scrapy\spiders\__init__.py", line 90, in 
parse raise NotImplementedError
NotImplementedError
2017-08-05 01:12:28 [scrapy.core.engine] INFO: Closing spider (finished)
2017-08-05 01:12:28 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 435,
 'downloader/request_count': 2,
 'downloader/request_method_count/GET': 2,

'downloader/response_bytes': 9282, '下载者/response_count': 2, 'downloader/response_status_count/200': 1, '下载器/response_status_count/301': 1, 'finish_reason': '完成', 'finish_time': datetime.datetime(2017, 8, 4, 19, 42, 28, 837000), 'log_count/DEBUG': 3, “日志计数/错误”：1， 'log_count/INFO': 7, 'response_received_count': 1, “调度程序/出队”：2， '调度程序/出队/内存'：2， '调度程序/排队'：2， '调度程序/排队/内存'：2， 'spider_exceptions/NotImplementedError': 1, 'start_time': datetime.datetime(2017, 8, 4, 19, 42, 25, 976000)} 2017-08-05 01:12:28 [scrapy.core.engine] 信息：蜘蛛关闭（完成）

Answer 1

当你实现了 parse_item 函数时，Scrapy 正在寻找 parse 函数。将 parse_item 更改为 parse 可能会起作用，或者您可以覆盖 parse 函数。

here 的另一个解决方案是使用 CrawlSpider