刮钥匙错误

Question

我在 Scrapy 教程中创建 Scrapy 蜘蛛时遇到问题：

http://doc.scrapy.org/en/latest/intro/tutorial.html#our-first-spider

这是我的 spiders/dmoz_spider.py 文件中的内容：

class DmozSpider(object):
  name = "dmoz"
  allowed_domains = ["dmoz.org"]
  start_urls = [
    "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
    "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
  ]

  @classmethod
  def from_crawler(cls, crawler):
    spider = crawler.spiders
    return cls(spider)

  def parse(self, response):
    filename = response.url.split("/")[-2]
    open(filename, 'wb').write(response.body)

好消息是我很确定蜘蛛正在被创建。坏消息是我收到此错误：

(scrapestat)unknownc8e0eb148153:tutorial christopherspears$ scrapy crawl dmoz
Traceback (most recent call last):
  File "/Users/christopherspears/.virtualenvs/scrapestat/bin/scrapy", line 4, in <module>
    execute()
  File "/Users/christopherspears/.virtualenvs/scrapestat/lib/python2.7/site-packages/scrapy/cmdline.py", line 143, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "/Users/christopherspears/.virtualenvs/scrapestat/lib/python2.7/site-packages/scrapy/cmdline.py", line 89, in _run_print_help
    func(*a, **kw)
  File "/Users/christopherspears/.virtualenvs/scrapestat/lib/python2.7/site-packages/scrapy/cmdline.py", line 150, in _run_command
    cmd.run(args, opts)
  File "/Users/christopherspears/.virtualenvs/scrapestat/lib/python2.7/site-packages/scrapy/commands/crawl.py", line 48, in run
    spider = crawler.spiders.create(spname, **opts.spargs)
  File "/Users/christopherspears/.virtualenvs/scrapestat/lib/python2.7/site-packages/scrapy/spidermanager.py", line 44, in create
    raise KeyError("Spider not found: %s" % spider_name)
KeyError: 'Spider not found: dmoz'

不确定是什么问题。有什么提示吗？

Answer 1

DmozSpider 应该继承自 BaseSpider（或 Spider，取决于你的 scrapy 版本）。因此，请对您的代码进行以下更改：

from scrapy.spider import BaseSpider

class DmozSpider(BaseSpider):
    ...

我自己尝试过，当蜘蛛类从引发 KeyError 的对象继承时。