【问题标题】:Scrapy 2.0.1: how to define an output order?Scrapy 2.0.1:如何定义输出顺序?
【发布时间】:2020-03-29 03:05:40
【问题描述】:

抓取时,我的输出顺序与我在蜘蛛/项目文件中的书面顺序不匹配。

例如:

  def parse(self, response):

        complete_article = response.xpath('//div[@class="storywrapper"]')

        for article in complete_article:
            dachzeile = article.xpath('.//div[@class="meldungHead"]/h1/...
            headline = article.xpath('.//div[@class="meldungHead"]/h1/...
            date = article.xpath('//meta[@name="date"]...
            datum = date.split("T")[0]
            uhrzeit = date.split("T")[1]
            ueberschrift = article.xpath('.//div[@class="mod ....
            text = article.xpath('//div[@class="storywra...
            relative_image = article.xpath('//div[@class="media ...
            final_image = self.base_url + relative_image
            url = response.url.encode('utf-8')

            items = testItem()

            items['Dachzeile'] = dachzeile
            items['Titel'] = headline
            items['Datum'] = datum
            items['Zeit'] = uhrzeit
            items['Einleitung'] = ueberschrift
            items['Artikel'] = text
            items['Bild'] = final_image
            items['Adresse'] = url

            yield items

但是 json 文件中的输出看起来像:

[
  {
    "Artikel": "....",
    "Einleitung": "...",
    "Titel": "...",
    "Zeit": "19:43:10",
    "Datum": "2020-03-28",
    "Adresse": "....html",
    "Bild": "...,
    "Dachzeile": "...,
  }
]

如何设置输出文件的顺序?

提前致以最诚挚的问候和感谢!

【问题讨论】:

标签: python scrapy export


【解决方案1】:

您可以使用OrderedDict维护秩序

从集合导入 OrderedDict

    for article in complete_article:
        ... your code

        items = OrderedDict()

        items['Dachzeile'] = dachzeile
        items['Titel'] = headline
        items['Datum'] = datum
        items['Zeit'] = uhrzeit
        items['Einleitung'] = ueberschrift
        items['Artikel'] = text
        items['Bild'] = final_image
        items['Adresse'] = url

        yield items

【讨论】:

  • 非常感谢,Umair!这成功了。也许这个问题使我成为一个血腥的新手;-)
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2018-05-02
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多