【问题标题】:Not saving lines when writing to CSV file using for-loop使用 for 循环写入 CSV 文件时不保存行
【发布时间】:2021-03-04 05:44:07
【问题描述】:

所以我正在尝试运行一个网站。刮刀运行得很好。但是,每当我尝试将抓取的信息/行写入 csv 文件时,它都会删除前一行。我最终只是在最后的文件中获得了最后的刮擦结果。我确定这只是一个缩进错误?我还是 Python 的新手,所以任何帮助都将不胜感激!

代码:

# create general field names
fields = ['name', 'about', 'job_title', 'location','company',
          'education','accomplishments','linkedin_url']

with open('ScrapeResults.csv', 'w') as f:
    # using csv.writer method from CSV package
    write = csv.writer(f)
    write.writerow(fields)
f.close()

# Loop-through urls to scrape multiple pages at once
for individual,link in contact_dict.items():

    ## assign ##
    the_name = individual
    the_link = link
    # scrape peoples url:
    person = Person(the_link, driver=driver, close_on_complete=False)

    # rows to be written
    rows = [[person.name, person.about, person.job_title, person.location, person.company,
             person.educations, person.accomplishments, person.linkedin_url]]
    # write
    with open('ScrapeResults.csv', 'w') as f:
    # using csv.writer method from CSV package
        write = csv.writer(f)
        write.writerows(rows)
        f.close()

【问题讨论】:

    标签: python csv for-loop web-scraping


    【解决方案1】:

    您需要以追加模式打开文件。

    改变

    with open('ScrapeResults.csv', 'w') as f:
    

    with open('ScrapeResults.csv', 'a') as f:
    

    【讨论】:

      【解决方案2】:

      我认为您正在覆盖文件而不是附加它。 for 循环中的附加模式应该可以解决问题

      with open('ScrapeResults.csv', "a")
      

      请参考python IO文档:https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files

      【讨论】:

        【解决方案3】:

        您必须以追加模式打开文件。

        替换

        with open('ScrapeResults.csv', 'w') as f:

        with open('ScrapeResults.csv', 'a') as f:

        【讨论】:

          猜你喜欢
          • 2018-01-19
          • 2018-03-14
          • 2018-01-12
          • 1970-01-01
          • 1970-01-01
          • 2015-08-12
          • 1970-01-01
          • 2023-03-25
          • 2013-03-21
          相关资源
          最近更新 更多