【发布时间】:2021-03-04 05:44:07
【问题描述】:
所以我正在尝试运行一个网站。刮刀运行得很好。但是,每当我尝试将抓取的信息/行写入 csv 文件时,它都会删除前一行。我最终只是在最后的文件中获得了最后的刮擦结果。我确定这只是一个缩进错误?我还是 Python 的新手,所以任何帮助都将不胜感激!
代码:
# create general field names
fields = ['name', 'about', 'job_title', 'location','company',
'education','accomplishments','linkedin_url']
with open('ScrapeResults.csv', 'w') as f:
# using csv.writer method from CSV package
write = csv.writer(f)
write.writerow(fields)
f.close()
# Loop-through urls to scrape multiple pages at once
for individual,link in contact_dict.items():
## assign ##
the_name = individual
the_link = link
# scrape peoples url:
person = Person(the_link, driver=driver, close_on_complete=False)
# rows to be written
rows = [[person.name, person.about, person.job_title, person.location, person.company,
person.educations, person.accomplishments, person.linkedin_url]]
# write
with open('ScrapeResults.csv', 'w') as f:
# using csv.writer method from CSV package
write = csv.writer(f)
write.writerows(rows)
f.close()
【问题讨论】:
标签: python csv for-loop web-scraping