将 html 抓取到 csv 文件中

Question

下面的代码从以下页面抓取数据： "http://www.gbgb.org.uk/resultsMeeting.aspx?id=136005.

它会抓取所有相关字段并将它们打印到屏幕上。但是我想尝试将表格形式的数据打印到 csv 文件中，以导出到电子表格或数据库中。

在网站源 HTML 中，赛道、日期、日期时间（比赛时间）等级、距离和奖品来自 div 类“resultsBlockheader”，并在网页上形成比赛卡的顶部区域。

源 HTML 中的比赛正文来自 div 类“resultsBlock”，其中包括完成位置（Fin）灰狗、陷阱、SP、时间/秒和时间距离。

最终会是这个样子

track,date,datetime,grade,distance,prize,fin,greyhound,trap,SP,timeSec,time distance

这可能吗，或者我必须先将其以表格形式打印到屏幕上，然后才能将其导出为 csv。

 from urllib import urlopen
 from bs4 import BeautifulSoup

 html = urlopen("http://www.gbgb.org.uk/resultsMeeting.aspx?id=136005")
 bsObj = BeautifulSoup(html, 'lxml')

 nameList = bsObj. findAll("div", {"class": "track"})
 for name in nameList:
 List = bsObj. findAll("div", {"class": "distance"})
 for name in nameList:
     print(name. get_text())
 nameList = bsObj. findAll("div", {"class": "prizes"})
 for name in nameList:
     print(name. get_text())
 nameList = bsObj. findAll("li", {"class": "first essential fin"})
 for name in nameList:
     print(name. get_text())
 nameList = bsObj. findAll("li", {"class": "essential greyhound"})
 for name in nameList:
     print(name. get_text())
 nameList = bsObj. findAll("li", {"class": "trap"})
 for name in nameList:
     print(name. get_text())
 nameList = bsObj. findAll("li", {"class": "sp"})
 for name in nameList:
     print(name. get_text())
 nameList = bsObj. findAll("li", {"class": "timeSec"})
 for name in nameList:
     print(name. get_text())
 nameList = bsObj. findAll("li", {"class": "timeDistance"})
 for name in nameList:
     print(name. get_text())

 nameList = bsObj. findAll("li", {"class": "essential trainer"})
 for name in nameList:
     print(name. get_text())

 nameList = bsObj. findAll("li", {"class": "first essential comment"})
 for name in nameList:
     print(name. get_text())

 nameList = bsObj. findAll("div", {"class": "resultsBlockFooter"})
 for name in nameList:
     print(name. get_text())

 nameList = bsObj. findAll("li", {"class": "first essential"})
 for name in nameList:
     print(name. get_text())

Answer 1

不知道您为什么没有按照this answer 中建议的代码解决您之前的问题 - 它实际上解决了将字段分组在一起的问题。

这是将track、date 和greyhound 转储到csv 中的后续代码：

import csv

from bs4 import BeautifulSoup
import requests


html = requests.get("http://www.gbgb.org.uk/resultsMeeting.aspx?id=135754").text
soup = BeautifulSoup(html, 'lxml')

rows = []
for header in soup.find_all("div", class_="resultsBlockHeader"):
    track = header.find("div", class_="track").get_text(strip=True).encode('ascii', 'ignore').strip("|")
    date = header.find("div", class_="date").get_text(strip=True).encode('ascii', 'ignore').strip("|")

    results = header.find_next_sibling("div", class_="resultsBlock").find_all("ul", class_="line1")
    for result in results:
        greyhound = result.find("li", class_="greyhound").get_text(strip=True)

        rows.append({
            "track": track,
            "date": date,
            "greyhound": greyhound
        })


with open("results.csv", "w") as f:
    writer = csv.DictWriter(f, ["track", "date", "greyhound"])

    for row in rows:
        writer.writerow(row)

运行代码后results.csv的内容：

Sheffield,02/02/16,Miss Eastwood
Sheffield,02/02/16,Sapphire Man
Sheffield,02/02/16,Swift Millican
...
Sheffield,02/02/16,Geelo Storm
Sheffield,02/02/16,Reflected Light
Sheffield,02/02/16,Boozed Flame

请注意，我在这里使用的是requests，但如果您愿意，可以使用urllib2。