【发布时间】:2021-10-06 10:26:07
【问题描述】:
您好,我正在this page 上使用 Python 中的 NBA 数据进行网络抓取。篮球参考的一些元素很容易被刮掉,但是这个元素给我带来了一些麻烦,因为我缺乏 python 知识。
我能够获取我想要的数据和列标题,但我最终得到了 2 个需要按索引组合的数据列表(我认为?),以便 player_injury_info 的索引 0 与索引 0 对齐player_names 等,我不知道该怎么做。
下面我粘贴了一些代码,你可以跟着看。
from urllib.request import urlopen
from bs4 import BeautifulSoup
import pandas as pd
from datetime import datetime, timezone, timedelta
url = "https://www.basketball-reference.com/friv/injuries.fcgi"
html = urlopen(url)
soup = BeautifulSoup(html)
# this correctly gives me the 4 column headers i want (Player, Team, Update, Description)
headers = [th.getText() for th in soup.findAll('tr', limit=2)[0].findAll('th')]
# 2 lists - player_injury_info and player_names. they need to be combined.
rows = soup.findAll('tr')
player_injury_info = [[td.getText() for td in rows[i].findAll('td')]
for i in range(len(rows))]
player_injury_info = player_injury_info[1:] # removing first element bc dont need it
player_names = [[th.getText() for th in rows[i].findAll('th')]
for i in range(len(rows))]
player_names = player_names[1:] # removing first element bc dont need it
### joining the lists in the correct order- the part i dont know how to do
player_list = player_names.append(player_injury_info)
### this should give me the data frame i want if i can get player_injury_info into the right format.
injury_data = pd.DataFrame(player_injury_info, columns = headers)
可能有一种更简单的方法可以将数据抓取到所有 1 个列表/数据框中?或者,就像我正在尝试做的那样,将 2 个列表一起加入就可以了。但是,如果有人能够跟进并提供解决方案,我将不胜感激!
【问题讨论】:
标签: python pandas list web-scraping append