【问题标题】:Appending elements of a list into a multi-dimensional list将列表的元素附加到多维列表中
【发布时间】:2021-10-06 10:26:07
【问题描述】:

您好,我正在this page 上使用 Python 中的 NBA 数据进行网络抓取。篮球参考的一些元素很容易被刮掉,但是这个元素给我带来了一些麻烦,因为我缺乏 python 知识。

我能够获取我想要的数据和列标题,但我最终得到了 2 个需要按索引组合的数据列表(我认为?),以便 player_injury_info 的索引 0 与索引 0 对齐player_names 等,我不知道该怎么做。

下面我粘贴了一些代码,你可以跟着看。

from urllib.request import urlopen
from bs4 import BeautifulSoup
import pandas as pd
from datetime import datetime, timezone, timedelta

url = "https://www.basketball-reference.com/friv/injuries.fcgi"
html = urlopen(url)
soup = BeautifulSoup(html)

# this correctly gives me the 4 column headers i want (Player, Team, Update, Description)
headers = [th.getText() for th in soup.findAll('tr', limit=2)[0].findAll('th')]

# 2 lists - player_injury_info and player_names.  they need to be combined.
rows = soup.findAll('tr')
player_injury_info = [[td.getText() for td in rows[i].findAll('td')]
            for i in range(len(rows))]
player_injury_info = player_injury_info[1:] # removing first element bc dont need it

player_names = [[th.getText() for th in rows[i].findAll('th')]
            for i in range(len(rows))]
player_names = player_names[1:]             # removing first element bc dont need it

### joining the lists in the correct order- the part i dont know how to do
player_list = player_names.append(player_injury_info)

### this should give me the data frame i want if i can get player_injury_info into the right format.
injury_data = pd.DataFrame(player_injury_info, columns = headers)

可能有一种更简单的方法可以将数据抓取到所有 1 个列表/数据框中?或者,就像我正在尝试做的那样,将 2 个列表一起加入就可以了。但是,如果有人能够跟进并提供解决方案,我将不胜感激!

【问题讨论】:

    标签: python pandas list web-scraping append


    【解决方案1】:

    我想你想要这个(元组列表),使用 zip:

    players = ["joe", "bill"]
    injuries = ["tooth-ache", "mental break"]
    list(zip(players, injuries))
    

    结果:

    [('joe', 'tooth-ache'), ('bill', 'mental break')]
    

    【讨论】:

      【解决方案2】:

      让 pandas 为您解析表格。

      import pandas as pd
      
      url = "https://www.basketball-reference.com/friv/injuries.fcgi"
      injury_data = pd.read_html(url)[0]
      

      输出:

      print(injury_data)
                    Player  ...                                        Description
      0     Onyeka Okongwu  ...  Out (Shoulder) - The Hawks announced that Okon...
      1       Jaylen Brown  ...  Out (Wrist) - The Celtics announced that Brown...
      2         Coby White  ...  Out (Shoulder) - The Bulls announced that Whit...
      3     Taurean Prince  ...  Out (Ankle) - The Cavaliers announced F Taurea...
      4       Jamal Murray  ...  Out (Knee) - Murray is recovering from a torn ...
      5      Klay Thompson  ...  Out (Right Achilles) - Thompson is on track to...
      6      James Wiseman  ...  Out (Knee) - Wiseman is on track to be ready b...
      7        T.J. Warren  ...  Out (Foot) - Warren underwent foot surgery and...
      8        Serge Ibaka  ...  Out (Back) - The Clippers announced Serge Ibak...
      9      Kawhi Leonard  ...  Out (Knee) - The Clippers announced Kawhi Leon...
      10    Victor Oladipo  ...  Out (Knee) - Oladipo could be cleared for full...
      11  Donte DiVincenzo  ...  Out (Foot) - DiVincenzo suffered a tendon inju...
      12    Jarrett Culver  ...  Out (Ankle) - The Timberwolves announced Culve...
      13    Markelle Fultz  ...  Out (Knee) - Fultz will miss the rest of the s...
      14    Jonathan Isaac  ...  Out (Knee) - Isaac is making progress with his...
      15       Dario Šarić  ...  Out (Knee) - The Suns announced that Sario has...
      16      Zach Collins  ...  Out (Ankle) - The Blazers announced that Colli...
      17     Pascal Siakam  ...  Out (Shoulder) - The Raptors announced Pascal ...
      18       Deni Avdija  ...  Out (Leg) - The Wizards announced that Avdija ...
      19     Thomas Bryant  ...  Out (Left knee) - The Wizards announced that B...
      
      [20 rows x 4 columns]
      

      但如果您要自己进行迭代,我只需获取行(<tr> 标签),然后在<a> 标签中获取玩家姓名,并将其与该行的<td> 标签组合。然后从这些列表中创建您的数据框:

      from urllib.request import urlopen
      from bs4 import BeautifulSoup
      import pandas as pd
      from datetime import datetime, timezone, timedelta
      
      url = "https://www.basketball-reference.com/friv/injuries.fcgi"
      html = urlopen(url)
      soup = BeautifulSoup(html)
      
      headers = [th.getText() for th in soup.findAll('tr', limit=2)[0].findAll('th')]
      
      trs = soup.findAll('tr')[1:]
      rows = []
      for tr in trs:
          player_name = tr.find('a').text
          data = [player_name] + [x.text for x in tr.find_all('td')]
          rows.append(data)
      
      injury_data = pd.DataFrame(rows, columns = headers)
      

      【讨论】:

        猜你喜欢
        • 1970-01-01
        • 2021-10-02
        • 2017-12-18
        • 2020-09-02
        • 2015-02-05
        • 1970-01-01
        • 1970-01-01
        • 2016-09-17
        • 2017-06-23
        相关资源
        最近更新 更多