将网页抓取转换为 excel？

Question

更新：我尝试在 Pycharm 上安装 pandas 模块并出现错误？ （索引错误：列表索引超出范围）。

Pandas error message 我也尝试在命令提示符窗口中安装，但没有使用 C:> pip install pandas

我也试过这个cmd.exe?

我终于可以让 pip install pandas 工作了，但它仍然说我没有模块...pip install pandas

我正在尝试将此信息自动保存到类似于 Sample excel 的 excel 文件中

import requests
from bs4 import BeautifulSoup

r = requests.get('https://cumberlink.com/sports/high-school/football/pa-football-writers-all-state- 
team-class-a-a-and/article_4d286757-a501-5b5b-b3be-cfebc06ef455.html').text
soup = BeautifulSoup(r, 'html.parser')

for item in soup.findAll('div', {"class": "subscriber-only"}):
    print(item.text)

我的输出如下：我怎样才能让它自动上传到一个excel文件中，最好是类似于我添加的图片的图表？

Player of the Year: Kyle McCord, St. Joseph’s Prep
Coach of the Year: Calvin Everett, Harrisburg
Quarterback
Kyle McCord, St. Joseph’s Prep – 6-3, 195 sophomore
Kane Everson, Harrisburg – 5-11, 180 senior
Cade Pribula, Central York – 6-1, 200 senior
Ricky Ortega, Coatesville – 6-0, 190 junior
Tommy Friberg, State College – 6-5, 200 junior
Running Back
Nyzair Smith, Dallastown – 5-10, 187 senior
Aaron Young, Coatesville – 5-11, 190 senior
Shamar Edwards, North Penn – 5-11, 190 senior
Damon Horton, Hazleton – 5-10, 200 senior
Elijah Morales, Wilson-West Lawn – 5-9, 160 senior
Wide Receiver
Dapree Bryant, Coatesville – 5-8, 175 junior
Johnny Freeman, St. Joseph’s Prep – 6-1, 205 junior
Tommy Manos, Nazareth – 6-0, 180 senior
Jacob Hunsinger, Hazleton – 6-1, 185 senior
Marvin Harrison Jr., St. Joseph’s Prep – 6-3, 175 sophomore
Tight End
Nick Gallo, Council Rock South – 6-4, 235 senior
Offensive Line
Andrew Kristofic, Pine-Richland – 6-6, 275 senior
Ramsey Kifolo, State College – 6-3, 315 junior
Mike Katic, Pine-Richland – 6-3, 280 senior
Raymond Christas, Dallastown – 6-2, 260 senior
Ricky Santiago, Coatesville – 6-5, 305 senior
Athlete
Marques Mason, St. Joseph’s Prep – 5-8, 165 senior
Dresyn Green, State College – 5-10, 170 sophomore
Specialist
Jack Rodenberger, Manheim Twp. – 6-0, 175 senior
Defensive Line
Trevor Harris, Liberty – 6-3, 235 senior
Dionte Nichols, Harrisburg – 6-2, 230 senior
Cade Brennan, Garnet Valley – 6-4, 235 senior
Brian Dallas, Pittsburgh C.C. – 6-4, 225 senior
Linebacker
Tristan Wheeler, Bethlehem Freedom – 6-2, 225 senior
Andre White Jr., Harrisburg – 6-3, 215 senior
Tyler King, Pine-Richland – 6-1, 220 senior
Myles Talley, St. Joseph’s Prep – 5-10, 200 senior
Elijah Workinger, Red Lion – 6-5, 250 senior
Adam Burkhart, Central Dauphin – 5-11, 215 senior
Jeremiah Trotter Jr., St. Joseph’s Prep – 6-0, 200 sophomore
Defensive Back
Joey Porter Jr., North Allegheny – 6-2, 187 senior
Keaton Ellis, State College – 6-1, 185 senior
Sammy Knipe, State College – 6-1, 210 sophomore
Dom Dodson, Pittsburgh C.C. – 5-11, 185 senior
Isaiah Gilmore, Wilson-West Lawn – 5-11, 185 senior
Athlete
Cohen Russell, State College – 5-8, 175 senior
Donte Kent, Harrisburg – 5-11, 170 junior
CLASS 4A
Player of the Year: Billy Lucas, Cathedral Prep
Coach of the Year: Mike Mischler, Cathedral Prep
OFFENSE
Quarterback
Jamie Diven, South Fayette – 6-1, 195 senior
Chase Diehl, Bishop McDevitt – 6-0, 205 senior
Isaac Rumery, Clearfield – 6-1, 205 senior
Deven Bollinger, Northwestern Lehigh – 6-4, 195 junior
Running Back
Billy Lucas, Cathedral Prep – 6-0, 205 senior
Ian Csicsek, Saucon Valley – 5-10, 185 senior
Max Shaw, Thomas Jefferson – 6-0, 190 senior
Mason Kordish, Valley View – 5-8, 170 senior
Tavion Banks, Bethlehem Catholic – 5-10, 187 junior
Wide Receiver
Nazir Burnett, Bishop McDevitt – 6-2, 175 senior
Mike Trimbur, South Fayette – 5-11, 180 senior
Dan Deabner, Thomas Jefferson – 6-0, 180 junior
Yusuf Terry, Imhotep Charter – 6-4, 175 senior
Tight End
Demonte Martin, Eastern York – 6-5, 195 senior
Offensive Line
Chris Oliver, Cathedral Prep – 6-4, 293 senior
Tim Weaver, Saucon Valley – 6-5, 285 senior
Dalton Daddona, Bethlehem Catholic – 6-3, 280 senior
Quentin Bloom, Clearfield – 6-0, 307 junior
Shane Murphy, Berks Catholic – 6-4, 270 senior
Marlon Westcott, Imhotep Charter – 6-2, 305 senior
Athlete
Janaasah Boone, Valley View – 6-0, 175 junior
Abdul MacFoy, Berks Catholic – 5-7, 165 junior
Specialist
Ryan Coe, South Fayette – 6-2, 225 senior
DEFENSE
Defensive Line
Joey Scarabino, Cathedral Prep – 6-3, 241 senior
Caleb Rockey, Bellefonte – 6-3, 245 senior
Ryan Bodolus, Pottsgrove – 6-1, 220 senior
David Tomb, Jersey Shore – 6-0, 205 senior
Luke Painton, Berks Catholic – 6-7, 245 senior
Carter Christopher, Bishop McDevitt – 6-3, 285 senior
Linebacker
Brandon George, Berks Catholic – 6-4, 240 senior
Paul Carreras, Bethlehem Catholic – 6-0, 225 senior
Traon Jones, Valley View – 5-5, 155 senior
Hunter O’Connor, Jersey Shore – 6-0, 225 senior
Dray Donley, Bishop McDevitt – 6-0, 215 senior
Defensive Back
Teagan Wilk, Berwick – 5-11, 180 junior
Alex MacKenzie, Conrad Weiser – 6-3, 195 senior
Nehemiah Figueroa, Pottstown – 5-10, 170 senior
Jaden Duplessis, Valley View – 6-1, 175 senior
Athlete
Tykee Smith, Imhotep Charter – 6-0, 190 senior
Kenyon Johnson, West Perry – 5-11, 180 junior
CLASS 2A
Player of the Year: Julian Fleming, Southern Columbia
Coach of the Year: Brandon Phillian, Wilmington
OFFENSE
Quarterback
Stone Hollenbach, Southern Columbia – 6-3, 200 senior
Skyy Moore, Shady Side Academy – 5-11, 185 senior
Aaron Tobias, Penns Valley – 6-2, 195 junior
John Caldwell, Ligonier Valley – 5-10, 150 senior
Logan Pfister, Chestnut Ridge – 6-1, 163 sophomore
Derek Johncour, Avonworth – 6-3, 181 senior
Running Back
Kobe Brish, Schuylkill Haven – 5-9, 200 junior
Gaige Garcia, Southern Columbia – 5-10, 195 junior
Christian Clutter, McGuffey – 6-0, 180 senior
Kam Williams, Steel Valley – 6-0, 185 senior
Kobe Joseph, Sharpsville – 6-3, 220 senior
Wide Receiver
Bryan Dworek, Brookville – 5-11, 175 senior
Aaron Tutino, Ligonier Valley – 5-11, 170 senior
Julian Fleming, Southern Columbia – 6-3, 215 junior
Cabe Park, Brookville – 5-9, 165 senior
Tight End
Logan Snyder, Penns Valley – 6-2, 200 junior
Offensive Line
Ja’Quay Hubbard, Sharpsville – 6-6, 315 senior
Austin Homan, Schuylkill Haven – 6-0, 243 senior
Troy Donlan, Southern Columbia – 6-0, 260 senior
Dylan Dininny, Richland – 6-5, 255 senior
Oak Six, Southern Columbia – 6-3, 265 senior
Athlete
Ryan Ripka, Penns Valley – 5-8, 160 senior
Aidan Houser, Wellsboro – 5-9, 165 junior
Specialist
Zane Puhala, Brockway – 5-10, 175 senior
DEFENSE
Defensive Line
Michael Hood, Mount Carmel – 6-0, 200 senior
Jimmy Reed, Wilmington – 6-2, 225 senior
Collin Instone, Richland – 6-3, 260 senior
Jordan Gray, Wilmington – 6-1, 235 senior
Harris Kohl, York Catholic – 6-0, 200 senior
Hunter Brown, Wellsboro – 6-4, 220 senior
Linebacker
Cal Haladay, Southern Columbia – 6-1, 190 junior
Colton Richards, Wilmington – 6-2, 270 senior
Todd Hill, Steel Valley – 5-10, 225 senior
Max Tillett, Southern Columbia – 6-0, 190 junior
Garrett David, North Penn-Mansfield – 6-3, 225 senior
Jarred Kohl, York Catholic – 6-1, 205 senior
Defensive Back
Cameron Marett, Wilmington – 5-8, 175 senior
Macen Akers, Everett – 6-1, 180 senior
Daniel Walker, Washington – 6-3, 210 senior
Daunte Allegretto, Ridgway – 5-8, 145 senior
Athlete
Robert Pontius, Wilmington – 6-4, 195 senior

Answer 1

import requests
from bs4 import BeautifulSoup
import pandas as pd


r = requests.get('https://cumberlink.com/sports/high-school/football/pa-football-writers-all-state-team-class-a-a-and/article_4d286757-a501-5b5b-b3be-cfebc06ef455.html')
soup = BeautifulSoup(r.content, 'html.parser')

new = []
for item in soup.findAll('div', {"class": "subscriber-only"}):
    if '-' in item.text:
        data = [s.strip() for s in item.text.replace('–', ',').split(',')]
        data[-1:] = data[-1].split()
        new.append(data)


df = pd.DataFrame(new, columns=['Name', 'School', 'Height', 'Weight', 'Class'])
df['Year'] = '2018'
df.to_excel('output.xlsx')

在线查看：Check