【发布时间】:2021-07-18 15:38:51
【问题描述】:
我试图在 transfermarkt 网站上获取一些球员的比赛、进球和助攻数
在第一个代码中,我可以得到我想要的:
import requests
from bs4 import BeautifulSoup
# headers for be a nav
headers = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36'}
# url site
site = "https://www.transfermarkt.com/lionel-messi/leistungsdaten/spieler/28003/plus/0?saison=ges"
# download page
objeto_response = requests.get(site, headers=headers)
page = BeautifulSoup(objeto_response.content, 'html.parser')
player_name = []
tag_player = page.find("h1", {"itemprop": "name"})
player_name.append(tag_player.text)
print(player_name)
numbers_player = []
games = page.find_all("td", {"class": "zentriert"})
goals = page.find_all("td", {"class": "zentriert"})
assists = page.find_all("td", {"class": "zentriert"})
numbers_player.append(games[0].text)
numbers_player.append(goals[1].text)
numbers_player.append(assists[2].text)
print(numbers_player)
输出:
['Lionel Messi']
['772', '667', '292']
但我想从更多玩家那里获取这些数据 所以我这样尝试:
import requests
from bs4 import BeautifulSoup
# headers for be a nav
headers = {'User-Agent': 'Mozilla/5.0'}
response = requests.get("https://www.sikayetvar.com/onedio", headers=headers)
players = ['messi', 'cristiano', 'neymar']
for player in players:
i = requests.get(
"https://www.transfermarkt.com/schnellsuche/ergebnis/schnellsuche?query="+player+"&x=0&y=0")
player_page = BeautifulSoup(i.content, features="html.parser")
player_num = player_page.select(
"td.hauptlink > a.spielprofil_tooltip tooltipstered > href")
# download page
objeto_response = requests.get(i, headers=headers)
page = BeautifulSoup(objeto_response.content, 'html.parser')
player_name = []
tag_player = page.find("h1", {"itemprop": "name"})
player_name.append(tag_player.text)
print(player_name)
numbers_player = []
games = page.find_all("td", {"class": "zentriert"})
goals = page.find_all("td", {"class": "zentriert"})
assists = page.find_all("td", {"class": "zentriert"})
numbers_player.append(games[0].text)
numbers_player.append(goals[1].text)
numbers_player.append(assists[2].text)
print(numbers_player)
和:
Exception has occurred: InvalidURL
Failed to parse: <Response [404]>
File "C:\Python\Projetos\foot_project\foot2.py", line 18, in <module>
objeto_response = requests.get(i, headers=headers)
有人可以帮助我吗? 我认为这是非常愚蠢的事情,但我不明白
【问题讨论】:
标签: python parsing web-scraping python-requests html-parsing