分裂 - 元素不可点击，因为另一个元素 <p> 遮住了它答案

【问题标题】：Splinter - Element is not clickable because another element <p> obscures it分裂 - 元素不可点击，因为另一个元素 <p> 遮住了它
【发布时间】：2020-09-24 06:29:06
【问题描述】：

我正在尝试从一个网站（来自src）获取一些缩略图，并单击一个链接，以便稍后获取大图。

为此，我使用Splinter 和BeautifulSoup。

这是我需要获取的第一个元素的html：

为了做到这一点，我有以下代码：

executable_path = {"executable_path": "/path/to/geckodriver"}
browser = Browser("firefox", **executable_path, headless=False

def get_player_images():

    url = f'https://www.premierleague.com/players'

    # Initiate a splinter instance of the URL
    browser.visit(url)

    browser.find_by_tag('div[class="table playerIndex"]')
    soup = BeautifulSoup(browser.html, 'html.parser')
    for el in soup:
        td =  el.findAll('td')
        for each_td in td:
            link = each_td.find('a', href=True)
            if link:
                print (link['href'])
            image = each_td.find('img')
            if image:
                print(image['src'])
# run
get_player_images()

但是在浏览器打开后我遇到了 2 个问题：

我只为玩家访问前两个实际的src。之后，照片不见了，我不明白为什么。

/players/19970/Max-Aarons/overview
https://resources.premierleague.com/premierleague/photos/players/40x40/p232980.png
/players/13279/Abdul-Rahman-Baba/overview
https://resources.premierleague.com/premierleague/photos/players/40x40/p118335.png
/players/13286/Tammy-Abraham/overview
//platform-static-files.s3.amazonaws.com/premierleague/photos/players/40x40/Photo-Missing.png
/players/3512/Adam-Smith/overview
//platform-static-files.s3.amazonaws.com/premierleague/photos/players/40x40/Photo-Missing.png
/players/10905/Che-Adams/overview
....

另外，如果我尝试点击 href 链接，则：

if link:
   browser.click_link_by_partial_href(link['href'])

我得到错误：

selenium.common.exceptions.ElementClickInterceptedException: Message: Element <a class="playerName" href="/players/19970/Max-Aarons/overview"> is not clickable at point (244,600) because another element <p> obscures it

我做错了什么？我在使用 selenium 时遇到了很多麻烦。

【问题讨论】：

滚动页面时播放器的照片是否会动态加载？该页面是否有任何 cookie 栏或页面底部的其他任何内容，会阻止您点击？
@Jonah 是的，它们是动态加载的。也许我应该这样做：browser.execute_script("window.scrollTo(10000, document.body.scrollHeight);") 让页面继续向下滚动，不是吗？

标签： selenium beautifulsoup splinter

【解决方案1】：

播放器数据通过 Javascript 动态加载。您可以使用requests模块获取信息。

例如：

import re
import json
import requests
from bs4 import BeautifulSoup


url = 'https://footballapi.pulselive.com/football/players?pageSize=30&compSeasons=274&altIds=true&page={page}&type=player&id=-1&compSeasonId=274'
img_url = 'https://resources.premierleague.com/premierleague/photos/players/250x250/{player_id}.png'
headers = {'Origin': 'https://www.premierleague.com'}

for page in range(1, 10):       # <--- increase this to desired number of pages
    data = requests.get(url.format(page=page), headers=headers).json()

    # uncoment this to print all data:
    # print(json.dumps(data, indent=4))

    for player in data['content']:
        print('{:<50} {}'.format(player['name']['display'], img_url.format(player_id=player['altIds']['opta'])))

打印：

Ethan Ampadu                                       https://resources.premierleague.com/premierleague/photos/players/250x250/p199598.png
Joseph Anang                                       https://resources.premierleague.com/premierleague/photos/players/250x250/p447879.png
Florin Andone                                      https://resources.premierleague.com/premierleague/photos/players/250x250/p93284.png
André Gomes                                        https://resources.premierleague.com/premierleague/photos/players/250x250/p120250.png
Andreas Pereira                                    https://resources.premierleague.com/premierleague/photos/players/250x250/p156689.png
Angeliño                                           https://resources.premierleague.com/premierleague/photos/players/250x250/p145235.png
Faustino Anjorin                                   https://resources.premierleague.com/premierleague/photos/players/250x250/p223332.png
Michail Antonio                                    https://resources.premierleague.com/premierleague/photos/players/250x250/p57531.png
Cameron Archer                                     https://resources.premierleague.com/premierleague/photos/players/250x250/p433979.png
Archie Davies                                      https://resources.premierleague.com/premierleague/photos/players/250x250/p215061.png
Stuart Armstrong                                   https://resources.premierleague.com/premierleague/photos/players/250x250/p91047.png
Marko Arnautovic                                   https://resources.premierleague.com/premierleague/photos/players/250x250/p41464.png
Kepa Arrizabalaga                                  https://resources.premierleague.com/premierleague/photos/players/250x250/p109745.png
Harry Arter                                        https://resources.premierleague.com/premierleague/photos/players/250x250/p48615.png
Daniel Arzani                                      https://resources.premierleague.com/premierleague/photos/players/250x250/p200797.png

... and so on.

注意：要获得更小的缩略图，请将图片 URL 中的 250x250 更改为 40x40

【讨论】：