如何使用 selenium 和 python 在 `::before` 部分中获取文本？答案

【问题标题】：How to get text in `::before` section with selenium and python?如何使用 selenium 和 python 在 `::before` 部分中获取文本？
【发布时间】：2020-10-11 07:54:35
【问题描述】：

我想在variant__available-qty 类中访问::before 之后存储的文本。在示例中为“14”。

当我尝试：

variants = driver.find_elements_by_class_name("variant__available-qty")

或

variants = driver.find_elements_by_class_name("variants__container-item")

它以一个空列表作为响应，尽管应该至少包含三个元素。我尝试执行一些 JS 脚本 (driver.execute_script(...)) 但没有成功。

<div class="variants__container">
   <div class="variants__container-headers">
      <div class="variants__header--item">Rozmiar</div>
      <div class="variants__header--qty">Wybierz ilość</div>
   </div>
   <div class="variants__container-items">
      <div id="variant__70224" class="variants__container-item">
         <div class="variant__price">
            <div class="price">71.27&nbsp;zł</div>
         </div>
         <div class="variant__item">
            <div class="variant__attributes">35-37 </div>
            <div class="variant__sku">610306143389</div>
         </div>
         <div class="variant__qty">
            <form method="post" action="https://b2b.snapoutdoor.pl/checkout/cart/add/uenc/aHR0cHM6Ly9iMmIuc25hcG91dGRvb3IucGwvL3Jlc3QvVjEvZXh0ZW5kdmFyaWFudHN0b2NhcnQvODY1NDY_Xz0xNjAyMzE4NzI4Nzcz/product/86546/">
               <div class="qty">
                  <div class="qty-down">-</div>
                  <input type="number" name="qty" data-productid="70224" onfocus="this.value=''" value="0" min="0">
                  <div class="qty-up">+</div>
               </div>
               <input type="hidden" name="super_attribute[233]" value="2028"><input type="hidden" name="form_key" value="OPk3fByYxbAMTgnu"><button disabled="" class="add-to-cart action primary" type="submit">Do koszyka</button>
            </form>
         </div>
         <div class="variant__available-qty">12</div>
      </div>
      <div id="variant__70225" class="variants__container-item">
         <div class="variant__price">
            <div class="price">71.27&nbsp;zł</div>
         </div>
         <div class="variant__item">
            <div class="variant__attributes">38-40 </div>
            <div class="variant__sku">610306143396</div>
         </div>
         <div class="variant__qty">
            <form method="post" action="https://b2b.snapoutdoor.pl/checkout/cart/add/uenc/aHR0cHM6Ly9iMmIuc25hcG91dGRvb3IucGwvL3Jlc3QvVjEvZXh0ZW5kdmFyaWFudHN0b2NhcnQvODY1NDY_Xz0xNjAyMzE4NzI4Nzcz/product/86546/">
               <div class="qty">
                  <div class="qty-down">-</div>
                  <input type="number" name="qty" data-productid="70225" onfocus="this.value=''" value="0" min="0">
                  <div class="qty-up">+</div>
               </div>
               <input type="hidden" name="super_attribute[233]" value="2036"><input type="hidden" name="form_key" value="OPk3fByYxbAMTgnu"><button disabled="" class="add-to-cart action primary" type="submit">Do koszyka</button>
            </form>
         </div>
         <div class="variant__available-qty">14</div>
      </div>
      <div id="variant__70226" class="variants__container-item">
         <div class="variant__price">
            <div class="price">71.27&nbsp;zł</div>
         </div>
         <div class="variant__item">
            <div class="variant__attributes">41-43 </div>
            <div class="variant__sku">610306143402</div>
         </div>
         <div class="variant__qty">
            <form method="post" action="https://b2b.snapoutdoor.pl/checkout/cart/add/uenc/aHR0cHM6Ly9iMmIuc25hcG91dGRvb3IucGwvL3Jlc3QvVjEvZXh0ZW5kdmFyaWFudHN0b2NhcnQvODY1NDY_Xz0xNjAyMzE4NzI4Nzcz/product/86546/">
               <div class="qty">
                  <div class="qty-down">-</div>
                  <input type="number" name="qty" data-productid="70226" onfocus="this.value=''" value="0" min="0">
                  <div class="qty-up">+</div>
               </div>
               <input type="hidden" name="super_attribute[233]" value="2042"><input type="hidden" name="form_key" value="OPk3fByYxbAMTgnu"><button disabled="" class="add-to-cart action primary" type="submit">Do koszyka</button>
            </form>
         </div>
         <div class="variant__available-qty">6</div>
      </div>
   </div>
</div>

【问题讨论】：

如果你提供网站的网址会很有帮助。
@Sushil 你需要登录才能看到这个:(
哦...:(也许你可以做一件事。复制粘贴你问题中网站的html代码。如果不是整个html代码，至少是其中的相关部分。跨度>
假设只有 1 个元素的类名为 variant__available-qty 请参阅seleniumbyexamples.github.io/findel
@slackingslacker 正如我在帖子中所写，至少有三个元素，find_elements_by_class_name 返回空列表

标签： python-3.x selenium selenium-webdriver web-scraping

【解决方案1】：

你可以简单地使用BeautifulSoup：

from bs4 import BeautifulSoup

html = """
<div class="variants__container">
   <div class="variants__container-headers">
      <div class="variants__header--item">Rozmiar</div>
      <div class="variants__header--qty">Wybierz ilość</div>
   </div>
   <div class="variants__container-items">
      <div id="variant__70224" class="variants__container-item">
         <div class="variant__price">
            <div class="price">71.27&nbsp;zł</div>
         </div>
         <div class="variant__item">
            <div class="variant__attributes">35-37 </div>
            <div class="variant__sku">610306143389</div>
         </div>
         <div class="variant__qty">
            <form method="post" action="https://b2b.snapoutdoor.pl/checkout/cart/add/uenc/aHR0cHM6Ly9iMmIuc25hcG91dGRvb3IucGwvL3Jlc3QvVjEvZXh0ZW5kdmFyaWFudHN0b2NhcnQvODY1NDY_Xz0xNjAyMzE4NzI4Nzcz/product/86546/">
               <div class="qty">
                  <div class="qty-down">-</div>
                  <input type="number" name="qty" data-productid="70224" onfocus="this.value=''" value="0" min="0">
                  <div class="qty-up">+</div>
               </div>
               <input type="hidden" name="super_attribute[233]" value="2028"><input type="hidden" name="form_key" value="OPk3fByYxbAMTgnu"><button disabled="" class="add-to-cart action primary" type="submit">Do koszyka</button>
            </form>
         </div>
         <div class="variant__available-qty">12</div>
      </div>
      <div id="variant__70225" class="variants__container-item">
         <div class="variant__price">
            <div class="price">71.27&nbsp;zł</div>
         </div>
         <div class="variant__item">
            <div class="variant__attributes">38-40 </div>
            <div class="variant__sku">610306143396</div>
         </div>
         <div class="variant__qty">
            <form method="post" action="https://b2b.snapoutdoor.pl/checkout/cart/add/uenc/aHR0cHM6Ly9iMmIuc25hcG91dGRvb3IucGwvL3Jlc3QvVjEvZXh0ZW5kdmFyaWFudHN0b2NhcnQvODY1NDY_Xz0xNjAyMzE4NzI4Nzcz/product/86546/">
               <div class="qty">
                  <div class="qty-down">-</div>
                  <input type="number" name="qty" data-productid="70225" onfocus="this.value=''" value="0" min="0">
                  <div class="qty-up">+</div>
               </div>
               <input type="hidden" name="super_attribute[233]" value="2036"><input type="hidden" name="form_key" value="OPk3fByYxbAMTgnu"><button disabled="" class="add-to-cart action primary" type="submit">Do koszyka</button>
            </form>
         </div>
         <div class="variant__available-qty">14</div>
      </div>
      <div id="variant__70226" class="variants__container-item">
         <div class="variant__price">
            <div class="price">71.27&nbsp;zł</div>
         </div>
         <div class="variant__item">
            <div class="variant__attributes">41-43 </div>
            <div class="variant__sku">610306143402</div>
         </div>
         <div class="variant__qty">
            <form method="post" action="https://b2b.snapoutdoor.pl/checkout/cart/add/uenc/aHR0cHM6Ly9iMmIuc25hcG91dGRvb3IucGwvL3Jlc3QvVjEvZXh0ZW5kdmFyaWFudHN0b2NhcnQvODY1NDY_Xz0xNjAyMzE4NzI4Nzcz/product/86546/">
               <div class="qty">
                  <div class="qty-down">-</div>
                  <input type="number" name="qty" data-productid="70226" onfocus="this.value=''" value="0" min="0">
                  <div class="qty-up">+</div>
               </div>
               <input type="hidden" name="super_attribute[233]" value="2042"><input type="hidden" name="form_key" value="OPk3fByYxbAMTgnu"><button disabled="" class="add-to-cart action primary" type="submit">Do koszyka</button>
            </form>
         </div>
         <div class="variant__available-qty">6</div>
      </div>
   </div>
</div>
""" # The html code
soup = BeautifulSoup(html,'html5lib')

divs = soup.find_all('div',class_ = 'variant__available-qty')

for div in divs:
    print(div.text)

输出：

12
14
6

就我而言，我已经硬编码了 html。但是您可以使用driver.page_source 提取网页的html 代码。然后您可以将该 html 转换为 BeautifulSoup 对象并执行这些操作。如果您仍想使用 selenium，请尝试使用 xpaths 或 css selectors 而不是 class_names。希望这会有所帮助！

【讨论】：

我试过了，但它看不到元素。 ``` driver.get(product_link) html = driver.page_source soup = BeautifulSoup(html, 'html5lib') divs = soup.find_all('div', class_='variant__available-qty') print("divs: ", divs ) for div in divs: print(div.text) ``` divs 的结果是空列表
我看到 cmets 中的格式错误，我在上面的示例中添加了一个要点：gist.github.com/martasobota/5998a7b8a3e41781dd5e18eb8fcd85e6
@Marta 尝试截取页面 driver.save_screenshot("screenshot.png") 以查看页面是否按照您希望的方式加载
@slackingslacker @Sushil 是的，我截图后，页面没有完全加载，当我添加time.sleep(3)时，我可以看到正确的结果！！！谢谢！！！
很高兴听到它奏效了。下面是一些如何等待不同元素的示例。 seleniumbyexamples.github.io/wait