Beautiful Soup 中 find() 函数的困难

Question

我正在尝试从网页中提取值，但遇到了一些麻烦。 HTML 看起来像：

<table class="sal-sector-exposure__sector-table"><thead><tr><th>Sectors</th> <th><span class="sal-sector-exposure__col-icon sal-sector-exposure__col-icon--fund"></span> <span class="sal-sector-exposure__col-title">Fund</span> <span class="sal-sector-exposure__col-pct">%</span></th> <!----> <th><span class="sal-sector-exposure__col-icon sal-sector-exposure__col-icon--benchmark"></span> <span class="sal-sector-exposure__col-title sal-sector-exposure__col-title--long">Category</span> <span class="sal-sector-exposure__col-title sal-sector-exposure__col-title--short">Cat</span> <span class="sal-sector-exposure__col-pct">%</span></th></tr></thead> <tbody><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-basic-materials"></i></span> <span>Basic Materials</span></td> <td><span>5.46</span></td> <!----> <td><span>4.60</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-consumer-cyclical"></i></span> <span>Consumer Cyclical</span></td> <td><span>11.95</span></td> <!----> <td><span>12.04</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-financial-services"></i></span> <span>Financial Services</span></td> <td><span>17.97</span></td> <!----> <td><span>15.44</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-real-estate"></i></span> <span>Real Estate</span></td> <td><span>1.86</span></td> <!----> <td><span>5.16</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-communication-services"></i></span> <span>Communication Services</span></td> <td><span>8.61</span></td> <!----> <td><span>9.22</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-energy"></i></span> <span>Energy</span></td> <td><span>4.15</span></td> <!----> <td><span>3.14</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-industrials"></i></span> <span>Industrials</span></td> <td><span>12.08</span></td> <!----> <td><span>10.62</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-technology"></i></span> <span>Technology</span></td> <td><span>18.32</span></td> <!----> <td><span>19.33</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-consumer-defensive"></i></span> <span>Consumer Defensive</span></td> <td><span>5.25</span></td> <!----> <td><span>5.96</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-healthcare"></i></span> <span>Healthcare</span></td> <td><span>12.38</span></td> <!----> <td><span>12.03</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-utilities"></i></span> <span>Utilities</span></td> <td><span>1.96</span></td> <!----> <td><span>2.45</span></td></tr></tbody></table>

我想用漂亮的汤提取粗体的 11.95，但我不知道如何。它只嵌入在两个跨度标签中，我不知道如何更具体。请帮忙！

Answer 1

试试这个：

from bs4 import BeautifulSoup

html = """<table class="sal-sector-exposure__sector-table"><thead><tr><th>Sectors</th> <th><span class="sal-sector-exposure__col-icon sal-sector-exposure__col-icon--fund"></span> <span class="sal-sector-exposure__col-title">Fund</span> <span class="sal-sector-exposure__col-pct">%</span></th> <!----> <th><span class="sal-sector-exposure__col-icon sal-sector-exposure__col-icon--benchmark"></span> <span class="sal-sector-exposure__col-title sal-sector-exposure__col-title--long">Category</span> <span class="sal-sector-exposure__col-title sal-sector-exposure__col-title--short">Cat</span> <span class="sal-sector-exposure__col-pct">%</span></th></tr></thead> <tbody><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-basic-materials"></i></span> <span>Basic Materials</span></td> <td><span>5.46</span></td> <!----> <td><span>4.60</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-consumer-cyclical"></i></span> <span>Consumer Cyclical</span></td> <td><span>11.95</span></td> <!----> <td><span>12.04</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-financial-services"></i></span> <span>Financial Services</span></td> <td><span>17.97</span></td> <!----> <td><span>15.44</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-real-estate"></i></span> <span>Real Estate</span></td> <td><span>1.86</span></td> <!----> <td><span>5.16</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-communication-services"></i></span> <span>Communication Services</span></td> <td><span>8.61</span></td> <!----> <td><span>9.22</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-energy"></i></span> <span>Energy</span></td> <td><span>4.15</span></td> <!----> <td><span>3.14</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-industrials"></i></span> <span>Industrials</span></td> <td><span>12.08</span></td> <!----> <td><span>10.62</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-technology"></i></span> <span>Technology</span></td> <td><span>18.32</span></td> <!----> <td><span>19.33</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-consumer-defensive"></i></span> <span>Consumer Defensive</span></td> <td><span>5.25</span></td> <!----> <td><span>5.96</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-healthcare"></i></span> <span>Healthcare</span></td> <td><span>12.38</span></td> <!----> <td><span>12.03</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-utilities"></i></span> <span>Utilities</span></td> <td><span>1.96</span></td> <!----> <td><span>2.45</span></td></tr></tbody></table>
"""
css_class_selector = "ms-sector-consumer-cyclical"

soup = BeautifulSoup(html, 'lxml')
span_child = soup.find("i", {"class":css_class_selector})
target_row = span_child.parent.parent.parent
target_value = target_row.find_all("td")[1].text
print(target_value)

我已将您提供的 html 作为 字符串变量 包含在内，以便测试我的解决方案

Answer 2

有很多方法可以做到这一点。这是另一个不依赖 BeautifulSoup 的。这非常适合您的情况。

from lxml import etree
html ='<table class="sal-sector-exposure__sector-table"><thead><tr><th>Sectors</th> <th><span class="sal-sector-exposure__col-icon sal-sector-exposure__col-icon--fund"></span> <span class="sal-sector-exposure__col-title">Fund</span> <span class="sal-sector-exposure__col-pct">%</span></th> <!----> <th><span class="sal-sector-exposure__col-icon sal-sector-exposure__col-icon--benchmark"></span> <span class="sal-sector-exposure__col-title sal-sector-exposure__col-title--long">Category</span> <span class="sal-sector-exposure__col-title sal-sector-exposure__col-title--short">Cat</span> <span class="sal-sector-exposure__col-pct">%</span></th></tr></thead> <tbody><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-basic-materials"></i></span> <span>Basic Materials</span></td> <td><span>5.46</span></td> <!----> <td><span>4.60</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-consumer-cyclical"></i></span> <span>Consumer Cyclical</span></td> <td><span>11.95</span></td> <!----> <td><span>12.04</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-financial-services"></i></span> <span>Financial Services</span></td> <td><span>17.97</span></td> <!----> <td><span>15.44</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-real-estate"></i></span> <span>Real Estate</span></td> <td><span>1.86</span></td> <!----> <td><span>5.16</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-communication-services"></i></span> <span>Communication Services</span></td> <td><span>8.61</span></td> <!----> <td><span>9.22</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-energy"></i></span> <span>Energy</span></td> <td><span>4.15</span></td> <!----> <td><span>3.14</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-industrials"></i></span> <span>Industrials</span></td> <td><span>12.08</span></td> <!----> <td><span>10.62</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-technology"></i></span> <span>Technology</span></td> <td><span>18.32</span></td> <!----> <td><span>19.33</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-consumer-defensive"></i></span> <span>Consumer Defensive</span></td> <td><span>5.25</span></td> <!----> <td><span>5.96</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-healthcare"></i></span> <span>Healthcare</span></td> <td><span>12.38</span></td> <!----> <td><span>12.03</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-utilities"></i></span> <span>Utilities</span></td> <td><span>1.96</span></td> <!----> <td><span>2.45</span></td></tr></tbody></table>'
dom = etree.HTML(html)
x = dom.xpath('//table[contains(@class, "sal-sector-exposure__sector-table")]/tbody/tr[2]/td[2]/span')
print(x[0].text)

Answer 3

您可以使用 CSS 选择器找到包含字符串“Consumer Cyclical”的 <td>，然后找到下一个 <td> 兄弟：

from bs4 import BeautifulSoup

html_doc = """<table class="sal-sector-exposure__sector-table"><thead><tr><th>Sectors</th> <th><span class="sal-sector-exposure__col-icon sal-sector-exposure__col-icon--fund"></span> <span class="sal-sector-exposure__col-title">Fund</span> <span class="sal-sector-exposure__col-pct">%</span></th> <!----> <th><span class="sal-sector-exposure__col-icon sal-sector-exposure__col-icon--benchmark"></span> <span class="sal-sector-exposure__col-title sal-sector-exposure__col-title--long">Category</span> <span class="sal-sector-exposure__col-title sal-sector-exposure__col-title--short">Cat</span> <span class="sal-sector-exposure__col-pct">%</span></th></tr></thead> <tbody><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-basic-materials"></i></span> <span>Basic Materials</span></td> <td><span>5.46</span></td> <!----> <td><span>4.60</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-consumer-cyclical"></i></span> <span>Consumer Cyclical</span></td> <td><span>11.95</span></td> <!----> <td><span>12.04</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-financial-services"></i></span> <span>Financial Services</span></td> <td><span>17.97</span></td> <!----> <td><span>15.44</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-real-estate"></i></span> <span>Real Estate</span></td> <td><span>1.86</span></td> <!----> <td><span>5.16</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-communication-services"></i></span> <span>Communication Services</span></td> <td><span>8.61</span></td> <!----> <td><span>9.22</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-energy"></i></span> <span>Energy</span></td> <td><span>4.15</span></td> <!----> <td><span>3.14</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-industrials"></i></span> <span>Industrials</span></td> <td><span>12.08</span></td> <!----> <td><span>10.62</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-technology"></i></span> <span>Technology</span></td> <td><span>18.32</span></td> <!----> <td><span>19.33</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-consumer-defensive"></i></span> <span>Consumer Defensive</span></td> <td><span>5.25</span></td> <!----> <td><span>5.96</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-healthcare"></i></span> <span>Healthcare</span></td> <td><span>12.38</span></td> <!----> <td><span>12.03</span></td></tr><tr><td><span class="sal-sector-exposure__sector"><i class="ms ms-sector-utilities"></i></span> <span>Utilities</span></td> <td><span>1.96</span></td> <!----> <td><span>2.45</span></td></tr></tbody></table>"""
soup = BeautifulSoup(html_doc, "html.parser")

val = soup.select_one('td:-soup-contains("Consumer Cyclical") + td')
print(val.get_text(strip=True))

打印：

11.95

---

或者找到class="ms-sector-consumer-cyclical"然后下一个<td>：

val = soup.select_one(".ms-sector-consumer-cyclical").find_next("td")
print(val.get_text(strip=True))

---

编辑：从 API 加载数据：

import json
import requests


url = "https://api-global.morningstar.com/sal-service/v1/fund/portfolio/v2/sector/F00000MJ3L/data?languageId=en&locale=en&clientId=MDC&benchmarkId=category&component=sal-components-mip-sector-exposure&version=3.49.0"
headers = {"ApiKey": "lstzFDEOhfFNMLikKa0am9mgEKLBl49T"}

data = requests.get(url, headers=headers).json()

# uncomment this to print all data:
# print(json.dumps(data, indent=4))

print(data["EQUITY"]["fundPortfolio"]["consumerCyclical"])

打印：

11.95041