从 div 中抓取

Question

我正在尝试网页抓取，但在从一些嵌套的 div 类中抓取特定值时遇到了麻烦。我在 .NET Framework C# 控制台应用程序中使用 .NET HtmlAgilityPack 类库。这是div代码：

<div class="ds-nearby-schools-list">
    <div class="ds-school-row">
        <div class="ds-school-rating">
            <div class="ds-gs-rating-8">
                <span class="ds-hero-headline ds-schools-display-rating">8</span>
                <span class="ds-rating-denominator ds-legal">/10</span>
            </div>
        </div>
        <div class="ds-nearby-schools-info-section">
            <a class="ds-school-name ds-standard-label notranslate" href="https://www.greatschools.org/school?id=00870&amp;state=MD" rel="nofollow noopener noreferrer" target="_blank">Candlewood Elementary School</a>
            <ul class="ds-school-info-section">
                <li class="ds-school-info">
                    <span class="ds-school-key ds-body-small">Grades:</span>
                    <span class="ds-school-value ds-body-small">K-5</span>
                </li>
                <li class="ds-school-info">
                    <span class="ds-school-key ds-body-small">Distance:</span>
                    <span class="ds-school-value ds-body-small">0.8 mi</span>
                </li>
            </ul>
        </div>
    </div>
</div>

我想从ds-hero-headline ds-schools-display-rating 类中刮取“8”。我在 HtmlNode.HtmlDocument 类的 DocumentNode 对象上为 SelectNodes 方法制定选择器时遇到问题。

Answer 1

我猜你可能在编写 XPath 来选择节点时遇到了麻烦。使用 SelectNodes 方法尝试 //*[contains(@class, 'ds-hero-headline') and contains(@class, 'ds-schools-display-rating')]。

但是，如果您的目标页面也有类似ds-hero-headline-content 的类名（ds-hero-headline 可以部分匹配），则此 XPath 可能会出现问题。在这种情况下，请参阅How can I find an element by CSS class with XPath?中的解决方案

Answer 2

我会用它来提取0.8 mi

//div[@class='ds-nearby-schools-list']/div[@class='ds-school-row']/div[@class='ds-nearby-schools-info-section']/ul[@class='ds-school-info-section']/li[@class='ds-school-info']/span[@class='ds-school-value ds-body-small' and preceding-sibling::span[@class='ds-school-key ds-body-small' and text()='Distance:']]/text()

然后这个正则表达式来分组数据：

^[0-9\.]+ (.*)$

最后，您可以使用某种转换来节省与对象的距离。

Answer 3

您是否尝试过以下方法来获取 8。您可以使用类名搜索特定的 span 元素以获取内部文本。

注意：我使用文本文件从您的问题中加载了 html。

    string htmlFile = File.ReadAllText(@"TempFile.html");
    HtmlDocument doc = new HtmlDocument();
    doc.LoadHtml(htmlFile);
    HtmlNode htmlDoc = doc.DocumentNode;

    HtmlNode node = htmlDoc.SelectSingleNode("//span[@class='ds-hero-headline ds-schools-display-rating']");
    Console.WriteLine(node.InnerText);

    // output: 8

替代： 另一种方法是指定要从中获取值的路径，从 div 元素开始。

    HtmlNode node2 = htmlDoc.SelectSingleNode("//div[@class='ds-gs-rating-8']//span[@class='ds-hero-headline ds-schools-display-rating']");
    Console.WriteLine(subNode.InnerText);

输出

8