无法使用 lxml 从 HTML 中提取值

Question

我有这段 HTML（数字不同）：

<span class="ng-binding"> <b>Total:</b> 68.71€ (459 items) </span>

---

我想从中提取68.71€ (459 items)

---

到目前为止，我尝试使用这段代码进行此操作，只是将 xpath 从 Google Chrome 复制到上面显示的 span 类：

import urllib.request
from lxml import html
import os

ids =  ["ftpstorage1-730",
        "ftpstorage2-730",
        "ftpstorage3-730"]

for id in ids:

url = 'http://steam.tools/itemvalue/#/'+id
with urllib.request.urlopen(url) as response:
    site = response.read()
    tree = html.fromstring(site)
    data = tree.xpath('//*[@id="container"]/div[5]/span[1]/text()')

    print(data)

理论上这应该有效，但它不会，我得到的只是data 是：

[" {{(items | filter:dupesFilter | filter:typeFilter | filter:filterText |   sumByKey:'price':'count':
e}}\n\t\t\t\t({{items | filter:dupesFilter | filter:typeFilter |    filter:filterText | sumByKey:'count
[" {{(items | filter:dupesFilter | filter:typeFilter | filter:filterText | sumByKey:'price':'count':
e}}\n\t\t\t\t({{items | filter:dupesFilter | filter:typeFilter | filter:filterText | sumByKey:'count
[" {{(items | filter:dupesFilter | filter:typeFilter | filter:filterText | sumByKey:'price':'count':
e}}\n\t\t\t\t({{items | filter:dupesFilter | filter:typeFilter | filter:filterText | sumByKey:'count

---

知道我做错了什么吗？

这与生成的数字不是静态的有关吗？

如果是这样，我怎样才能提取数字？

Answer 1

您在控制台上看到的是 带有 AngularJS 绑定占位符的未渲染 HTML。您需要一个真正的浏览器来执行 javascript 并让 Angular 将真正的值放入占位符中。

或者，如果您深入了解如何检索和计算总价，则无需使用真正的浏览器即可解决此问题。向提供id 和app 参数的http://item-value10.appspot.com/ParseInv 端点发出GET 请求，解析JSON 响应并根据商品数量计算价格：

import requests


template_url = "http://item-value10.appspot.com/ParseInv"
ids = ["ftpstorage1-730", "ftpstorage2-730", "ftpstorage3-730"]

for id in ids:
    with requests.Session() as session:
        session.get('http://steam.tools/itemvalue/#/' + id)

        storage, app = id.split("-")
        url = template_url.format(storage=storage, app=app)

        response = session.get(url, params={
            "id": storage,
            "app": app
        }, headers={
            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36",
            "Referer": "http://steam.tools/itemvalue/"
        })

        data = response.json()
        total = sum(float(item["price"]) * int(item["count"]) for item in data["items"])
        print(total)

打印：

20.439999999999998
78.16
0