从 html <script> 中提取 JSON 对象

Question

我有 JSON 对象：

{
  "review_body": "Beef noodles realism weathered modem tanto hotdog dolphin long-chain hydrocarbons 8-bit euro-pop tank-traps Tokyo narrative.-space j-pop franchise otaku faded RAF girl artisanal hotdog denim ablative systemic smart-Kowloon. Man construct dome smart-computer pen monofilament beef noodles rain garage geodesic bicycle San Francisco wonton soup dissident nodal point tower. Boat uplink film dead man modem warehouse. Nodal point jeans euro-pop render-farm nano-fetishism semiotics hacker gang. Futurity narrative youtube otaku Kowloon free-market drugs. Fluidity assassin Tokyo bicycle media assault concrete industrial grade ablative lights boat BASE jump A.I. post-stimulate carbon. Physical computer narrative city youtube math-neural assassin modem.",
  "link": "http://www.getlost.com/store/acme/review/10607787#comment10607787",
  "seller_id": "104523",
  "survey_id": "9933447",
  "loggedin_user": 0,
  "store_rating": "8.02",
  "store_thumb": "http://www.getlost.com/store/thumbnail/acme.jpg",
  "store_name": "acme",
  "username": "ronin666",
  "rating": "1",
  "ref": "RR,acme,104523"
}

嵌入

<script LANGUAGE="javascript">
window.commentShare = $.extend((window.commentShare || {}), {
    10607787: {
        "review_body": "Beef noodles realism weathered modem tanto hotdog dolphin long-chain hydrocarbons 8-bit euro-pop tank-traps Tokyo narrative.-space j-pop franchise otaku faded RAF girl artisanal hotdog denim ablative systemic smart-Kowloon. Man construct dome smart-computer pen monofilament beef noodles rain garage geodesic bicycle San Francisco wonton soup dissident nodal point tower. Boat uplink film dead man modem warehouse. Nodal point jeans euro-pop render-farm nano-fetishism semiotics hacker gang. Futurity narrative youtube otaku Kowloon free-market drugs. Fluidity assassin Tokyo bicycle media assault concrete industrial grade ablative lights boat BASE jump A.I. post-stimulate carbon. Physical computer narrative city youtube math-neural assassin modem.",
        "link": "http:\/\/www.getlost.com\/store\/acme\/review\/10607787#comment10607787",
        "seller_id": "104523",
        "survey_id": "9933447",
        "loggedin_user": 0,
        "store_rating": "8.02",
        "store_thumb": "http:\/\/www.getlost.com\/store\/thumbnail\/acme.jpg",
        "store_name": "acme",
        "username": "ronin666",
        "rating": "1",
        "ref": "RR,acme,104523"
    }
});
</script>

我想提取上述 JSON 对象。如何做到这一点？我应该使用正则表达式吗？

如何获得这种类型的对象（通过Ipython，python 2.7）：

我基本上是在使用 BeautifulSoup 抓取评论网站 resellerratings.com 的任意商店。我获得了 soup 对象，并注意到有一些有用的 JSON 对象包含所选商店中每条评论的信息。但是，在调用 soup.find("script", language = "javascript") 时，我仍然保留了嵌入在脚本标签中的 JSON 对象。

from mechanize import Browser
import bs4
from bs4 import BeautifulSoup

br = Browser()
br.set_handle_robots(False)
br.set_handle_refresh(False)

example_url = 'http://www.resellerratings.com/store/My_Digital_Palace'

response = br.open(example_url)
soup = BeautifulSoup(response)
soup.find("script", language = "javascript")

这应该返回：

<script language="javascript">
window.commentShare = $.extend(
    (window.commentShare || {}), {
        375015: {
            "review_body": "I bought a Kodak LS443 form My Digital Palace in 2004.  I also purchased a 5 year warranty.  Now the camera does not work and I am unable to contact them.  What do I do???  Am I just screwed???<br><br>Margaret Fuller<br>margaret_fuller@sbcglobal.net",
            "link": "http:\/\/www.resellerratings.com\/store\/My_Digital_Palace\/review\/375015#comment375015",
            "seller_id": "6930",
            "survey_id": "385176",
            "loggedin_user": 0,
            "store_rating": "1.00",
            "store_thumb": "http:\/\/www.resellerratings.com\/store\/thumbnail\/My_Digital_Palace.jpg",
            "store_name": "My Digital Palace",
            "username": "maf1059",
            "rating": "1",
            "ref": "RR,My_Digital_Pala,6930"
        }
    }
);
</script>

Answer 1

很简单，只需去掉包装和多余的行，就可以得到多汁、多汁的 JSON 本身。下面删除了您的 javscript sn-p 的前四行和最后三行（同时还将最初的 { 放回丢失的位置）：

import json

raw = "{" + "\n".join(str(soup.find("script")).split("\n")[4:-3])

如果页面上的 <script> 对象没有以统一的方式编写（也就是说，前四行和后两行并不总是无关紧要的），您可能不得不求助于正则表达式或其他匹配。之后，您可以继续访问 JSON。

json_obj = json.loads(raw)

您的问题只是一个正则表达式/拆分问题。我认为人们对 Javascript 有点失望。 :)

Answer 2

如果您的页面上有此 JOSN 并希望通过 javascript 访问它，您可以通过遍历 window.commentShare 对象中的对象来实现。

这里有一个小测试功能供您添加到您的页面，以便您了解它是如何工作的。它会提醒您的 JSON 值之一。为了完整起见，我已将其添加到您示例的末尾。

<script  LANGUAGE="javascript">
window.commentShare = $.extend((window.commentShare || {}), {
    10607787: {
        "review_body": "Beef noodles realism weathered modem tanto hotdog dolphin long-chain hydrocarbons 8-bit euro-pop tank-traps Tokyo narrative.-space j-pop franchise otaku faded RAF girl artisanal hotdog denim ablative systemic smart-Kowloon. Man construct dome smart-computer pen monofilament beef noodles rain garage geodesic bicycle San Francisco wonton soup dissident nodal point tower. Boat uplink film dead man modem warehouse. Nodal point jeans euro-pop render-farm nano-fetishism semiotics hacker gang. Futurity narrative youtube otaku Kowloon free-market drugs. Fluidity assassin Tokyo bicycle media assault concrete industrial grade ablative lights boat BASE jump A.I. post-stimulate carbon. Physical computer narrative city youtube math-neural assassin modem.",
        "link": "http:\/\/www.getlost.com\/store\/acme\/review\/10607787#comment10607787",
        "seller_id": "104523",
        "survey_id": "9933447",
        "loggedin_user": 0,
        "store_rating": "8.02",
        "store_thumb": "http:\/\/www.getlost.com\/store\/thumbnail\/acme.jpg",
        "store_name": "acme",
        "username": "ronin666",
        "rating": "1",
        "ref": "RR,acme,104523"
    }
});

function test(){

for (var i in window.commentShare) {
    var myObj = window.commentShare[i];
    alert(myObj.review_body);
 }

}
test();

</script>