使用 python 抓取特定部分的网页

Question

我有一个使用soup findAll 创建的数组，它的第一个元素有以下信息。在这个列表中，我只需要地址信息，即
“54000 NANCY 47 RUE SERGENT BLANDAN”，我如何获得此信息？

 {
  "div": {
    "@class": "result-left",
    "h3": "Establishment(s)",
    "div": [
      {
        "label": "Status:",
        "#text": "Closed"
      },
      {
        "p": {
          "label": "Brand name:",
          "#text": "LE ZODIAC"
        }
      },
      {
        "p": {
          "label": "Usual name:"
        }
      },
      {
        "p": {
          "label": "Address:",
          "br": [
            "",
            "54000\r\n\t\t\t\t\t\t\t\t\t\t\tNANCY"
          ],
          "#text": "47\r\n\t\t\t\t\t\t\t\t\t\t\r\n\t\t\t\t\t\t\t\t\t\tRUE\r\n\t\t\t\t\t\t\t\t\t\tSERGENT BLANDAN"
        }
      },
      {
        "p": {
          "label": "Principal activity:",
          "#text": "47.78C - \r\n\t\t\t\t\t\t\t\t\t\t\t\r\n\t\t\t\t\t\t\t\t\t\t\tAutres commerces de détail spécialisés divers"
        }
      },
      {
        "p": {
          "label": {
            "sup": "*",
            "#text": [
              "Employee numbers",
              ":"
            ]
          }
        }
      },
      {
        "p": {
          "label": "Year employee numbers verified:"
        }
      }
    ]
  }
}

Answer 1

您可以在提取感兴趣的项目后使用您的字符串并使用 re 进行一些字符串清理。这是给定的 json 所特有的

import  re

s = {
  "div": {
    "@class": "result-left",
    "h3": "Establishment(s)",
    "div": [
      {
        "label": "Status:",
        "#text": "Closed"
      },
      {
        "p": {
          "label": "Brand name:",
          "#text": "LE ZODIAC"
        }
      },
      {
        "p": {
          "label": "Usual name:"
        }
      },
      {
        "p": {
          "label": "Address:",
          "br": [
            "",
            "54000\r\n\t\t\t\t\t\t\t\t\t\t\tNANCY"
          ],
          "#text": "47\r\n\t\t\t\t\t\t\t\t\t\t\r\n\t\t\t\t\t\t\t\t\t\tRUE\r\n\t\t\t\t\t\t\t\t\t\tSERGENT BLANDAN"
        }
      },
      {
        "p": {
          "label": "Principal activity:",
          "#text": "47.78C - \r\n\t\t\t\t\t\t\t\t\t\t\t\r\n\t\t\t\t\t\t\t\t\t\t\tAutres commerces de détail spécialisés divers"
        }
      },
      {
        "p": {
          "label": {
            "sup": "*",
            "#text": [
              "Employee numbers",
              ":"
            ]
          }
        }
      },
      {
        "p": {
          "label": "Year employee numbers verified:"
        }
      }
    ]
  }
}

result =  re.sub(r'\r\n\t+',' ',' '.join([s['div']['div'][3]['p']['br'][1], s['div']['div'][3]['p']['#text']]))
print(result)

Answer 2

有很多重复的制表符、CRLF 和其他空格似乎不太方便。 定义这个函数是值得的：

def simplify_ws(s: str):
    """Coalesces multiple whitespace, e.g. 'a   b c' --> 'a b c'."""
    return ' '.join(s.split())

你的字典很好，很完整， 所以它当然可以用于解决方案。 但是让 bs4 迭代只是你最喜欢的段落会更方便：

for p in soup.find_all('p'):
    txt = p.get_text()
    if 'Address:' in txt:
        print(simplify_ws(txt))

您可能希望在此基础上进行更多过滤和修改。