当一些 <ul> 没有 <li> 子元素时，从 <ul> 数组中抓取 <li>

Question

我正在尝试网络抓取数据，其中一些是 <li> 元素。我认为问题在于一些<ul> 父母没有<li> 孩子。

HTML 示例如下 -=

<div class="tab-pane predefined-carrier-DPDUK ">
    <img src="https://assets.easypost.com/assets/images/carriers/dpd-logo.c4b107116e903920a5794e69e1990827.svg" alt="DPD UK">
    <ul>
        <li>Parcel</li>
        <li>Pallet</li>
        <li>ExpressPak</li>
        <li>FreightParcel</li>
        <li>Freight</li>
    </ul>
</div>
<div class="tab-pane predefined-carrier-ChinaEMS ">
    <img src="https://assets.easypost.com/assets/images/carriers/china-ems-logo-ca.0c938786bd8d8f141e8fa9337a3362a4.png" alt="EMS">
    <p>No predefined packages for EMS.</p>
    <ul></ul>
</div>
<div class="tab-pane predefined-carrier-Estafeta ">
    <img src="https://assets.easypost.com/assets/images/carriers/estafeta-logo-ca.886242ba90c68a1d68f0e4e5a3a14419.png" alt="Estafeta">
    <ul>
        <li>ENVELOPE</li>
        <li>PARCEL</li>
    </ul>
</div>

所以某些<ul> 将不返回任何结果，即没有<li> 子级。我想出了几个“解决方案”

这个应该遍历每个<ul>，但它总是失败第二个try所以不返回<li>

import requests
from bs4 import BeautifulSoup

r = requests.get("https://www.easypost.com/docs/api#parcels", headers={'User-agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0'})
c = r.content

soup= BeautifulSoup(c, "html.parser")

all = soup.find_all("div", {"class":lambda L: L and L.startswith("tab-pane predefined-carrier")})

for item in all:
    print("\n".join([img['alt'] for img in item.find_all('img', alt=True)]))   
    
    try:
        print(item.find("p").text)
    except:
        print("HAS PACKAGES")
    
    try:
        for ul in all.find_all("ul"):
            for litag in ultag.find_all("li"):
                print(litag.text)
    except:
        print("has no list items")
    
    print("")

结果集是这样的：

DPD UK
HAS PACKAGES
has no list items

EMS
No predefined packages for EMS.
has no list items

Estafeta
HAS PACKAGES
has no list items

第二种解决方案是返回<li>，但我想不出办法让每个<li> 在新行上打印：

import requests
from bs4 import BeautifulSoup

r = requests.get("https://www.easypost.com/docs/api#parcels", headers={'User-agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0'})
c = r.content

soup= BeautifulSoup(c, "html.parser")

all = soup.find_all("div", {"class":lambda L: L and L.startswith("tab-pane predefined-carrier")})

for item in all:
    print("\n".join([img['alt'] for img in item.find_all('img', alt=True)]))   
    
    try:
        print(item.find("p").text)
    except:
        print("HAS PACKAGES")
    
    try:
            print(item.find_all("ul")[0].text)
    except:
        pass
    print("")

结果集类似这样：

DPD UK
HAS PACKAGES
ParcelPalletExpressPakFreightParcelFreight

EMS
No predefined packages for EMS.


Estafeta
HAS PACKAGES
ENVELOPEPARCEL

希望有人能让我走上正确的道路，TIA

Answer 1

这可能会对您有所帮助。我试图提取<ul> 并检查它是否有<li> 标签。只有这样我才会在新行上打印那些 <li> 内容。

import requests
from bs4 import BeautifulSoup

r = requests.get("https://www.easypost.com/docs/api#parcels", headers={'User-agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0'})
c = r.content

soup= BeautifulSoup(c, "html.parser")

div_tag = soup.find_all("div", {"class":lambda L: L and L.startswith("tab-pane predefined-carrier")})

for i in div_tag:
    uls = i.find('ul')
    # Checks if <ul> is empty. That is no <li> tags present
    if uls.text != '':
        # Gets all the <li> from <ul>
        li_tags = uls.findAll('li')
        # print the text of <li> one after other in new line
        for item in li_tags:
            print(item.text)
        print('\n')

Sample Output:

Parcel
Pallet
ExpressPak
FreightParcel
Freight

ENVELOPE
PARCEL

Parcel
Satchel