有时您尝试抓取的信息丢失时制作网络抓取工具

Question

当我尝试抓取页面上所有显卡的品牌时，它适用于前 15 个，但后来我得到了TypeError: 'NoneType' object is not subscriptable。

from bs4 import BeautifulSoup as soup
from urllib.request import urlopen as uReq

# the url we want to scrape and saves it to a variable
url = 'https://www.newegg.com/p/pl?d=graphics+card&RandomID=551877219014822520210210001440&PageSize=36'

# opens the url and returns a file object
uClient = uReq(url)

# reads the object and returns the html contents as a string
page_html = uClient.read()

# closes the file
uClient.close()

# html parsing
page_soup = soup(page_html, "html.parser")

# grabs each element with the class of item container and stores in a variable

containers = page_soup.findAll("div", {"class": "item-container"})

# scraping the brands of each graphics card from the website

for container in containers:
    brand = container.div.div.a.img["title"]
    print(brand)

Answer 1

在这种情况下，捕获Exception（在您的情况下为TypeError）可能是有意义的

try:
    something_which_may_raise()
except TypeError:  # specific Exception
    my_code_to_handle_exception()  # maybe do nothing
except Exception as ex:  # generic Exception
    # NOTE you can collect the Exception object here to interact with it!
    print("caught an unexpected Exception: {}".format(repr(ex))
    raise ex  # re-raise that Exception to the calling function

对于你的情况，也许这就是你要找的

...
for index, container in enumerate(containers):
    try:
        print(container.div.div.a.img["title"])  # brand
    except Exception as ex:
        print("couldn't read brand from container {}".format(index))