python web-crawler 猜测解析器警告答案

【问题标题】：python web-crawler Guessed at parser warningpython web-crawler 猜测解析器警告
【发布时间】：2020-08-23 13:41:40
【问题描述】：

我正在尝试使用 python (3.8) 制作一个网络爬虫，我认为我已经完成了，但是我遇到了这个错误，任何人都可以帮助我并提前感谢。

Python 代码：

import requests
from bs4 import BeautifulSoup


def aliexpress_spider (max_pages):
    page = 1
    while page <= max_pages:
        url = "https://www.aliexpress.com/af/ps4.html?trafficChannel=af&d=y&CatId=0&SearchText=ps4&ltype=affiliate&SortType=default&page=" + str(page)
        sourcecode = requests.get(url)
        plaintext = sourcecode.text
        soup = BeautifulSoup(plaintext)
        for link in soup.findAll('a' , {'class' : 'item-title'}):
            href = "https://www.aliexpress.com" + link.get("href")
            title  = link.string
            print(href)
            print(title)
        page += 1


aliexpress_spider(1)

错误按摩：

  GuessedAtParserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html.parser"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

The code that caused this warning is on line 11 of the file C:/Users/moham/PycharmProjects/moh/test.py. To get rid of this warning, pass the additional argument 'features="html.parser"' to the BeautifulSoup constructor.

  soup = BeautifulSoup(plaintext)

【问题讨论】：

这不一定是问题。您只是没有在第 11 行提供 praser 使用。您可以改为写 soup = BeautifulSoup(plaintext, 'html.parser)，警告将消失

标签： python beautifulsoup python-requests web-crawler

【解决方案1】：

import requests
from bs4 import BeautifulSoup


def aliexpress_spider (max_pages):
    page = 1
    while page <= max_pages:
        url = "https://www.aliexpress.com/af/ps4.html?trafficChannel=af&d=y&CatId=0&SearchText=ps4&ltype=affiliate&SortType=default&page=" + str(page)
        sourcecode = requests.get(url)
        
        soup = BeautifulSoup(sourcecode.text ,"html.parser")
        for link in soup.findAll('a' , {'class' : 'item-title'}):
            href = "https://www.aliexpress.com" + link.get("href")
            title  = link.string
            print(href)
            print(title)
        print(soup.title)
        page += 1



aliexpress_spider(1)

【讨论】：