【发布时间】：2022-08-22 02:22:13
【问题描述】：
我正在使用HTMLSession 进行网络爬虫项目，我计划使用一组用户定义的关键字来爬取 Ask 搜索引擎结果。我已经开始为我的爬虫编写代码了，这里是：
from requests_html import HTMLSession
class Scraper():
def scrapedata(self,tag):
url = f\'https://www.ask.com/web?q={tag}\'
s = HTMLSession()
r = s.get(url)
print(r.status_code)
qlist = []
ask = r.html.find(\'div.PartialSearchResults-item\')
for a in ask:
print(a.find(\'a.PartialSearchResults-item-title-link.result-link::text\', first = True ).text.strip())
ask = Scraper()
ask.scrapedata(\'ferrari\')
但是，当我运行此代码时，我没有得到与在我的终端中搜索的关键字相关的所有网页标题的列表，而是得到以下错误：
[Running] python -u \"c:\\Users\\user\\Documents\\AAprojects\\Whelpsgroups1\\Beauty\\scraper.py\"
200
Traceback (most recent call last):
File \"c:\\Users\\user\\Documents\\AAprojects\\Whelpsgroups1\\Beauty\\scraper.py\", line 19, in <module>
ask.scrapedata(\'ferrari\')
File \"c:\\Users\\user\\Documents\\AAprojects\\Whelpsgroups1\\Beauty\\scraper.py\", line 15, in scrapedata
print(a.find(\'a.PartialSearchResults-item-title-link.result-link::text\', first = True ).text.strip())
File \"C:\\Python310\\lib\\site-packages\\requests_html.py\", line 212, in find
for found in self.pq(selector)
File \"C:\\Python310\\lib\\site-packages\\pyquery\\pyquery.py\", line 261, in __call__
result = self._copy(*args, parent=self, **kwargs)
File \"C:\\Python310\\lib\\site-packages\\pyquery\\pyquery.py\", line 247, in _copy
return self.__class__(*args, **kwargs)
File \"C:\\Python310\\lib\\site-packages\\pyquery\\pyquery.py\", line 232, in __init__
xpath = self._css_to_xpath(selector)
File \"C:\\Python310\\lib\\site-packages\\pyquery\\pyquery.py\", line 243, in _css_to_xpath
return self._translator.css_to_xpath(selector, prefix)
File \"C:\\Python310\\lib\\site-packages\\cssselect\\xpath.py\", line 190, in css_to_xpath
return \' | \'.join(self.selector_to_xpath(selector, prefix,
File \"C:\\Python310\\lib\\site-packages\\cssselect\\xpath.py\", line 190, in <genexpr>
return \' | \'.join(self.selector_to_xpath(selector, prefix,
File \"C:\\Python310\\lib\\site-packages\\cssselect\\xpath.py\", line 222, in selector_to_xpath
xpath = self.xpath_pseudo_element(xpath, selector.pseudo_element)
File \"C:\\Python310\\lib\\site-packages\\cssselect\\xpath.py\", line 232, in xpath_pseudo_element
raise ExpressionError(\'Pseudo-elements are not supported.\')
cssselect.xpath.ExpressionError: Pseudo-elements are not supported.
[Done] exited with code=1 in 17.566 seconds
我什至不知道这意味着什么，我搜索了互联网，但遇到了与 IE7 相关的问题，我看不出与我的问题有什么关系，特别是因为我使用 Microsoft Edge 作为我的默认设置网页浏览器。另外，我希望依靠社区中更有经验的成员的帮助来帮助我解决问题。来自喀麦隆的感谢。
-
选择器的
::text部分是伪元素，正如错误消息所述，它不受支持。相反，使用.find来获取匹配的元素，然后获取每个的.text。您将遇到的另一个问题是您编写的代码好像您的.find将返回单个元素，但事实并非如此。您需要遍历返回的元素并获取每个元素的.text。
标签： python web-scraping python-requests