【发布时间】：2021-05-04 11:25:29
【问题描述】：
目前我正在编写一个小脚本，该脚本应该提取最便宜产品的名称、链接、价格和图片，并提供我所在国家/地区的价格比较网站链接。
这是我目前的代码：
#!/usr/bin/env python3
from urllib.request import Request, urlopen
from lxml import html
from lxml import etree
from lxml.etree import tostring
link = 'https://geizhals.at/?cat=monlcd19wide&xf=11939_23~11955_IPS~11963_144~14591_19201080&asuch=&bpmin=&bpmax=&v=e&hloc=at&plz=&dist=&mail=&sort=p&bl1_id=30#productlist'
link = 'https://geizhals.at/?cat=monlcd19wide&v=e&hloc=at&sort=p&bl1_id=30&xf=11939_23%7E11955_IPS%7E11963_240%7E14591_19201080'
link = 'https://geizhals.at/?cat=cpuamdam4&xf=25_6%7E5_PCIe+4.0%7E5_SMT%7E820_AM4'
def get_webSite():
req = Request(link, headers={'User-Agent': 'Mozilla/5.0'})
return urlopen(req).read()
webpage = get_webSite() # Contains all HTML from the site
root = html.fromstring(webpage)
price = root.xpath("//*[@id=\"product0\"]/div[6]/span/span")[0].text.strip()
name = root.xpath("//*[@id=\"product0\"]/div[2]/a/span")[0].text.strip()
link = "https://geizhals.at/" + root.xpath("//*[@id=\"product0\"]/div[2]/a/@href")[0]
picture = root.xpath("//*[@id=\"product0\"]/div[1]/a/div/picture/img/@big-image-url")[0]
# the @ refers to the attribute of the selected element, / slashes seem to separate the searched terms
# The [0] refers to the first element of a list, we use this because xPath returns a list with exactly one item
price = price.lstrip('€ ') # removes the euro sign and the space
price = price.replace(',', '.') # removes the comma with a dot
price = float(price) # converts price string to float
print(f"Price : {price}")
print("Name : " + (name))
print("Link : " + (link))
print("PictureLink : " + (picture))
除了图片缩略图的链接外，一切正常并打印到控制台中。 我尝试了正常的 xPath 和完整的 xPath，但无济于事。即使存在，也找不到这样的元素。
可能是什么问题？
【问题讨论】：
-
工作？ .....
标签： python web-scraping xpath html-parsing