【发布时间】:2018-11-08 08:11:37
【问题描述】:
此脚本在“while True:”处循环,通过单击底部的下一步按钮从多个页面中抓取数据,但我无法弄清楚如何构造代码以在分页时继续写入 HTML。相反,它会覆盖之前编写的 html 结果。感谢您的帮助。谢谢!
while True:
time.sleep(10)
golds = driver.find_elements_by_css_selector(".widgetContainer #widgetContent > div.singleCell")
print("found %d golds" % len(golds))
template = """\
<tr class="border">
<td class="image"><img src="{0}"></td>\
<td class="title"><a href="{1}" target="_new">{2}</a></td>\
<td class="price">{3}</td>
</tr>"""
lines = []
for gold in golds:
goldInfo = {}
goldInfo['title'] = gold.find_element_by_css_selector('#dealTitle > span').text
goldInfo['link'] = gold.find_element_by_css_selector('#dealTitle').get_attribute('href')
goldInfo['image'] = gold.find_element_by_css_selector('#dealImage img').get_attribute('src')
try:
goldInfo['price'] = gold.find_element_by_css_selector('.priceBlock > span').text
except NoSuchElementException:
goldInfo['price'] = 'No price display'
line = template.format(goldInfo['image'], goldInfo['link'], goldInfo['title'], goldInfo['price'])
lines.append(line)
try:
#clicks next button
driver.find_element_by_link_text("Next→").click()
except NoSuchElementException:
break
time.sleep(10)
html = """\
<html>
<body>
<table>
<tr class='headers'>
<td class='image'></td>
<td class='title'>Product</td>
<td class='price'>Price / Deal</td>
</tr>
</table>
<table class='data'>
{0}
</table>
</body>
</html>\
"""
f = open('./result.html', 'w')
f.write(html.format('\n'.join(lines)))
f.close()
【问题讨论】:
-
尝试用
f = open('./result.html', 'a')替换f = open('./result.html', 'w') -
谢谢安德森,完美。您想将其发布为我可以接受该答案的答案吗?
-
我认为this answer 信息量更大