【发布时间】:2016-01-23 06:22:01
【问题描述】:
所以,我正在尝试制作一个“用户友好”的自动刮刀。我基本上想要它做的是,刮一页,单击按钮,刮掉该页面,单击按钮。并且根据用户的需要多次循环。 (是的,我知道缩进已关闭,我正在测试一些东西。
from selenium import webdriver
import time
# Credit
# Vars
website_name = input("Webstie: ")
class_name = input("Class name: ")
button_xpath = input("what's the XPATH that you want to click (normally a button): ")
number_of_pages = int(input("How many pages would you like to scrape: "))
path = r"C:\Users\Skid\Desktop\chromedriver.exe"
driver = webdriver.Chrome(path)
driver.get(website_name + "")
def NormalScrape():
for x in range(number_of_pages):
time.sleep(5)
print("1")
posts = driver.find_elements_by_class_name(class_name)
time.sleep(2)
print("2")
driver.find_element_by_xpath('.' + button_xpath).click()
for post in posts:
print(post.text)
NormalScrape()
错误:
Traceback (most recent call last):
File "C:/Users/Skid/PycharmProjects/untitled/PureTest.py", line 34, in <module>
NormalScrape()
File "C:/Users/Skid/PycharmProjects/untitled/PureTest.py", line 31, in NormalScrape
print(post.text)
File "C:\Users\Skid\AppData\Local\Programs\Python\Python35-32\lib\site-packages\selenium\webdriver\remote\webelement.py", line 71, in text
return self._execute(Command.GET_ELEMENT_TEXT)['value']
File "C:\Users\Skid\AppData\Local\Programs\Python\Python35-32\lib\site-packages\selenium\webdriver\remote\webelement.py", line 454, in _execute
return self._parent.execute(command, params)
File "C:\Users\Skid\AppData\Local\Programs\Python\Python35-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 201, in execute
self.error_handler.check_response(response)
File "C:\Users\Skid\AppData\Local\Programs\Python\Python35-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 181, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
(Session info: chrome=47.0.2526.111)
(Driver info: chromedriver=2.20.353145 (343b531d31eeb933ec778dbcf7081628a1396067),platform=Windows NT 10.0 x86_64)
Exception ignored in: <bound method Service.__del__ of <selenium.webdriver.chrome.service.Service object at 0x00F4DB10>>
Traceback (most recent call last):
File "C:\Users\Skid\AppData\Local\Programs\Python\Python35-32\lib\site-packages\selenium\webdriver\common\service.py", line 136, in __del__
File "C:\Users\Skid\AppData\Local\Programs\Python\Python35-32\lib\site-packages\selenium\webdriver\common\service.py", line 120, in stop
File "C:\Users\Skid\AppData\Local\Programs\Python\Python35-32\lib\site-packages\selenium\webdriver\common\service.py", line 95, in send_remote_shutdown_command
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 954, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 887, in _find_spec
TypeError: 'NoneType' object is not iterable
【问题讨论】:
-
在使用
post.text之前可能检查post- 也许post是None
标签: python python-3.x selenium