使用 Selenium 抓取 React 表答案

【问题标题】：Scraping a React-table using Selenium使用 Selenium 抓取 React 表
【发布时间】：2020-11-03 11:54:49
【问题描述】：

我编写了一个代码，用于使用 python selenium 对 HTML React 表进行缩放。但它无法捕获表中的值（仅限 DOM 元素）。这是网站https://nonfungible.com/market/history/decentraland?filter=saleType%3D&length=10&sort=blockTimestamp%3Ddesc&start=0 这是我的代码：

from selenium import webdriver
dr = webdriver.PhantomJS(r'PATH_TO_PHANTOM/phantomjs-2.1.1-macosx/bin/phantomjs')
dr.get("https://nonfungible.com/market/history/decentraland?filter=saleType%3D&length=10&sort=blockTimestamp%3Ddesc&start=0")
table = dr.find_elements_by_class_name("rt-tr-group")
for item in table:
    headers = ",".join([th.text for th in item.find_elements_by_class_name("rt-td")])
    print(headers)

它什么也不打印。

感谢您的帮助！

【问题讨论】：

标签： selenium web-scraping webdriverwait react-table expected-condition

【解决方案1】：

如果您打算使用ChromeDriver 和google-chrome，您可以使用以下Locator Strategy：

代码块：

driver.get("https://nonfungible.com/market/history/decentraland?filter=saleType%3D&length=10&sort=blockTimestamp%3Ddesc&start=0")
print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.rt-tr-group div.rt-td")))])

控制台输出：

['LAND', '115792089237316195423570985008687907834894736851909887012435355206597646221309', '', '', '', '$798.462', '20,000 (MANA)', '1 hr ago 7/14/2020, 12:23:58 PM', 'DCLENS', '43469741616618090506261857019714019688977185639939981493731790027408795972144', '', '', '', '$13.141', '333 (MANA)', '3 hrs ago 7/14/2020, 10:48:46 AM', 'DCLENS', '89838368294314845493020349053893353532486044687731983038462074466093608387295', '', '', '', '$11.627', '295 (MANA)', '3 hrs ago 7/14/2020, 10:44:24 AM', 'DCLENS', '25527909519625043332149265050323189196974713653740181118935970777552699458072', '', '', '', '$4.035', '100 (MANA)', '5 hrs ago 7/14/2020, 8:23:03 AM', 'DCLXMAS2019', '349', '', '', '', '$50.91', '1,251 (MANA)', '7 hrs ago 7/14/2020, 6:11:31 AM', 'LAND', '1361129467683753853853498429727072845789', '', '', '', '$897.357', '22,000 (MANA)', '9 hrs ago 7/14/2020, 4:48:01 AM', 'DCLLNCH', '3629', '', '', '', '$8.164', '200 (MANA)', '10 hrs ago 7/14/2020, 3:04:35 AM', 'LAND', '115792089237316195423570985008687907837276713420356456256678977458620023701615', '', '', '', '$609.161', '14,960 (MANA)', '11 hrs ago 7/14/2020, 2:38:29 AM', 'DCLENS', '93012029374970918876221125714905812452590679293106342049263811906231821801921', '', '', '', '$4.086', '100 (MANA)', '12 hrs ago 7/14/2020, 1:39:25 AM', 'DCLMNSHT2020', '2260', '', '', '', '$19.138', '0.08 (ETH)', '13 hrs ago 7/14/2020, 1:00:43 AM']

【讨论】：