无法解析 h4 标签内的数据：Python3

Question

我在从第三个表的“文学”选项卡中解析数据时遇到问题。我到达餐桌的步骤：

• 转到 ibl.mdanderson.org/fasmic/#!

• 键入并选择 AKT1（3 个突变）（注意：'GO' 按钮不起作用，请单击下拉菜单中的选项）

• 单击带有文本“MS”的绿色按钮，将出现一个新表格。

• 在这个新表中，会有一个名为literature的选项卡，我需要文学文本和PMID。

我尝试了以下代码，但它给出了一个空列表：

xyz= driver.find_element_by_xpath("//*[contains(text(),'Literature')]").click()
    for elements in driver.find_elements_by_xpath('//div[@class="tab-pane ng-scope active"]'):

        soup = BeautifulSoup(driver.page_source, 'lxml')
        table = soup.find('div', attrs={'id': "literature_div"})
        table_body = table.find('h4')
        rows = table.find_all('h4')
        for row in rows:
             cols = row.find_all('h4')
             # cols = [ele.text.strip() for ele in cols]
             litrature.append([ele for ele in cols if ele])  # Get rid of empty value
             print("Data from COLUMN 1:")
             print(litrature)

我该如何解决这个问题？

更新 当我尝试单击“文献”表下的“下一步”按钮时，出现以下错误： “消息：的元素引用已过时；要么元素不再附加到 DOM，它不在当前框架上下文中，要么文档已被刷新“ 以下是我添加以单击“NEXT”按钮的行：driver.find_element_by_xpath('//a[@ng-click="selectPage(page + 1, $event)"]').click()

我该如何解决这个问题？

Answer 1

你需要等待3次

from selenium import webdriver

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC


driver = webdriver.Chrome()
driver.get('https://ibl.mdanderson.org/fasmic/#!/')

WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.XPATH , '//input')))
input = driver.find_element_by_xpath("//input")
input.send_keys("AKT1\n")

button = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.CLASS_NAME , 'btn-tab-avail')))
button.click()
driver.find_element_by_xpath("//*[contains(text(),'Literature')]").click()

WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.CSS_SELECTOR, '#literature_div h4')))
rows = driver.find_elements_by_css_selector("#literature_div h4")

litrature = []

for item in rows:
    item = item.text
    litrature.append(item)
    print("Data from COLUMN 1:")
    print item

Answer 2

像这样？对 python 等待有更多了解的人当然可以改进我的等待线。

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC

url = "https://ibl.mdanderson.org/fasmic/#!/"
d = webdriver.Chrome()
wait = WebDriverWait(d, 10)
d.get(url)
d.find_element_by_css_selector('[type=text]').send_keys('AKT1 (3 mutations)')
d.find_element_by_css_selector("input[type='text']").send_keys(Keys.RETURN)
btn = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-default.btn-tab-avail")))
btn.click()
d.find_element_by_css_selector("[heading=Literature]").click()

ele = wait.until(EC.text_to_be_present_in_element((By.CSS_SELECTOR, "#literature_div [ng-repeat]"), "PMID"))
eles = d.find_elements_by_css_selector("#literature_div [ng-repeat]")

for item in eles:
    print(item.text,"\n")

d.quit()