Python BeautifulSoup4 解析：雅虎财经上隐藏的 html 元素

Question

我正在雅虎财经上分析亚马逊的资产负债表。它包含嵌套行，我无法提取所有这些行。工作表如下所示：

我使用 BeautifulSoup4 和 Selenium Web 驱动程序得到以下输出：

以下是代码：

import pandas as pd
from bs4 import BeautifulSoup
import re
from selenium import webdriver
import string
import time

# chart display specifications w/ Panda
pd.options.display.float_format = '{:.0f}'.format
pd.set_option('display.width', None)

is_link = 'https://finance.yahoo.com/quote/AMZN/balance-sheet/'

chrome_path = r"C:\\Users\\hecto\\Documents\\python\\drivers\\chromedriver.exe"
driver = webdriver.Chrome(chrome_path)
driver.get(is_link)

html = driver.execute_script('return document.body.innerHTML;')
soup = BeautifulSoup(html,'lxml')

features = soup.find_all('div', class_='D(tbr)')

headers = []
temp_list = []
label_list = []
final = []
index = 0
#create headers
for item in features[0].find_all('div', class_='D(ib)'):
    headers.append(item.text)
#statement contents
while index <= len(features)-1:
    #filter for each line of the statement
    temp = features[index].find_all('div', class_='D(tbc)')
    for line in temp:
        #each item adding to a temporary list
        temp_list.append(line.text)
    #temp_list added to final list
    final.append(temp_list)
    #clear temp_list
    temp_list = []
    index+=1
df = pd.DataFrame(final[1:])
df.columns = headers

#function to make all values numerical
def convert_to_numeric(column):
    first_col = [i.replace(',','') for i in column]
    second_col = [i.replace('-','') for i in first_col]
    final_col = pd.to_numeric(second_col)

    return final_col

for column in headers[1:]:
    df[column] = convert_to_numeric(df[column])
final_df = df.fillna('-')

print(df)

再一次，我似乎无法在我的输出中获得资产负债表的所有行（即现金、总流动资产）。我哪里做错了？我错过了什么吗？

Answer 1

您可能必须单击“全部展开”按钮才能查看其他行。参考这个线程看看如何在 Selenium 中模拟点击：python selenium click on button