我试图从这个网站https://chanakyya.com/Election-Results?electionType=Assembly抓取。但我无法通过检查访问表格以进行网络抓取。
【问题讨论】：
selenium for that frombeautifulsoup 你不会得到任何输出，因为需要输入，所以从 selenium 尝试
标签： python-3.x web-scraping beautifulsoup
您可以抓取填充表格的 json 数据，然后读入数据帧。
注意：我使用了一个包choice 来获取特定状态的输入：
import requests
import pandas as pd
#pip install choice
import choice
states = requests.get('https://chanakyya.com/Chanakya/states.json').json()
state_list = [x['stateDisplayName'] for x in states]
state_choice = choice.Menu(state_list).ask()
stateName = [x['stateName'] for x in states if x['stateDisplayName'] == state_choice][0]
url = f'https://chanakyya.com/Chanakya/{stateName}/{stateName}.json'
resultsData = requests.get(url).json()
tables = {}
for key, value in resultsData['ELECTION_DATA']['stateLevelData'].items():
table = pd.DataFrame(value)
tables[key] = table
for key, table in tables.items():
print(f'*** {key} ***')
print(table,'\n\n')
# Assembly Data
assembly_list = resultsData['ASSEMBLY_NAME_DATA'].keys()
assembly_choice = choice.Menu(assembly_list).ask()
assembly_choice = resultsData['ASSEMBLY_NAME_DATA'][assembly_choice].split('.json')[0]
assembly_url = f'https://chanakyya.com/Chanakya/{stateName}/AssemblyData/{assembly_choice}_Details.json'
resultsAssemblyData = requests.get(assembly_url).json()
print(resultsAssemblyData)
输出：
Make a choice:
0: Andhra Pradesh
1: Assam
2: Bihar
3: Chhattisgarh
4: Goa
5: Gujarat
6: Haryana
7: Himachal Pradesh
8: Jammu & Kashmir
9: Jharkhand
Enter number or name; return for next page
?
0: Karnataka
1: Kerala
2: Madhya Pradesh
3: Maharashtra
4: New Delhi
5: Odisha
6: Punjab
7: Rajasthan
8: Tamilnadu
9: Telangana
Enter number or name; return for next page
?
0: Uttarakhand
1: Uttar Pradesh
2: West Bengal
Enter number or name; return for next page
? 1
*** 2009_Parliament ***
partyName numberOfSeatLeading votePercentage
0 BSP 100 27.42
1 SP 118 23.26
2 INC 95 18.26
3 BJP 62 17.50
4 IND 5 4.53
5 RLD 21 3.27
6 PECP 1 0.98
7 AD 0 0.85
8 SBSP 0 0.55
9 JD(U) 0 0.30
10 RSBP 1 0.28
11 JPS 0 0.20
12 CPI 0 0.16
13 MD 0 0.12
14 IJP 0 0.11
15 RTKP 0 0.11
16 RPI(A) 0 0.10
17 PMSP 0 0.10
18 ASP 0 0.07
19 EKSP 0 0.05
*** 2012_Assembly ***
partyName numberOfSeatLeading votePercentage
0 SP 224 29.13
1 BSP 80 25.91
2 BJP 47 15.00
3 INC 28 11.65
4 IND 6 4.14
5 PECP 4 2.35
6 RLD 9 2.33
7 AD 1 0.90
8 GED 2 0.55
9 NCP 1 0.33
10 IEMC 1 0.25
11 CPI 0 0.13
12 CPIM 0 0.09
*** 2014_Parliament ***
partyName numberOfSeatLeading votePercentage
0 BJP 328 42.32
1 SP 42 22.19
2 BSP 9 19.63
3 INC 15 7.48
4 IND 0 1.75
5 AAP 0 1.02
6 AD 9 1.01
7 RLD 0 0.86
8 NOTA 0 0.74
9 PECP 0 0.62
10 QED 0 0.44
11 BMUP 0 0.19
12 CPI 0 0.16
13 SBSP 0 0.14
14 AITC 0 0.13
15 RPD 0 0.12
16 JD(U) 0 0.07
17 RUC 0 0.07
18 SHS 0 0.04
19 LD 0 0.04
20 NAP 0 0.04
21 BSCP 0 0.04
22 MD 0 0.03
*** 2017_Assembly ***
partyName numberOfSeatLeading votePercentage
0 BJP 312 39.70
1 BSP 19 22.20
2 SP 47 21.80
3 INC 7 6.20
4 IND 3 2.60
5 RLD 1 1.80
6 AD 9 1.00
7 SPSP 4 0.71
8 Nirbal Indian Shoshit Hamara Aam Dal 1 0.60
9 PECP 0 0.30
*** 2019_Parliament ***
partyName numberOfSeatLeading votePercentage
0 BJP 275 49.80
1 BSP 65 19.40
2 SP 40 18.10
3 INC 8 6.30
4 RLD 4 1.71
5 ADAL 9 1.20
6 JDL 2 0.20
【讨论】：
choice 包的意义所在。单击返回转到下一页的选择，直到您获得所需状态的页面。然后选择北方邦。或者只是硬编码。