【发布时间】：2017-12-10 00:59:27
【问题描述】：
我有类似的 HTML 代码：
<!-- Snippet snippets/search_result_text.html end -->
</h2>
<p class="filter-list">
<span class="facet">Organisations:</span>
<span class="filtered pill">**Reserve Bank of Australia**
<a href="/dataset?groups=business" class="remove" title="Remove"><i class="icon-remove"></i></a>
</span>
<span class="facet">Groups:</span>
<span class="filtered pill">**Business Support and Regulation**
<a href="/dataset?organization=reservebankofaustralia" class="remove" title="Remove"><i class="icon-remove"></i></a>
</span>
</p>
</form>
<!-- Snippet snippets/search_form.html end -->
<!-- Snippet snippets/search_package_list.html start -->
<ul class="dataset-list unstyled">
<!-- Snippet snippets/package_item.html start -->
<li class="dataset-item">
<div class="dataset-content">
<h3 class="dataset-heading">
<a href="/dataset/banks-assets">**Banks – Assets**</a>
</h3>
<div>These data are derived from returns submitted to the Australian Prudential Regulation Authority (APRA) by banks authorised under the Banking Act 1959. APRA assumed...</div>
</div>
<ul class="dataset-resources unstyled">
<li>
<a href="/dataset/banks-assets" class="label" data-format="xls">XLS</a>
</li>
</ul>
</li>
<!-- Snippet snippets/package_item.html end -->
<!-- Snippet snippets/package_item.html start -->
<li class="dataset-item">
<div class="dataset-content">
<h3 class="dataset-heading">
<a href="/dataset/consolidated-exposures-immediate-and-ultimate-risk-basis">**Consolidated Exposures – Immediate and Ultimate Risk Basis**</a>
</h3>
<div>In March 2003, banks and selected Registered Financial Corporations (RFCs) began reporting their international assets, liabilities and country exposures to APRA in ARF/RRF 231...</div>
</div>
<ul class="dataset-resources unstyled">
<li>
<a href="/dataset/consolidated-exposures-immediate-and-ultimate-risk-basis" class="label" data-format="xls">XLS</a>
</li>
</ul>
</li>
<!-- Snippet snippets/package_item.html end -->
我想提取上面用粗体字母表示的数据，并希望以 csv 特定格式写入，例如：
Group Organisation Title
Business Support and Regulation Reserve Bank of Australia Banks-Assets
Business Support and Regulation Reserve Bank of Australia Consolidated Exposures – Immediate and Ultimate Risk Basis
等等.... 我有我的 python 代码，它提供了两个不同的文件。
webpage_urls = ["https://data.gov.au/dataset?q=&groups=business&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&organization=reservebankofaustralia&_groups_limit=0",
"https://data.gov.au/dataset?q=&groups=business&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&organization=department-of-finance&_groups_limit=0",
"https://data.gov.au/dataset?q=&groups=business&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&organization=departmentofagriculturefisheriesandforestry&_groups_limit=0",
"https://data.gov.au/dataset?organization=department-of-communications&q=&groups=business&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&_groups_limit=0",
"https://data.gov.au/dataset?organization=ip-australia&q=&groups=business&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&_groups_limit=0",
"https://data.gov.au/dataset?q=&organization=australiancommunicationsandmediaauthority&groups=business&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&_groups_limit=0",
"https://data.gov.au/dataset?q=&organization=www-mitchellshirecouncil-vic-gov-au&groups=business&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&_groups_limit=0",
"https://data.gov.au/dataset?q=&groups=business&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&organization=digital-transformation-agency&_groups_limit=0"]
# fetching data from all urls
data = []
dfs = []
for i in webpage_urls:
wiki2 = i
page= urllib.request.urlopen(wiki2)
soup = BeautifulSoup(page)
lobbying = {}
data2 = soup.find_all('h3', class_="dataset-heading")
for element in data2:
lobbying[element.a.get_text()] = {}
data2[0].a["href"]
prefix = "https://data.gov.au"
for element in data2:
print()
lobbying[element.a.get_text()]["link"] = prefix + element.a["href"]
#print(lobbying)
df = pd.DataFrame.from_dict(lobbying, orient='index').rename_axis('Titles').reset_index()
dfs.append(df)
df = pd.concat(dfs, ignore_index=True)
df1 = df.drop_duplicates(subset = 'Titles')
print (df1)
df1.to_csv('D:/output2.csv')
for i in webpage_urls:
wiki2 = i
page= urllib.request.urlopen(wiki2)
soup = BeautifulSoup(page)
# fetching organisations
data3 = soup.find_all('li', class_="nav-item active")
lobbying1 = []
for element in data3:
lobbying1.append(element.span.get_text())
data.append(lobbying1)
df_ = pd.DataFrame(data, columns = ['Organisations', 'Groups'])
df2 = df_.drop_duplicates(subset = 'Organisations')
with pd.option_context('display.max_rows', 999):
print (df2)
df2.to_csv('D:/output_new.csv')
上面也给出了链接。请帮助在具有三列的单个 csv 中获得所需的格式。
【问题讨论】：
标签： python csv pandas beautifulsoup