【发布时间】：2017-12-19 23:36:49
【问题描述】：
仍在学习使用 Python 代码和 beautifulsoup 进行网页抓取，但偶然发现了格式问题。
代码正在从网站中提取正确的数据，但没有将其放置在正确的列中。
例如：
列 "unit_size" 应该有 ==> 5' x 8' x 10'
但它每隔一行写入维度（以及应该在以下列中的其他信息）。
"unit_type" 列应该有 ==> "Drive Up 1st Floor Outside Level No Climate"
列 "online_price" 应该有 ==> "$74.95"
"street_address" 列应该有 ==> 1224 N Tryon St Charlotte NC 28206"
你们是一个很好的帮助。
下面是python代码：
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
urls = ['https://www.uhaul.com/Locations/Self-Storage-near-Charlotte-NC-28206/780052/'
, 'https://www.uhaul.com/Locations/Self-Storage-near-Charlotte-NC-28212/780063/']
filename = "u_haul.csv"
open(filename, 'w').close()
f = open(filename, "a")
num = 0
headers = "unit_size, unit_type, online_price, street_address\n"
f.write(headers)
for my_url in urls:
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
page_soup = soup(page_html, "html.parser")
street_address = page_soup.find("div", {"class": "address"}).text
#store_city = page_soup.find("span", {"": ""}).text
#store_postalcode = page_soup.find("span", {"": ""}).text
containers = page_soup.findAll("div", {"class": "row"})
for container in containers:
title_container = container.findAll("div", {"class": "medium-4 medium-offset-2 small-7 columns"})
unit_type = container.findAll("p", {"class": "collapse"})
online_price = container.findAll("div", {"class": "medium-3 column"})
for item in zip(title_container, unit_type, online_price ):
csv = item[0].text + "," + item[1].text + "," + item[2].text + "," + street_address + "\n"
f.write(csv)
num += 1
f.close()
以下是容器的 HTML：
<div class="row">
<div class="medium-6 columns">
<button class="pull-left toggle-trigger no-toggle-icon show-for-small-only" data-keep-events="" data-toggle-id="mainMenu" id="menuToggle">
<i class="fa fa-bars"></i>
</button>
<!-- mp_trans_remove_start -->
<button class="pull-right toggle-trigger no-toggle-icon show-for-small-only" data-keep-events="" data-toggle-id="searchBox" id="searchToggle">
<i class="fa fa-search"></i>
</button>
<!-- mp_trans_remove_end -->
<a aria-label="Shopping Cart" class="pull-right button show-for-small-only" href="/Cart.aspx" id="header_cart_mobilie">
<i class="fa fa-shopping-cart"></i>
</a>
<div class="logo">
<a class="show-for-medium-up" href="/" id="header_logo_desktop">
<img alt="U-Haul" src="/Images/uhaul-logo.png?v=1290732713"/>
<img alt="Your moving and storage resource." src="/Images/uhaul_tagline.png?v=629728584"/>
</a>
<a class="show-for-small-only" href="/" id="header_logo_mobile">
<img alt="U-Haul" src="/Images/uhaul_logo_white.png?v=291560867"/>
</a>
</div>
</div>
<div class="medium-6 columns">
<ul class="inline text-right show-for-medium-up">
<li>
<a href="/Cart.aspx" id="header_cart">
<i class="fa fa-shopping-cart"></i>
Cart
</a>
</li>
<li>
<a href="/Orders/" id="header_signinlookup">
<i class="fa fa-sign-in"></i>
Sign in / look up order
</a>
</li>
<li>
<a href="/Locations/" id="header_locations">
<i class="fa fa-map-marker"></i>
Locations
</a>
</li>
</ul>
</div>
</div>下面是地址的 HTML：
[ < div class = "address" >
<
p class = "collapse" >
<
span > 1224 N Tryon St < /span> <
br / >
<
span > Charlotte < /span>, <
span > NC < /span> <
span > 28206 < /span><br/ >
<
/p>这是“unit_size”和“unit_type”列的 HTML：
<div class="medium-4 medium-offset-2 small-7 columns">
<h4 class="">
5' x 8' x 10'
</h4>
<p class="collapse">
Drive Up 1st Floor Outside Level No Climate <br/> Miscellaneous Storage (up to 2 rooms) <br/>
<em></em>
</p>
</div>最后是“online_price”列的 HTML：
<div class="medium-3 column">
<p>
<strong class="text-large ">
$74.95
</strong>
<br/> per month
</p>
</div>【问题讨论】：
-
网络浏览器不关心空格和制表符。它总是将许多空格显示为一个空格 - 但您必须使用标准字符串函数（如
strip()、split()、join()、replace()等）删除它们。
标签： python html web-scraping beautifulsoup