【发布时间】：2017-10-09 22:17:41
【问题描述】：
所以我的抓取工具只会将某个范围内的最后一个 URL 发送回 CSV 写入器。 我无法弄清楚我在哪里错过了错误。希望有一双新的眼睛可能会有所帮助。
代码如下：
import requests
from bs4 import BeautifulSoup
import csv
urls = ["https://www.realestate.com.au/property/1-125-mansfield-st-berwick-vic-3806",
"https://www.realestate.com.au/property/1-13-park-ave-mosman-nsw-2088",
"https://www.realestate.com.au/property/1-17-sarton-rd-clayton-vic-3168",
"https://www.realestate.com.au/property/1-2-bridge-st-northcote-vic-3070",
"https://www.realestate.com.au/property/1-2-marara-rd-caulfield-south-vic-3162",]
results = {}
for url in urls:
resp = requests.get(url)
if resp.status_code != 200:
print('Failed in url {}'.format(url))
continue
soup = BeautifulSoup(resp.text, 'html')
link = soup.find(name='a', attrs={'class': lambda x: x and 'property-value__btn-listing' in x}) # find just takes the first one, so no repeated links
href = link.get('href')
results[url] = href
print href
【问题讨论】：
-
缩进你的代码。不缩进python代码是没有意义的
标签： python csv beautifulsoup