【发布时间】:2016-11-13 02:50:49
【问题描述】:
试图抓取一个站点(俄语,西里尔文)并将所有内容保存在 csv 中,但出现错误
Traceback(最近一次调用最后一次): 文件“/Users/kr/PycharmProjects/education_py/credit_parser.py”,第 30 行,在 base64.b64decode(listing_title[0].encode('utf-8')), 文件“/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/base64.py”,第 76 行,在 b64decode 引发类型错误(味精) TypeError: 不正确的填充
我的代码
# coding: utf8
import requests
from lxml.html import fromstring
import csv
import base64
headers = {
'User-Agent': "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/601.6.17 (KHTML, like Gecko) Version/9.1.1 Safari/601.6.17"
}
csvfile = open('credit-listing.csv', 'wb')
writer = csv.writer(csvfile, quotechar='|', quoting=csv.QUOTE_ALL)
i = 1
while i < 2:
url = requests.get("http://credit-board.ru/index.php?page=search&sCategory=116&iPage={}".format(i), headers=headers)
page_html = fromstring(url.content)
all_listings = page_html.xpath('//*[@id="listing-card-list"]/li')
listings_list = []
for listing in all_listings:
listing_urls = listing.xpath('./div/div/div/div/a/@href')[0]
listing_request = requests.get(listing_urls)
listing_html = fromstring(listing_request.content)
listing_title = listing_html.xpath('//*[@id="item-content"]/h1/strong/text()')
listing_text = listing_html.xpath('//*[@id="description"]/p[1]/text()')
listing_meta = listing_html.xpath('//*[@id="custom_fields"]/div/div/text()')
listings_list.append([listing_title, listing_text, listing_meta])
writer.writerow([
base64.b64decode(listing_title[0].encode('utf-8')),
base64.b64decode(listing_text[0].encode('utf-8')),
base64.b64decode(listing_meta[0].encode('utf-8'))
])
i+=1
print i
【问题讨论】:
-
有些行的长度小于 4 个字符,你确定
incoming data isn't encrypted ! -
我该如何解决?我无法更改网站上的内容。
标签: python python-2.7 csv encoding base64