打开 URL 时解码响应

Question

我正在使用以下代码打开一个 url 并检索它的响应：

def get_issue_report(query):
    request = urllib2.Request(query)
    response = urllib2.urlopen(request)
    response_headers = response.info()
    print response.read()

我得到的回复如下：

<?xml version='1.0' encoding='UTF-8'?><entry xmlns='http://www.w3.org/2005/Atom' xmlns:gd='http://schemas.google.com/g/2005' xmlns:issues='http://schemas.google.com/projecthosting/issues/2009' gd:etag='W/&quot;DUUFQH47eCl7ImA9WxBbFEg.&quot;'><id>http://code.google.com/feeds/issues/p/chromium/issues/full/2</id><published>2008-08-30T16:00:21.000Z</published><updated>2010-03-13T05:13:31.000Z</updated><title>Testing if chromium id works</title><content type='html'>&lt;b&gt;What steps will reproduce the problem?&lt;/b&gt;
&lt;b&gt;1.&lt;/b&gt;
&lt;b&gt;2.&lt;/b&gt;
&lt;b&gt;3.&lt;/b&gt;

&lt;b&gt;What is the expected output? What do you see instead?&lt;/b&gt;


&lt;b&gt;Please use labels and text to provide additional information.&lt;/b&gt;
 </content><link rel='replies' type='application/atom+xml' href='http://code.google.com/feeds/issues/p/chromium/issues/2/comments/full'/><link rel='alternate' type='text/html' href='http://code.google.com/p/chromium/issues/detail?id=2'/><link rel='self' type='application/atom+xml' href='https://code.google.com/feeds/issues/p/chromium/issues/full/2'/><author><name>rah...@google.com</name><uri>/u/@VBJVRVdXDhZCVgJ%2FF3tbUV5SAw%3D%3D/</uri></author><issues:closedDate>2008-08-30T20:48:43.000Z</issues:closedDate><issues:id>2</issues:id><issues:label>Type-Bug</issues:label><issues:label>Priority-Medium</issues:label><issues:owner><issues:uri>/u/kuchhal@chromium.org/</issues:uri><issues:username>kuchhal@chromium.org</issues:username></issues:owner><issues:stars>4</issues:stars><issues:state>closed</issues:state><issues:status>Invalid</issues:status></entry>

我想去掉 &lt、&gt 等字符。我尝试使用

response.read().decode('utf-8')

但这并没有多大帮助。

以防万一，response.info() 会打印以下内容：

Content-Type: application/atom+xml; charset=UTF-8; type=entry
Expires: Fri, 01 Jul 2011 11:15:17 GMT
Date: Fri, 01 Jul 2011 11:15:17 GMT
Cache-Control: private, max-age=0, must-revalidate, no-transform
Vary: Accept, X-GData-Authorization, GData-Version
GData-Version: 1.0
ETag: W/"DUUFQH47eCl7ImA9WxBbFEg."
Last-Modified: Sat, 13 Mar 2010 05:13:31 GMT
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Connection: close

这是网址：https://code.google.com/feeds/issues/p/chromium/issues/full/2

Answer 1

使用

xml.sax.saxutils.unescape()

http://docs.python.org/library/xml.sax.utils.html#module-xml.sax.saxutils

Answer 2

Sentinel 已经解释了如何解码像 < 这样的实体引用，但问题远不止于此。

您提供的示例表明您正在阅读 Atom 提要。如果你想在 Python 中可靠地做到这一点，那么我建议使用 Mark Pilgrim 的Universal Feed Parser。

在您的示例中阅读提要的方式如下：

>>> import feedparser
>>> d = feedparser.parse('http://code.google.com/feeds/issues/p/chromium/issues/full/2')
>>> len(d.entries)
1
>>> print d.entries[0].title
Testing if chromium id works
>>> print d.entries[0].description
<b>What steps will reproduce the problem?</b>
<b>1.</b>
<b>2.</b>
<b>3.</b>

<b>What is the expected output? What do you see instead?</b>


<b>Please use labels and text to provide additional information.</b>

使用feedparser 可能比尝试自己进行 XML 解析、实体解码、日期解析、HTML 清理等更可靠、更方便。

Answer 3

from HTMLParser import HTMLParser
import urllib2


query="http://code.google.com/feeds/issues/p/chromium/issues/full/2"

def get_issue_report(query):
    request = urllib2.Request(query)
    response = urllib2.urlopen(request)
    response_headers = response.info()
    return response.read()

s = get_issue_report(query)

p = HTMLParser()

print p.unescape(s)

p.close()