【发布时间】:2017-06-29 06:09:46
【问题描述】:
我正在尝试从某个网站获取一些新闻,但出现了问题。
import urllib.request
user_agent = 'Chrome/58.0(compatible;IE 10.0; Windows 10)'
headers = {'User-Agent': user_agent}
url = 'http://www.1905.com/list-p-catid-221.html'
request=urllib.request.Request(url,headers)
response=urllib.request.urlopen(request)
first=response.read()
print(first)
f = open('WebCrawler/1905/film/index.2.txt','wb')
f.write(first)
问题是:
Traceback (most recent call last):
File "C:/Users/Heidy/PycharmProjects/untitled1/m2.py", line 7, in <module>
response=urllib.request.urlopen(request)
....
TypeError: can't concat bytes to str
我刚开始学习Python和爬虫,如果你愿意帮助我,非常感谢!
我在response.read() 中添加了.decode('utf-8'),但是没有用。
这是完整的追溯:
enter code here
Traceback (most recent call last):
File "C:/Users/Heidy/PycharmProjects/untitled1/m2.py", line 7, in <module>
response=urllib.request.urlopen(request)
File "C:\Users\Heidy\AppData\Local\Programs\Python\Python36-
32\lib\urllib\request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\Heidy\AppData\Local\Programs\Python\Python36-
32\lib\urllib\request.py", line 526, in open
response = self._open(req, data)
File "C:\Users\Heidy\AppData\Local\Programs\Python\Python36-
32\lib\urllib\request.py", line 544, in _open
'_open', req)
File "C:\Users\Heidy\AppData\Local\Programs\Python\Python36-
32\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "C:\Users\Heidy\AppData\Local\Programs\Python\Python36-
32\lib\urllib\request.py", line 1346, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "C:\Users\Heidy\AppData\Local\Programs\Python\Python36-
32\lib\urllib\request.py", line 1318, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "C:\Users\Heidy\AppData\Local\Programs\Python\Python36-
32\lib\http\client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\Heidy\AppData\Local\Programs\Python\Python36-
32\lib\http\client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\Heidy\AppData\Local\Programs\Python\Python36-
32\lib\http\client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\Heidy\AppData\Local\Programs\Python\Python36-
32\lib\http\client.py", line 1064, in _send_output
+ b'\r\n'
TypeError: can't concat bytes to str
【问题讨论】:
-
请提供完整的回溯。
-
使用:
request=urllib.request.Request(url, headers=headers) -
但是我得到的文字是乱码???
标签: web-crawler