python Requests库总结

这个总结主要是结合慕课网的视频和自己查资料总结而成

什么是Requests库？

requests库github地址：https://github.com/requests/requests

Reqyests库主要用来准备Request和处理Response。

为什么要学习Requests库？

web开发和爬虫都需要学习的东西，在服务端编程中理解好Requests库可以更好的编写Restful API的程序，还是自动化测试的工具箱。

安装Requests库

pip install requests

这个是安装requests库的

pip install gunicorn

gunicorn是一个python Wsgi http server，只支持在Unix系统上运行，来源于Ruby的unicorn项目。

pip install httpbin

httpbin是一个http库的测试工具

gunicorn httpbin:app

通过gunicorn启动httpbin,可以通过127.0.0.1/8000访问

简单了解http协议

http协议:HyperText Transfer Protocl 超文本传输协议.

http协议是应用层上的一个无状态的协议，是一种为分布式，协作式，多媒体信息服务的协议。

> GET / HTTP/1.1
> Host: www.imooc.com
> User-Agent: curl/7.47.0
> Accept: */*

Request:

第一行：分别是方法:GET，地址：/，协议：HTTP/1.1。

二三四行以key:value的形式组成headers。

< HTTP/1.1 200 OK
< Server: nginx
< Date: Sun, 16 Sep 2018 14:36:46 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 249314
< Connection: keep-alive
< Vary: Accept-Encoding
< Vary: Accept-Encoding
< X-Varnish: 636943726 641514266
< Age: 20
< Via: 1.1 varnish (Varnish/6.0)
< X-Cache: HIT from CS42
< Accept-Ranges: bytes

Response:

start line:状态码，具体解释

后面的也是组成一个headers，告诉浏览器怎么具体解析

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>慕课网-程序员的梦工厂</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge, chrome=1">
<meta name="renderer" content="webkit" />
<meta name="mobile-agent" content="format=wml"; url="https://m.imooc.com/">
<link rel="alternate" media="only screen and (max-width: 640px)" href="https://m.imooc.com/">
<meta name="mobile-agent" content="format=xhtml"; url="https://m.imooc.com/">
<meta name="mobile-agent" content="format=html5"; url="https://m.imooc.com/">
<meta property="qc:admins" content="77103107776157736375" />
<meta property="wb:webmaster" content="c4f857219bfae3cb" />
<meta http-equiv="Access-Control-Allow-Origin" content="*" />
<meta http-equiv="Cache-Control" content="no-transform " />
<meta http-equiv="Cache-Control" content="no-siteapp" />
<link rel="dns-prefetch" href="//www.imooc.com" />
<link rel="dns-prefetch" href="//img.imooc.com" />
<link rel="dns-prefetch" href="//img.mukewang.com" />
<link rel="apple-touch-icon" sizes="76x76" href="/static/img/common/touch-icon-ipad.png">
<link rel="apple-touch-icon" sizes="120x120" href="/static/img/common/touch-icon-iphone-retina.png">
<link rel="apple-touch-icon" sizes="152x152" href="/static/img/common/touch-icon-ipad-retina.png">
<meta name="Keywords" content="" />
<meta name="Description" content="慕课网（IMOOC）是IT技能学习平台。慕课网(IMOOC)提供了丰富的移动端开发、php开发、web前端、android开发以及html5等视频教程资源公开课。并且富有交互性及趣味性，你还可以和朋友一起编程。" />

Message Body

urllib

urllib和urllib2是相互独立的模块，在使用的时候需要两个结合使用

requests库使用了urllib3，urllib3则多了一些功能，比如多次请求重复使用一个socket，提升效率。

# _*_ coding: utf-8 _*_
import urllib2
import urllib

URL_IP = 'http://127.0.0.1:8000/ip'
URL_GET = 'http://127.0.0.1:8000/get'

def use_simple_urllib2():
    response = urllib2.urlopen(URL_IP)
    print '>>>>Response Headers:'
    print response.info()
    print '>>>>Response Body:'
    print ''.join([line for line in response.readlines()])

def use_params_urllib2():
    #构建请求参数
    params = urllib.urlencode({'param1':'hello', 'param2':'world'})
    print 'Request Params:'
    print params
    #发送请求
    response = urllib2.urlopen('?'.join([URL_GET, '%s'])%params)
    #处理响应
    print '>>>>Response Headers:'
    print response.info()
    print '>>>>Status Code:'
    print response.getcode()
    print '>>>>Response Body:'
    print ''.join([line for line in response.readlines()])

if __name__ == '__main__':
    print '>>>>Use simple urllib2:'
    use_simple_urllib2()
    print '>>>>Use params urllib2:'
    use_params_urllib2()

先启动gunicorn，然后运行上面的程序，

>>>>Use simple urllib2:
>>>>Response Headers:
Server: gunicorn/19.9.0
Date: Mon, 17 Sep 2018 03:18:04 GMT
Connection: close
Content-Type: application/json
Content-Length: 23
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

>>>>Response Body:
{"origin":"127.0.0.1"}

>>>>Use params urllib2:
Request Params:
param2=world&param1=hello
>>>>Response Headers:
Server: gunicorn/19.9.0
Date: Mon, 17 Sep 2018 03:18:04 GMT
Connection: close
Content-Type: application/json
Content-Length: 245
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

>>>>Status Code:
>>>>Response Body:
{"args":{"param1":"hello","param2":"world"},"headers":{"Accept-Encoding":"identity","Connection":"close","Host":"127.0.0.1:8000","User-Agent":"Python-urllib/2.7"},"origin":"127.0.0.1","url":"http://127.0.0.1:8000/get?param2=world&param1=hello"}

View result

可以得到上面的结果

urllib提供了一系列功能可以对URL进行操作，利用程序去执行各种http请求

简单使用requests

#-*- coding: utf-8 -*-
import requests

URL_IP = 'http://127.0.0.1:8000/get'

def use_simple_requests():
    response = requests.get(URL_IP)
    print '>>>>Response Headers:'
    print response.headers
    print '>>>>Response Body:'
    print response.text

def use_params_requests():
    params = {'param1':'hello', 'param2':'world'}
    #发送请求
    response = requests.get(URL_IP,params=params)
    #处理响应
    print '>>>>Response Headers:'
    print response.headers
    print '>>>>Status Code:'
    print response.status_code
    print '>>>>Response Body:'
    print response.json()

if __name__ == '__main__':
    print '>>>>Use simple requests:'
    use_simple_requests()

    print '>>>>Use params requests'
    use_params_requests()

View Code