python爬虫之request and BeautifulSoup

1.爬虫的本质是什么?

模仿浏览器的行为,爬取网页信息。

 

2.requests

1.get请求

无参数实例
import requests
ret = requests.get('https://github.com/timeline.json')
print ret.text
  
  
有参数实例
import requests
ret = requests.get("http://httpbin.org/get", params= {'key1': 'value1', 'key2': 'value2'})

print ret.text

get