python---基础知识回顾(十)进程和线程(py2中自定义线程池和py3中的线程池使用)

python---基础知识回顾(十)进程和线程(协程gevent:线程在I/O请求上的优化)

python---异步IO(asyncio)协程

二:异步I/O模块的了解(同协程:线程在I/O请求上的优化)

(一)asyncio模块的使用

1.使用简单实例

import asyncio,time

async def func1(num):
    print(num,"before func1.....")
    await asyncio.sleep(num)
    print(num,"after func1......")

tasks = [
    asyncio.ensure_future(func1(1)),
    asyncio.ensure_future(func1(3)),
]

begin = time.time()

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(*tasks))

loop.close()

end = time.time()

print(end-begin)
1 before func1.....
3 before func1.....
1 after func1......
3 after func1......
3.003171682357788

(二).异步获取网站资源(不支持requests模块),使用asyncio模块的socket请求完成

不支持requests请求的原因:

异步的实现,需要自动切换协程,需要用到await或者yield,
而requests模块不提供切换协程的条件,所以当第一条协程执行到请求部分就停下来了,等服务器响应,
不会切到其他的协程去执行。
requests会阻塞asyncio循环,所以会出现收不到服务器响应的情况。可以使用aiohttp替代requests解决这个问题。

 

使用asyncio实现requests请求获取资源

import asyncio,time

async def func1(host,url='/'):
    print(host,url)
    reader, writer = await asyncio.open_connection(host,80)
    request_header_content = '''GET %s HTTP/1.0\r\nHost: %s\r\n\r\n'''%(url,host)
    request_header_content = bytes(request_header_content,encoding="utf-8")
    writer.write(request_header_content)
    await writer.drain()
    text = await reader.read()
    print(host,url,text)
    writer.close()

tasks = [
    asyncio.ensure_future(func1('www.cnblogs.com','/ssyfj/')),
    asyncio.ensure_future(func1('www.baidu.com')),
    asyncio.ensure_future(func1('www.ckook.com','/forum.php')),
]

begin = time.time()

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(*tasks))

loop.close()

end = time.time()

print(end-begin)
www.cnblogs.com /ssyfj/
www.baidu.com /
www.ckook.com /forum.php
www.baidu.com / b'HTTP/1.0 200 OK\r\nAccept-Ranges: bytes\r\nCache-Control: no-cache\r\nContent-Length: 14615\r\nContent-Type: text/html\r\nDate: Sun, 24 Jun 2018 10:57:33 GMT\r\nLast-Modified: Mon, 11 Jun 2018 11:19:00 GMT\r\nP3p: CP=" OTI DSP COR IVA OUR IND COM "\r\nPragma: no-cache\r\nServer: BWS/1.1\r\nSet-Cookie: BAIDUID=3C070A482979333DD407E21A39ECA46A:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com\r\nSet-Cookie: BIDUPSID=3C070A482979333DD407E21A39ECA46A; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com\r\nSet-Cookie: PSTM=1529837853; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com\r\nVary: Accept-Encoding\r\nX-Ua-Compatible: IE=Edge,chrome=1\r\n\r\n<!DOCTYPE html><!--STATUS OK-->\r\n<html>\r\n<head>\r\n\t<meta http-equiv="content-type" content="text/html;charset=utf-8">\r\n\t<meta http-equiv="X-UA-Compatible" content="IE=Edge">\r\n\t<link rel="dns-prefetch" href="//s1.bdstatic.com"/>\r\n\t<link rel="dns-prefetch" href="//t1.baidu.com"/>\r\n\t<link rel="dns-prefetch" href="//t2.baidu.com"/>\r\n\t<link rel="dns-prefetch" href="//t3.baidu.com"/>\r\n\t<link rel="dns-prefetch" href="//t10.baidu.com"/>\r\n\t<link rel="dns-prefetch" href="//t11.baidu.com"/>\r\n\t<link rel="dns-prefetch" href="//t12.baidu.com"/>\r\n\t<link rel="dns-prefetch" href="//b1.bdstatic.com"/>\r\n\t<title>\xe7\x99\xbe\xe5\xba\xa6\xe4\xb8\x80\xe4\xb8\x8b\xef\xbc\x8c\xe4\xbd\xa0\xe5\xb0\xb1\xe7\x9f\xa5\xe9\x81\x93</title>\r\n\t<link href="http://s1.bdstatic.com/r/www/cache/static/home/css/index.css" rel="stylesheet" type="text/css" />\r\n\t<!--[if lte IE 8]><style index="index" >#content{height:480px\\9}#m{top:260px\\9}</style><![endif]-->\r\n\t<!--[if IE 8]><style index="index" >#u1 a.mnav,#u1 a.mnav:visited{font-family:simsun}</style><![endif]-->\r\n\t<script>var hashMatch = document.location.href.match(/#+(.*wd=[^&].+)/);if (hashMatch && hashMatch[0] && hashMatch[1]) {document.location.replace("http://"+location.host+"/s?"+hashMatch[1]);}var ns_c = function(){};</script>\r\n\t<script>function h(obj){obj.style.behavior=\'url(#default#homepage)\';var a = obj.setHomePage(\'//www.baidu.com/\');}</script>\r\n\t<noscript><meta http-equiv="refresh" content="0; url=/baidu.html?from=noscript"/></noscript>\r\n\t<script>window._ASYNC_START=new Date().getTime();</script>\r\n</head>\r\n<body link="#0000cc"><div >
www.cnblogs.com /ssyfj/ b'HTTP/1.1 200 OK\r\nDate: Sun, 24 Jun 2018 10:57:33 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 14922\r\nConnection: close\r\nVary: Accept-Encoding\r\nCache-Control: private, max-age=10\r\nExpires: Sun, 24 Jun 2018 10:57:43 GMT\r\nLast-Modified: Sun, 24 Jun 2018 10:57:33 GMT\r\nX-UA-Compatible: IE=10\r\n\r\n\r\n<!DOCTYPE html>\r\n<html lang="zh-cn">\r\n<head>\r\n<meta charset="utf-8"/>\r\n<meta name="viewport" content="width=device-width, initial-scale=1" />\r\n<title>\xe5\xb1\xb1\xe4\xb8\x8a\xe6\x9c\x89\xe9\xa3\x8e\xe6\x99\xaf - \xe5\x8d\x9a\xe5\xae\xa2\xe5\x9b\xad</title>\r\n<link type="text/css" rel="stylesheet" href="/bundles/blog-common.css?v=-hy83QNg62d4qYibixJzxMJkbf1P9fTBlqv7SK5zVL01"/>\n<link >
www.ckook.com /forum.php b'HTTP/1.1 301 Moved Permanently\r\nDate: Sun, 24 Jun 2018 10:57:33 GMT\r\nServer: Apache/2.4.23 (Win32) OpenSSL/1.0.2j PHP/5.4.45\r\nX-Powered-By: PHP/5.4.45\r\nLocation: https://www.ckook.com/forum.php\r\nContent-Length: 133\r\nContent-Type: text/html\r\n\r\n<br />\n<b>Notice</b>:  Undefined index: HTTP_USER_AGENT in <b>C:\\www\\web\\www\\source\\class\\class_core.php</b> on line <b>19</b><br />\n'
5.589319705963135
输出

相关文章:

  • 2021-07-16
  • 2021-10-26
  • 2021-10-06
  • 2021-06-24
  • 2022-02-01
  • 2021-11-15
  • 2021-12-11
  • 2021-07-31
猜你喜欢
  • 2021-07-30
  • 2022-12-23
  • 2021-06-09
  • 2021-11-08
  • 2021-07-23
  • 2021-07-18
  • 2021-12-03
相关资源
相似解决方案