api请求的初学者异步/等待问题

Question

我想加快一些 API 请求的速度...为此，我试图弄清楚如何去做并复制一些运行的代码，但是当我尝试自己的代码时，它不再是异步的。也许有人发现失败了？

复制代码（从stackoverflow猜测）：

#!/usr/bin/env python3
import asyncio

@asyncio.coroutine
def func_normal():
    print('A')
    yield from asyncio.sleep(5)
    print('B')
    return 'saad'

@asyncio.coroutine
def func_infinite():
    for i in range(10):
        print("--%d" % i)
    return 'saad2'

loop = asyncio.get_event_loop()
tasks = func_normal(), func_infinite()
a, b = loop.run_until_complete(asyncio.gather(*tasks))
print("func_normal()={a}, func_infinite()={b}".format(**vars()))
loop.close()

我的“自己的”代码（我需要在最后返回一个列表并合并所有函数的结果）：

import asyncio
import time


@asyncio.coroutine
def say_after(start,count,say,yep=True):
    retl = []
    if yep:
        time.sleep(5)
    for x in range(start,count):
        retl.append(x)
        print(say)
    return retl


def main():
    print(f"started at {time.strftime('%X')}")
        
    loop = asyncio.get_event_loop()
    tasks = say_after(10,20,"a"), say_after(20,30,"b",False)
    a, b = loop.run_until_complete(asyncio.gather(*tasks))
    print("func_normal()={a}, func_infinite()={b}".format(**vars()))
    loop.close()
    c =  a + b

    #print(c)
    print(f"finished at {time.strftime('%X')}")

main()

或者我完全错了，应该用多线程解决这个问题？对于返回我需要合并的列表的 API 请求，最好的方法是什么？

Answer 1

为每个需要改进的部分添加了评论。删除了一些简单的代码。

事实上，我没有发现使用包裹在协程中的 range() 和使用 async def 有任何性能提升，可能值得进行更繁重的操作。

import asyncio
import time

# @asyncio.coroutine IS DEPRECATED since python 3.8
@asyncio.coroutine
def say_after(wait=True):
    result = []

    if wait:
        print("I'm sleeping!")
        time.sleep(5)
        print("'morning!")
        # This BLOCKs thread, but release GIL so other thread can run.
        # But asyncio runs in ONE thread, so this still harms simultaneity.

    # normal for is BLOCKING operation.
    for i in range(5):
        result.append(i)
        print(i, end='')
    print()

    return result


def main():
    start = time.time()

    # Loop argument will be DEPRECATED from python 3.10
    # Make main() as coroutine, then use asyncio.run(main()).
    # It will be in asyncio Event loop, without explicitly passing Loop.
    loop = asyncio.get_event_loop()
    tasks = say_after(), say_after(False)

    # As we will use asyncio.run(main()) from now on, this should be await-ed.
    a, b = loop.run_until_complete(asyncio.gather(*tasks))

    print(f"Took {time.time() - start:5f}")

    loop.close()


main()

更好的方法：

import asyncio
import time


async def say_after(wait=True):
    result = []

    if wait:
        print("I'm sleeping!")
        await asyncio.sleep(2)  # 'await' a coroutine version of it instead.
        print("'morning!")

    # wrap iterator in generator - or coroutine
    async def asynchronous_range(end):
        for _i in range(end):
            yield _i

    # use it with async for
    async for i in asynchronous_range(5):
        result.append(i)
        print(i, end='')
    print()

    return result


async def main():
    start = time.time()

    tasks = say_after(), say_after(False)
    a, b = await asyncio.gather(*tasks)

    print(f"Took {time.time() - start:5f}")


asyncio.run(main())

---

结果

您的代码：

DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
  def say_after(wait=True):
I'm sleeping!
'morning!
01234
01234
Took 5.003802

更好的异步代码：

I'm sleeping!
01234
'morning!
01234
Took 2.013863

请注意，固定代码现在在其他任务处于休眠状态时完成其工作。