【问题标题】:Python async process getting executed in different ordersPython异步进程以不同的顺序执行
【发布时间】:2021-08-28 14:36:47
【问题描述】:

我有一个异步函数,它调用一个异步子进程,稍后在执行时,同一个函数执行文件读取。子进程执行后,应该会生成文件,但是当我出于某种原因运行该函数时,文件读取操作在调用子进程之前执行,这会导致错误提示No file named X

async def job():
     # creates and calls my CPU intensive command which should take some
     # time to perform
     cmd = "my time consuming command"
     await run_shell_process(cmd)
     
     # opening a file that would be generated as an output
     # to the command, but returning a file not found 
     # I checked and made sure that the file IS getting
     # getting generated, but the order of the process is
     # causing the issue
     with open(my_file, 'r') as file:
         file.read()


async def run_command_shell(command):
    process = await asyncio.create_subprocess_shell(
        command, stderr=asyncio.subprocess.STDPIPE
    )

    # Wait for the subprocess to finish
    stdout, stderr = await process.communicate()

    # Result
    result = stdout.decode().strip()

    # Return stdout
    return result

有什么方法可以确保我的异步子进程在运行以下文件打开操作之前完全执行

注意:该作业还使用app.add_task(job()) 作为Sanic 中的后台任务运行。我尝试使用多进程回调方法在未来执行后调用回调,但是因为它已经作为子进程运行,所以它抛出了一个错误,说不能为已经运行的子进程生成子进程。另外,我正在运行的命令是一个非常占用 CPU 的命令,这可能会导致重新调度线程并且它们以不同的顺序执行

【问题讨论】:

  • 你检查过stdout是否有错误吗?
  • @dirn 没有,并且该命令按预期工作。它还生成必要的输出和文件...

标签: python python-3.x concurrency python-asyncio sanic


【解决方案1】:

有一系列工具https://docs.python.org/3/library/asyncio-sync.html,其中 Event 看起来很合适。

import asyncio

async def slow_must_finish_first(wev, rev):
    global file
    await wev.wait()
    print('slow writer writing')
    await asyncio.sleep(1.0)
    print('slow writer finished')
    rev.set()

async def fast_must_finish_last(rev):
    await rev.wait()
    print('fast reader reading')

async def main():
    write_event = asyncio.Event()
    read_event = asyncio.Event()
    tasks = [
        asyncio.create_task(fast_must_finish_last(read_event)),
        asyncio.create_task(slow_must_finish_first(write_event, read_event))
    ]
    write_event.set()
    await asyncio.gather(*tasks)

if __name__ == '__main__':
    asyncio.run(main())

输出:

slow writer writing
slow writer finished
fast reader reading

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2013-03-15
    • 2015-09-28
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多