【问题标题】:Python in AWS Lambda: multiprocessing, terminate other processes when one finishedAWS Lambda中的Python:多处理,一个完成时终止其他进程
【发布时间】:2021-01-18 05:44:11
【问题描述】:

我从this other question了解到AWS Lambda不支持multiprocessing.Pool和multiprocessing.Queue。

我还在 AWS Lambda 中研究 Python 多处理。但我的问题是:当第一个子进程返回时,我们如何终止主进程? (所有子进程将以不同的执行时间返回)

我这里有什么:

import time
from multiprocessing import Process, Pipe

class run_func():
    
    number = 0

    def __init__(self, number):
        self.number = number

    def subrun(self, input, conn):
         # subprocess function with different execution time based on input.
        response = subprocess(input)
        conn.send([input, response])
        conn.close()

    def run(self):
        number = self.number
        processes = []
        parent_connections = []
        for i in range(0, number):
            parent_conn, child_conn = Pipe()
            parent_connections.append(parent_conn)
            process = Process(target=self.subrun, args=(i, child_conn,))
            processes.append(process)

        for process in processes:
            process.start()
        for process in processes:
            process.join()

        results = []
        for parent_connection in parent_connections:
            resp = parent_connection.recv()
            print(resp)
            results.append((resp[0],resp[1]))
        return results

def lambda_handler(event, context):
    starttime = time.time()
    results = []
    work = run_func(int(event['number']))
    results = work.run()
    print("Results : {}".format(results))
    print('Time: {} seconds'.format(time.time() - starttime))
    return output

当前程序将返回,直到所有子进程完成(for parent_connection in parent_connections)。但我想知道如何在第一个子进程完成时终止? (至少终止主进程,其他子进程 - 可以让它继续运行)

添加: 明确地说,我的意思是第一个返回的子进程(可能不是第一个创建的子进程)。

【问题讨论】:

    标签: python amazon-web-services aws-lambda multiprocessing


    【解决方案1】:

    所以 join() 循环是等待所有子进程完成的循环。

    如果我们在完成第一个子进程后打破它并强制终止所有其他进程,它将为您工作

    class run_func():
        
        number = 0
    
        def __init__(self, number):
            self.number = number
    
        def subrun(self, input, conn):
             # subprocess function with different execution time based on input.
            response = subprocess(input)
            conn.send([input, response])
            conn.close()
    
        def run(self):
            number = self.number
            processes = []
            parent_connections = []
            for i in range(0, number):
                parent_conn, child_conn = Pipe()
                parent_connections.append(parent_conn)
                process = Process(target=self.subrun, args=(i, child_conn,))
                processes.append(process)
    
            for process in processes:
                process.start()
            for process in processes:
                process.join()
                break
            results = []
            for parent_connection in parent_connections:
                resp = parent_connection.recv()
                print(resp)
    

    【讨论】:

    • 也许我的解释会导致一些混乱,我的意思是第一个返回的进程,而不是第一个创建的进程。我该怎么办?
    猜你喜欢
    • 1970-01-01
    • 2021-09-05
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2018-11-27
    • 1970-01-01
    • 2015-05-12
    • 2015-11-10
    相关资源
    最近更新 更多