使用带有地图的 ThreadPoolExecutor 获取错误答案

【问题标题】：get error using ThreadPoolExecutor with map使用带有地图的 ThreadPoolExecutor 获取错误
【发布时间】：2021-09-17 13:11:49
【问题描述】：

我注意到当 ThreadPoolExecutor() 调用 function(x) 内部出现错误时，我的代码不会输出错误或中断：

import concurrent.futures

def function(x):
   # do sth invalid such as
   x = y + 1 #there is no y variable

input_list = [1,2,3]
with concurrent.futures.ThreadPoolExecutor() as executor: 
    executor.map(function,input_list)

如何使用 map() 而不是 submit() 获取执行程序的错误输出？

【问题讨论】：

标签： python dictionary concurrency multiprocessing threadpoolexecutor

【解决方案1】：

executer.map 返回一个 iterable，必须对其进行迭代才能获得单独的结果，包括可能引发的任何异常：

import concurrent.futures

def function(x):
    # do sth invalid such as
    if x == 2:
        raise ValueError("I don't like 2")
    return x, x ** 2

input_list = [1,2,3]
with concurrent.futures.ThreadPoolExecutor(3) as executor:
    results = executor.map(function,input_list)
    try:
        for x, return_value in results:
            print(f'{x} ** 2 = {return_value}')
    except Exception as e:
        print(e)

打印：

1 ** 2 = 1
I don't like 2

或者您可以稍微不同地迭代并获得相同的结果：

import concurrent.futures

def function(x):
    # do sth invalid such as
    if x == 2:
        raise ValueError("I don't like 2")
    return x, x ** 2

input_list = [1,2,3]
with concurrent.futures.ThreadPoolExecutor(3) as executor:
    results = executor.map(function,input_list)
    while True:
        try:
            x, return_value = next(results)
        except StopIteration:
            break
        except Exception as e:
            print(e)
        else:
            print(f'{x} ** 2 = {return_value}')

但是，如果您使用包 multiprocessing.pool 中的类 ThreadPool，那么您可以获得所有结果，即获得超出可能引发的任何异常的结果（使用 imap ):

from multiprocessing.pool import ThreadPool

def function(x):
    # do sth invalid such as
    if x == 2:
        raise ValueError("I don't like 2")
    return x, x ** 2

input_list = [1,2,3]
with ThreadPool(3) as executor:
    results = executor.imap(function,input_list)
    while True:
        try:
            x, return_value = next(results)
        except StopIteration:
            break
        except Exception as e:
            print(e)
        else:
            print(f'{x} ** 2 = {return_value}')

打印：

1 ** 2 = 1
I don't like 2
3 ** 2 = 9

【讨论】：