【发布时间】:2020-02-14 12:21:02
【问题描述】:
我有一个简短的 sn-p 呼叫 pool.map 挂断。它不会在任何合理的时间内返回。这个想法是有几个点列表并找到与另一个点列表对应的度量。
import numpy as np
import multiprocessing
import scipy.spatial as ss
def dist(args): # returns the sum of distances of the nearest neighbors.
kdSet, points = args
dist, _ = kdSet.query(points)
return np.sum(dist)
#just define some points to play with
points = np.array([[13.27, 25.49], [13.18, 25.39], [13.08, 25.39],
[12.99, 25.39], [12.89, 25.39], [12.80, 25.39],
[12.71, 25.39], [12.61, 25.30], [12.52, 25.30]])
pointList = [points + idx for idx in range(3)] #creates a list of points
#create a list of KDTrees for fast lookup. Add some number to make them a little different from pointList
kdTreeList = [ss.KDTree(pointsLocal + idx/10) for idx, pointsLocal in enumerate(pointList)]
#this part works fine: serial execution
distances = list(map(dist, zip(kdTreeList, pointList)))
print(distances)
#this part hangs, if called as multiprocess, expected the same result as above
with multiprocessing.Pool() as pool:
print('Calling pool.map')
distancesMultiprocess = pool.map(dist, zip(kdTreeList, pointList)) #<-This call does not return, but uses the CPU 100%
print(distancesMultiprocess)
按顺序调用函数可以正常工作。但是如果在 pool.map 函数中调用它既不会给出错误也不会返回任何东西。它只是挂在那里。 CPU 负载达到 100%,所以发生了一些事情。
出于测试目的删除 KDTree 类并没有显示不同的结果。
由于没有抛出错误,我不知道这里出了什么问题。有什么提示吗?
【问题讨论】:
-
你的代码对我来说很好,使用 python-3.5。我记得较旧的 python (with ... as pool 语法上有问题。这可能是一个线索吗?
-
你在windows上吗?你需要在 windows 上使用
if __name__ == "__main__"守卫,否则你会创建一个多处理炸弹 -
你也可以试试
pool.join() ; pool.close()。例如阅读stackoverflow.com/questions/20387510/… 或stackoverflow.com/questions/38271547/…
标签: python numpy multiprocessing python-multiprocessing