我在下面附加了一些代码,对您提供的代码的 sn-p 进行了一些基本优化。它大约快 1.4 倍,不是您正在寻找的数量级,但希望它能给您一些想法。
请注意,您提供的代码不起作用,因为您使用了 undefined discrete_A discrete_B 但我认为这些应该是您上面定义的 discreteA 和 discreteC。
有几个原则:
- 使用 numpy 时,如果您可以对数组执行向量操作,它通常比索引到数组并在 python 中执行计算要快。 numpy 是一个 C 库,库内的计算将受益于编译时优化。对数组执行操作也受益于处理器预测缓存(您的 CPU 期望您使用相邻的内存向前并预加载它;如果您零散地访问数据,您可能会失去这种好处)。
- 通过缓存中间结果,避免在不需要时多次执行操作。一个很好的例子是矩阵转置,当您只需要执行一次相同的操作时,您执行了 20 次。
听起来代码中可能有一些范围调用 this 来应用这些原则。如果您经常调用此代码,则可能值得在更高级别进行缓存,并可能查看具有更大数据块的缓存操作。我不确定您的 product 方法是什么,所以我无法真正发表评论。
除此之外,熟悉 python see here 中的分析器是非常值得的。探查器是了解代码中花费时间的关键。
以下代码中的改进步骤是(以秒为单位的时间):
v1: 9.801263800123706
v2: 8.354220200097188
v3: 7.2868248000741005
v4: 7.5897450998891145
v5: 6.721231299918145
代码更改:
import timeit
import numpy as np
q = np.array([0]*10 + [1] + [0]*10)
W = np.array([[0, 1], [1, 0]])
discreteA = {'Prior': 0.6153846153846154,
'Prob': np.array([0.0125, 0., 0., 0.0125, 0.025, 0.0125, 0.025, 0.0375, 0.075, 0.1, 0.2125, 0.1375, 0.15, 0.1, 0.0875, 0.0125, 0., 0., 0., 0., 0.])}
discreteC = {'Prior': 0.38461538461538464,
'Prob': np.array([0., 0., 0., 0.02, 0.02, 0.22, 0.46, 0.16, 0.1, 0.02, 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
def v1():
return float(np.sum([np.dot(W.T[x,:], (discreteA['Prob'][i]*discreteA['Prior'], discreteC['Prob'][i]*discreteC['Prior'])) for i,x in enumerate(q)]))
# instead of doing the Prob * Prior calculation one at a time on each loop, do it as a vector operation
# first, then index into the resultant vector. The vector multiply is faster as it happens in C without
# having to come back to python
def v2():
prodA = discreteA['Prob'] * discreteA['Prior']
prodC = discreteC['Prob'] * discreteC['Prior']
return float(np.sum([np.dot(W.T[x,:], (prodA[i], prodC[i])) for i,x in enumerate(q)]))
# there are only two possible transposed matrices so don't recalulate every time
def v3():
prodA = discreteA['Prob'] * discreteA['Prior']
prodC = discreteC['Prob'] * discreteC['Prior']
trans = (W.T[0,:], W.T[1,:])
return float(np.sum([np.dot(trans[x], (prodA[i], prodC[i])) for i,x in enumerate(q)]))
# there's no need to enumerate, you can just index directly into q
def v4():
prodA = discreteA['Prob'] * discreteA['Prior']
prodC = discreteC['Prob'] * discreteC['Prior']
trans = (W.T[0,:], W.T[1,:])
return np.sum([np.dot(trans[q[i]], (prodA[i], prodC[i])) for i in range(len(q))])
# sum from a generator rather than creating a list. note np.sum(generator) is depreciated and np.sum(np.fromiter(generator)) or built-in sum is preferred
# note this changes result from 0.5153846153846154 to 0.5153846153846153 due to differences in python and numpy sum
def v5():
prodA = discreteA['Prob'] * discreteA['Prior']
prodC = discreteC['Prob'] * discreteC['Prior']
trans = (W.T[0,:], W.T[1,:])
return sum((np.dot(trans[q[i]], (prodA[i], prodC[i])) for i in range(len(q))))
if (res := v1()) == v2() and res == v3() and res == v4() and abs(res - v5()) < 0.000000000000001:
print(f'Results match.')
print(f'v1: {timeit.timeit(v1, number=100000)}')
print(f'v2: {timeit.timeit(v2, number=100000)}')
print(f'v3: {timeit.timeit(v3, number=100000)}')
print(f'v4: {timeit.timeit(v4, number=100000)}')
print(f'v5: {timeit.timeit(v5, number=100000)}')