为什么 numpy 向量化函数不适用于每个元素答案

【问题标题】：Why numpy vectorised function does not apply to each element为什么 numpy 向量化函数不适用于每个元素
【发布时间】：2020-03-29 22:55:21
【问题描述】：

我正在尝试将矢量化函数应用于一维 NumPy 数组（测试）。

如果某个元素大于某个阈值，则应用该函数，否则返回 0。

在整个数组上传递函数会得到一个零数组（结果 #1）

但这适用于大数组（结果 #2）和小数组（结果 #3）的某些切片

你能帮我理解这是为什么吗？

import numpy as np
test = np.array([-58.08281  , -47.07844  , -39.38589  , -38.244213 , -36.04118  ,
       -35.17719  , -47.651756 , -47.123497 , -38.47037  , -31.427711 ,
       -35.980206 , -39.04678  , -43.247276 , -29.217781 , -26.16616  ,
       -23.175611 , -19.073223 , -19.573145 , -19.291908 , -19.084608 ,
       -24.24286  , -26.768343 , -29.40547  , -42.254036 , -32.5126   ,
       -27.8232   , -26.521381 , -18.53816  , -16.300032 , -14.897881 ,
       -11.96727  , -11.895884 , -11.958228 , -11.689035 , -19.331993 ,
       -22.528988 , -14.850136 , -10.7898   , -10.738896 , -11.510415 ,
       -11.297523 , -14.9558525, -18.261246 , -20.11386  , -35.434853 ,
       -36.547577 , -29.713285 , -35.055378 , -19.717499 , -15.524372 ,
       -14.905738 , -11.690297 , -12.295127 , -14.571337 , -14.457521 ,
       -20.896961 , -35.145    , -39.106945 , -20.592056 , -19.292147 ,
       -21.957949 , -20.131369 , -31.953508 , -24.577961 , -23.88112  ,
       -16.549093 , -16.742077 , -22.181223 , -21.692726 , -34.572075 ,
       -20.111103 , -18.57012  , -12.833547 , -11.325545 , -12.807129 ,
       -11.844269 , -19.830124 , -21.79983  , -18.484238 , -12.855567 ,
       -11.830711 , -14.83697  , -14.618052 , -19.990686 , -30.934792 ,
       -27.72318  , -17.222315 , -14.099125 , -16.516563 , -15.129327 ,
       -19.21385  , -41.145554 , -37.12835  , -20.674335 , -17.670841 ,
       -26.641182 , -26.721628 , -29.708376 , -16.29707  , -15.220005 ,
       -11.475418 ,  35.859955 , -10.404102 ,  35.160667 , -11.339685 ,
       -17.627815 , -18.65314  , -25.346134 , -38.297813 , -22.460407 ,
       -21.334377 , -16.922516 , -10.733174 ,  35.263527 ,  35.078003 ,
        35.26928  ,  35.44266  ,  35.89205  , -10.965962 , -16.772722 ,
       -10.638295 ,  35.37294  ,  35.32364  ,  35.271263 ,  35.900078 ,
        35.145794 , -12.282233 , -14.206524 , -18.138363 , -37.339016 ,
       -26.27323  , -27.531588 , -25.00942  , -13.963585 , -12.315678 ,
       -10.978365 ,  35.439877 , -10.534686 , -11.77856  , -12.630129 ,
       -22.29188  , -32.74709  , -29.052572 , -16.526686 , -18.223225 ,
       -19.174236 , -18.920668 , -34.266537 , -23.23388  , -19.992903 ,
       -13.9729805, -16.85691  , -20.88271  , -21.805904 , -24.517344 ,
       -17.412155 , -15.050234 ,  35.047886 , -10.27907  , -10.765995 ,
       -11.394721 , -34.574    , -18.185272 , -15.156159 , -10.370025 ,
       -11.406872 , -13.781429 , -13.863158 , -24.35263  , -29.509377 ,
       -24.758411 , -14.150916 , -13.686075 , -15.366934 , -14.149103 ,
       -22.916718 , -35.810047 , -33.369896 , -17.931974 , -18.65556  ,
       -28.330248 , -27.015589 , -23.890095 , -15.020579 , -13.920487 ,
        35.49385  ,  35.613037 ,  35.326546 ,  35.1469   , -12.024554 ,
       -17.770742 , -18.414755 , -31.574192 , -35.00205  , -20.591629 ,
       -21.097118 , -14.166552 ,  35.61772  ,  35.196175 ,  35.884003 ,
        35.032402 ,  35.289963 ,  35.18595  , -36.364285 , -10.158181 ,
        35.040634 ,  35.349873 ,  35.31796  ,  35.87602  ,  35.88828  ,
        35.086105 , -12.404961 , -13.550255 , -20.19417  , -35.630135 ,
       -23.762396 , -27.673418 , -19.928736 , -12.206515 , -11.781338 ,
        35.307823 ,  35.67385  , -10.780588 , -11.199528 , -13.561855 ,
       -24.982666 , -30.838753 , -25.138466 , -16.61114  , -20.002995 ,
       -18.823566 , -21.581133 , -25.644733 , -22.914455 , -17.489904 ,
       -13.714966 , -18.483316 , -20.454823 , -25.238888 , -20.592503 ,
       -17.511456 , -13.5111885,  35.399975 , -10.711888 , -10.577221 ,
       -13.2071705, -27.878649 , -16.227467 , -13.394671 ,  35.33075  ,
       -10.933496 , -12.903596 , -13.261947 , -23.191689 , -36.082005 ,
       -26.252464 , -14.935854 , -14.955426 , -16.291502 , -15.563564 ,
       -27.91648  , -30.43707  , -27.09887  , -16.93166  , -19.03229  ,
       -26.68034  , -26.50705  , -22.435007 , -15.312309 , -13.67744  ,
        35.70387  ,  35.197517 ,  35.21866  ,  35.759956 , -12.934032 ,
       -18.348143 , -19.073929 , -36.864773 , -32.881073 , -20.560263 ,
       -20.530846 , -13.128365 ,  35.65545  ,  35.465275 ,  35.028538 ,
        35.842434 ,  35.676643 , -17.01441  , -17.217728 ,  35.667717 ,
        35.871662 ,  35.92965  ,  35.316013 ,  35.096027 ,  35.02661  ,
        35.988937 , -12.0597515, -13.201061 , -20.259245 , -28.855875 ,
       -21.791933 , -25.400242 , -17.618946 , -11.611944 , -11.329423 ,
        35.063614 ,  35.825493 , -10.553531 , -10.820301 , -13.883024 ,
       -22.231556 , -26.921532 , -31.872276 , -18.039211 , -19.713062 ,
       -20.517511 , -21.620483 , -26.919012 , -20.787134 , -17.330051 ,
       -13.198881 , -15.984946 , -19.181019 , -21.50328  , -25.311642 ,
       -18.11811  , -14.696231 , -10.136784 , -10.480961 , -11.110486 ,
       -13.739718 , -28.865023 , -15.966995 , -13.198223 ,  35.18759  ,
       -10.803377 , -12.718526 , -13.597855 , -23.472122 , -34.405643 ,
       -24.122065 , -14.643904 , -14.425363 , -15.651573 , -15.197855 ,
       -25.13602  , -33.207695 , -26.908777 , -17.217882 , -19.061764 ,
       -27.06517  , -28.88142  , -21.721449 , -14.84623  , -12.997027 ,
        35.853565 ,  35.51484  ,  35.660423 ,  35.982292 , -12.461762 ,
       -17.52755  , -19.008127 , -32.69878  , -30.82928  , -20.193447 ,
       -19.172876 , -12.901536 ,  35.05082  ,  35.915546 ,  35.254303 ,
        35.797028 , -14.470562 , -22.461277 , -15.07134  ,  35.970448 ,
        35.198704 ,  35.945583 ,  35.362762 ,  35.306732 ,  35.064957 ,
        35.10975  , -11.703257 , -13.411005 , -20.08778  , -28.905445 ,
       -22.59493  , -25.155657 , -17.814808 , -11.842859 , -11.154184 ,
        35.989094 ,  35.854362 , -10.2389765, -10.827884 , -14.010275 ,
       -25.168896 , -33.99552  , -22.858255 , -16.562387 , -19.22073  ,
       -18.317003 , -23.036928 , -25.22068  , -21.934307 , -16.469448 ,
       -13.88927  , -18.307293 , -20.485218 , -29.06332  , -19.628113 ,
       -16.496414 , -12.351503 ,  35.66623  , -10.330103 , -10.866837 ,
       -16.813847 , -21.454565 , -15.892494 , -12.269305 ,  35.174488 ,
       -11.898882 , -13.1494465, -15.517578 , -35.11971  , -29.069548 ,
       -19.153015 , -13.194953 , -14.334308 , -14.483275 , -15.592762 ,
       -30.123589 , -38.262245 , -24.752253 , -17.36696  , -22.627728 ,
       -29.787828 , -44.489254 , -17.438164 , -13.678364 , -11.26264  ,
        35.92086  ,  35.600876 ,  35.231567 ,  35.960655 , -13.438512 ,
       -16.794493 , -19.414097 , -33.008324 , -23.844492 , -18.63253  ,
       -17.060545 , -10.566847 ,  35.735447 ,  35.061024 ,  35.95225  ,
       -11.117262 , -18.978222 , -39.73106  , -11.048341 ,  35.58616  ,
        35.699783 ,  35.32885  ,  35.09172  ,  35.119743 ,  35.753242 ,
        35.73512  , -12.641587 , -14.861554 , -25.59355  , -29.808552 ,
       -24.463276 , -26.617489 , -15.665792 , -11.706967 , -11.054789 ,
        35.413254 ,  35.13033  , -10.968152 , -11.514641 , -17.074472 ,
       -31.623056 , -40.51703  , -18.116985 , -15.995826 , -18.33452  ,
       -17.266975 , -28.274193 , -24.104795 , -21.711021 , -15.209898 ,
       -15.003292 , -20.39471  , -21.562126 , -34.197975 , -16.957975 ,
       -14.923981 , -10.418877 ,  35.874657 , -10.214642 , -11.880876 ])

test2 = np.array([5,5,5])

Thresh = -10
Ratio = 5

#Defining function
def gr(x):
    if x >=Thresh:
        return Thresh + (x-Thresh)/Ratio
    else:
        return 0
#vectorising function
gr_v = np.vectorize(gr)

#RESULTS
##1
print(sum(gr_v(test))) #0
##2
print(sum(gr_v(test2)))
##3
print(sum(gr_v(test[200:250])))

【问题讨论】：

这很有趣。这是类型推断。真是个难题。请参阅下面的答案。
我知道vectorize 文档很长而且很无聊，但是阅读它是值得的。最近多次出现这个错误，

标签： python numpy

【解决方案1】：

顺便说一句，知道np.vectorize“提供函数主要是为了方便，而不是为了性能。实现本质上是一个for循环。”

因此与以下相同：

l = [gr(x) for x in test]
sum(l)
-80.41708859999999

要使用矢量化实现，您可以这样做：

np.sum(Thresh + (test[test>=Thresh]-Thresh)/Ratio)
-80.41708859999999

就时间表现而言：

%timeit np.sum(Thresh + (test[test>=Thresh]-Thresh)/Ratio)
13.8 µs ± 864 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

%timeit sum(gr_v(test))
217 µs ± 5.32 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

使用此test 大小，实际矢量化实现速度提高了 15 倍。

【讨论】：

【解决方案2】：

这与矢量化的固有属性有关。如果向量化函数的第一个输出是整数 0，那么整个输出都是关闭的。虽然vectorise 的这种行为有点让人记忆深刻，但我可以提供一个相当简单的修复方法。而不是使用

def gr(x):
    if x >=Thresh:
        return Thresh + (x-Thresh)/Ratio
    else:
         return 0

0
-21.0
-12.044083

将函数改写为：

def gr(x):
    if x >=Thresh:
         return Thresh + (x-Thresh)/Ratio
    else:
        return 0.0

它返回：

-80.41708859999999
-21.0
-12.044083

返回0.0 而不是0 的目的是输出一个float(0) 而不是int(0)。它还解释了为什么您的代码适用于 test2 和 test[200:250] 数组，因为它们的第一个元素都是正数，因此 gr_v 返回的是浮点数，而不是 int(0)。希望对您有所帮助！

【讨论】：

你可以只做0.0或者在定义向量化函数时指定输出类型：gr_v = np.vectorize(gr, otypes=[float])

【解决方案3】：

这与类型推断有关：

试试这个：

gr_v = np.vectorize(gr, otypes=[np.float])

这修复了它（注意 .0）：

Thresh = -10.0
Ratio = 5.0

#Defining function
def gr(x):
    if x >=Thresh:
        return Thresh + (x-Thresh)/Ratio
    else:
        return 0.0
#vectorising function
gr_v = np.vectorize(gr)

【讨论】：