sklearn 成对距离结果意外不对称答案

【问题标题】：sklearn's pairwise distance result is unexpectedly asymmetricalsklearn 成对距离结果意外不对称
【发布时间】：2017-05-31 08:38:07
【问题描述】：

我正在计算向量元素之间的欧几里得成对距离。我使用 sklearn 包中的 pairwise_distances 函数。然而，某些元素的结果矩阵只是近似对称的：在一个示例中，应该相等的元素的值仅等于小数点后 15 位。

当我在假设输入矩阵对称的下游分析中遇到错误时，我意识到了这一点。我知道我可以四舍五入，但这是什么原因造成的？！

这是我试图计算成对距离的向量（它是熊猫数据框的一列）：

lag_measure_data[['bios_level']].values

array([[ 0.76881030949999995538490793478558771312236785888671875 ],
   [ 0.                                                      ],
   [ 0.67783090619999997183953155399649403989315032958984375 ],
   [ 0.3228176074999999922710003374959342181682586669921875  ],
   [ 0.75822395549999999087020796650904230773448944091796875 ],
   [ 0.469808621599999975959605080788605846464633941650390625],
   [ 0.989529862699999984698706612107343971729278564453125   ],
   [ 0.                                                      ],
   [ 0.5575436799999999859522858969285152852535247802734375  ],
   [ 0.9756440299999999954394525047973729670047760009765625  ],
   [ 0.66511863289999995085821637985645793378353118896484375 ],
   [ 0.978062709200000046649847718072123825550079345703125   ],
   [ 0.473957179800000016900440868994337506592273712158203125],
   [ 0.82409385540000001935112550199846737086772918701171875 ],
   [ 0.56548685279999999497846374651999212801456451416015625 ],
   [ 0.399505730399999980928527065771049819886684417724609375],
   [ 0.474232963900000026313819034839980304241180419921875   ],
   [ 0.34276307189999999369689476225175894796848297119140625 ],
   [ 0.9985316859999999739017084721126593649387359619140625  ],
   [ 0.9063241512999999915933813099400140345096588134765625  ],
   [ 0.                                                      ]])

这是我用来获取距离矩阵的命令：

d_matrix_lag = pairwise_distances(lag_measure_data[['bios_level']].values)

我没有在这里打印输出距离矩阵，因为它太乱了，但作为第一行中的示例，第 4 列的值是

0.445992701999999907602756366031826473772525787353515625

而第4行第一列的值为

0.4459927019999998520916051347739994525909423828125

【问题讨论】：

什么是向量给出这样的距离？
所以如果你比较我打印的数组的第一个和第四个元素，你可以希望重现这些结果。 pairwise_distances（0.3228176074999999922710003374959342181682586669921875,0.76881030949999995538490793478558771312236785888671875）停止[700]：阵列（[[0.4459927019999998520916051347739994525909423828125]]）pairwise_distances（0.76881030949999995538490793478558771312236785888671875,0.3228176074999999922710003374959342181682586669921875）停止[701]：阵列（[[0.445992701999999907602756366031826473772525787353515625]]）跨度>
在上面的评论中，我对两个浮点数进行了两次成对计算，同时切换了两个参数的位置，结果略有不同

标签： python pandas numpy scikit-learn

【解决方案1】：

我可以在对称性测试中重现您的错误：

import numpy as np

a = np.array([[ 0.76881030949999995538490793478558771312236785888671875 ],
   [ 0.                                                      ],
   [ 0.67783090619999997183953155399649403989315032958984375 ],
   [ 0.3228176074999999922710003374959342181682586669921875  ],
   [ 0.75822395549999999087020796650904230773448944091796875 ],
   [ 0.469808621599999975959605080788605846464633941650390625],
   [ 0.989529862699999984698706612107343971729278564453125   ],
   [ 0.                                                      ],
   [ 0.5575436799999999859522858969285152852535247802734375  ],
   [ 0.9756440299999999954394525047973729670047760009765625  ],
   [ 0.66511863289999995085821637985645793378353118896484375 ],
   [ 0.978062709200000046649847718072123825550079345703125   ],
   [ 0.473957179800000016900440868994337506592273712158203125],
   [ 0.82409385540000001935112550199846737086772918701171875 ],
   [ 0.56548685279999999497846374651999212801456451416015625 ],
   [ 0.399505730399999980928527065771049819886684417724609375],
   [ 0.474232963900000026313819034839980304241180419921875   ],
   [ 0.34276307189999999369689476225175894796848297119140625 ],
   [ 0.9985316859999999739017084721126593649387359619140625  ],
   [ 0.9063241512999999915933813099400140345096588134765625  ],
   [ 0.                                                      ]])

from sklearn.metrics.pairwise import pairwise_distances
dist_sklearn = pairwise_distances(a)
print((dist_sklearn.transpose() == dist_sklearn).all())

得到 False 作为输出。尝试改用 scipy.spatial.distance 。您将获得成对距离计算的距离向量，但可以使用 squareform() 将其转换为距离矩阵

from scipy.spatial.distance import pdist, squareform

dist = pdist(a)
sq = squareform(dist)
print((sq.transpose() == sq).all())

这给了我对称矩阵。希望这会有所帮助

【讨论】：