Python pandas：如何在符号更改时选择行并具有最小值？

Question

我试图找出函数与 x=0 交叉的位置。我利用了当函数穿过 x 轴时，它的符号会发生变化的事实。

现在，我有一个这样的数据框，我想找到最接近零的两行，因为该函数在两个点穿过 x 轴。

     A     value
0  105  0.662932
1  105  0.662932
2  107  0.052653 # sign changes here when A is 107
3  108 -0.228060 # among these two A 107 is closer to zero
4  110 -0.740819
5  112 -1.188906
6  142 -0.228060 # sign changes here when A is 142
7  143  0.052654 # among these two, A 143 is closer to zero
8  144  0.349638

需要的输出：

     A     value
2  107  0.052653 
7  143  0.052654 

Answer 1

import pandas as pd

data = [
    [105,  0.662932],
    [105,  0.662932],
    [107,  0.052653], # sign changes between here
    [108, -0.228060], # and here; first row has `value` closer to 0
    [110, -0.740819],
    [112, -1.188906],
    [142, -0.228060], # sign changes between here
    [143,  0.052654], # and here; second row has `value` closer to 0
    [144,  0.349638],
]

df = pd.DataFrame(data, columns=["A", "value"])

# where the sign is the same between two elements, the diff is 0
# otherwise, it's either 2 or -2 (doesn't matter which for this use case)
# use periods=1 and =-1 to do a diff forwards and backwards

sign = df.value.map(np.sign)
diff1 = sign.diff(periods=1).fillna(0)
diff2 = sign.diff(periods=-1).fillna(0)

# now we have the locations where sign changes occur. We just need to extract
# the `value` values at those locations to determine which of the two possibilities
# to choose for each sign change (whichever has `value` closer to 0)

df1 = df.loc[diff1[diff1 != 0].index]
df2 = df.loc[diff2[diff2 != 0].index]
idx = np.where(abs(df1.value.values) < abs(df2.value.values), df1.index.values, df2.index.values)
df.loc[idx]

    A   value
2   107 0.052653
7   143 0.052654

感谢@Vince W. 提到应该使用np.where；我最初采用的是一种更复杂的方法。

编辑 - 请参阅下面的@useruser3483203 的答案，它比这快很多。通过在 numpy 数组而不是 pandas Series 上执行前几个操作（diff、abs、比较相等性），您甚至可以提高一点（当我重新运行他们的时间时快 2 倍）。不过，numpy 的 diff 与 pandas 中的不同，因为它删除了第一个元素，而不是为它返回 NaN。这意味着我们取回符号变化的第一行的索引，而不是第二行，并且需要添加一个才能获得下一行。

def find_min_sign_changes(df):
    vals = df.value.values
    abs_sign_diff = np.abs(np.diff(np.sign(vals)))
    # idx of first row where the change is
    change_idx = np.flatnonzero(abs_sign_diff == 2)
    # +1 to get idx of second rows in the sign change too
    change_idx = np.stack((change_idx, change_idx + 1), axis=1)

    # now we have the locations where sign changes occur. We just need to extract
    # the `value` values at those locations to determine which of the two possibilities
    # to choose for each sign change (whichever has `value` closer to 0)

    min_idx = np.abs(vals[change_idx]).argmin(1)
    return df.iloc[change_idx[range(len(change_idx)), min_idx]]

Answer 2

您可以使用numpy 概括该方法：

a = df.value.values
u = np.sign(df.value)
m = np.flatnonzero(u.diff().abs().eq(2))

g = np.stack([m-1, m], axis=1)
v = np.abs(a[g]).argmin(1)

df.iloc[g[np.arange(g.shape[0]), v]]

     A     value
2  107  0.052653
7  143  0.052654

---

此解决方案也将更加高效，尤其是随着规模的扩大。

In [122]: df = pd.concat([df]*100)

In [123]: %timeit chris(df)
870 µs ± 10 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [124]: %timeit nathan(df)
2.03 s ± 10.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [125]: %timeit df.loc[find_closest_to_zero_idx(df.value.values)]
1.81 ms ± 12.4 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Answer 3

我设法得到了一个简单的解决方案：

import numpy as np
import pandas as pd

data = [
    [105,  0.662932],
    [105,  0.662932],
    [107,  0.052653], # sign changes between here
    [108, -0.228060], # and here; first row has `value` closer to 0
    [110, -0.740819],
    [112, -1.188906],
    [142, -0.228060], # sign changes between here
    [143,  0.052654], # and here; second row has `value` closer to 0
    [144,  0.349638],
]

df = pd.DataFrame(data, columns=["A", "value"]

解决方案

def find_closest_to_zero_idx(arr):
    fx = np.zeros(len(arr))
    fy = np.array(arr)

    # lower index when sign changes in array
    idx = np.argwhere((np.diff(np.sign(fx - fy)) != 0) )
    nearest_to_zero = []

    # test two values before and after zero which is nearer to zero
    for i in range(len(idx)):
        if abs(arr[idx[i][0]]) < abs(arr[idx[i][0]+1]):
            nearer = idx[i][0]
            nearest_to_zero.append(nearer)
        else:
            nearer = idx[i][0]+1
            nearest_to_zero.append(nearer)


    return nearest_to_zero

idx = find_closest_to_zero_idx(df.value.values)

结果

idx = find_closest_to_zero_idx(df.value.values)

df.loc[idx]

     A     value
2  107  0.052653
7  143  0.052654

缓慢但纯粹的 pandas 方法

df['value_shifted'] = df.value.shift(-1)
df['sign_changed'] = np.sign(df.value.values) * np.sign(df.value_shifted.values)

# lower index where sign changes
idx = df[df.sign_changed == -1.0].index.values

# make both lower and upper index from the a-axis negative so that
# we can groupby later.
for i in range(len(idx)):
    df.loc[ [idx[i], idx[i]+1], 'sign_changed'] = -1.0 * (i+1)

df1 = df[ np.sign(df.sign_changed) == -1.0]
df2 = df1.groupby('sign_changed')['value'].apply(lambda x: min(abs(x)))
df3 = df2.reset_index()

answer = df.merge(df3,on=['sign_changed','value'])
answer
     A     value  value_shifted  sign_changed
0  107  0.052653      -0.228060          -1.0
1  143  0.052654       0.349638          -2.0