计算熊猫数据框的滚动相关性

Question

是否可以使用 pandas 中的滚动窗口和相关函数将较短的数据帧或序列与较长的数据帧或序列进行相关，并沿着较长的时间序列得到结果？基本上做 numpy.correlate 方法所做的事情，但不是互相关，而是做成对相关。

x= [0,1,2,3,4,5,4,7,6,9,10,5,6,4,8,7]
y= [4,5,4,5]
print(x)
print(y)
corrs = []
for i in range(0,len(x)-3):
    corrs.append( np.corrcoef(x[i:i+4],y)[0,1] )

结果是：

[0.4472135954999579, 0.4472135954999579, 0.4472135954999579, 0.0, 0.8164965809277259, -0.4472135954999579, 0.8320502943378437, 0.0, -0.24253562503633297, 0.24253562503633297, -0.7683498199278325, 0.8451542547285166, -0.50709255283711]

windows 和 pairwise 的每个组合要么给出一系列 NAN，要么给出“ValueError: Length mismatch”。在我制作的简单测试用例中，它总是 NAN 或单个结果，但没有窗口。

x = pd.DataFrame(x)
y = pd.DataFrame(y)

corr = y.rolling(np.shape(y)[0]).corr(x)
print(corr)
corr = y.rolling(np.shape(x)[0]).corr(x)
print(corr)
corr = x.rolling(np.shape(x)[0]).corr(y)
print(corr)
corr = x.rolling(np.shape(y)[0]).corr(y)
print(corr)
corr = y.rolling(np.shape(y)[0]).corr(x,pairwise=True)
print(corr)
corr = y.rolling(np.shape(x)[0]).corr(x,pairwise=True)
print(corr)
corr = x.rolling(np.shape(x)[0]).corr(y,pairwise=True)
print(corr)
corr = x.rolling(np.shape(y)[0]).corr(y,pairwise=True)
print(corr)

Answer 1

将Rolling.apply 与np.corrcoef 或Series.corr 与y 等相同的索引值一起使用-因此Series.reset_index 与drop=True 是必要的：

x= [0,1,2,3,4,5,4,7,6,9,10,5,6,4,8,7]
y= [4,5,4,5]

corrs = []
for i in range(0,len(x)-3):
    corrs.append( np.corrcoef(x[i:i+4],y)[0,1] )

x = pd.Series(x)
y = pd.Series(y)

corr1 = x.rolling(np.shape(y)[0]).apply(lambda x: np.corrcoef(x, y)[0,1], raw=True)
corr2 = x.rolling(np.shape(y)[0]).apply(lambda x: x.reset_index(drop=True).corr(y), raw=False)

---

print (pd.concat([pd.Series(corrs).rename(lambda x: x + 3), corr1, corr2], axis=1))
           0         1         2
0        NaN       NaN       NaN
1        NaN       NaN       NaN
2        NaN       NaN       NaN
3   0.447214  0.447214  0.447214
4   0.447214  0.447214  0.447214
5   0.447214  0.447214  0.447214
6   0.000000  0.000000  0.000000
7   0.816497  0.816497  0.816497
8  -0.447214 -0.447214 -0.447214
9   0.832050  0.832050  0.832050
10  0.000000  0.000000  0.000000
11 -0.242536 -0.242536 -0.242536
12  0.242536  0.242536  0.242536
13 -0.768350 -0.768350 -0.768350
14  0.845154  0.845154  0.845154
15 -0.507093 -0.507093 -0.507093