为什么“自我预测”比“从输入预测”差？

Question

我尝试将@Daniel Möller 提供的代码实现到我的数据中。这是使用 LSTM 学习的时间序列预测问题。 https://github.com/danmoller/TestRepo/blob/master/TestBookLSTM.ipynb

import numpy as np, pandas as pd, matplotlib.pyplot as plt
from sklearn.preprocessing import MinMaxScaler
from keras.models import Sequential
from keras.layers import LSTM, Dense, TimeDistributed, Bidirectional
from sklearn.metrics import mean_squared_error, accuracy_score
from scipy.stats import linregress
from keras.callbacks import EarlyStopping

fi = 'pollution.csv'
raw = pd.read_csv(fi, delimiter=',')
raw = raw.drop('Dates', axis=1)

print (raw.shape)

scaler = MinMaxScaler(feature_range=(-1, 1))
raw = scaler.fit_transform(raw)
n_rows = raw.shape[0] 
n_feats = raw.shape[1]
time_shift = 7
train_size = int(n_rows * 0.8)
train_data = raw[:train_size, :] 
test_data = raw[train_size:, :] 
x_train = train_data[:-time_shift, :] 
x_test = test_data[:-time_shift,:] 
x_predict = raw[:-time_shift,:] 
y_train = train_data[time_shift:, :] 
y_test = test_data[time_shift:,:]
y_predict_true = raw[time_shift:,:]
x_train = x_train.reshape(1, x_train.shape[0], x_train.shape[1]) 
y_train = y_train.reshape(1, y_train.shape[0], y_train.shape[1])
x_test = x_test.reshape(1, x_test.shape[0], x_test.shape[1])
y_test = y_test.reshape(1, y_test.shape[0], y_test.shape[1])
x_predict = x_predict.reshape(1, x_predict.shape[0], x_predict.shape[1])
y_predict_true = y_predict_true.reshape(1, y_predict_true.shape[0], y_predict_true.shape[1])

print (x_train.shape)
print (y_train.shape)
print (x_test.shape)
print (y_test.shape)

model = Sequential()
model.add(LSTM(64,return_sequences=True,input_shape=(None, n_feats)))
model.add(LSTM(32,return_sequences=True))
model.add(LSTM(n_feats,return_sequences=True)) 

stop = EarlyStopping(monitor='loss',min_delta=0.000000000001,patience=30) 

model.compile(loss='mse', optimizer='Adam')
model.fit(x_train,y_train,epochs=10,callbacks=[stop],verbose=2,validation_data=(x_test,y_test))

newModel = Sequential()
newModel.add(LSTM(64,return_sequences=True,stateful=True,batch_input_shape=(1,None,n_feats)))
newModel.add(LSTM(32,return_sequences=True,stateful=True))
newModel.add(LSTM(n_feats,return_sequences=False,stateful=True))

newModel.set_weights(model.get_weights())
newModel.reset_states()

lastSteps = np.empty((1, n_rows, n_feats))  
lastSteps[:,:time_shift] = x_predict[:,-time_shift:] 

newModel.predict(x_predict).reshape(1,1,n_feats)

rangeLen = n_rows - time_shift  
for i in range(rangeLen):
    lastSteps[:,i+time_shift] = newModel.predict(lastSteps[:,i:i+1,:]).reshape(1,1,n_feats)

forecastFromSelf = lastSteps[:,time_shift:,:]
print (forecastFromSelf.shape)
forecastFromSelf = scaler.inverse_transform(forecastFromSelf.reshape(forecastFromSelf.shape[1],forecastFromSelf.shape[2]))

y_predict_true = scaler.inverse_transform(y_predict_true.reshape(y_predict_true.shape[1],y_predict_true.shape[2]))
plt.plot(y_predict_true[:,0], color='b', label='True') 
plt.plot(forecastFromSelf[:,0],color='r', label='Predict')
plt.legend()
plt.title("Self forcast (Feat 1)")
plt.show()


newModel.reset_states()
newModel.predict(x_predict) 
newSteps = []
for i in range(x_predict.shape[1]):
    newSteps.append(newModel.predict(x_predict[:,i:i+1,:]))
forecastFromInput = np.asarray(newSteps).reshape(1,x_predict.shape[1],n_feats)
print (forecastFromInput.shape)
forecastFromInput = scaler.inverse_transform(forecastFromInput.reshape(forecastFromInput.shape[1],forecastFromInput.shape[2]))

plt.plot(y_predict_true[:,0], color='b', label='True')
plt.plot(forecastFromInput[:,0], color='r', label='Predict')
plt.legend()
plt.title("Forecast from input (Feat 1)")
plt.show()

可以通过增加模型层和时期数来增加预测。 然而，这里的问题是，为什么“自我预测”比“输入预测”更糟糕？

污染数据在这里：https://github.com/sirjanrocky/some-neural-tests-for-study/blob/master/pollution.csv 此代码运行没有错误。你也可以试试

Answer 1

假设您的数据在步骤 1000 结束，并且您没有更多数据。

但即使在步骤 1100 之前，您也希望进行预测。如果您没有输入数据，您将不得不依赖预测的输出。

在“自我预测”中，你将无中生有地预测 100 步，没有任何基础

但是每个预测都有一个相关的错误（它并不完美）。当您根据预测进行预测时，您提供的输入有错误，而输出的错误更大。这是不可避免的。

从预测中预测，从预测中预测，从预测中预测......这会累积很多错误。

如果你的模型能很好地进行这种预测，那么你可以肯定地说它已经学到了很多关于序列的知识。

---

当您根据已知输入进行预测时，您所做的事情就会安全得多。你有真实的数据要输入，真实的数据是没有错误的。

因此，您的预测虽然肯定有错误，但并不是“累积”错误。您不是根据有错误的数据进行预测，而是根据真实数据进行预测。

---

在图片中：

自我预测（错误累积）

   true input     --> model --> predicted output step 1001 (a little error)
(steps 1 to 1000)                                |
                                                 V
                                               model
                                                 |
                                                 V
                                predicted output step 1002 (more error)
                                                 |
                                                 V
                                               model
                                                 |
                                                 V
                                predicted output step 1003 (even more error)

根据输入预测（错误不会累积）

    true input    --> model --> predicted output step 1001 (a little error)
(steps 1 to 1000) 

    true input    --> model --> predicted output step 1002 (a little error) 
    (step 1001) 

    true input    --> model --> predicted output step 1003 (a little error)
    (step 1002)                      

---

在预测部分发现的错误（自我预测）

如果要预测测试数据进行比较：

（cmets中的数字只是为了我自己的方向，他们假设训练数据长度为1000，测试数据长度为200，总共1200个元素）

lastSteps = np.empty((1,n_rows-train_size,n_feats))   #test size = 200
lastSteps[:,:time_shift] = y_train[:,-time_shift:]    #0 to 6 = 993 to 999

newModel.predict(x_train)    #predict 999

rangeLen = n_rows - train_size - time_shift
for i in range(rangeLen):
    lastSteps[:,i+time_shift] = newModel.predict(lastSteps[:,i:i+1,:]).reshape(1,1,n_feats)
        #el 7 (1000) <- pred from el 0 (993)
forecastFromSelf = lastSteps[:,time_shift:,:] #1000 forward

如果要预测结束后的未知数据：

你应该训练整个数据（用x_predict, y_predict_true训练）

lastSteps = np.empty((1,new_predictions + time_shift,n_feats))   
lastSteps[:,:time_shift] = y_predict_true[:,-time_shift:]    #0 to 6 = 1193 to 1199

newModel.predict(x_predict)    #predict 1199

rangeLen = new_predictions 
for i in range(rangeLen):
    lastSteps[:,i+time_shift] = newModel.predict(lastSteps[:,i:i+1,:]).reshape(1,1,n_feats)
        #el 7 (1200) <- pred from el 0 (1193)
forecastFromSelf = lastSteps[:,time_shift:,:] #1200 forward