使用 CNN+LSTM 模型和 TimeDistributed 层包装器的 Keras 时间序列预测

Question

我有几个人类活动识别数据的数据文件，其中包含按时间顺序记录的原始样本行。每行有 8 列 EMG 传感器数据和 1 列对应的目标传感器数据。我正在尝试将 8 通道的 EMG 传感器数据输入 CNN+LSTM 深度模型，以预测 1 通道的目标数据。我通过将数据集（下图中的 a）分解为 50 行原始样本窗口（下图中的 b），然后将这些窗口重塑为4 个窗口的块，作为模型 LSTM 部分的时间步长（下图中的 c）。下图有望更好地解释它：

我一直在关注这里关于如何实现我的模型的教程：https://medium.com/smileinnovation/how-to-work-with-time-distributed-data-in-a-neural-network-b8b39aa4ce00

我已经重新调整了数据并建立了模型，但不断出现以下错误，我无法弄清楚如何解决：

    "ValueError: Error when checking target: expected FC_out to have 2 dimensions, but got array with shape (808, 50, 1)"

我的代码如下，是使用 Keras 和 Tensorflow 用 Python 编写的：

    from keras.models import Sequential
    from keras.layers import CuDNNLSTM
    from keras.layers.convolutional import Conv2D
    from keras.layers.core import Dense, Dropout
    from keras.layers import Flatten
    from keras.layers import TimeDistributed

    #Code that reads in file data and shapes it into 4-window blocks omitted. That code produces the following arrays:
    #x_train  - shape of (808, 4, 50, 8) which equates to (samples, time steps, window length, number of channels)
    #x_valid  - shape of (223, 4, 50, 8) which equates to the same as x_train
    #y_train  - shape of (808, 50, 1) which equates to (samples, window length, number of target channels)


    # Followed machine learning mastery style for ease of reading
    numSteps = x_train.shape[1]
    windowLength = x_train.shape[2]
    numChannels = x_train.shape[3]
    numOutputs = 1

    # Reshape x data for use with TimeDistributed wrapper, adding extra dimension at the end
    x_train = x_train.reshape(x_train.shape[0], numSteps, windowLength, numChannels, 1)
    x_valid = x_valid.reshape(x_valid.shape[0], numSteps, windowLength, numChannels, 1)


    # Build model
    model = Sequential()
    model.add(TimeDistributed(Conv2D(64, (3,3), activation=activation, name="Conv2D_1"), 
                                     input_shape=(numSteps, windowLength, numChannels, 1)))

    model.add(TimeDistributed(Conv2D(64, (3,3), activation=activation, name="Conv2D_2")))
    model.add(Dropout(0.4, name="CNN_Drop_01"))

    # Flatten for passing to LSTM layer
    model.add(TimeDistributed(Flatten(name="Flatten_1")))

    # LSTM and Dropout
    model.add(CuDNNLSTM(28, return_sequences=True, name="LSTM_01"))
    model.add(Dropout(0.4, name="Drop_01"))

    # Second LSTM and Dropout
    model.add(CuDNNLSTM(28, return_sequences=False, name="LSTM_02"))
    model.add(Dropout(0.3, name="Drop_02"))

    # Fully Connected layer and further Dropout
    model.add(Dense(16, activation=activation, name="FC_1"))
    model.add(Dropout(0.4)) # For example, for 3 outputs classes 

    # Final fully Connected layer specifying outputs
    model.add(Dense(numOutputs, activation=activation, name="FC_out"))

    # Compile model, produce summary and save model image to file
    # NOTE: coeffDetermination refers to a function for calculating R2 and is not included in this code
    model.compile(optimizer='Adam', loss='mse', metrics=[coeffDetermination])


    # Now train the model
    history_cb = model.fit(x_train, y_train, validation_data=(x_valid, y_valid), epochs=30, batch_size=64)

如果有人能弄清楚我做错了什么，我将不胜感激。还是我只是以不正确的方式解决这个问题，尝试使用此模型配置进行时间序列预测？

Answer 1

"ValueError: 检查目标时出错：预期 FC_out 为 2 维，但得到的数组形状为 (808, 50, 1)"

• 您的输入是 (808, 4, 50, 8, 1)，输出是 (808, 50, 1)
• 但是，从 model.summary() 显示输出形状应该是 (None, 4, 1)
• 由于时间步数为 4，因此 y_train 应该类似于 (808, 4, 1)。
• 或者，如果您想拥有 (888, 50, 1)，则需要更改模型以获取最后一部分为 (None, 50, 1)。

Model: "sequential_10"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
time_distributed_18 (TimeDis (None, 4, 48, 6, 64)      640       
_________________________________________________________________
time_distributed_19 (TimeDis (None, 4, 46, 4, 64)      36928     
_________________________________________________________________
CNN_Drop_01 (Dropout)        (None, 4, 46, 4, 64)      0         
_________________________________________________________________
time_distributed_20 (TimeDis (None, 4, 11776)          0         
_________________________________________________________________
LSTM_01 (LSTM)               (None, 4, 28)             1322160   
_________________________________________________________________
Drop_01 (Dropout)            (None, 4, 28)             0         
_________________________________________________________________
Drop_02 (Dropout)            (None, 4, 28)             0         
_________________________________________________________________
FC_1 (Dense)                 (None, 4, 16)             464       
_________________________________________________________________
dropout_3 (Dropout)          (None, 4, 16)             0         
_________________________________________________________________
FC_out (Dense)               (None, 4, 1)              17        
=================================================================
Total params: 1,360,209
Trainable params: 1,360,209
Non-trainable params: 0

对于不同序列长度的多对多序列预测，请查看此链接https://github.com/keras-team/keras/issues/6063

dataX or input : (nb_samples, nb_timesteps, nb_features) -> (1000, 50, 1)
dataY or output: (nb_samples, nb_timesteps, nb_features) -> (1000, 10, 1)

model = Sequential()  
model.add(LSTM(input_dim=1, output_dim=hidden_neurons, return_sequences=False))  
model.add(RepeatVector(10))
model.add(LSTM(output_dim=hidden_neurons, return_sequences=True))  
model.add(TimeDistributed(Dense(1)))
model.add(Activation('linear'))   
model.compile(loss='mean_squared_error', optimizer='rmsprop', metrics=['accuracy'])