Tensorflow 2 LSTM 模型不使用序列学习

Question

我目前正在使用 LSTM 模型通过 Tensorflow 2.2.0 进行时间序列预测

我一直在使用大型数据集，一切运行良好。 但是，数据集创建需要大量 RAM，我想使用 tensorflow.keras.utils.Sequence 来解决问题，我的问题如下：

使用序列时，我的模型不再学习（它预测整个数据集上真实信号的平均值）

我的数据集是从两个 python 列表 x_train_flights 和 y_train_flights 创建的，每个列表都包含 pandas DataFrames。对于此列表中的每个(x_train_flight, y_train_flight)：

• 形状为(-1, features) 的x_train_flight 包含features 信号
• 形状为(-1, 1) 的y_train_flight 包含一个与来自x_train_flights 的信号在时间上对齐的信号

系统如下（我不允许分享真实数据，我使用伪随机信号重新创建了图形）：

这里是features=2（蓝色和橙色线）和look_back=5。也就是说，矩形中的10个点（来自x_train_flights）用于预测黄金点（在训练阶段与y_train_flights中的对应点进行比较）。灰色点是之前的预测。

为了创建我的数据集，我一直在使用这些函数：

def lstm_shapify(sequence, look_back, features):
    res = np.empty((look_back, len(sequence), features), dtype=np.float32)

    for i in range(look_back):
        res[i] = np.roll(sequence, -i * features)

    return np.transpose(res, axes=(1, 0, 2))[:-look_back + 1]


def make_dataset(x_flights, y_flights, look_back, features):
    x = np.empty((0, look_back, features), dtype=np.float32)
    y = np.empty((0, 1), dtype=np.float32)

    for i in range(len(x_flights)):
        x_sample = x_flights[i].values
        y_sample = y_flights[i].values[look_back - 1:]

        x = np.concatenate([x, lstm_shapify(x_sample, look_back, features)])
        y = np.concatenate([y, y_sample])

    return x, y

我的网络适合以下内容：

model.fit(
    x_train,
    y_train,
    epochs=7,
    batch_size=batch_size
)

所以，我创建了这个自定义序列：

class LSTMGenerator(Sequence):

    def __init__(
            self,
            x_flights: List[DataFrame],
            y_flights: List[DataFrame],
            look_back: int,
            batch_size: int,
            features: int
    ):
        self.x_flights = x_flights
        self.y_flights = []
        self.look_back = look_back
        self.batch_size = batch_size
        self.features = features
        self.length = 0

        for y_flight in y_flights:
            y = y_flight.iloc[look_back - 1:].to_numpy()
            self.y_flights.append(y)
            self.length += len(y) // batch_size

    def __getitem__(self, index):
        flight_index = 0

        while True:
            n = len(self.y_flights[flight_index]) // self.batch_size
            if index < n:
                break
            flight_index += 1
            index = index - n

        start_index = index * self.batch_size
        x_batch = lstm_shapify(
            self.x_flights[flight_index]
                .iloc[start_index:start_index + self.batch_size + self.look_back - 1]
                .to_numpy(),
            self.look_back,
            self.features
        )
        y_batch = self.y_flights[flight_index][start_index:start_index + self.batch_size]

        return x_batch, y_batch

    def __len__(self):
        return self.length

它返回的每个元组（x，y）分别是两个形状为 (batch_size, look_back, features) 和 (batch_size, 1) 的 numpy 数组。

现在我正在尝试适应它：

model.fit(
    LSTMGenerator(x_train_flights, y_train_flights, look_back, batch_size, features),
    epochs=epochs
)

这是我的模型：

model = Sequential()    

model.add(LSTM(
    100,
    input_shape=(look_back, features),
    kernel_regularizer=regularizers.l2(1e-3),
    bias_regularizer=regularizers.l2(1e-4)
))
model.add(Dropout(0.2))
model.add(BatchNormalization())
model.add(Dense(1, activation='tanh'))

model.compile(optimizer='adam', loss='mse')

希望你能帮助我

编辑：有关数据集的更多详细信息

Answer 1

这是一个工作示例：

from tensorflow.keras import *
from tensorflow.keras.layers import *
from tensorflow.keras.utils import *
import numpy as np
import tensorflow as tf

np.random.seed(1234)
tf.random.set_seed(1234)

features = 3
lookback = 7

model = Sequential()
model.add(LSTM(500, input_shape = (lookback, features)))
model.add(Dense(1, activation='tanh'))

XS = np.random.randn(200, features)
YS = np.random.randn(200)

class LookbackSeq(Sequence):
    def __init__(self, XS, YS, batch_size, lookback):
        self.XS = XS
        self.YS = YS
        self.batch_size = batch_size
        self.lookback = lookback

    def __len__(self):
        n_windows = self.XS.shape[0] - self.lookback
        return int(np.ceil(n_windows / self.batch_size))

    def __getitem__(self, i):
        base = i * self.batch_size

        n_windows = self.XS.shape[0] - self.lookback
        batch_size = min(n_windows - base, self.batch_size)

        X = np.zeros((batch_size, self.lookback, self.XS.shape[1]))
        Y = np.zeros((batch_size, 1))
        for i in range(batch_size):
            for j in range(self.lookback):
                X[i, j] = self.XS[base + i + j]
            Y[i] = self.YS[base + i + self.lookback]
        return X, Y

model.compile(optimizer='adam', loss='mse')

# ALL SAMPLES IN MEMORY
X, Y = [], []
for i in range(len(XS) - lookback):
    X.append(XS[i:i+lookback])
    Y.append(YS[i+lookback])
X, Y = np.array(X), np.array(Y)
model.fit(X, Y, epochs = 10, batch_size = 4, shuffle = False)

# GENERATED ON THE FLY
# gen = LookbackSeq(XS, YS, 4, lookback)
# model.fit(x = gen,
#           steps_per_epoch = len(gen),
#           shuffle = False,
#           epochs = 10)

我假设您的输入数据的形状为 X = (n_points, n_features) 和 Y = (n_points,)。 LookbackSeq 为您进行批处理和窗口化（回溯）。

您可以对相关行进行注释和取消注释，以使用动态生成的样本进行训练，或者将它们全部存储在内存中。你应该得到相同的结果。

Answer 2

我通过休息并再次查看代码解决了它（我意识到这是一个愚蠢的错误）：我的Sequence 的问题来自每批中的样本是时间连续的样本，而我的计算一切数据集的批次，很好地打乱了。

我的Sequence 有问题，因为批次是在随机数据集中的随机索引处选择的。现在我从一个随机数据集中选择一个随机索引处的每个样本来创建一个批次。