Keras模型在训练后不学习

Question

我正在为句子分类任务训练一个 keras 模型。问题是尽管它给出了 94% 的准确率，但它并没有学到任何东西。当我给出一个新句子（数据集中不存在）时，它给出了相同的概率（在model.prediction 步骤中）。我不知道为什么会这样。

这是我的模型

model = Sequential()
model.add(Embedding(max_words, 30, input_length=max_len))
model.add(BatchNormalization())
model.add(Activation('tanh'))
model.add(Dropout(0.5))
model.add(Bidirectional(LSTM(32)))
model.add(BatchNormalization())
model.add(Activation('tanh'))
model.add(Dropout(0.5))
model.add(Dense(2, activation='sigmoid'))
model.summary()

这里max_words = 2000和max_len=300

这是模型摘要

Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_3 (Embedding)      (None, 300, 30)           60000     
_________________________________________________________________
batch_normalization_5 (Batch (None, 300, 30)           120       
_________________________________________________________________
activation_5 (Activation)    (None, 300, 30)           0         
_________________________________________________________________
dropout_3 (Dropout)          (None, 300, 30)           0         
_________________________________________________________________
bidirectional_3 (Bidirection (None, 64)                16128     
_________________________________________________________________
batch_normalization_6 (Batch (None, 64)                256       
_________________________________________________________________
activation_6 (Activation)    (None, 64)                0         
_________________________________________________________________
dropout_4 (Dropout)          (None, 64)                0         
_________________________________________________________________
dense_3 (Dense)              (None, 2)                 130       
=================================================================
Total params: 76,634
Trainable params: 76,446
Non-trainable params: 188

这里是代码，我的数据集大小是 20k，有 10% 在测试中。

model.compile(loss='sparse_categorical_crossentropy', metrics=['accuracy'], optimizer = 'adam')
history = model.fit(sequences_matrix, Y_train, batch_size=256, epochs=50, validation_split=0.1)

Answer 1

尝试将最后一层的激活函数从 sigmoid 更改为 softmax。它与您使用的损失（分类交叉熵）不太匹配。如果你使用sigmoid，那么你只需要一个单元并且应该使用二元交叉熵损失。