为什么即使对于训练集，简单的神经网络也不能给出可接受的结果？

Question

我想解决二元分类问题：网络的输入是 35x10，输出是 1 或 0，我想训练它每个 epoch 添加 1 个新样本并删除 1 个旧样本（如滑动窗口）。这是我的代码：

from keras.models import Sequential
from keras.layers.core import Dense, Activation, Flatten

df = pd.read_csv('all.csv')

X = df.iloc[:, 0:10].values
X = np.array(X)
Y = df.iloc[:, 10].values
Y = np.array(Y)

model = Sequential()
model.add(Flatten(input_shape=(35, 10)))
model.add(Dense(35))
model.add(Activation('relu'))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=["accuracy"])

print(model.summary())
batch_size = 1
nb_classes = 2
nb_epoch = 1
input_size = 35

X = X[:1015]
Y1 = np.where(Y=="Move_scaner", 1, 0)[:1015]
no_of_samples = X.shape[0]
for i in range(no_of_samples-input_size):
    if i != 0:
        model.load_weights('./my_checkpoint')
    y = 1 if 1 in Y1[i:(i+input_size)] else 0
    x = X[i:(i+input_size)].reshape(1, 35, 10)
    model.fit(x, [y],
              batch_size, nb_epoch)
    model.save_weights('./my_checkpoint')


eval_results = []
accuracy = []
for j in range(no_of_samples-input_size):
    y = 1 if 1 in Y1[j:(j + input_size)] else 0
    x = X[j:(j + input_size)].reshape(1, 35, 10)
    ev_res =model.evaluate(x, [y], verbose=0)
    eval_results.append(ev_res)
    accuracy.append(ev_res[1]*100)
    print("Loss on train data: {} Accuracy: {}%".format (ev_res[0],  ev_res[1]*100))


unique, counts = np.unique(accuracy, return_counts=True)
d = dict(zip(unique, counts))
print(d)

模型总结：

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_1 (Flatten)          (None, 350)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 35)                12285     
_________________________________________________________________
activation_1 (Activation)    (None, 35)                0         
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 36        
_________________________________________________________________
activation_2 (Activation)    (None, 1)                 0         
=================================================================
Total params: 12,321
Trainable params: 12,321
Non-trainable params: 0

每个 epoch 我得到准确度 = 1 和损失 = 0 但这是运行评估后的输出：

Loss on train data: 116.31327819824219 Accuracy: 0.0%
Loss on train data: 107.40914154052734 Accuracy: 0.0%
Loss on train data: 101.50980377197266 Accuracy: 0.0%
....# here a lot of zeros
Loss on train data: 65.45806121826172 Accuracy: 0.0%
Loss on train data: 65.6653060913086 Accuracy: 0.0%
Loss on train data: 73.23053741455078 Accuracy: 0.0%
Loss on train data: 1.7928523758674385e-32 Accuracy: 100.0%
Loss on train data: 6.712029695306438e-35 Accuracy: 100.0%
Loss on train data: 4.285122127232612e-33 Accuracy: 100.0%
Loss on train data: 1.7302612471686554e-35 Accuracy: 100.0%
Loss on train data: 1.5139565830394122e-37 Accuracy: 100.0%
Loss on train data: 0.0 Accuracy: 100.0%
Loss on train data: 0.0 Accuracy: 100.0%
...
Loss on train data: 0.0 Accuracy: 100.0%
Loss on train data: 0.0 Accuracy: 100.0%
Loss on train data: 0.0 Accuracy: 100.0%

我得到的结果是：368 个零准确度的结果，612 个 100% 准确度的结果。显然，模型没有正确训练。它有什么问题？

Answer 1

也许你的模型由于学习率不合适而出现振荡，所以我向你建议两件事：

1.- 使用 SGD 代替 Adam 来测试模型是否能够学习某些东西。 2.- 调整你的学习率，当你遇到问题并且不知道模型是否能够概括知识时，小的值通常会更好。

额外：尝试批量学习，可能以 20 个元素或类似的方式进行批量学习。