为什么在模型上加载保存的权重时会出现问题

Question

我正在尝试使用许多工具（dropout、autoencoder 等）修改分类器模型，以分析获得最佳结果的方法。因此，我使用save_weights 和load_weights 方法。

我第一次启动我的模型时，它运行良好。然而，在加载权重时，fit 没有做任何事情。损失在整个训练过程中停滞不前。

我知道我一定做错了什么，但我不知道是什么。 我首先认为这是梯度消失的问题，因为我首先遇到了自动编码数据集的问题。但经过多次调整和尝试，我觉得问题在于权重加载。自己看看（这显然是在运行时重启之后）：

# Classifier

model = Sequential()

model.add(Dense(50, activation= 'relu', input_dim= x.shape[1]))
model.add(Dense(50, activation= 'relu'))
model.add(Dense(50, activation= 'relu'))
model.add(Dense(50, activation= 'relu'))
model.add(Dense(10, activation= 'softmax'))

model.compile(optimizer='adam', loss = 'categorical_crossentropy', metrics = ['acc'])

model.save_weights('/content/drive/My Drive/Colab Notebooks/Weights/KagTPOneStart')

第一次拟合（加载初始权重十拟合。是的，我知道此时初始权重已经存在，但我在这里留了线以证明这次它不会造成问题）：

model.load_weights('/content/drive/My Drive/Colab Notebooks/Weights/KagTPOneStart')

model.fit(x,y_train,epochs=10,batch_size=20, validation_split=0.15)

model.save_weights('/content/drive/My Drive/Colab Notebooks/Weights/KagTPOneNormal')

结果：

Train on 35700 samples, validate on 6300 samples
Epoch 1/10
35700/35700 [==============================] - 5s 128us/step - loss: 1.0875 - acc: 0.8036 - val_loss: 0.3275 - val_acc: 0.9067
Epoch 2/10
35700/35700 [==============================] - 4s 120us/step - loss: 0.2792 - acc: 0.9201 - val_loss: 0.3186 - val_acc: 0.9079
Epoch 3/10
35700/35700 [==============================] - 4s 122us/step - loss: 0.2255 - acc: 0.9357 - val_loss: 0.1918 - val_acc: 0.9444
Epoch 4/10
35700/35700 [==============================] - 4s 121us/step - loss: 0.1777 - acc: 0.9499 - val_loss: 0.1977 - val_acc: 0.9465
Epoch 5/10
35700/35700 [==============================] - 4s 121us/step - loss: 0.1530 - acc: 0.9549 - val_loss: 0.1718 - val_acc: 0.9478
Epoch 6/10
35700/35700 [==============================] - 4s 121us/step - loss: 0.1402 - acc: 0.9595 - val_loss: 0.1847 - val_acc: 0.9510
Epoch 7/10
35700/35700 [==============================] - 4s 122us/step - loss: 0.1236 - acc: 0.9637 - val_loss: 0.1675 - val_acc: 0.9546
Epoch 8/10
35700/35700 [==============================] - 4s 121us/step - loss: 0.1160 - acc: 0.9660 - val_loss: 0.1776 - val_acc: 0.9586
Epoch 9/10
35700/35700 [==============================] - 4s 120us/step - loss: 0.1109 - acc: 0.9683 - val_loss: 0.1928 - val_acc: 0.9492
Epoch 10/10
35700/35700 [==============================] - 4s 120us/step - loss: 0.1040 - acc: 0.9701 - val_loss: 0.1749 - val_acc: 0.9570
WARNING:tensorflow:This model was compiled with a Keras optimizer (<tensorflow.python.keras.optimizers.Adam object at 0x7fb76ca35080>) but is being saved in TensorFlow format with `save_weights`. The model's weights will be saved, but unlike with TensorFlow optimizers in the TensorFlow format the optimizer's state will not be saved.

Consider using a TensorFlow optimizer from `tf.train`.

第二次训练（加载初始权重然后拟合）：

model.load_weights('/content/drive/My Drive/Colab Notebooks/Weights/KagTPOneStart')

model.fit(x,y_train,epochs=10,batch_size=20, validation_split=0.15)

model.save_weights('/content/drive/My Drive/Colab Notebooks/Weights/KagTPOneNormal')

结果：

Train on 35700 samples, validate on 6300 samples
Epoch 1/10
35700/35700 [==============================] - 4s 121us/step - loss: 14.4847 - acc: 0.1011 - val_loss: 14.5907 - val_acc: 0.0948
Epoch 2/10
35700/35700 [==============================] - 4s 122us/step - loss: 14.5018 - acc: 0.1003 - val_loss: 14.5907 - val_acc: 0.0948
Epoch 3/10
35700/35700 [==============================] - 4s 120us/step - loss: 14.5018 - acc: 0.1003 - val_loss: 14.5907 - val_acc: 0.0948
Epoch 4/10
35700/35700 [==============================] - 4s 121us/step - loss: 14.5018 - acc: 0.1003 - val_loss: 14.5907 - val_acc: 0.0948
Epoch 5/10
35700/35700 [==============================] - 4s 121us/step - loss: 14.5018 - acc: 0.1003 - val_loss: 14.5907 - val_acc: 0.0948
Epoch 6/10
35700/35700 [==============================] - 4s 121us/step - loss: 14.5018 - acc: 0.1003 - val_loss: 14.5907 - val_acc: 0.0948
Epoch 7/10
35700/35700 [==============================] - 4s 122us/step - loss: 14.5018 - acc: 0.1003 - val_loss: 14.5907 - val_acc: 0.0948
Epoch 8/10
35700/35700 [==============================] - 4s 121us/step - loss: 14.5018 - acc: 0.1003 - val_loss: 14.5907 - val_acc: 0.0948
Epoch 9/10
35700/35700 [==============================] - 4s 122us/step - loss: 14.5018 - acc: 0.1003 - val_loss: 14.5907 - val_acc: 0.0948
Epoch 10/10
35700/35700 [==============================] - 5s 130us/step - loss: 14.5018 - acc: 0.1003 - val_loss: 14.5907 - val_acc: 0.0948
WARNING:tensorflow:This model was compiled with a Keras optimizer (<tensorflow.python.keras.optimizers.Adam object at 0x7fb76ca35080>) but is being saved in TensorFlow format with `save_weights`. The model's weights will be saved, but unlike with TensorFlow optimizers in the TensorFlow format the optimizer's state will not be saved.

Consider using a TensorFlow optimizer from `tf.train`.

提前感谢您的帮助:)

PS：这里有数据供参考，但我真的不认为这是问题所在。这是 google 在 kaggle 上提供的类似 MNIST 的数据集。 （我相信这正是 MNIST 但不是所有样本）：

import pandas as pd 
df=pd.read_csv('/content/drive/My Drive/Colab Notebooks/IA/Kaggle TP1/train.csv')
data = df.values
data.shape        #(42000, 785)
y = data[:,0]
y_train =  np_utils.to_categorical(y, 10)
x = data[:,1:]

Answer 1

要重新开始对 fit() 函数已使用的模型进行训练，您必须重新编译它。

model.compile(optimizer='adam', loss = 'categorical_crossentropy', metrics = ['acc'])

原因是模型分配了一个优化器，它已经处于某种状态。这个状态表示训练的进度，所以如果不重新编译模型，就会在这个状态下继续训练。如果您的模型在第一次训练中确实卡住了，它几乎肯定会继续卡住（学习率太低等）。

编译定义了损失函数、优化器和指标，与分配给层的权重无关。