减少 CNN 模型中的验证损失

Question

import tensorflow as tf
import tensorflow.keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D
import pickle
import numpy as np
from keras.models import model_from_json
from keras.models import load_model
import matplotlib.pyplot as plt

# Opening the files about data
X = pickle.load(open("X.pickle", "rb"))
y = pickle.load(open("y.pickle", "rb"))

# normalizing data (a pixel goes from 0 to 255)
X = X/255.0

# Building the model
model = Sequential()
# 3 convolutional layers
model.add(Conv2D(32, (3, 3), input_shape = X.shape[1:]))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))


model.add(Conv2D(64, (3, 3)))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.9))

# 5 hidden layers
model.add(Flatten())

model.add(Dense(128))
model.add(Activation("relu"))

model.add(Dense(128))
model.add(Activation("relu"))

model.add(Dense(128))
model.add(Activation("relu"))

model.add(Dense(128))
model.add(Activation("relu"))

model.add(Dense(128))
model.add(Activation("relu"))

# The output layer with 7 neurons, for 7 classes
model.add(Dense(13))
model.add(Activation("softmax"))

# Compiling the model using some basic parameters
model.compile(loss="sparse_categorical_crossentropy",
                optimizer="adam",
                metrics=["accuracy"])

# Training the model, with 40 iterations
# validation_split corresponds to the percentage of images used for the validation phase compared to all the images

print("X = " + str(len(X)))
print("y = " + str(len(y)))

history = model.fit(X, y, batch_size=32, epochs=1000, validation_split=0.1)

# Saving the model
model_json = model.to_json()
with open("model.json", "w") as json_file :
    json_file.write(model_json)

model.save_weights("model.h5")

print("Saved model to disk")

model.save('CNN.model')

# Printing a graph showing the accuracy changes during the training phase
print(history.history.keys())

plt.show()

plt.plot(history.history['accuracy'])

plt.plot(history.history['loss'])

plt.title('model accuracy')

plt.ylabel('accuracy')

plt.xlabel('epoch')

plt.legend(['train', 'validation'], loc='upper left')

plt.show()

问题在于，我的训练损失较低，但验证准确度很高。而且验证的准确性也极低。我该如何解决这个问题？我试图将下降值增加到 0.9，但损失仍然高得多。我也尝试使用线性函数进行激活，但没有用。

请帮忙。

Answer 1

正如已经提到的，在没有看到数据的情况下很难给出好的建议。

我会尝试以下方法： - 移除 maxpooling 层后的 Dropout - 去除一些致密层 - 在密集之间添加 dropout

如果仍然过拟合，则在密集层之间添加 dropout

编辑： 在我看到损失和准确度图之后，我会提出以下建议：

1. 最高优先级是获取更多数据。
2. 然后使用数据增强甚至增加您的数据集
3. 如果额外数据无济于事，则进一步降低神经网络的复杂性（但我认为随着数据的增加，训练速度会变慢，并且验证损失也会在更长的时期内减少）

Answer 2

数据增强是减少过拟合的最佳技术。尝试使用数据生成器进行训练和验证集，以减少损失并提高准确性。

要了解有关增强和可用转换的更多信息，请查看https://github.com/keras-team/keras-preprocessing

# Add our data-augmentation parameters to ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255.,
                               rotation_range = 40,
                               width_shift_range = 0.2,
                               height_shift_range = 0.2,
                               shear_range = 0.2,
                               zoom_range = 0.2,
                               horizontal_flip = True)

# Note that the validation data should not be augmented!
test_datagen = ImageDataGenerator(rescale = 1./255.)

# Flow training images in batches of 20 using train_datagen generator
train_generator = train_datagen.flow_from_directory(train_dir,
                                                batch_size = 20,
                                                class_mode = 'binary', 
                                                target_size = (150, 150))     

# Flow validation images in batches of 20 using test_datagen generator
validation_generator =  test_datagen.flow_from_directory(validation_dir,
                                                      batch_size  = 20,
                                                      class_mode  = 'binary', 
                                                      target_size = (150, 150)) 

# Now fit the training, validation generators to the CNN model
history = model.fit_generator(train_generator,
        validation_data = validation_generator,
        steps_per_epoch = 100,
        epochs = 3,
        validation_steps = 50,
        verbose = 2,callbacks=[callbacks])