如何找出为什么我的 tensorflow 模型具有如此高的损失和低准确度？

Question

背景： 我正在创建一个测试应用程序，它在很大程度上复制了here 描述的功能。

我能够运行上面链接的教程中的代码，我发现损失和准确性是合理的，即使在几个 epoch 之后也是如此。

Tutorial Code: Early into the training of the two-headed CNN, losses and accuracy look good

这是因为代码以 VGG16 模型和已经训练的权重开始，它冻结了这些层，因此核心分类不需要学习。

我的测试代码很大程度上复制了教程结构。它使用完全相同的数据集和已经训练好的 VGG16 权重。但是，我使用生成器加载图像数据集（而不是像教程那样将所有数据拉入内存）。

您可以在here 提供的答案中找到我是如何创建这些生成器的。我苦苦挣扎了一阵子，终于把它弄到了我认为正确的地步。

问题： 当我训练我的模型时，分类损失和准确率符合预期，但是边界框损失会增加，并且边界框准确度并没有提高。

My Code: Even after just a couple epochs you see the bounding box loss starting to grow

更多详情： 我花了很多时间查看生成器生成的 (image, target) 元组，我认为我正在正确处理生成的数据（包括 unitrect）。

A pycharm view of the images and target tuples yielded by generator

事实上，我还添加了一个调试模式，允许我显示输入到培训课程中的图像和矩形。 A motorcycle with the bounding box as computed from the unit rectangle bounding box loaded from CSV into the dataframe (df); df is an input to flow_from_dataframe

我正在使用的模型：

imodel = tf.keras.applications.vgg16.VGG16(weights=None, include_top=False,
                                           input_tensor=Input(shape=(224, 224, 3)))
imodel.load_weights(weights, by_name=True)
imodel.trainable = False
# flatten the max-pooling output of VGG
flatten = imodel.output
flatten = Flatten()(flatten)
# construct a fully-connected layer header to output the predicted
# bounding box coordinates
bboxHead = Dense(128, activation="relu")(flatten)
bboxHead = Dense(64, activation="relu")(bboxHead)
bboxHead = Dense(32, activation="relu")(bboxHead)
bboxHead = Dense(4, activation="sigmoid",
                 name="bounding_box")(bboxHead)
# construct a second fully-connected layer head, this one to predict
# the class label
softmaxHead = Dense(512, activation="relu")(flatten)
softmaxHead = Dropout(0.5)(softmaxHead)
softmaxHead = Dense(512, activation="relu")(softmaxHead)
softmaxHead = Dropout(0.5)(softmaxHead)
softmaxHead = Dense(len(classes), activation="softmax",
                    name="class_label")(softmaxHead)
# put together our model which accept an input image and then output
# bounding box coordinates and a class label
model = Model(
    inputs=imodel.input,
    outputs=(bboxHead, softmaxHead))
# define a dictionary to set the loss methods -- categorical
# cross-entropy for the class label head and mean absolute error
# for the bounding box head
losses = {
    "class_label": "categorical_crossentropy",
    "bounding_box": "mean_squared_error",
}
# define a dictionary that specifies the weights per loss (both the
# class label and bounding box outputs will receive equal weight)
lossWeights = {
    "class_label": 1.0,
    "bounding_box": 1.0
}
# initialize the optimizer, compile the model, and show the model
# summary
opt = Adam(lr=learning_rate)
model.compile(loss=losses, optimizer=opt, metrics=["accuracy"], loss_weights=lossWeights)

我对“适合”的呼吁

model.fit(x=train_generator[0], steps_per_epoch=train_generator[1],
          validation_data=validation_generator[0], validation_steps=validation_generator[1],
          epochs=epochs, verbose=1)

我加载的权重是我在其他实验中使用并从kaggle 下载的 - （参见 vgg16_weights_tf_dim_ordering_tf_kernels.h5）。

我的发电机：

def generate_image_generator(generator, data_directory, df, subset, target_size, batch_size, shuffle, seed):
genImages = generator.flow_from_dataframe(dataframe=df, directory=data_directory, target_size=target_size,
                                          x_col="file",
                                          y_col=['cls_onehot', 'bbox'],
                                          subset=subset,
                                          class_mode="multi_output",
                                          batch_size=batch_size, shuffle=shuffle, seed=seed)

while True:
    images, labels = genImages.next()

    targets = {
        'class_label': labels[0],
        'bounding_box': np.array(labels[1], dtype="float32")
    }
    yield images, targets   

def get_train_and_validate_generators(self, data_directory, files, max_images, validation_split, shuffle, seed, target_size):
        generator = ImageDataGenerator(validation_split=validation_split,
                                       rescale=1./255.)

        df = get_dataframe(data_directory, files)

        if max_images:
            df = df.head(max_images)

        train_generator = generate_image_generator(generator, data_directory, df, "training",
                                                   target_size,
                                                   self.batch_size,
                                                   shuffle, seed)

        valid_generator = generate_image_generator(generator, data_directory, df, "validation",
                                                   target_size,
                                                   self.batch_size,
                                                   shuffle, seed)

从 CSV 列表中加载数据框

def get_dataframe(data_directory, files):
frames=[]
for di in files:
    df = pd.read_csv(data_directory+di["file"])
    frames.append(df)
df = pd.concat(frames)

df['cls_onehot'] = df['cls'].str.get_dummies().values.tolist()
df['bbox'] = df[['sxu', 'syu', 'exu', 'eyu']].values.tolist()
return df

CSV 的 sn-p：

    id,file,sx,sy,ex,ey,cls,sxu,syu,exu,eyu,w,h
0,motorcycle.0001.jpg,31,19,233,141,motorcycle,0.1183206106870229,0.11801242236024845,0.8893129770992366,0.8757763975155279,262,161
1,motorcycle.0002.jpg,32,15,232,142,motorcycle,0.12167300380228137,0.09259259259259259,0.8821292775665399,0.8765432098765432,263,162
2,motorcycle.0003.jpg,30,20,234,143,motorcycle,0.11406844106463879,0.12269938650306748,0.8897338403041825,0.8773006134969326,263,163
3,motorcycle.0004.jpg,30,15,231,132,motorcycle,0.11450381679389313,0.1,0.8816793893129771,0.88,262,150
4,motorcycle.0005.jpg,31,19,232,145,motorcycle,0.1183206106870229,0.1144578313253012,0.8854961832061069,0.8734939759036144,262,166

当我从“imagenet”加载权重，而不是使用从 kaggle 收到的权重时，我发现边界框损失同样增加

    imodel = tf.keras.applications.vgg16.VGG16(weights="imagenet", include_top=False,
                                           input_tensor=Input(shape=(224, 224, 3)))

问题： 请提供有关如何隔离此边界框损失增长问题的建议。

Answer 1

好的。看来我的发电机根本没有问题。代码很好，除了一个愚蠢的疏忽。我仍然有一个旧的调用来编译运行。我第一次使用复合损失函数正确调用了编译。然后我再次调用它，严格使用分类交叉熵作为代价，有效地忽略了我的边界框。

无论如何，如果有人偶然发现这篇文章，我希望他们能找到有关如何使用生成器函数进行分类和对象检测的完整视图。

我用正确的细节编辑了上述问题。所以它现在反映了正确的答案。

我仍然希望获得专家的观点，他们必须深入研究模型的工作原理以更好地了解导致损失计算的潜在细节。

现在我开始从高层次上理解 tensorflow，它清楚地知道如何识别事情何时运行。不清楚如何诊断事情何时不正常。