Keras model.pop() 不工作答案

【问题标题】：Keras model.pop() not workingKeras model.pop() 不工作
【发布时间】：2017-10-30 16:05:22
【问题描述】：

我正在使用 Python 3.6.2 和 Tensorflow-gpu 1.3.0 运行 Keras 2.0.6。

为了对 Vgg16 模型进行微调，我在手动构建了一个 vgg16 架构并加载了权重之后运行了这段代码，但我还没有调用 compile()：

model = self.model
model.pop()
for layer in model.layers: layer.trainable=False
model.add(Dense(num, activation='softmax'))
self.compile()

当我在 Tensorboard 中查看图表时，我看到（查看附件图片的左上角）dense_3 连接到 dropout_2 但自身悬空。然后在它旁边我看到了 dense_4，也连接到 dropout_2。

Tensorboard model graph

按照 joelthchao 在 2016 年 5 月 6 日的建议 here，我尝试将 pop() 替换为下面的 pop_layer() 代码。不幸的是，Tensorboard 中显示的图表变得一团糟。

def pop_layer(model):
    if not model.outputs:
        raise Exception('Sequential model cannot be popped: model is empty.')

    model.layers.pop()
    if not model.layers:
        model.outputs = []
        model.inbound_nodes = []
        model.outbound_nodes = []
    else:
        model.layers[-1].outbound_nodes = []
        model.outputs = [model.layers[-1].output]
    model.built = False

我知道有些东西不能正常工作，因为我在 Kaggle 猫狗比赛中运行此代码时准确性较低，我徘徊在 90% 左右，而其他人在 Theanos 之上运行此代码（它改编自 fast.ai）很容易得到97%。也许我的准确性问题来自其他地方，但我仍然认为dense_3 不应该在那里晃来晃去，我想知道这是否可能是我的精度问题的根源。

我怎样才能确定断开并删除dense_3？

运行代码之前和之后的 model.summary() 见下文，为微调做准备。我们不再看到 dense_3，但我们确实在 tensorboard 图中看到了它。

运行前

Layer (type)                 Output Shape              Param #   
=================================================================
lambda_1 (Lambda)            (None, 3, 224, 224)       0         
_________________________________________________________________
zero_padding2d_1 (ZeroPaddin (None, 3, 226, 226)       0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 64, 224, 224)      1792      
_________________________________________________________________
zero_padding2d_2 (ZeroPaddin (None, 64, 226, 226)      0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 64, 224, 224)      36928     
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 64, 112, 112)      0         
_________________________________________________________________
zero_padding2d_3 (ZeroPaddin (None, 64, 114, 114)      0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 128, 112, 112)     73856     
_________________________________________________________________
zero_padding2d_4 (ZeroPaddin (None, 128, 114, 114)     0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 128, 112, 112)     147584    
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 128, 56, 56)       0         
_________________________________________________________________
zero_padding2d_5 (ZeroPaddin (None, 128, 58, 58)       0         
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 256, 56, 56)       295168    
_________________________________________________________________
zero_padding2d_6 (ZeroPaddin (None, 256, 58, 58)       0         
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 256, 56, 56)       590080    
_________________________________________________________________
zero_padding2d_7 (ZeroPaddin (None, 256, 58, 58)       0         
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 256, 56, 56)       590080    
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 256, 28, 28)       0         
_________________________________________________________________
zero_padding2d_8 (ZeroPaddin (None, 256, 30, 30)       0         
_________________________________________________________________
conv2d_8 (Conv2D)            (None, 512, 28, 28)       1180160   
_________________________________________________________________
zero_padding2d_9 (ZeroPaddin (None, 512, 30, 30)       0         
_________________________________________________________________
conv2d_9 (Conv2D)            (None, 512, 28, 28)       2359808   
_________________________________________________________________
zero_padding2d_10 (ZeroPaddi (None, 512, 30, 30)       0         
_________________________________________________________________
conv2d_10 (Conv2D)           (None, 512, 28, 28)       2359808   
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 512, 14, 14)       0         
_________________________________________________________________
zero_padding2d_11 (ZeroPaddi (None, 512, 16, 16)       0         
_________________________________________________________________
conv2d_11 (Conv2D)           (None, 512, 14, 14)       2359808   
_________________________________________________________________
zero_padding2d_12 (ZeroPaddi (None, 512, 16, 16)       0         
_________________________________________________________________
conv2d_12 (Conv2D)           (None, 512, 14, 14)       2359808   
_________________________________________________________________
zero_padding2d_13 (ZeroPaddi (None, 512, 16, 16)       0         
_________________________________________________________________
conv2d_13 (Conv2D)           (None, 512, 14, 14)       2359808   
_________________________________________________________________
max_pooling2d_5 (MaxPooling2 (None, 512, 7, 7)         0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 25088)             0         
_________________________________________________________________
dense_1 (Dense)              (None, 4096)              102764544 
_________________________________________________________________
dropout_1 (Dropout)          (None, 4096)              0         
_________________________________________________________________
dense_2 (Dense)              (None, 4096)              16781312  
_________________________________________________________________
dropout_2 (Dropout)          (None, 4096)              0         
_________________________________________________________________
dense_3 (Dense)              (None, 1000)              4097000   
=================================================================
Total params: 138,357,544
Trainable params: 138,357,544
Non-trainable params: 0

运行后

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
lambda_1 (Lambda)            (None, 3, 224, 224)       0         
_________________________________________________________________
zero_padding2d_1 (ZeroPaddin (None, 3, 226, 226)       0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 64, 224, 224)      1792      
_________________________________________________________________
zero_padding2d_2 (ZeroPaddin (None, 64, 226, 226)      0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 64, 224, 224)      36928     
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 64, 112, 112)      0         
_________________________________________________________________
zero_padding2d_3 (ZeroPaddin (None, 64, 114, 114)      0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 128, 112, 112)     73856     
_________________________________________________________________
zero_padding2d_4 (ZeroPaddin (None, 128, 114, 114)     0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 128, 112, 112)     147584    
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 128, 56, 56)       0         
_________________________________________________________________
zero_padding2d_5 (ZeroPaddin (None, 128, 58, 58)       0         
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 256, 56, 56)       295168    
_________________________________________________________________
zero_padding2d_6 (ZeroPaddin (None, 256, 58, 58)       0         
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 256, 56, 56)       590080    
_________________________________________________________________
zero_padding2d_7 (ZeroPaddin (None, 256, 58, 58)       0         
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 256, 56, 56)       590080    
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 256, 28, 28)       0         
_________________________________________________________________
zero_padding2d_8 (ZeroPaddin (None, 256, 30, 30)       0         
_________________________________________________________________
conv2d_8 (Conv2D)            (None, 512, 28, 28)       1180160   
_________________________________________________________________
zero_padding2d_9 (ZeroPaddin (None, 512, 30, 30)       0         
_________________________________________________________________
conv2d_9 (Conv2D)            (None, 512, 28, 28)       2359808   
_________________________________________________________________
zero_padding2d_10 (ZeroPaddi (None, 512, 30, 30)       0         
_________________________________________________________________
conv2d_10 (Conv2D)           (None, 512, 28, 28)       2359808   
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 512, 14, 14)       0         
_________________________________________________________________
zero_padding2d_11 (ZeroPaddi (None, 512, 16, 16)       0         
_________________________________________________________________
conv2d_11 (Conv2D)           (None, 512, 14, 14)       2359808   
_________________________________________________________________
zero_padding2d_12 (ZeroPaddi (None, 512, 16, 16)       0         
_________________________________________________________________
conv2d_12 (Conv2D)           (None, 512, 14, 14)       2359808   
_________________________________________________________________
zero_padding2d_13 (ZeroPaddi (None, 512, 16, 16)       0         
_________________________________________________________________
conv2d_13 (Conv2D)           (None, 512, 14, 14)       2359808   
_________________________________________________________________
max_pooling2d_5 (MaxPooling2 (None, 512, 7, 7)         0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 25088)             0         
_________________________________________________________________
dense_1 (Dense)              (None, 4096)              102764544 
_________________________________________________________________
dropout_1 (Dropout)          (None, 4096)              0         
_________________________________________________________________
dense_2 (Dense)              (None, 4096)              16781312  
_________________________________________________________________
dropout_2 (Dropout)          (None, 4096)              0         
_________________________________________________________________
dense_4 (Dense)              (None, 2)                 8194      
=================================================================
Total params: 134,268,738
Trainable params: 8,194
Non-trainable params: 134,260,544

【问题讨论】：

标签： image-processing deep-learning tensorflow-gpu kaggle keras-2

【解决方案1】：

我相信这是一个problem，在使用 tensorflow 后端时在 Keras 中实现了 layers.pop()。现在这里是一个按名称删除最后一层的解决方法：

name_last_layer = str(model1.layers[-1])

model2 = Sequential()
for layer in model1.layers:
    if str(layer) != name_last_layer:
        model2.add(layer)

其中 model1 是您的原始模型，而 model2 是没有最后一层的同一模型。在此示例中，我将 model2 设为 Sequential 模型，但您当然可以更改它。

【讨论】：