如何使用 Keras 中的功能 API 在快捷连接中添加卷积层？

Question

我正在尝试使用 Keras 中的功能 API 实现一个自定义 CNN 模型，如附图所示。我已经编写了实现主分支的代码，但是在添加 1x1 卷积作为快捷连接时遇到了问题。为每一对卷积块添加快捷卷积，就在最大池化层之前。代码如下：

input_shape = (256,256,3)
model_input = Input(shape=input_shape)
print(model_input) 

def custom_cnn(model_input):
    x = Conv2D(16, (3, 3), strides = (2,2), padding = 'same')(model_input)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2D(16, (3, 3), strides = (2,2), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(3, 3), strides=(2,2))(x)
    
    x = Conv2D(32, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2D(32, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(3, 3), strides=(2,2))(x)
    
    x = Conv2D(48, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2D(48, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(3, 3), strides=(2,2))(x)
    
    x = Conv2D(64, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2D(64, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(3, 3), strides=(2,2))(x)
    
    x = Conv2D(80, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2D(80, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(3, 3), strides=(2,2))(x)
            
    x = GlobalAveragePooling2D()(x)
    x = Dense(num_classes, activation='softmax')(x)
    model = Model(inputs=model_input, outputs=x, name='custom_cnn')
    return model

#instantiate the model
custom_model = custom_cnn(model_input)

#display model summary
custom_model.summary() 

Answer 1

这里是按照架构在网络中实现残差块：

num_classes = 3
input_shape = (256,256,3)
model_input = Input(shape=input_shape)

def custom_cnn(model_input):
    
    x = Conv2D(16, (3, 3), strides = (2,2), padding = 'same')(model_input)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2D(16, (3, 3), strides = (2,2), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    xx = Conv2D(16, (1,1), strides= (4,4), padding = 'same')(model_input)
    x = Add()([x,xx])
    xx = MaxPooling2D(pool_size=(3, 3), strides=(2,2))(x)
    
    x = Conv2D(32, (3, 3), strides = (1,1), padding = 'same')(xx)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2D(32, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    xx = Conv2D(32, (1,1), strides= (1,1), padding = 'same')(xx)
    x = Add()([x,xx])
    xx = MaxPooling2D(pool_size=(3, 3), strides=(2,2))(x)
    
    x = Conv2D(48, (3, 3), strides = (1,1), padding = 'same')(xx)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2D(48, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    xx = Conv2D(48, (1,1), strides= (1,1), padding = 'same')(xx)
    x = Add()([x,xx])
    xx = MaxPooling2D(pool_size=(3, 3), strides=(2,2))(x)
    
    x = Conv2D(64, (3, 3), strides = (1,1), padding = 'same')(xx)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2D(64, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    xx = Conv2D(64, (1,1), strides= (1,1), padding = 'same')(xx)
    x = Add()([x,xx])
    xx = MaxPooling2D(pool_size=(3, 3), strides=(2,2))(x)
    
    x = Conv2D(80, (3, 3), strides = (1,1), padding = 'same')(xx)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2D(80, (3, 3), strides = (1,1), padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    xx = Conv2D(80, (1,1), strides= (1,1), padding = 'same')(xx)
    x = Add()([x,xx])
    xx = MaxPooling2D(pool_size=(3, 3), strides=(2,2))(x)
            
    x = GlobalAveragePooling2D()(xx)
    x = Dense(num_classes, activation='softmax')(x)
    model = Model(inputs=model_input, outputs=x, name='custom_cnn')
    return model

#instantiate the model
custom_model = custom_cnn(model_input)

#display model summary
custom_model.summary() 

Answer 2

您可以使用keras.layers.concatenate 函数合并多个层，因此如果您有两个“平行”层x 和y，您可以像这样将它们合并到层z：

z = layers.concatenate([x, y])

在您的代码中，它看起来像这样：

首先，创建一个“平行”层（请注意，x 和之后的y 两个层都将应用于从model_input 层开始的同一层）

x = Conv2D(16, (3, 3), strides = (2,2), padding = 'same')(model_input)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = Conv2D(16, (3, 3), strides = (2,2), padding = 'same')(x)
x = BatchNormalization()(x)
x = Activation('relu')(x)

然后您将创建第二个“平行”层：

y = Conv2D(...)(model_input)

合并它们

z = layers.concatenate([x, y])

并在z 上应用最大池化（在作为连接结果的层上）

z = MaxPooling2D(pool_size=(3, 3), strides=(2,2))(z)

再一次，创建两个分支，都将z 作为输入（其中z 等同于上一次迭代中的model_input）并在需要应用连接的每个块中重复。