使用 CNN 进行图像回归

Question

我的直接问题是，我尝试过的所有各种 CNN 回归模型总是返回相同（或非常相似）的值，我正试图找出原因。但我愿意接受广泛的建议。

我的数据集如下所示：

• x: 64x64 灰度图像排列成 64 x 64 x n ndarray
• y: 0 到 1 之间的值，每个对应一个图像（可以将其视为某种比例）
• weather：每张照片拍摄时的 4 个天气读数（环境温度、湿度、露点、气压）

目标是使用图像和天气数据来预测y。由于我正在处理图像，因此我认为 CNN 是合适的（如果这里有其他策略，请告诉我）。

据我了解，CNN 最常用于分类任务——将它们用于回归是相当不寻常的。但从理论上讲，它应该不会有太大的不同——我只需要将损失函数更改为 MSE/RMSE 并将最后一个激活函数更改为线性（尽管这里可能使用 sigmoid 更合适，因为y 介于 0 和 1 之间)。

我遇到的第一个障碍是试图弄清楚如何合并天气数据，自然的选择是将它们合并到第一个全连接层中。我在这里找到了一个例子：How to train mix of image and data in CNN using ImageAugmentation in TFlearn

我遇到的第二个障碍是确定架构。通常我会选择一篇论文并复制它的架构，但我在 CNN 图像回归上找不到任何东西。所以我尝试了一个（相当简单的）具有 3 个卷积层和 2 个全连接层的网络，然后我尝试了 https://github.com/tflearn/tflearn/tree/master/examples 的 VGGNet 和 AlexNet 架构

现在我遇到的问题是我尝试的所有模型都输出相同的值，即训练集的平均值 y。看看张量板，损失函数很快就变平了（大约 25 个 epoch 之后）。你知道这里发生了什么吗？虽然我确实了解每一层的基本功能，但对于特定数据集或任务的良好架构，我没有直觉。

这是一个例子。我正在使用 tflearn 示例页面中的 VGGNet：

tf.reset_default_graph()

img_aug = ImageAugmentation()
img_aug.add_random_flip_leftright()
img_aug.add_random_flip_updown()
img_aug.add_random_90degrees_rotation(rotations=[0, 1, 2, 3])

convnet = input_data(shape=[None, size, size, 1], 
                     data_augmentation=img_aug, 
                     name='hive')
weathernet = input_data(shape=[None, 4], name='weather')

convnet = conv_2d(convnet, 64, 3, activation='relu', scope='conv1_1')
convnet = conv_2d(convnet, 64, 3, activation='relu', scope='conv1_2')
convnet = max_pool_2d(convnet, 2, strides=2, name='maxpool1')

convnet = conv_2d(convnet, 128, 3, activation='relu', scope='conv2_1')
convnet = conv_2d(convnet, 128, 3, activation='relu', scope='conv2_2')
convnet = max_pool_2d(convnet, 2, strides=2, name='maxpool2')

convnet = conv_2d(convnet, 256, 3, activation='relu', scope='conv3_1')
convnet = conv_2d(convnet, 256, 3, activation='relu', scope='conv3_2')
convnet = conv_2d(convnet, 256, 3, activation='relu', scope='conv3_3')
convnet = max_pool_2d(convnet, 2, strides=2, name='maxpool3')

convnet = conv_2d(convnet, 512, 3, activation='relu', scope='conv4_1')
convnet = conv_2d(convnet, 512, 3, activation='relu', scope='conv4_2')
convnet = conv_2d(convnet, 512, 3, activation='relu', scope='conv4_3')
convnet = max_pool_2d(convnet, 2, strides=2, name='maxpool4')

convnet = conv_2d(convnet, 512, 3, activation='relu', scope='conv5_1')
convnet = conv_2d(convnet, 512, 3, activation='relu', scope='conv5_2')
convnet = conv_2d(convnet, 512, 3, activation='relu', scope='conv5_3')
convnet = max_pool_2d(convnet, 2, strides=2, name='maxpool5')

convnet = fully_connected(convnet, 4096, activation='relu', scope='fc6')
convnet = merge([convnet, weathernet], 'concat')
convnet = dropout(convnet, .75, name='dropout1')

convnet = fully_connected(convnet, 4096, activation='relu', scope='fc7')
convnet = dropout(convnet, .75, name='dropout2')

convnet = fully_connected(convnet, 1, activation='sigmoid', scope='fc8')

convnet = regression(convnet, 
                     optimizer='adam', 
                     learning_rate=learning_rate, 
                     loss='mean_square', 
                     name='targets')

model = tflearn.DNN(convnet, 
                    tensorboard_dir='log', 
                    tensorboard_verbose=0)

model.fit({
            'hive': x_train,
            'weather': weather_train  
          },
          {'targets': y_train}, 
          n_epoch=1000, 
          batch_size=batch_size,
          validation_set=({
              'hive': x_val,
              'weather': weather_val
          }, 
                          {'targets': y_val}), 
          show_metric=False, 
          shuffle=True,
          run_id='poop')

要了解我的对象是什么：

• x_train 是形状为 (n, 64, 64, 1) 的 ndarray
• weather_train 是形状为 (n, 4) 的 ndarray
• y_train 是形状为 (n, 1) 的 ndarray

过度拟合是另一个问题，但鉴于模型在训练集上的表现不佳，我想我以后可以担心。

Answer 1

解决您对测试集中所有实例的相同预测值的担忧。这里有几个选项不涉及更改 conv 网络的结构：

1. 您可以使用 sklearn StandardScaler() 重新调整目标变量（通过移除均值并缩放到单位方差来标准化特征）
2. 缩放像素数据；通常性能会随着像素数据的缩放而提高，根据经验，总是将像素数据除以 255.0（显示在帖子末尾）
3. 您可以使用学习率和误差函数（CNN 为所有预测输出相同值的原因是因为它已确定这是最小误差点）

下一步。如果您尝试执行回归，请确保您的最终全连接层使用线性激活函数而不是乙状结肠。线性激活函数将神经元的输入乘以神经元权重，并创建与输入成比例的输出。

convnet = fully_connected(convnet, 1, activation='linear', scope='fc8')

最后。我最近在 Keras 中为回归任务实现了 ResNet50。这是该网络的构造，此版本不允许加载预训练的权重，它必须接收形状为 (224, 224, 3) 的图像。

from keras.layers.normalization import BatchNormalization
from keras.layers.convolutional import Conv2D, MaxPooling2D, DepthwiseConv2D
from keras.layers.core import Activation, Dropout, Dense
from keras.layers import Flatten, Input, Add, ZeroPadding2D, GlobalAveragePooling2D, GlobalMaxPooling2D
from keras.models import Model
from keras import backend


def block1(x, filters, kernel_size=3, stride=1, conv_shortcut=True, name=None):
    """
    A residual block

    :param x: input tensor
    :param filters: integer, filters of the bottleneck layer
    :param kernel_size: kernel size of bottleneck
    :param stride: stride of first layer
    :param conv_shortcut: use convolution shortcut if true, otherwise identity shortcut
    :param name: string, block label
    :return: Output tensor of the residual block

    """

    # bn_axis = 3 if backend.image_data_format() == 'channels_last' else 1

    bn_axis = -1

    if conv_shortcut is True:
        shortcut = Conv2D(4 * filters, 1, strides=stride, name=name+'_0_conv')(x)
        shortcut = BatchNormalization(axis=bn_axis, epsilon=1.001e-5, name=name+'_0_bn')(shortcut)
    else:
        shortcut = x

    x = Conv2D(filters, 1, strides=stride, name=name+'_1_conv')(x)
    x = BatchNormalization(axis=bn_axis, epsilon=1.001e-5, name=name+'_1_bn')(x)
    x = Activation('relu', name=name+'_1_relu')(x)

    x = Conv2D(filters, kernel_size, padding='SAME', name=name+'_2_conv')(x)
    x = BatchNormalization(axis=bn_axis, epsilon=1.001e-5, name=name+'_2_bn')(x)
    x = Activation('relu', name=name+'_2_relu')(x)

    x = Conv2D(4 * filters, 1, name=name+'_3_conv')(x)
    x = BatchNormalization(axis=bn_axis, epsilon=1.001e-5, name=name+'_3_bn')(x)

    x = Add(name=name+'_add')([shortcut, x])
    x = Activation('relu', name=name+'_out')(x)

    return x


def stack1(x, filters, blocks, stride1=2, name=None):
    """
    a set of stacked residual blocks

    :param x: input tensor
    :param filters: int, filters fof the bottleneck layer in the block
    :param blocks: int, blocks in the stacked blocks,
    :param stride1: stride of the first layer in the first block
    :param name: stack label
    :return: output tensor for the stacked blocks

    """

    x = block1(x, filters, stride=stride1, name=name+'_block1')

    for i in range(2, blocks+1):
        x = block1(x, filters, conv_shortcut=False, name=name+'_block'+str(i))

    return x

def resnet(height, width, depth, stack_fn, use_bias=False, nodes=256):
    """
    :param height: height of image, int
    :param width: image width, int
    :param depth: bn_axis or depth, int
    :param stack_fn: function that stacks residual blocks
    :param nodes: width of nodes included in top layer of CNN, int
    :return: a Keras model instance
    """

    input_shape = (height, width, depth)

    img_input = Input(shape=input_shape)

    x = ZeroPadding2D(padding=((3, 3), (3, 3)), name='conv1_pad')(img_input)
    x = Conv2D(64, 7, strides=2, use_bias=use_bias, name='conv1_conv')(x)

    x = ZeroPadding2D(padding=((1, 1), (1, 1)), name='pool1_pad')(x)
    x = MaxPooling2D(3, strides=2, name='pool1_pool')(x)

    x = stack_fn(x)

    # top layer
    x = GlobalAveragePooling2D(name='avg_pool')(x)
    x = Dense(nodes, activation='relu')(x)

    # perform regression
    x = Dense(1, activation='linear')(x)

    model = Model(img_input, x)

    return model


def resnet50(height, width, depth, nodes):

    def stack_fn(x):
        x = stack1(x, 64, 3, stride1=1, name='conv2')
        x = stack1(x, 128, 4, name='conv3')
        x = stack1(x, 256, 6, name='conv4')
        x = stack1(x, 512, 3, name='conv5')
        return x

    return resnet(height, width, depth, stack_fn, nodes=nodes)

这可以使用一些 x_train, x_test, y_train, y_test 数据来实现（其中 x_train/test 是图像数据，y_train,y_test 数据是区间 [0, 1] 上的数值。

scaler = MinMaxScaler()
images = load_images(df=target, path=PATH_features, resize_shape=(224, 224), quadruple=True)
images = images / 255.0  # scale pixel data to [0, 1]
images = images.astype(np.float32)
imshape = images.shape

target = target[Target]
target = quadruple_target(target, target=Target)

x_train, x_test, y_train, y_test = train_test_split(images, target, test_size=0.3, random_state=101)

y_train = scaler.fit_transform(y_train)
y_test = scaler.transform(y_test)

model = resnet50(imshape[1], imshape[2], imshape[3], nodes=256)

opt = Adam(lr=1e-5, decay=1e-5 / 200)
model.compile(loss=lossFN, optimizer=opt)

history = model.fit(x_train, y_train, validation_data=(x_test, y_test), verbose=1, epochs=200)

pred = model.predict(x_test)