将非图像特征添加到 Inception 网络

Question

我正在使用 Inception V3 模型训练一个二元分类器，我想将我的数据集的一些非图像特征输入到网络中。

我之前用这些特征训练了一个逻辑回归模型，效果很好，我想看看我是否可以通过组合这些模型来改进我的 cnn。

看起来 inception 在 softmax 之前有一个完全连接的层（logits 层），我相信我应该将一些节点连接到该层上以输入我的特征。但是，我从来没有这样做过。

logits 层在此处构建 - 初始代码的 sn-p

# Final pooling and prediction
        with tf.variable_scope('logits'):
          shape = net.get_shape()
          net = ops.avg_pool(net, shape[1:3], padding='VALID', scope='pool')
          # 1 x 1 x 2048
          net = ops.dropout(net, dropout_keep_prob, scope='dropout')
          net = ops.flatten(net, scope='flatten')
          # 2048
          logits = ops.fc(net, num_classes, activation=None, scope='logits',
                          restore=restore_logits)
          # 1000
          end_points['logits'] = logits
          if FLAGS.mode == '0_softmax':
            end_points['predictions'] = tf.nn.softmax(logits, name='predictions')

制作全连接层的函数：

@scopes.add_arg_scope
def fc(inputs,
       num_units_out,
       activation=tf.nn.relu,
       stddev=0.01,
       bias=0.0,
       weight_decay=0,
       batch_norm_params=None,
       is_training=True,
       trainable=True,
       restore=True,
       scope=None,
       reuse=None):
  """Adds a fully connected layer followed by an optional batch_norm layer.

  FC creates a variable called 'weights', representing the fully connected
  weight matrix, that is multiplied by the input. If `batch_norm` is None, a
  second variable called 'biases' is added to the result of the initial
  vector-matrix multiplication.

  Args:
    inputs: a [B x N] tensor where B is the batch size and N is the number of
            input units in the layer.
    num_units_out: the number of output units in the layer.
    activation: activation function.
    stddev: the standard deviation for the weights.
    bias: the initial value of the biases.
    weight_decay: the weight decay.
    batch_norm_params: parameters for the batch_norm. If is None don't use it.
    is_training: whether or not the model is in training mode.
    trainable: whether or not the variables should be trainable or not.
    restore: whether or not the variables should be marked for restore.
    scope: Optional scope for variable_scope.
    reuse: whether or not the layer and its variables should be reused. To be
      able to reuse the layer scope must be given.

  Returns:
     the tensor variable representing the result of the series of operations.
  """
  with tf.variable_scope(scope, 'FC', [inputs], reuse=reuse):
    num_units_in = inputs.get_shape()[1]
    weights_shape = [num_units_in, num_units_out]
    weights_initializer = tf.truncated_normal_initializer(stddev=stddev)
    l2_regularizer = None
    if weight_decay and weight_decay > 0:
      l2_regularizer = losses.l2_regularizer(weight_decay)
    weights = variables.variable('weights',
                                 shape=weights_shape,
                                 initializer=weights_initializer,
                                 regularizer=l2_regularizer,
                                 trainable=trainable,
                                 restore=restore)
    if batch_norm_params is not None:
      outputs = tf.matmul(inputs, weights)
      with scopes.arg_scope([batch_norm], is_training=is_training,
                            trainable=trainable, restore=restore):
        outputs = batch_norm(outputs, **batch_norm_params)
    else:
      bias_shape = [num_units_out,]
      bias_initializer = tf.constant_initializer(bias)
      biases = variables.variable('biases',
                                  shape=bias_shape,
                                  initializer=bias_initializer,
                                  trainable=trainable,
                                  restore=restore)
      outputs = tf.nn.xw_plus_b(inputs, weights, biases)
    if activation:
      outputs = activation(outputs)
    return outputs

我的模型有 10 个非图像特征，所以我想我会使用 num_units_out + 10？我不确定如何处理输入。我假设我会将特征数据直接添加到这一层，方法是将其添加到已经来自前一层的输入中。所以本质上我将有两个输入层。

Answer 1

在 FC 层之前添加您的功能：

net = ops.flatten(net, scope='flatten')
 # assuming both tensors have a shape like <batch>x<features>
net = tf.concat([net, my_other_features], axis=-1)

这会将现有的 FC 部分与 Input->FC->Sigmoid 部分组合成一个单层。另一种说法是，最终的逻辑层（FC->Sigmoid）将获得一个特征向量输入，其中包含您的特征和 CNN 从图像中计算出的特征。