Tensorflow 模型在推理过程中输出不同的值

Question

在训练期间，我记录了我的回归模型针对训练数据输出的预测值。当我在预测模式下运行相同的数据集时，模型输出的值范围大不相同：

张量板

在这里我们看到模型一直在预测 (140, 250) 范围内的值。

对同一数据集的预测

这里我们的模型预测值介于 (17, 23) 之间。什么给了？

我怀疑估算器 API 在使用 tf.layers.batch_normalization 时不会神奇地保存 moving_mean 和 moving_variance。

我的模型：

def model_fn(features, labels, mode, params):
  training = mode == tf.estimator.ModeKeys.TRAIN
  extra_update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)

  x = tf.reshape(features, [-1, 32, 32, 3])
  x = tf.layers.batch_normalization(x, training=training, name='norm_128')

  i = 1
  for filters in [32, 64]:
    x = tf.layers.conv2d(x, filters=filters, kernel_size=3, activation=None, name='conv{}'.format(i))
    x = tf.layers.batch_normalization(x, training=training, name='norm{}'.format(i))
    x = tf.nn.relu(x, name='act{}'.format(i))
    i += 1

    x = tf.layers.conv2d(x, filters=filters * 2, kernel_size=3, strides=2, activation=None, name='pool{}'.format(i))
    x = tf.layers.batch_normalization(x, training=training, name='norm{}'.format(i))
    x = tf.nn.relu(x, name='act{}'.format(i))
    i += 1

  flat = tf.contrib.layers.flatten(x, scope='flatten')
  dropout = tf.layers.dropout(flat, rate=params['dropout_rate'], training=training, name='dropout')
  output_layer = tf.layers.dense(dropout, units=1, name='output_layer')

  predictions = tf.reshape(output_layer, [-1])

  predictions_dict = {
    'pred': predictions,
  }

  if mode == tf.estimator.ModeKeys.PREDICT:
    return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions_dict)

  loss = tf.losses.mean_squared_error(labels=labels, predictions=predictions)

  tf.summary.scalar('loss', loss)
  tf.summary.histogram('prediction', predictions)
  tf.summary.scalar('prediction', tf.reduce_mean(predictions))

  optimizer = tf.train.AdamOptimizer(learning_rate=params['learning_rate'])
  with tf.control_dependencies(extra_update_ops):
    train_op = optimizer.minimize(loss=loss, global_step=tf.train.get_global_step())
  eval_metric_ops = {
    'rmse_val': tf.metrics.root_mean_squared_error(labels=tf.cast(labels, tf.float32), predictions=predictions)
  }

  tf.summary.scalar('rmse_train', eval_metric_ops['rmse_val'][1])


  return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op, eval_metric_ops=eval_metric_ops)

Answer 1

__已编辑__

您的代码中唯一的随机点是drop out。在训练之后和预测时间，将退出的保持概率设置为1。因为 drop out 层随机选择传递给它们的变量子集，并且在该子集上进行训练以防止过度拟合。