使用 MNIST 数据集更改卷积层和池化层的 Tensorflow 数量

Question

我使用的是 Windows 10 专业版、python 3.6.2rc1、Visual Studio 2017 和 Tensorflow。我正在使用以下链接中的教程中的 Tensorflow 示例：

https://www.tensorflow.org/tutorials/layers

在展平最后一层（第 3 层）之前，我添加了另一层卷积和池化，以查看准确性是否发生变化。

我添加的代码如下：

## Input Tensor Shape: [batch_size, 7, 7, 64]
## Output Tensor Shape: [batch_size, 7, 7, 64]
conv3 = tf.layers.conv2d(
    inputs=pool2,
    filters=64,
    kernel_size=[3, 3],
    padding=1,
    activation=tf.nn.relu)

pool3 = tf.layers.max_pooling2d(inputs=conv3, pool_size=[2, 2], strides=1)
pool3_flat = tf.reshape(pool3, [-1, 7* 7 * 64])

我将 padding 更改为 1 并将 stride 更改为 1 的原因是为了确保输出的大小与输入的大小相同。但是在添加这个新层后，我收到以下警告并且没有显示任何结果，程序结束：

Estimator 与 Scikit Learn 界面分离，方法是移至 单独的类 SKCompat。参数 x、y 和 batch_size 仅 在 SKCompat 类中可用，Estimator 将只接受 input_fn。 转换示例： est = Estimator(...) -> est = SKCompat(Estimator(...)) WARNING:tensorflow:From E:\Apps\DA2CNNTest\TFHWDetection WITH More Layers\TFClassification\TFClassification\TFClassification.py:179：调用 BaseEstimator.fit（来自 tensorflow.contrib.learn.python.learn.estimators.estimator）与 batch_size 是已弃用并将在 2016-12-01 之后删除。 更新说明： 通过移动到 Estimator 与 Scikit Learn 界面分离 单独的类 SKCompat。参数 x、y 和 batch_size 仅 在 SKCompat 类中可用，Estimator 将只接受 input_fn。 转换示例： est = Estimator(...) -> est = SKCompat(Estimator(...)) 线程“MainThread”(0x5c8) 已退出，代码为 0 (0x0)。 程序“[13468] python.exe”已退出，代码为 1 (0x1)。

如果不添加此层，它可以正常工作。为了解决这个问题，我改变了conv3和pool3如下：

conv3 = tf.layers.conv2d(
    inputs=pool2,
    filters=64,
    kernel_size=[5, 5],
    padding="same",
    activation=tf.nn.relu)

# Input Tensor Shape: [batch_size, 7, 7, 64]
# Output Tensor Shape: [batch_size, 3, 3, 64]
pool3 = tf.layers.max_pooling2d(inputs=conv3, pool_size=[2, 2], strides=2)
pool3_flat = tf.reshape(pool3, [-1, 3* 3 * 64])

但后来我遇到了另一个错误

nist_classifier.fit(
    x=train_data,
    y=train_labels,
    batch_size=100,
    steps=20000,
    monitors=[logging_hook])

如下：

tensorflow.python.framework.errors_impl.NotFoundError：检查点中未找到密钥 conv2d_2/bias [[节点：save/RestoreV2_5 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_5/tensor_names, save/恢复V2_5/shape_and_slices)]]

错误是指monitors=[logging_hook]。

我的整个代码如下，如您所见，我已经用 padding=1 注释了前一个代码。

如果你能指导我我的错误是什么以及为什么会这样，我真的很感激。此外，我在第 3 层的输入和输出的维度是正确的吗？

完整代码：

"""Convolutional Neural Network Estimator for MNIST, built with tf.layers."""

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import numpy as np
import tensorflow as tf

from tensorflow.contrib import learn
from tensorflow.contrib.learn.python.learn.estimators import model_fn as model_fn_lib



tf.logging.set_verbosity(tf.logging.INFO)

def cnn_model_fn(features, labels, mode):
    """Model function for CNN."""

    input_layer = tf.reshape(features, [-1, 28, 28, 1])


# Input Tensor Shape: [batch_size, 28, 28, 1]
# Output Tensor Shape: [batch_size, 28, 28, 32]
conv1 = tf.layers.conv2d(
    inputs=input_layer,
    filters=32,       
    kernel_size=[5, 5],
    padding="same",
    activation=tf.nn.relu)


# Input Tensor Shape: [batch_size, 28, 28, 32]
# Output Tensor Shape: [batch_size, 14, 14, 32]
pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2], strides=2)

# Convolutional Layer #2
# Input Tensor Shape: [batch_size, 14, 14, 32]
# Output Tensor Shape: [batch_size, 14, 14, 64]

conv2 = tf.layers.conv2d(
    inputs=pool1,
    filters=64,
    kernel_size=[5, 5],
    padding="same",
    activation=tf.nn.relu)

# Pooling Layer #2
# Input Tensor Shape: [batch_size, 14, 14, 64]
# Output Tensor Shape: [batch_size, 7, 7, 64]
pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2], strides=2)


'''Adding a new layer of conv and pool'''
## Input Tensor Shape: [batch_size, 7, 7, 32]
## Output Tensor Shape: [batch_size, 7, 7, 64]
#conv3 = tf.layers.conv2d(
#    inputs=pool2,
#    filters=64,
#    kernel_size=[3, 3],
#    padding=1,
#    activation=tf.nn.relu)


## Input Tensor Shape: [batch_size, 7, 7, 64]
## Output Tensor Shape: [batch_size, 7, 7, 64]
#pool3 = tf.layers.max_pooling2d(inputs=conv3, pool_size=[2, 2], strides=1)
#pool3_flat = tf.reshape(pool3, [-1, 7* 7 * 64])


# Input Tensor Shape: [batch_size, 7, 7, 64]
# Output Tensor Shape: [batch_size, 7, 7, 64]
conv3 = tf.layers.conv2d(
    inputs=pool2,
    filters=64,
    kernel_size=[5, 5],
    padding="same",
    activation=tf.nn.relu)

# Input Tensor Shape: [batch_size, 7, 7, 64]
# Output Tensor Shape: [batch_size, 3, 3, 64]
pool3 = tf.layers.max_pooling2d(inputs=conv3, pool_size=[2, 2], strides=2)


'''End of manipulation'''


# Input Tensor Shape: [batch_size, 3, 3, 64]
# Output Tensor Shape: [batch_size, 3 * 3 * 64]
pool3_flat = tf.reshape(pool3, [-1, 3* 3 * 64])

# Input Tensor Shape: [batch_size, 3 * 3 * 64]
# Output Tensor Shape: [batch_size, 1024]
# dense(). Constructs a dense layer. Takes number of neurons and activation function as arguments.
dense = tf.layers.dense(inputs=pool3_flat, units=1024, activation=tf.nn.relu)

# Add dropout operation; 0.6 probability that element will be kept
dropout = tf.layers.dropout(
    inputs=dense, rate=0.4, training=mode == learn.ModeKeys.TRAIN)


logits = tf.layers.dense(inputs=dropout, units=10)

loss = None
train_op = None

# Calculate Loss (for both TRAIN and EVAL modes)
if mode != learn.ModeKeys.INFER:
    onehot_labels = tf.one_hot(indices=tf.cast(labels, tf.int32), depth=10)
    loss = tf.losses.softmax_cross_entropy(
        onehot_labels=onehot_labels, logits=logits)

# Configure the Training Op (for TRAIN mode)
if mode == learn.ModeKeys.TRAIN:
    train_op = tf.contrib.layers.optimize_loss(
    loss=loss,
    global_step=tf.contrib.framework.get_global_step(),
    learning_rate=0.001,
    optimizer="SGD")

# Generate Predictions
# The logits layer of our model returns our predictions as raw values in a [batch_size, 10]-dimensional tensor.
predictions = {
    "classes": tf.argmax(
        input=logits, axis=1),
    "probabilities": tf.nn.softmax(
        logits, name="softmax_tensor")
}

# Return a ModelFnOps object
return model_fn_lib.ModelFnOps(
    mode=mode, predictions=predictions, loss=loss, train_op=train_op)

def main(unused_argv):
# Load training and eval data
mnist = learn.datasets.load_dataset("mnist")
train_data = mnist.train.images  # Returns np.array
train_labels = np.asarray(mnist.train.labels, dtype=np.int32)
eval_data = mnist.test.images  # Returns np.array
eval_labels = np.asarray(mnist.test.labels, dtype=np.int32)

# Create the Estimator
mnist_classifier = learn.Estimator(
    model_fn=cnn_model_fn, model_dir="/tmp/mnist_convnet_model")

# Set up logging for predictions
# Log the values in the "Softmax" tensor with label "probabilities"
tensors_to_log = {"probabilities": "softmax_tensor"}
logging_hook = tf.train.LoggingTensorHook(
    tensors=tensors_to_log, every_n_iter=50)

# Train the model
mnist_classifier.fit(
    x=train_data,
    y=train_labels,
    batch_size=100,
    steps=20000,
    monitors=[logging_hook])

# Configure the accuracy metric for evaluation
#change metrics variable name
metricss = {
    "accuracy":
        learn.MetricSpec(
            metric_fn=tf.metrics.accuracy, prediction_key="classes"),
}

#Evaluate the model and print results
#for i in range(100)
eval_results = mnist_classifier.evaluate(
    x=eval_data[0:100], y=eval_labels[0:100], metrics=metricss)
print(eval_results)

if __name__ == "__main__":
    tf.app.run()

Answer 1

一个简单的解决方法是为模型定义一个自定义检查点目录，如下所示。

tf.train.generate_checkpoint_state_proto("/tmp/","/tmp/mnist_convnet_model")

这解决了 MNIST 示例的问题，还让您可以访问可以控制检查点的位置。

Answer 2

错误看起来像model_dir 中可用的训练模型与当前图形更改冲突。 Estimator 从保存的模型目录加载检查点，并从之前保存的模型继续训练。因此，每当您对模型进行更改时，您都需要删除旧模型并重新开始训练。