TFSlim - 为 VGG16 加载保存的检查点时出现问题答案

【问题标题】：TFSlim - problems loading saved checkpoint for VGG16TFSlim - 为 VGG16 加载保存的检查点时出现问题
【发布时间】：2017-03-14 00:42:37
【问题描述】：

(1) 我正在尝试使用 TFSlim 微调 VGG-16 网络，方法是将预训练的权重加载到除fc8 层之外的所有层中。我通过使用 TF-SLIm 函数实现了这一点，如下所示：

import tensorflow as tf
import tensorflow.contrib.slim as slim
import tensorflow.contrib.slim.nets as nets

vgg = nets.vgg

# Specify where the Model, trained on ImageNet, was saved.
model_path = 'path/to/vgg_16.ckpt'

# Specify where the new model will live:
log_dir = 'path/to/log/'

images = tf.placeholder(tf.float32, [None, 224, 224, 3])
predictions = vgg.vgg_16(images)

variables_to_restore = slim.get_variables_to_restore(exclude=['fc8'])
restorer = tf.train.Saver(variables_to_restore)




init = tf.initialize_all_variables()

with tf.Session() as sess:
   sess.run(init)
   restorer.restore(sess,model_path)
   print "model restored"

只要我不更改 VGG16 模型的 num_classes，它就可以正常工作。我想做的是将num_classes从1000更改为200。我的印象是，如果我通过定义一个新的vgg16-modified类来替换fc8来产生200个输出，（沿着使用variables_to_restore = slim.get_variables_to_restore(exclude=['fc8']) 表示一切都会好起来的。但是，tensorflow 抱怨尺寸不匹配：

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [1,1,4096,200] rhs shape= [1,1,4096,1000]

那么，如何真正做到这一点？ TFSlim 的文档非常不完整，并且在 Github 上散布着几个版本 - 所以在那里没有得到太多帮助。

【问题讨论】：

标签： tensorflow tf-slim

【解决方案1】：

你可以试试slim的恢复方式——slim.assign_from_checkpoint。

对应部分：

*************************************************
* Fine-Tuning Part of a model from a checkpoint *
*************************************************
Rather than initializing all of the weights of a given model, we sometimes
only want to restore some of the weights from a checkpoint. To do this, one
need only filter those variables to initialize as follows:
  ...
  # Create the train_op
  train_op = slim.learning.create_train_op(total_loss, optimizer)
  checkpoint_path = '/path/to/old_model_checkpoint'
  # Specify the variables to restore via a list of inclusion or exclusion
  # patterns:
  variables_to_restore = slim.get_variables_to_restore(
      include=["conv"], exclude=["fc8", "fc9])
  # or
  variables_to_restore = slim.get_variables_to_restore(exclude=["conv"])
  init_assign_op, init_feed_dict = slim.assign_from_checkpoint(
      checkpoint_path, variables_to_restore)
  # Create an initial assignment function.
  def InitAssignFn(sess):
      sess.run(init_assign_op, init_feed_dict)
  # Run training.
  slim.learning.train(train_op, my_log_dir, init_fn=InitAssignFn)

更新

我尝试了以下方法：

import tensorflow as tf
import tensorflow.contrib.slim as slim
import tensorflow.contrib.slim.nets as nets
images = tf.placeholder(tf.float32, [None, 224, 224, 3])
predictions = nets.vgg.vgg_16(images)
print [v.name for v in slim.get_variables_to_restore(exclude=['fc8']) ]

得到这个输出（缩短）：

[u'vgg_16/conv1/conv1_1/weights:0',
 u'vgg_16/conv1/conv1_1/biases:0',
 …
 u'vgg_16/fc6/weights:0',
 u'vgg_16/fc6/biases:0',
 u'vgg_16/fc7/weights:0',
 u'vgg_16/fc7/biases:0',
 u'vgg_16/fc8/weights:0',
 u'vgg_16/fc8/biases:0']

所以看起来你应该在作用域前加上vgg_16:

print [v.name for v in slim.get_variables_to_restore(exclude=['vgg_16/fc8']) ]

给出（缩短）：

[u'vgg_16/conv1/conv1_1/weights:0',
 u'vgg_16/conv1/conv1_1/biases:0',
 …
 u'vgg_16/fc6/weights:0',
 u'vgg_16/fc6/biases:0',
 u'vgg_16/fc7/weights:0',
 u'vgg_16/fc7/biases:0']

更新 2

执行没有错误的完整示例（在我的系统上）。

import tensorflow as tf
import tensorflow.contrib.slim as slim
import tensorflow.contrib.slim.nets as nets

s = tf.Session(config=tf.ConfigProto(gpu_options={'allow_growth':True}))

images = tf.placeholder(tf.float32, [None, 224, 224, 3])
predictions = nets.vgg.vgg_16(images, 200)
variables_to_restore = slim.get_variables_to_restore(exclude=['vgg_16/fc8'])
init_assign_op, init_feed_dict = slim.assign_from_checkpoint('./vgg16.ckpt', variables_to_restore)
s.run(init_assign_op, init_feed_dict)

在上面的示例中，vgg16.ckpt 是tf.train.Saver 为 1000 类 VGG16 模型保存的检查点。

将此检查点与 200 类模型的所有变量（包括 fc8）一起使用会产生以下错误：

init_assign_op, init_feed_dict = slim.assign_from_checkpoint('./vgg16.ckpt', slim.get_variables_to_restore())
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
      1 init_assign_op, init_feed_dict = slim.assign_from_checkpoint(
----> 2       './vgg16.ckpt', slim.get_variables_to_restore())

/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/variables.pyc in assign_from_checkpoint(model_path, var_list)
    527     assign_ops.append(var.assign(placeholder_value))
    528
--> 529     feed_dict[placeholder_value] = var_value.reshape(var.get_shape())
    530
    531   assign_op = control_flow_ops.group(*assign_ops)

ValueError: total size of new array must be unchanged

【讨论】：

我已经尝试过这种方法。它仍然给我同样的错误：InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [1,1,4096,200] rhs shape= [1,1,4096,1000] [[Node: save_1/Assign_32 = Assign[T=DT_FLOAT, _class=["loc:@vgg_16/fc8/weights"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](vgg_16/fc8/weights, save_1/restore_slice_32/_3)]]
嗨，谢谢。这似乎可以解决问题，只要num_classes 与 VGG16 一致。如果您使用 200 个而不是 1000 个类来初始化 vgg_16 的实例，错误仍然会出现。
对我来说，排除 vgg_16/fc8 范围有效。我在答案中添加了另一个更新。
@dm0_ 你的系统是什么，你用的是什么版本的tensorflow？
@ThiagoBalbo 系统是 Ubuntu 14.04。我不确定TF版本。我认为是当时的最新版本。可能是 0.12。